______________________________________________________________________ DRAFT TRANSCRIPT Routing SIG Wednesday 7 September 2005 11.00am ______________________________________________________________________ PHILIP SMITH: OK, good afternoon, everybody. I think we should make a start to the Routing SIG. My name is Philip Smith. I'm one of the chairs of the SIG. My co-chair is Randy Bush and Randy is sitting over guarding the door this afternoon. Just some housekeeping announcements before we actually get into the presentations. I would firstly like to thank the sponsor for the day, the gold sponsor is Nominum so thanks very much for their support for this afternoon. Just to remind you about the Nominum lucky draw - submit your entries at the registration desk and you can possibly win an iPod. Oh, yes - with that, the box will be closed at noon on Friday, so make sure you get your entries in by then and the winner will be announced at the end of the Annual Member Meeting on the Friday afternoon. What else? Tea is available this afternoon outside the meeting room here. For those of you who are coming to the APOPS BoF session this evening, it's moved to the Function Room 7 starting at six o'clock. And hopefully finishing by seven o'clock, certainly in time for the social event. And the social event tonight is sponsored by VNNIC. It's being held at the Vietnam Museum of Ethnology. If you haven't purchased a ticket and would like to go, please see the registration desk. You must have a ticket. The first bus leaves at 6:30 and the last bus leaves at 7:10 so please don't miss it. Otherwise you have a rather long walk or you'll just miss the social. It is an open-air event so I would actually recommend that you wear a long-sleeved shirt and long trousers - keep yourself well covered-up. Mosquito repellent is available if you don't have your own. What else? The MyAPNIC demo is running all day at the help desk. The help desk available during break times and don't forget the onsite notice board for last-minute changes and so forth. OK, moving on to the Routing SIG. The mailing list for the routing SIG is sig-routing@lists.apnic.net. If you're not a member of the routing SIG mailing list and you'd like to join it, the interface is available through the APNIC mailman system. If you want to contact either Randy or myself, we have a mailing list, http://mailman.apnic.net/mailman/listinfo/sig-routing. Now, we have two sessions today - the first session is from now till 3:30 and we have three presenters. Second session will be from 4:00 to 5:30. Now, before I invite Geoff up to give the first presentation, I'd like to remind everybody who is speaking in the room here to speak clearly, both for the benefit of the stenographers and also for the benefit of online viewers and listeners. When you come up to the microphone, if you would state your name and the organisation you work for that would be helpful so we can identify who is actually asking the question. So, this afternoon, we have two presentations by Geoff Huston. The first one is about the auto-detection of hijacked prefixes and the second one is on AS consumption patterns. And, after that, we have another presentation by APNIC from George Michaelson about the APNIC resource certification. All three presentations are informational and I would like Geoff to come up and prepare himself. GEOFF HUSTON: Good afternoon. My name is Geoff Huston. I'm with APNIC. This afternoon, I'd actually like to firstly talk about what I think is a failed experiment. I was actually trying to find if it was possible to detect various forms of hijacking of addresses by looking inside the routing table and looking for pointers as to what it might look like. But, as you will see, I failed. So let's just go through the experiment anyway and see what lessons can be learned from this particular piece of failed work. Firstly, what is address hijacking and why is it important, I suppose? Increasingly, as you're probably aware, the Internet is a relatively hostile place and there are certainly large amounts of various forms of attack whether it's spam, whether it's deliberately trying to reroute addresses, whether it's all kinds of use of the network which I think any of us would consider to be more abuse than use. This hijacking is certainly one way that a party can try and hide its identity - just use someone else's address space for the period of time for which you want to do something naughty. What is a hijacked address? Well, here's what I would consider to be a hijacked address - it's when someone is using an address prefix that the original controller of that resource has no knowledge and didn't authorise its use. So it's in the routing system. It's not a bogon. It's not that this address prefix is drawn from a block currently held in IANA. It's not an unallocated piece of address. It actually is an address that, some time in the past, the RIRs handed out to someone. It may involve playing games with your local RIR to try and get them to change the record of ownership in the databases but I'm not necessarily looking at those databases, so it may include some aspect of identity fraud but I'm not really trying to attack the problem from there. And generally, as I've said, hijacking is normally a part and parcel of an attack that involves identity cloaking, in other words trying to hide who you are. We have seen this in terms of folk using addresses generating large amounts of spam and then dropping away from that address and going to use the next one. In terms of various attack-launching systems where you're trying to enlist the help of a few million zombie machines to do your bidding, again using a hijacked address tends to be a little bit tempting in terms of trying to hide who you are. The real question I suppose, is how prevalent is this? And can we actually try and find it? Certainly, I would suggest it's been very hard to isolate these incidents. So I thought, "Well, what kind of signature are you looking for if you're looking for an address that's been hijacked?" Put yourself on the other side. What sort of addresses would be tempting to use? You really would like to use an address that's been dormant for a long time, an address that no-one's actually been using and probably may not notice if you actually invade there and use it for a short period. I thought, “Well, maybe, the signature of a hijacked address prefix is one that hasn't been advertised for quite some time.” So there's a long idle period between its last use and when the hijack happens. Also, what we've seen, certainly, is when you hijack an address, you don't use it for a long period of time. You use it for a short period of time and then run away. Pretty much you're probably not hijacking an AS number as well. They're actually harder to do, I think. I thought, "Maybe what I'm looking for is advertisements that come from entirely origin to first hop AS to the last time I saw it." So this address prefix that was stably announced at one time in the Net all of a sudden appears in a completely different location for a short period of time. It's more than likely, if you're going to do this, you're trying not to use something more specific of someone else's. They're not covered been an aggregate announcement. So that's the kind of signature that I thought would be a reasonable target to say, "If I can find some of these, maybe they are hijacked prefixes." So the next thing is what database can I use or what set of data is available to apply this? And what I found is that actually the routeviews route collection is very, very useful for this. Because, if you actually have a look at a resource like routeviews, the University of Oregon, it gives you snapshots of the BGP system going all the way back to November 1997. So, if you actually do differences on each successive snapshot over that nine years, there's a little bit of data but you can comb your way through it. You can actually start to answer some basic questions about each prefix. Over that period, we've seen some 650,000 prefixes that have been announced. Now, because there are only 160,000 in the BGP table today, that means an awful lot of prefixes have been announced that are not announced any more. So, in looking at that data collection, you can actually find out when was the prefix first advertised and when was it last advertised? When was it withdrawn? You can also, for each prefix, work out the announcing autonomous system number and, of course, if you look at the autonomous system card, you can also see the first hop. And at the same time you can also look at what other prefixes were being advertised by that same autonomous system number. So, it's a remarkable valuable resource and if you're prepared to do the data mining, you can get useful information about the use of any individual address prefix. So how can I do this? One way is to actually look at the update logs themselves, look at the entire BGP traffic that actually occurs with routeviews so you can see every individual announcement and every individual withdrawal. The problem that I see is that if you actually look at the BGP protocol - and Randy Bush has done some presentations in previous APNICs that describe this in some detail and they've been very valuable - what you actually find is that there are two contributors to an update or a withdrawal. One is a network event and the other one is the BGP protocol itself oscillating until it converges once more into some state or other. There is a high frequency component of the updates where you see a whole bunch of updates and withdrawals. And there's a low frequency component, which is actually more about network connectivity. So if you use the updates, the problem is you've actually got to apply a high-frequency noise filter and try and figure out when you get a series of updates and withdrawals what was the root cause event that actually caused that and only look at the root cause events. There have been a number of papers I've noticed about trying to do root cause analysis and root cause identification. It's very hard. And the amount of data you've got to go through and the amount of analysis you've got to do is extremely detailed and for this particular work, I thought, "Well, maybe I can use a short cut and simply try and look at less data." So what I've done is actually use the successive BGP snapshots, which are taken every two hours, and simply do a diff on those so the highest frequency I've got is now just two hours. So, in that respect, I'm hoping I'm eliminating a fair deal of the noise caused by simply BGP protocol doing what BGP protocol likes to do. The first step in noise reduction is to actually look at the snapshots, not the updates. So, OK, the first graph (refers to slide). How many prefixes go away and then come back at an entirely different location? And how long did they go away before they come back? So this graph here, which is a pretty classic exponential decay - on the left-hand side, the access is from zero to 7,000 prefixes and they're in months - one month, seven months, one year, two years and so on. So an awful lot of prefixes go dormant for a month and then reappear again - over 6,000, which is a lot. But there are some prefixes - and there are at least 200 or 300 - that went quiet for three years and then came back. And, in between, there are a whole bunch of other measurements between sort of 1,000 and 4,000 of prefixes that disappeared for two months up to three years and then came back. The other aspect of this is it's an exponential decay curve. What that means is that, when you have a prefix that disappears from the routing system, it's equally likely to reappear today or tomorrow. In other words, the probability of a prefix reappearing is constant. That is very surprising. I didn't realise - and I'm not sure many else of us did - that there is this continual reappearance of old addresses back into the routing system from different autonomous system numbers. In other words, three a fair deal of address movement. Now, all of that could be hijacking but that seems to be unlikely. That's a very, very big number. So it's probably not. This is probably just behaviour. So I thought, "Well, OK, maybe I'll sharpen up what I'm looking at. Maybe what I'm looking for are prefixes that have been disappeared, went away for at least two months and then came up for only three days or less, and then disappeared again." So these are prefixes that sort of spike. You think, "Well, that would be pretty interesting. Maybe I can isolate 100 at most of these prefixes and maybe I'm in a hijack-capable set." (Refers to slide) The red line shows that refinement against the original data set. There are at least 1,500 prefixes over this period that were down for two months, back up for a couple of days and then down for at least another month. Even trying to sharpen it, I've still got – (a) - an exponential decay pattern and – (b) - no real signature of what a hijacked prefix might be. That data doesn't tell me anything. Not doing very well. So then I said, "OK, maybe I'll change this a little bit more - down for two months, up for 5 to 14 days, so it was up a while and did some damage - and then again for another month." These are prefixes that come up for a week or two and come down. Exponential decay. Admittedly, there's only a few hundred now, around the 2-month to 3-month period, there are a couple of hundred prefixes that came up for a couple of weeks and then disappeared again. They were up for at least a week. It doesn't tell me anything. It's still the same exponential decay. I can't see anything in that data. What it does say, though, is that the behaviour of addresses is not - at least in this area - what many of us may think. You do your announcements in BGP and let it go. At the edge of your autonomous system, you're doing stack-based announcements using some kind of aggregate statement and you try to make your prefix announcement as stable as possible. But, inside the Internet, there are a bunch of prefixes, somewhere around there and there (refers to slide) That deliberately have come up for a small perilled of time and then disappear again. It seems to be normal behaviour. I'm certainly not getting anywhere with looking at hijacking. So I sort of came to the conclusion that I think I'm looking in the wrong spot for that kind of activity, that address announcement patterns actually are a very good indicator of what might be an address hijack. There's no clear signature I can see in that data that says, "If you apply this, you've got a pretty good indication that the address is not what it appears to be." Is that someone asking a question up there? TOM VEST: Did you do any AS analysis? Are there any usual suspects? Just curious if you identified the usual suspects in terms of originating ASes for those recurrent... GEOFF HUSTON: No, I haven't done work on what the originating AS is and what the new AS was, nor even the first time I was a little bit shaken in that I couldn't see any reliable pattern in that data and stopped looking at that point, no, I haven't looked at that area. Not sure if there's anything in there but certainly the data is available if you want to start looking. RANDY BUSH: Randy Bush, IIJ. We have done such analysis and, among the more - they're the obvious, of course, but also, military services, of large - people with large militaries, specifically, what looks like conducted experiments if they can hijack address space of other countries. GEOFF HUSTON: I didn't really want to hear that answer, but I'm sure, yes, what can I say? TOM VEST: Nothing. GEOFF HUSTON: Nothing. I don't think, in and of itself, this technique will give you sure fire pointers, but I do think there is something there that, if you're an ISP and folk are asking you, "Please route this prefix," then you can look at its prefix history and, if it has been dormant for an awfully long time, and its originating AS when you last saw it was completely different, you certainly have grounds to say, "Maybe I should check out this address prefix in a little bit more detail, and have a closer look at the registry data and anything else that might help me in figuring out whether or not I should accept that routing request." It's certainly not THE indicator but it's useful. Careful checking of where the address came from, who originally owned it, what its past is, is probably good sense. But it's not, as I said, in and of itself, enough. And, quite frankly too, you also have to look at it from the perspective of an ISP. The ISP industry is not the richest industry in the world and certainly having the skilled staff to do data mining for address prefixes costs money. Certainly, doing that kind of checking work is quite a tedious and expensive job, or at least it can be. So, will ISPs want to do this is a very good question. Somehow, it doesn't reassure me, and it probably shouldn't reassure you, that there's no clear way, using this kind of technique, it actually see if a routing request is real or not, to see if an address being advertised may actually be a hijack as opposed to normal business. So I think a deeper issue that I've been skirting around, that I hope George will tell us a lot more about, which is that the problem is really address and routing security and, from an ISP's point of view, when a customer comes to you and says, "Please route..." there are a few basic questions that need to be asked. Is the address prefix they're routing or attempting to route to you valid? Did it come through the system? Is it a normal prefix? Where is it coming from? Who's injecting it? Who injected it last time? Do they have the necessary credentials? Because it may well be that the real address-holder has asked an ISP, who is your customer, to inject it in. So can you test that the real address-holder has, indeed, authorised the person who is contacting you to say, "Please accept this prefix. Please route it"? And the next thing is, "When I see a prefix coming through the routing system, is that real? Does it represent an acceptable way of forwarding, or is someone trying to disrupt the forwarding system?" The fact that you can't see it very clearly doesn't mean address hijacking doesn't happen. It does. It happens, I think, more than most of us are comfortable with. Is a relatively consistent level as far as we can tell, of activity about disruption and subversion in the routing system. Certainly one of the easiest ways of causing mayhem and disruption in the Internet is indeed to advertise other people's prefixes. So, yes, address hijacking is one part of this insecurity and it is a deep and significant problem. But I don't think looking at prefix history announcements is actually an efficient, reliable and good way of going about it. What I personally would like to see is I think stuff that's been worked on in the IETF in the routing protocol security requirements are but also in many other forums - it seems that, while public key infrastructure has been largely derided and cast as being irrelevant for many applications, I actually think for ISPs and for routing and addressing, public key infrastructure is actually your friend and is actually going to be useful. Because what you can do, I believe, is automate a fair deal of the checking. What you're trying to do, instead of trying to figure out whether a request is bogus or not, whether someone is lying, a good way of doing it is to insist that, when you get requests, they are accompanied with the credentials to say, "It's me, the address is authentic. These are the signatures of the folk who have control of the prefix." And here is the equivalent signature of the autonomous system number that's originating the advertisement and here is the cross-matching authority, here is the key for the sign authority from the address owner to that originating address to say, "Please route this prefix." If you had those credentials and had a trust point where you can validate the private keys against the public key, you could, indeed, very efficiently check what's real and what's not. So in actual fact, the public key infrastructure provides an answer that I think is much stronger. Indeed, it would also be good if, in effect, the attestation referred to an address allocation path. What I mean from that is, when I see someone claiming, "I have this address prefix," I can find a signed address certification which says, “This blocks has a certificate which hands it to the ISP, and here's another one that says, "This ISP handed that to that particular end point". So I can see a chain of allocations, signed, in terms of certificates, that allow me to validate that that address prefix is, indeed, correctly identified from the controller, or that particular prefix. It would be even cooler if that was associated with the route advertisement. So, rather than having a manual process of "please route", when I see a BGP update with that prefix, there are some credentials in the BGP update message that I can check against the certificate repository or some other similar mechanism, so now I can automate this that, if I see an update with an attached signature block, I can actually have some indication of whether that is validatable, whether it's authentic, if I had that and, all of a sudden, I have a preference - if I see the same prefix coming from two different peers and one has a signature chain attached, I have a strong preference to say, "At least I understand the validity of what I'm hearing from the left. Even though what I'm hearing from the right may or may not be real, without attached key blocks, it's hard to understand that." It would be good if that was part of the indicator. And if it formed part of the pref sit. The situation today is nothing like that, is it? Quite frankly, what's inside the BGP table, it's very, very hard to understand that everything inside there is real. More like than just protecting your routers. It's more than just checking sums on BGP peering sessions. It's more, I think, than even the routing databases. They really aren't the full story. Quite frankly, it's a case of not only protecting your routers, but understanding the authenticity and validity of the protocol payload, what is actually being described in BGP. It seems to me that it's a case of sticking your fingers in the dyke and hoping the water will flow around you. It has, it will and there's a lot of it. This is no way to do security in BGP. All of that effort, to my mind, appears to be wasted effort. What would be really good is if we actually do some basic security functions in the routing domain that address and AS certificate public key infrastructure underpins the routing system, so that then I know that what I'm passing around in the routing system has authenticity. And it would be really good when I actually pass that around in the routing system that I can verify the information as it comes through to me. That would be absolutely excellent. It would make the whole issue of hijacking really hard. Because, in such a system, if I honestly wanted to use your address, your prefix, and advertise it, I have to steal your private key. I have to steal an enormous amount of information from you that hopefully I could never, ever steal from you. So, in that case, the real answer here doesn't, to my mind, involve mucking around with the history of prefix announcements. I think it's diverse. The true work, as far as I can see, is actually about public key infrastructure and certification of resources and using that in BGP to actually understand what's good data and what is potential hijacking. Any questions? Tom? TOM VEST: Geoff, have you also been looking - with this analysis and routeview stuff, I guess about 680 million which were routed at the beginning of that archive. So it's sort of those really old archive that's from legacy prefixes, probably, I would imagine, would be a little more opaque for this kind of analysis - you can't see where they started. GEOFF HUSTON: The original ones, the 1997, I have no idea when they started. TOM VEST: Right. The ones in production then. GEOFF HUSTON: What I'm looking for is not the length of time they were in production. I'm looking at the length of time they were dormant. They go to sleep for some time. And I was actually second guessing and thinking, "If I was going to hijack a prefix and try not to get noticed immediately, the most likely ones are the ones that are valid, the ones that were assigned, that hadn't been used for some time so that the true owner might not notice me." So I was looking for extended dormant periods and then announcements. It didn't seem to get anywhere. RANDY BUSH: Geoff, I think part of what's happening in this discussion is you may be trying to focus on hijackers who are trying to gain space and use it in a long-term - in a pseudo-legitimate fashion. In other words, somebody who found some dormant space, grabbed it and was using it for three years now, etc. It's interesting that what we've been working on instead - I believe that's very good work and very important, by the way - it's interesting that, as usual, you're at the macro end of the game and I'm at the micro. What we've been working on in Oregon is looking at people who pop up an announcement for 10 minutes and take it away and go to a different space - in other words, the spammers. So we are looking at the advertisements, not at the table dumps. And the noise - one of the ways we're qualifying it is, "Is the original AS different than the origin AS of the covering prefix?" OK. Is that origin AS doing this multiple times, multiple places, etc, etc? We're finding an immense amount of this stuff going on. GEOFF HUSTON: The problem with an immense amount is you start to gain at least the informal feeling that this isn't abuse. This is actually folk doing things, whatever they may be doing. RANDY BUSH: Well, I think the question is what do we mean by abuse? You, possibly, the study you're doing, could be construed as abuse of the process by which one gets address space and, on the other hand, what we're saying is the abuse of BGP announcements to do not-nice things on the net. GEOFF HUSTON: Right. RANDY BUSH: Right and that's what's interesting in contrasting the two. GEOFF HUSTON: Yeah. Absolutely. Any other questions? I'll sort of park that topic there then because George is actually going to continue with this in his presentation. So can we switch over to the next one? PHILIP SMITH: Yes, please, if you would. GEOFF HUSTON: The next presentation is actually about autonomous system number consumption patterns and this is some of the work that I've been doing - there's also some parallel work being done by RIPE at this point in time looking at the autonomous system number pool and what's happening to it. So let me just really sort of go over this in terms of what it is and why it's interesting. Everyone who announces in BGP, of course, publicly, as distinct from private use, has to use a number drawn from the autonomous number system pool. There are actually only 64,510. 1,024 have drawn away for private use, the top ones. Zero is drawn away as well, so that's 1,025. And also autonomous system number 23,456 is reserved for use in the 4-byte transition. So there are 64,51 in use and right now we've got through 39,934 of them, which is getting well beyond the halfway point. So there are 24,576 remaining. By the look of it, we're going to get through them. Sooner or later, that is going to get exhausted. So the question is what are we going to do about it? And when do we need to do something about it? This has been identified quite some years ago in the IETF and Enke Chen did some good work in the routing group and documented in this particular draft - draft-ietf-idr-as4bytes-10.txt - which describes how to use a 32-bit field and also describes in some detail the transition arrangements as to how to make this happen. It's been sitting there for some time as a draft and it hasn't actually ever made it into the standards tract. And part of the reason is that although it's been out there, very few folk have implemented it and, for a long time, only one implementation existed. In talking about this some months back, it also emerged that there is a second implementation. So we can do the next step in the formal IETF standards process of producing an implementation report of the two independent implementations and saying, "Please take this out as a draft and publish it as an RFC." Because, you see, getting it to a standard is only really a small part of the step. You could argue it's not a particularly necessary step. There's lots of other work to be done. A lot of you, in fact, all of us, rely on BGP to work for the Internet to work and when you start playing around with a different autonomous number system pool, most of you would agree, I think, that some amount of testing - testing of the implementation, testing of the transition plans, testing of the way it works in reality - is probably necessary. It also would be good if we had time to actually set up the new AS number registries, which don't happen overnight. They take a little bit longer than that. And we actually need to start deployment. The good news is that, if you're running BGP today and you have an autonomous system number today, you don't need to do anything right now. The way in which the transition plan works - which is really quite neat, I thought - is that the folk who actually are deploying these extended AS numbers, that can't map back into 2-byte, those are the ones that really need to have new versions of BGP. So everyone in this room - you don't need to do anything until you, yourself, are given a 4-byte AS number, although it would be nice if you changed a little bit earlier than that. But this does take time. And you can either do it at the last second or you can plan a little bit beforehand and do this in a more orderly fashion. So what sort of time frame are we talking about? Well, obviously, we'd like to get this done before we run out. Because, if you haven't got BGP implementations to support it, handing you a 4-byte AS number won't help you and it won't help anyone else. So we do need some lead time here to test this deployment, to test the implementations and so on to make sure that we've got everything working. What would be reasonable these days as distinct from what would be reasonable five years ago, I think, has changed. Trying to get a change in your production systems, a quick, urgent, "Oh, my God, must be done" change is probably around three months. An "Oh, well, let's plan this properly" takes about a year for whatever reason. As an industry, we're slower than we used to be and, as far as I can see, it would be comfortable if we started doing this work around three to four years before the first 4-byte AS numbers will definitely be allocated before we run out. So the next question is when do we run out? The rest of this talk will look at trying to get a reasonable predictor of what kind of date before we get through the AS numbers. There's no shortage of data to look at this. You can look at the AS numbers in the BGP table and the rate at which new ASes are appearing in the table. You can look at the rate at which IANA hands out blocks of AS numbers to the registry. Or you can look in he registry files in daily log of activity delegations, which have dates in there, and do extrapolations from the data sets. So the first one, the BGP routing table - this is the last eight years of number of autonomous system numbers in the Internet. In late '96, we had 1,900 autonomous system numbers in the BGP routing table. Today, it's a little over 20,000. That's a really interesting curve. You can sort of see that two things happened and what looked like being exponential growth turned into being almost linear and the date is certainly not coincidental. March 2001, if you look at your stock indices, was about the time NASDAQ went down again. So the exponential growth rate was actually the Internet boom and the latter part of the growth is post-boom. This is, if you will, business getting back to normal. So it looks like the most recent data and the earlier data is a different model of the industry. There are two kinds of ways of projecting this forward and it's certainly not clear if the growth rate of autonomous system numbers is purely linear or whether there's a slight acceleration. So, at this point, I actually did two projections. One was just straight linear - the Internet will keep on added autonomous system numbers at a constant rate until we run out. And that will take us into 2025 if we could use every last number. Or you can say that, because of multi-homing, because of the increasing density of interconnectivity, because of the increasing number of players who are playing BGP, you are going to continue to have an accelerating growth curve. And, if you use the previous data and extrapolate that into growth curves, it draws back into July 2014. So there's a variation there, somewhere between nine and 20 years. This is the rate at which IANA is allocating address blocks. That's from 1992 until the present. You notice that IANA is still allocating pretty hard and, again, if you extrapolate forward, using both linear and exponential, IANA will run out of AS blocks to hand to RIRs somewhere between 2011 on a constantly accelerating pattern, or 2014, if it's just handing them out at a constant rate. And the last one to look at, which is actually where the RIPE folk have looked very, very hard, is to actually look at the rate at which RIRs assign AS numbers and report them in the delegated file. The curve looks pretty clean. There's certainly a lot of consistency. But there is one step function that makes you worry about the data. The step function was actually a transfer of autonomous numbers as part of ERX-related activity from ARIN to RIPE and when they got transferred, they got a new date and you have to worry, "Are the dates I'm looking at good or bad?" Hard to say. That's certainly the data I've seen. Again, you can extrapolate forward. That's the same as with IANA. Somewhere between 2010 and 2014, the RIRs will run out of AS numbers. Can we improve the projection in I've got two projection windows and I'm going, "Which model makes sense? What's the best fit to the data?" So I started looking again at "What more can I extract out of this?" Here is everything happening at once. (Refers to slide) The top curve is IANA, allocating out blocks of autonomous system numbers to RIRs. The red curve is actually the RIR assignments, not advertised, because, interestingly, a large number of autonomous system numbers never appear in the routing table. The green line is actually the autonomous system numbers that you can't see. Sorry - the green one is what I can see and the purple is what you can't see. So there is something going on there that is probably worth looking at. RIRs keep around 5,000 AS numbers in their pool. 13,000 approximately AS numbers are in the BGP routing table. Where are they? I have no idea. So I was kind of wondering, "What happens? Why do these 13,000 AS numbers, 40% of the allocated numbers - where are they? Why aren't they being used? Is it people saying, "That's an old number, I want a new one”? Are folk hoarding AS numbers? God knows why. But it creates some uncertainty. So I started to look hard at the numbers I can see - the red line - and the numbers I can't see that are out there somewhere - the green line - and going, "Why is that green line rising? What's happening here? What's the ratio?” That's a bit interesting. Between 1996 and 2002, the per cent of unadvertised AS numbers declined rapidly. Somehow, they were being used. But, over the last three years, it's consistent - around 40% of AS numbers aren't seen. I can extrapolate that forward and go, “There's a slight linear curve to that so the ratio of advertised to unadvertised appears to be linear, slowly declining but linear.” Why? How about if I do it by date? So now the solid blue is the unadvertised autonomous system numbers by the date when they were assigned and the red is the advertised ones. So you notice that in the very recent past, the last couple of months, there's a whole lot of unadvertised ones, ones that have been handed out that are yet to appear. Way back, there's a lot that, if you like, were assigned but I can't see them. Different view to make it far more graphic. (Refers to slide) This is the entire autonomous system number space. The yellow bit at the right are the ones that we've yet to consume. The blue is what I see in the routing table. The red is what has been allocated at some point, but I can't see it. So the oldest numbers are actually the ones that are unadvertised. It seems that AS numbers disappear over time. Here's the graphic of what's going on. It seems that the older the AS number, the more likely it's disappeared. It seems that, as an industry, you don't like old numbers and 5% of the numbers drop out each year. They're old. You like new shiny ones. I don't know why - they're just numbers. But, for some reason, we just have this constant attrition. So the low number ranges are the highest ratio of unannounced to announced. It takes around four months generally for a recent AS number to hit the routing system. So what it means is, when I project forward, I should look at that particular ratio and work from that. So I start combining these things to go, "Here's the rate at which the RIRs assign ASes for use. Here's the growth of the advertised table and here's those AS numbers kicking around there." I'm still talking somewhere around 2010 to 2014 or so, but now I'm trying to figure out which model is best. Are we accelerating? Are we constant? Here's a linear model for the last two years. The business is going - the entire Internet industry is growing at a constant rate, but you'd be wrong, ever so subtly wrong. In the last two years, we were doing around nine AS numbers a day and these days we're somewhere between 10 and 12 AS numbers a day. So we're still expanding at a rate slightly higher than linear. So is it a compound growth factor? They look almost the same but that red curve is slightly bowed. It is an exponential factor, very slowly, and that's the curve fit and this time it's actually quite a decent fit. So what does that say? That says that the current extrapolation from 2003 onwards gives us a growth rate which is indeed compound and, if you actually do all the modelling, we will end up running out of autonomous system numbers somewhere around 11 August 2010, give or take a day or two. That assumes that we don't try and get back the old ones. I don't think the old ones are even reclaimable. It assumes that we simply continue with the current policies and it does assume that the Internet business pattern is continuously growing at a very slow rate, but growing, and there is a continuing consumption of AS numbers. What does that mean? Well, if we really are looking at 2010 and we want about three to four years to plan a transition, that actually means that you and I should be thinking about what we need to do some time in the next year. Which is probably a lot closer than most of you had figured. So certainly at this particular point, I would say that the data is pointing to the fact that, if we want to do this in an orderly fashion, if we want to do this without panic, maybe we should be looking at the necessary changes to BGP to support this some time in the next year or so, so that we're there in time. Any questions? CHE-HOO CHENG: Just a small point regarding your point about unadvertised AS numbers. Think there are a number of unadvertised AS numbers which actually appear on local exchanges but they don't appear on the global routing table. GEOFF HUSTON: That's very few and certainly, when you look at somewhere like routeviews, you'd logically expect most of the time that an AS will appear globally if it's going to appear at all. But, in routeviews, each of the different peers differ by around 10 to 20 AS numbers so, certainly, there's some locality abuse of AS numbers rather than everything being global. The other thing I've noticed too, if you read the MPLS VPN, you will find a VPN ID can be constructed from an AS number and I have a suspicion that unannounced AS numbers are being used in MPLS VPNs as a VPN. TOM VEST: You might want to take special note to August 2004 and going forward to see if the dynamic changes - I think we may be entering into a situation where we have a bit of a recursive self-reinforcing dynamic for AS proliferation within the industry. Peering criteria are moving - or they're expanding beyond some of the old traffic and infrastructure metrics and now including an explicit AS count, number of ASes, which can custom ASes as an explicit criteria for some peering. And it could be that, as we have seen other peering criteria gained in the past, that this may actually encourage some operators to have their customers get ASNs faster than they might have in the past. It's something to look for. GEOFF HUSTON: There are a number of discontinuities. This is actually the daily rate of AS numbers appearing in the routing table and this is since 2003 until a couple of days ago. You'll notice there was a jump late 2003 and a very pronounced jump early 2005 in the daily rate. So there are a number of events somewhere in the world that prompted a whole bunch of people to say, "I'm going to get my own AS number and use it." And it sort of bounds around for a while and then all of a sudden again. So this is not as smooth as we thought and there are particular events in particular parts of the world that cause a jump in AS number consumption for a while and it may well be exchanges and it may be peering needs. Thank you. MAEMURA AKINORI: Akinori Maemura from France Telecom. Let me look at the last page before the “thank you” page. I can’t find why the date we should start for the preparation of deployment of 4-byte BGP number is 2006. GEOFF HUSTON: I know that you work for a very large carrier and I have spent quite some years in a very similar environment and to slide in new versions of a new protocol as critical as BGP takes some months on your production network. Everyone is very scared about sliding in changes and having the entire network collapse around your feet. We are now a very, very slow and conservative industry in making changes to critical protocols. And I was simply, I suppose, forecasting that, when we try and do this as an industry globally, the longer the period we have to actually test and stage the deployment, the happeer everyone is going to be. And what I'm suggesting is that current trends seem to indicate that, if the industry growth rate continues as it does, there's a really hard wall out there and somewhere around 2010 we'd better be running this stuff and I'm trying to be nice to everyone and saying, "Look, if we really are taking some time to get our critical protocols deployed with new versions, then the longer the lead time, the happeer everyone will be." That's how I rolled up to a date of around 2006 as a good date to start working on this as an industry. MAEMURA AKINORI: You don't mean that should be as soon as possible? GEOFF HUSTON: I don't think it would be as soon as possible but, if we are going to do this right, it would be good to start allocating actual 4-byte AS numbers to folk who are ready for it by late 2008, early 2009. In other words, move before you have to. So that the folk who say, "Look I really want a 2-byte number,” you can give it to them, and the folk who say, "Look, I'm ready, give me a 4-byte number," we can satisfy that requirement. That's why I'm suggesting we really do have to start working on this some time in the next 12 months. PHILIP SMITH: Geoff, I have one question as well - Philip Smith, Cisco - and that's really from the vendor's side. Because, doing a straw poll within the vendor space, the feedback is generally, “No new features will be implemented until customers ask for it.” And what happens is it tends to be the larger customers have the bigger influence. But, then again, the larger customers are quite happy with all the AS numbers they've got, thank you very much, and it's really the new kids on the block in the ISP industry who need this feature, but don't have the sway with the vendors to make them commit stuff. I'm not quite sure what the answer is there either. I think both sides of the industry are sitting waiting for each other in some respects. GEOFF HUSTON: That's certainly my frustration as well and I have heard from a number of vendors that, these days, their business model for putting new features in BGP is based on customer demand rather than anticipation of Internet-related need. And, yes, you're quite right - the larger customers out there on the Internet have their 2-byte numbers and don't see the need. And even if they read the transition plan very carefully, they would realise that, while the rest of the world can do 4-byte ASes with new versions of BGP, they don't have to change if they already have an AS number and don't need any more. So, yes, there is a certain amount of people looking at each other wondering who's going to jump first. On the other hand, there this really is a small change. It would be really good if some of these vendors realised that it would be an industry-good thing if they actually jumped a little bit harder and faster than waiting for customers to moan at them. RANDY BUSH: Randy Bush, IIJ. I think ascribing numbers to ISPs at this point is ill-advised. Nobody is more for or against it. Many of the large ISPs have specifically said to vendors, "Would you please do it?" It's a small act on the vendor's part. It's not a big deal for the vendors. They're waiting for the Internet Vendor Task Force to actually say, "This is it," and it will come out to release them in plenty of time. The real problem will be rolling it out, as you make quite clear, and getting folk to try it and finding out the other 19 complex features in your vendor implementation that it accidentally broke. PHILIP SMITH: OK, are there any other questions or comments for Geoff? If not, thank you very much for both your presentations. GEORGE MICHAELSON: Hello everyone. I'm George Michaelson from the technical group at APNIC and I'm here to give you information on a presentation on the project of resource certification. I'm going to run through some background issues about what we're doing and what we hope to achieve, give you an oversight of the RFC which covers the area, talk about the certificate format and the phases of the project we think is going to be a long-lived activity and look at the immediate deliverables and discuss issues which have come up recently and for is room for lots of questions. I would like to stress this is an informational presentation and it is likely in the longer term more substantive outcomes will come from this activity, possibly including policy. It is quite clear from the amount of time it has been talked about that routing security is getting important. An integral part of this is going to be the ability to check for the TV A S and do this as automatically as possible. RFC 3779 tries to address this problem. It is a definition of the extension to the public key X 509 certificate format as a means of representing AS numbers and IP addresses and provides coverage for AFIs but I don't think anybody is using them in is sense that matters here. An aspect of the RFC that's important for us is it's very explicit is the authority hierarchy, the way the delegation of authority is passed down should follow the delegation authority for the actual resources themselves. It should follow the natural hierarchy with IANA, the registries, the LIRs, the members and it there the way through the downstream of delegations. This is quite explicitly a hierarchical model at least in this part of the process. So what are we planning on doing? Well, our objective is to have a project to provide a service which is centred around issues with complying certificates with this RFC to our membership. We want to set up the infrastructure, policies, processes to make it possible for people to use these certificates for routing community for other purposes as well and we have to think about the certification practice statement - what commitments are we going to make to you, the membership, the general public, we have to think about the repository, the place we're going to hold and construct and maintain the certificates and think about the revocation - when can we tell you when certificates are superseded? We think there are going to be tools made valuable to people and example certificates. We expect to provide support for certification by the NIRs and membership and other ISPs. RFC 3779, the abstract basically says it all. This is reformatted into bullet points and it is right up front to the RFC, it is two certificate extensions, a way to present address blocks or prefixes subject to a certificate and a way to represent to subject. By having these extensions you can provide information about the authority to use the systems. To give you an idea of the kind of framework we're looking at, the idea is that IANA is going to be making an assertion about delegations down to the registries so we're going to have certificates that quite clearly name IANA's authority to be giving us resource management controls over blocks address and we're going to issue certificates directly to ISPs for their use, ISP's for end users and in some cases directly to end user entities and to the RIRs for further subdivision. The certificates have got quite an interesting format. The version is about the version of the encoding, the serial number is globally unique identity. Make sure these things are always unique and can be detected. There is the naming element which for us is going to be C for CA or O for APNIC. There's the algorithms used to design the resources and validity period. You'll notice this has an end of life. These are not open-ended assertions for the validity of resource. They do expire. RANDY BUSH: When? GEORGE MICHAELSON: I'll be getting there. There is a possibility we could be using this as a large number space - as well as having a structure to identify these, we have a large number field and can say the certificates are actable against these if we can deliver global uniqueness. This might make processes easy. The extensions is the stand-out part of the certificates and includes features to define behaviour of the certificate. The basic constraints we're going to be making use of is we're going to be using something called the CA bit, the ability to use a certificate to create and sign other certificates. It is making it is certification authority. If you have resource that we understand can be split up and given on to other people, we're going to turn on the bit in the certificate that makes it possible to use it to create down stream signings. If you have a block of resource that's integral and not designed to be split up, we're not going to turn that bit on. There's also flags that can identify these attachments are mandatory and must be present and checked. This is a feature which is going to help us prevent the certificates from being used in non-conforming ways and make it clear if you can't understand the extensions the certificate isn't generally valid for other purposes. We need be very sure the certificates are not used for general purpose identity but are completely related to management of the trust relation 147s in Internet number resource. So the project - we've got three or four phases and the first phase we're basically entering now is a trial period. We're looking at participating with early adopters, developers and router designers. We don't want major changes but it is understandable if we have misunderstood a major part of the projects there may be changes to the scoping of the project and that may be the certificate format's change. This is a published RFC but these things happen. Maybe there is a Bliss version coming out and we have to take account of that. We think this will be in the state where it is a stable service next year, wider deployment and have the certificate practice statement that would have all the checks in place and we would think the certificate format would be much more stable by that point and it is likely people will be coding changes into the routing architecture to take account of this. Full service - we're trying to aim for the mid-point of 2006 and that is going to be something where we promote this in a much wider context. The deliverables in phase one, we want to be conformant with the RFC for the certificates and issue these. Essentially I think you'll self-select. If you're interested in this activity, put your hand up for it and we'll talk to you about work wing the initial certificates. We are going to be working on tools and utilities we expect to put into the open source community. We might be targeting these to my APNIC, or own web-based portal. If there is more general facilities, we would like to make them available. Open SSL is the most ubiquitous suite of software for doing activity in X 509 keys and doesn't currently handle extensions well. It is likely to make it easier for people to construct certificates with these extensions. The tools are going to address things like certificate requests. You can't use a normal browser to make a request. If you do, the request isn't going to look quite right so they provide tools to assist with certificate requests and have tools on my APNIC to help people further subdelegate resources issue certificates and there needs to be certificate delegation tools because it is likely to first phase of people using it is going to be outside of the routing crowd using this to check the validity of materials they're putting in. We're going to be presenting this at future routing SIGs if that's OK with everyone. It may lead to policy proposals about adoption of the activities as part of normal practice for delegation and resources. Issues have come up already people a have talked to us about in corridors or in the background. One question we often get what is happening with the routing community is? We have been in discussion with two large vendors and we're actively seeking cooperation with anyone with an interest in coding BGB. We think it is like 3 be the beginnings of liaison. We will give them test certificates to make sure they conform to expectations. Certificate lifetime is often raised on the floor of APNIC. We have to be conscious the lifetime of the certificate relates to the lifetime of our relationship with the people we give them to. Where we have a relationship with you either as a member or some others relationship, that's bounded in time. We don't have an open-ended relationship and for the life of that relationship and some kind of hang time - we're not saying it ends day 365 but there will be some hang time beyond which if we said to you, "We need to renew our relationship," and that doesn't happen, the certificates are going to be invalid. We thought about for the fact the time being the membership relationship is essentially year by year and it is clear some people would like to have more certainty and say, "We need to know about our routing stability for three or five years because that's how we do business but we don't want to exclude talk to people coming into relationships of some kind to make that work. That does not necessarily mean you have to give us five years membership up front. We're talking about sensible business process to be with people to understand what the long-term commitment is. There are ways to do this. The next thing is are we going to be doing the subdelegations, warrant and certify the delegations you guys make about the things you give out? No, we're attesting we gave this resource and the certificate about this resource to you. We are going to give you software you can use in that system to make the attestations you have to give out about the subdistribution but that's something you're going to be attesting to, not something we're attesting to. We're only attesting on a given date we were in a relationship with you and we know you have control over the resourcing question. That's where it's going to end for the time being. Thank you. Any questions? Randy. RANDY BUSH: You surprised? GEORGE MICHAELSON: Not at all. RANDY BUSH: Serial numbers not globally unique. GEORGE MICHAELSON: That's true. The serial number is unique in the CA, that's the sign-in. RANDY BUSH: The question of whether you will turn on the CA bit in the certificate you give me, you might be giving me a certificate to go with some address space that I'm just an end organisation and therefore should not be re-delegating but I may still want the CA bit on because I have different roles in my organisation. Who's entitled to ask you to change the DNS delegation? It's not just if I am subdelegating, it is also if I intend to divide up my own organisation in a certificate attestation sense. GEORGE MICHAELSON: I would not want to pre-empt a proper discussion but I think you make a valid point. RANDY BUSH: I want to back register those certificates to you. In other words, if you give me a certificate I use to create a certificate for who may change the DNS, I have to give you that creative certificate. GEORGE MICHAELSON: Remember the certificates about who may change the DNS are identity certificates that control access to web servers. These certificates are resource management and are subtly different. It may seem a fine distinction but we would not expect these certificates to necessarily give you a right to manage DNS delegation. That again is something we really might need to talk about if you're saying you want resource management to tie with things like that. I think we would have to think about that. That's not what we're looking at. RANDY BUSH: I think you should think about it. In other words, I'm the one who should say who can talk to you about my DNS. I should have to sign that certificate. GEORGE MICHAELSON: I think there is room for a conversation and lots of issues there. RANDY BUSH: Thank you for the tools etc. I think that's very necessary for two aspects. One is so you develop the tools and the greater advantage of open source and being able to have common fixes and the understanding of the security status of those tools. GEORGE MICHAELSON: It is in no sense an announcement but an observation is we've recently decided to participate in the O’Riley open source conference series and they've given us recognition as a community benefit not for profit so we're trying to pick up in the atmosphere for that community and do the right thing. RANDY BUSH: That's really commendable. The one place I think we're going to continue to have disagreement is of course expiration. The long-standing statement I agree with by the registries - of the registries don't set routing policy, and indeed what's going to happen here if we're not careful is you will be telling me as an ISP how routable my AS and address space are. GEORGE MICHAELSON: I think there's three separate things to be said here. One is it's really fundamental to the model we're providing that be believe there has to be a lifetime. This isn't just an interesting conversation, it is probably a fundamental conversation. That's one thing. The second thing is it's in X 509. I don't know if anything in X 509, by it these certificates or public certificates that has no end of life. Every X 509 object has finite life. If you're saying you would like the life to be 2153- RANDY BUSH: No, I'm saying -- GEORGE MICHAELSON: You're saying things like automatic renewal? RANDY BUSH: Renewal and role of the mechanisms -- GEORGE MICHAELSON: If you roll back to two meetings where we were discussing v6 allocation windows for registries for IANA, people were making observations short lifetimes were good for them because it provided the review mechanism and that raised some hackles in my community because I'd like to see other people's abilities to do that and now I'm in this role I can think there are some traits here. It is good to have a continuing relationship with someone and one way to do that is have mutual obligation. RANDY BUSH: The implication I can make behind that is, oh, legacy people - people who took up space way before APNIC existed will now have a continuing obligation to APNIC. GEORGE MICHAELSON: I am thinking specifically if you want to be issued a certificate you have it got to be in some kind of relationship with APNIC and that relationship is going to be bound and the certificates are going to be bounded by the life of the relationship and hanging time. We are thinking of ways to have long life relationships but it is hard to see a way to make this open-ended. RANDY BUSH: I encourage short certificate lifetimes for technical reasons. That's not the issue. The issue is whether I have to go to my management and say the routability on secure routing -- when secure routing comes in, the routability and viability of the organisation depends upon a business relationship with APNIC for legacy space and what will happen is if we in the next few months don't understand that - we as ISPs - is the relationship appearing now is in fact the trust relationship is between the ISPs and the ISPs can start to certify each other without needing APNIC. I'm not saying that's a good thing. I would much rather see APNIC meet the need but not in a way that looks like you can threaten us and control us in a business fashion. GEORGE MICHAELSON: I have this characterised as red and blue nets. There is going to be a lot of nets that are blue. If you're faced with a route that comes in for a block of address that has no assertion and a block that comes in with a certificate assertion, you're going to prefer the certificate assertion. OK, so randy doesn't choose to renew and a large block comes out of status, do you chuck it away? No but you have a higher preference for a certificate against it. RANDY BUSH: Those people get to tell their management they're easily hijackable. Does not fly. GEORGE MICHAELSON: I think this is a continuing conversation. EDWARD LEWIS: It is not the expiration date that's the issue, it is the renewal process. If you want to focus - I think it's not so much the date that's the issue, it is how easy is it going to be the renew the certificate when you want to redo the key you have. RANDY BUSH: Specifically with those with long-term legacy AS numbers and IP space. GEORGE MICHAELSON: I think that's potentially where the policy aspects of what we're talking about need to be explored. I don't think it materially affects the project now. For the first quarter and maybe through 2006 this has low impact. I take what you're saying seriously and clearly there's got to be further discussion. I think we're done. PHILIP SMITH: Are there any more questions for George? If not, thank you very much, George. GEORGE MICHAELSON: Thank you. PHILIP SMITH: So this brings us up to the end of the first session for the routing SIG. On the screen I've shown the agenda for the second half, what's coming after the tea break. So first up we'll have Gaurab talking about some of his research work, followed by Tomoya Yoshida talking about the IRR in Japan and finishing up with myself talking about Flap Damping and Randy talking about allocation announcement he's been looking at. Without further ado, let's have a tea break and be back here at 4:00 sharp for the second session of the routing SIG. Thank you. PHILIP SMITH: The social event is sponsored by VNNIC and will be held at the Vietnam Museum of Ethnology. The first bus leaves at 6:30 and the last bus leaves at 7:10. All of you going to the APOP BGP key-signing party, don't run away for the first bus. The social event is an open-air event so cover up with mosquito repellent and try to cover exposed limbs as much as you can. Also, check the onsite noticeboard for any updates to the conference program. So onto the second session. We've got four presenters. First up we have Gaurab who will be talking about determining Tier 1 status through the ASPATH analysis so Gaurab, if you're ready. GAURAB RAJ UPADHAYA: Good afternoon, everyone. I'm trying to do something that's not been done before when I looked at different past presenters. This is a work in progress and this was first presented - the idea was first presented about a month ago, month and a half ago. Between then and now we've done much more work to get some more concrete figures. Before we go into the thing, I'll make it clear we're talking about Tier 1 which is not a binary property, according to Bill Woodcock. Also flame shields, there is a lot of debate on the Internet routing table and let's take this as granted there is no single routing table on the Internet. It differs. It can't be the same. Our work is based on multiple use from multiple sources which means it is provided across the Internet from multiple sources. There is no such thing as Tier 1 across the entire Internet. Also our analysis here, the content providers are not proportionately represented in this analysis. Their role in the Internet or how they are being Tier 1, they are still not seen or they're still not visible in this analysis and I will see why. Our hypothesis is based on this fact that if an ASN or company claims it is in Tier 1 on any given AS spot there can be more than one so-called ASN. We're trying to lose the S fart to see whether that is true or not true. We'll take an algorithm from Japan and this is what I use from my slides - route views. Whenever I refer to pfs, that is Japan and oix is route views. We have a list of so called Tier 1s, possible Tier 1s against which we have used the routing table. We have to build a starting point. How to do that? We took the 20 most frequently occurring ASNs on all observed ASPATHs and we intercepted them and got the data and got a list of ASNs. Looking at the ASN, we also have to make sure we remove the feet wires so they're wiring to the collectors. We found the 20 most frequently occurring ASNs in all observed ASPATHs after removing the immediate next hop transit providers. We used data for the third of each month in 2005 was used for oix route views and pfs Japan. This is what we have analysed, the data collected on the third of each month from January to September. We found the intersection from the two lists, the top 10. This is the top 10 we got from the two data sets. We see all the user so-called Tier 1s in this list. Nothing surprising there. So we took a sample again and then looked at something like this here. As I said earlier, there can be more than two Tier 1s on the same ASPATH. Say for example AS174 which, if you just take the top 20 ASNs, only the Japan subset of the data you see AS174 is one of the highest occurring ASNs. If you run the algorithm against the others you can see there are more than three of those in the same table. Not so difficult to see that here between 2516 which is the feed provider for the data, 1239, 2914 and 1 74 - 174 doesn't seem to be in Tier 1. It is much more likely these two players are more likely to be Tier 1s than AS 174. If you go on this data or a large subset or the entire table, there are a lot of examples of this occurrence where taking an AS number and running it through the algorithm tells whether there are multiple Tier 1s from a list of Tier 1s occurring earlier on the same ASPATH or not. Let's look at some interesting things we got out of this. When we created a list of ISPs, we didn't have AOL on it. This is because we created the list on the fact they were the top 10 most occurring. AOL doesn't provide transit to downstream network and would probably not be seen on that. We added AS 1668 (AOL) to the Tier 1 top 10 ISPs and ran the test and we did see an example where AS 1668 and any of our other 10 ASNs occurred more than twice in any ASPATH. It helped us see AS 1668, if it claims to be Tier 1 using this algorithm, that is true. How you define Tier 1 is something else. When AS 1668 was on the ASPATH, at most one of the A SN from our top 10 list were visible on the ASPATH. We did see more than 1 AS on the top 10 when comparing it with AS 1668. While doing this, we found a couple of interesting results. For example, we saw customer ASN between two ASN on the Tier 1 list. What I call them is anomalies on these slides. Let's see some examples here. This is an example I brought out of Zoom data from Japan. This is where you see this interesting combination where there is UUNET, Gblx and Sprint who are members of the top 10 list and then we see ASN 11664 which is Telemax, 14551 is also an ASN we use in Latin America. We have companies using more than one ASN, we treat them as one. This is an interesting example. What makes it more interesting is a lot of anomalies we detected, this is actually also the best selected part. This was the best selected path. This is definitely an exception but this happened immediately. After a week or so, this was no longer there. People probably went and corrected the mistake but these things happen. Here's some more data. There is this network 204.13.189.0 with two ASNs in the middle, going through a small provider in the area. There is definitely an anomaly or something that's quite not right. Then in this case, this is 38 Tier 1, 2914, 701 and 1239 is sprint and we have this in the middle. This is from the route views data from August 3 and this is the providers to use and then we see these anomalies. We see anomalies - this is from three days ago. Still you see that. The interesting thing to note is between the two datas - between last month and this month the number of anomalies tend to remain around the same number. They keep on sending. Other people keep on fixing the networks or once in a while it is happening. It is not a consistent thing, though they are there. We'll go through examples when we have time. One of the things interesting to see in a lot of data is whatever we see in the middle here are most from the same ASNs. These are particular ASNs. They have probably misconfigured the routers so that they are leaking routes from two sides. We counted the number of anomalies and from the Japanese data set we only saw one anomaly and that was the one we saw in June. From the Oregon route views data set, these are the numbers. You can see they're quite large numbers from January to September and they are all around the same numbers almost except in April when it went up pretty high to 155 and July 85. You see these anomalies - what I term anomalies. What I did with this was - because this is an exceptionally high number of anomalies, I look at the Japanese data set and there's one in every six months. Route views had an exceptionally high number of anomalies. What we did was the bottom-most - on our Tier 1 list AS 4637 was the bottom one. We removed one ASN which was 4637 and the number of anomalies went down significantly. As you can see, it went from 140 to being 2 and almost in a similar consistent fashion it went down very significantly except in May which is due to some other issue with some other providers. So how do you treat these exceptions? Removing AS 4637 from Tier 1 list yields significantly fewer anomalies. This tends to confirm the hypothesis that Reach may not be very Tier 1 on the global scale. When we remove Reach from the list of Tier 1 on the algorithm we get far fewer exceptions which tend to verify or tend to go towards the hypothesis which says Reach may not really be a Tier 1. There's too many exceptions. If we're only looking at the Japan data, it is making it more Tier 1 in their specific region. If we do this run against data from Hong Kong and look at that part of your data, Reach doesn't yield any exceptions. There might be more Tier 1 than they are in the rest of the world. So what do you think? Tier 1 in a socioeconomic concept is probably something being generated or something marketing people came up with. This technique I have just talked about can be used to find inter AS relationships and determine the value of particular ASNs. If someone gives you a network and claims they're Tier 1, running through this algorithm may give you certain ideas of their value in a particular region. What we're trying to do is look deeply into the data from multiple sources or multiple points, say Europe, the U.S, Latin America and see if we can determine reasonable Tier 1s or ASNs that are valuable in a particular region, though they am might not be the same on a closed scale. What next? We probably need to run more tests and hopefully data from other regions. One of the things we plan to do - as you know route view gets data from all over the world and we haven't run analysis on all the data. In lack of separate data from all the sources, we might run part analysis and run it through again. We might run full routes intersection with peering routes. We have route collectors which only collect peering routes and one thing we're looking at is see the exception of these routes and see what kind of resource we can get. One of the things we also plan to do is we probably would review the earlier research done on these topics and presented at SIGCOMM and recent meetings. We have not done this so far. We have done this recently and not looked at all the prior research done in this topic extensively. We'll probably look at that and see what we can get. Thanks. Thanks to all the people who helped me in this. PHILIP SMITH: Do we have any questions for Gaurab? RANDY BUSH: Two things. One is not knowing if a route view's peer is announcing their internal routes, the routes they announce to customers or the routes they announce only to peers hides from you whether - if 20 million and 14 treated route users appear, you would only see 7 of them downstream and that's a continuing problem with all the peering collectors. It makes me crazy. Number two is I thought at the beginning of your presentation you were going to do the reverse, you were going to look at the routing data and say for these ASes, these are the ones that appear only once or twice in the path and therefore they are Tier 1s, not going from the exception. Your list that your hold in your head who you think are Tier 1s and using it to validate the routing data. What happens if you reverse it? When you go to SIGCOMM and that stuff, it is reversed and trying to infer the relationships from the routing data. You have inferred the relationships and you're validating the routing data which is good in itself but it would be interesting if you did the resource. GAURAB RAJ UPADHAYA: We looked into that aspect. The work we did on that would make a presentation only after six months or so after we have a better idea and better collection of data. Randy is right in the sense that what people give is something only they know already, we don't know already. What I'm trying to do is get more independent data sources from either people like the right data or the internal transit customer data and use our own authority tables and do lots of analysis on the routing data to see if we can get something that works. Thank you, Randy. GEOFF HUSTON: The work done by Lexion Gao when associated with Sprint labs was inferring relationships from the Internet and went a bit further than you're looking at and made a relatively clean supposition no one's in this business to lose money, therefore what do I do with my customer’s routes? Announce it to my upstreams. What do I do with my upstreams? Announce to my customers. What do I do with my peers? Don't announce my customers. Every ASPATH is actually almost a hill. When you look at an ASPATH, you see a bunch of customer upstream relationships and perhaps a peering relationship and a bunch of reverse customer provider relationships. Their analysis was run through the entire set of ASPATHs and inferred customer provider across the entire set of Ases and look at which ones emerge at the top of the relationship. That work applied the algorithm they published and came up with 15 Tier 1s, of which Reach is not. They had an automatic system of doing it and the anomalies they saw were quite small and this was when you saw actual leakage that should never have happened - upstreams got leaked to either other upstreams or across to peers. The reason that was interesting was not only did it give you the Tier 1s but the Tier 2s, 3s, 4s and 5s onwards and worked out where in the hierarchical relationship the AS lies. I encourage you to look at their data as well and compare the technique with yours. RANDY BUSH: That paper was very old and shut down in about three series of papers since there. Most recent would be the paper about two SIGCOMMS ago and the problem with those is the valley free assumption which does not hold. There are cases and agreements between people we would all agree are Tier 1 providers to provide each other transit back-up in local failures et cetera so when you look at the routing data, betting your life you will not see certain things is not wise. GEOFF HUSTON: Her paper was more about statistically than absolutely. RANDY BUSH: I understand. GAURAB RAJ UPADHAYA: We are trying to look at a particular ASN. You have an upstream and they sit at the top and have downstreams on both sides. One of the things we want to do with the research is try to find which is the upstream and where the peering relationship actually happens. This is looking at where the peering relationship actually lies. To do this, there are a lot of anomalies that are not really reflected in the ASPATH and the data tables. It will probably take some time. We'll look into the earlier work done and see what results we get with the new data, the more data we have. CHE HOO: What you are doing is very interesting. We look at the issue from the routing tables point of view but there are a lot of complicated commercial arrangements and they are the kind of standard peering - I don't think you can have a clean conclusion. GAURAB RAJ UPADHAYA: That's what we actually looked at. I know for sure something like Reach is not a U.S. provider. If you look in the table, one of the providers are probably Tier 2 in the global scale. If I look at the ASPATH network, Reach - the counterpart in the north American region probably will be a Tier 1 in that region. As I said earlier, I think doing this on a global scale makes it more difficult. I'm trying to bring it down to a scale where we can apply this logic and get data that can be verified. PHILIP SMITH: Any other questions or comments for Gaurab? GAURAB RAJ UPADHAYA: We will have the data on the website soon if you are interested in the data. Something I should mention - this is work in collaboration with a friend of mine who is a mathematics graduate at the Univeristy of Kathmandu. Thank you. PHILIP SMITH: Next up we have Tomoya Yoshida who is speaking about deployment of IRR in Japan. TOMOYA YOSHIDA: Can you hear me? So this is from NTT communications and the Chair of the IRR planning team. This is based on the JANOG meeting, presented at the last SANOG about one month before. The title is the deployment of IRR in JP and routing security. This routing security means focusing on the routing hijack. So inter-AS routing - we change the routing information by BGP. The route generated somewhere in the world is transmitted all over the world and in many cases we cannot find in advance which or what flow will come because the Internet is always continuing changing. So from here, we focus on the hijack. We have two - I think we have two types of hijack. One is the malicious case so they hijack the route information and use the spam send, the address. Or a site network, they hijack the total site or network. And another one is the miss configuration so sometimes the router bug - so in some cases I saw the - so someone announce to some people and then they receive the redistribute route and they register with OSPF and then re-register with the BGP so this is hijack. And the time is almost often - the type miss is often found. From where hijack route comes - the peer is often found and many ISP use the prefix filtering for customers and often find disappearing upstream. However, the prefix length of hijack route, if the prefix length is the same, so it depends on the form of connection, so when you see the hijack route from peer or your customers route, so you prefer the customer always so it is not - the hijack length is longer and in the case of the shorter, all can be disregarded as unpleasant. In case of the ASPATH length of hijack route, this depends on the form of connections. Origin AS of hijack route, if the origin AS of hijack route is the same it may be hard to detect. If different, it may not be hard to detect. For MED value of hijack route, it may depend on the aforementioned. I couldn't find the community hijack but if it happened it would depend on the filtering. Some ISP filtering the exact BGP community or if include in some BGP community, they observe measuring so it depend on the filtering way. You can see the - some case of the pollution condition of router so the red one is the totally hijacked, so I mean the red router - I can explain this for you. So the red router has the only hijack route. So completely polluted. And the orange one is the - they have the hijack route and the right route so in this case the hijack route is the best route. Yell so the send out in those routers, the hijack route and the right route. In this case the hijack route is the best route, in this case safe but dangerous. The green is the right route only. You can see this topology - sorry, the colour is not good. Here is the right AS 200 and the left side in the hijack AS 666. So this RC is the route client so this receives the right route out of this route reflector. This route reflector has the right AS only. This router reflector has the two prefixes. The one is the right router route 200 and the other is hijack route. In this case, the best path is right AS. This is partly polluted but not in the best parts so then data reflector announce the route information to the route client here. But in this router reflector, this is polluted so we then choose the hijack perfect for the best path because in this case, the hijack is caused. So here for hijack route, its number is the near. So this router with the client has only the best possible hijack route so this is totally polluted. Even in the one AS we see the different condition so this is very difficult to follow the hijack on the same AS. So what can we do as the wrong route is detected promptly? This is very important. What is the right route and how to detect the hijack route? Maintain the information of IR and IRR suitably so we can insist the right route. This is important. The scope of which can detect by only IRR. So IRR has the origin information and mostly the perfect length information. Can detect the different prefix length and different prefix length. ASPATH depend on the IRR object. The way the hijack route origin ends is the same and the prefix length is the same so it is difficult to detect is hijack by only IRR. So currently the many IRRs scattered around the world so they store the routing information. In many cases the justification of data is not checked so we need reliable information so what is reliable information? I think the assignment and allocation information - information from Internet registry. IRR Internet registry only knows the reliability of those information. So the opinion of here is the route may be hijacked and the technique of checking the justification of the route is required. So reliable information is of course required to checking the justification. Then we concluded IR, Internet registry, do the IRR because Internet registry ensured the reliability of the routing information. So this is the experiment IRR service by JPNIC. So currently our service is a free service for JP community and mirroring, we mirror to APNIC, RIPE NCC and RADB and the number of objects is increasing, especially this year it is more increasing. So the total number of mntnr object is 60. So how to make reliable information? It is necessary to cooperate with the database of an IP address in order to keep the information on IRR healthy and why we choose to operate IRR? By practical use of IRR, danger such as mistake operation is reduced and it can contribute to keep the Internet safe. So this is a part of the JP IRR implementation to cooperate with the IP address database. We check the status every five minutes so we change some route information so just that description field we change and then check the changes of the description field through other IRR. So we are checking every five minutes. The mirror link checking is very important, I think because if IRR user registers JP IRR, but does not reflect to another IRR, it is not good condition. And also we compare the BGP information so that that information - we compare those information. So this is the attestation mechanism of JP IRR and IP registry system. So here's the LIR so LIR receive the assignment from JPNIC and here's the resource administrator and he appoints to some operator, JP IRR administrator. So he only can access to the JP IRR web so by using the certification information - so this is the security of the registration and if some query received of JP IRR, we check this code of the changes between the IP register system, this Whois system, and JP IRR system and then respond to the operator. This is an example of the attestation acquisitions. Sorry for this all being in Japanese. Then we can use the browser to import the certification to the browser and they can access the JP IRR. So first of all, the JP information is stored in JP IRR so IPv6 also do from now so routing information database or BGP is made in JP so this is the goal of our JP IRR. And also I insist on the necessity for international cooperation. It is not the problem solved even if only JPNIC do because some cases, the people who does not receive the route information from JPNIC, we cannot check the reliability so we cooperate around the other community or other countries so it is necessity for international cooperation. An operation based on the hierarchy of the registry structure - so each registry manages exactly the route/AS information under management of own country or himself. So in my case - in our case, JPNIC store the JP information and some other country stores their route information. So with JPNIC, open the certification and maintain the Japanese resource management and Internet resource management and we do into the routing registry database management. Each country do this and then we can share the information through the structure from the Internet registry, IR structure. So currently the one problem - one big problem is the mirror link. So we mirror to APNIC and APNIC mirror to RIPE NCC for example but this JPNIC information does not reflect directly because JPNIC does not have the mirroring. This is the - this is the mirroring problem. We have to mirror every IRR. So now I am proposing about the CRISP - CRISP is the Cross Registry Information Services Protocol - when CRISP prevented the IRR, we got the IR information through the hierarchy only one. So in IRR, when you see the router information, you need to search another IRR but if CRISP implemented to these - so we can get the right information from these structures from the top of Internet registry, higher up the hierarchy structure. So this is our goal. I think the - like the SPGP, we want to have the PKI structure so I think the - this is the same goal so this is the conclusion and suggestions. So the reliable information will be made so we need reliable information to check the hijack route and high flexibility BGP operation is realised by using Internet routing information database so this is an important and I want to insist that we want to cooperate in the Asian region so not only in JP but in other countries in other regions so we want to cooperate these IRR implementation. Thank you. PHILIP SMITH: Are there any questions? EDWARD LEWIS: There is discussion going on right now with the CRISP working group about this topic and I have talked to people this week about this. To me mirroring means copying all the data from one place to another place, replication. The CRISP protocol known as iris doesn't do that. It lets me reference, saying if I go to the database of JPNIC, it says you can go to APNIC and get information. That isn't copying and mirroring the data back and forth, it is referencing the information. It is a slightly different goal. One of the considerations here is sometimes when you have information in a database like JPNIC that has a Japanese routing information and I need to know about something coming out of France so I want to go to the RIPE database, the question is sometimes can I get to the right database had I need to if there's a routing problem? The question is whether or not referencing in a situation like this is a good idea or do I really want to replicate the data back and forth? I'm not sure there's been a lot - I'm not sure what the debate is on that. If you want to collect all the data in one place, it might not stick. On the other hand, I have had experience where if you don't have the data you need for security on hand when you need it, it might not be able to get to it. I guess break it down into the mirroring CRISP I think is a misnomer, it is not the right use of the word. CRISP is for referencing information from different places. If you're going to do mirroring, there's other database replication programs for doing that. The question is whether you want to do that. It is a 2-part question. You don't necessarily have to answer those here but they're just considerations. I think mirroring CRISP, you don't want to tie those two together. That's the one point I would make about the slide. TOMOYA YOSHIDA: I have to mirror so every IRR - I mean the more than 50 IRR I have to mirror each so the basic concept of the CRISP is the - we don't need to resist the several IRRs so in current case - for my case, I register the several IRRs. One is JP IRR and one RADB. This is not the good condition, I think. For ensuring the route information, I think the Internet register only ensures the exact route information so this is the information JPNIC start IRR so for JP community we store mostly JP IRR information. So then if another country do run the National Internet Registry or APNIC do, so the information stored in those countries - also that information is very reliable because the Internet registry operate IRR. So when we use those models, we have to mirror each other. This is not - I think this is not good way or good conditions so if we apply the CRISP implementation for Internet Routing Registry, we can start on the one query for - so some Japanese ISP query to JP IRR. The CRISP search other Internet registry so this is the - yeah, I think. EDWARD LEWIS: The way you're describing CRISP is correct. When I see mirroring in CRISP, it is actually mirroring or CRISP. TOMOYA YOSHIDA: I think those - if JPNIC has several IRRs, we check the mirroring so in the search - for a search, we can change the mirroring to CRISP. EDWARD LEWIS: Maybe I'm unclear about the word ‘mirroring’. We can talk about that later. TOMOYA YOSHIDA: OK. PHILIP SMITH: Any other questions? Can I just have one simple question? In seeking cooperation or support from other ISPs in the region, is there - should they contact you or do you see some other mechanism people should use to support this effort? TOMOYA YOSHIDA: Of course please contact me or JPNIC but we want to discuss about the operation of the Internet registry, IRR operation of the national - in this AP region, the National Internet Registry especially. But I find the APNIC and JPNIC IRR operator so how about another National Internet Registry. So if some comments or suggestions from the NIRs, I really appreciate it. If you have some comments or suggestions, please contact me or use the mailing list. PHILIP SMITH: I think nobody wants to comment at the moment but by all means contact Tomoya Yoshida through email. I think that would be the best to do. Maybe bring it up at the NIR SIG at the APNIC 21 meeting. That's a possibility. OK, thank you very much. TOMOYA YOSHIDA: Thank you very much PHILIP SMITH: I'm going to fly through this pretty quickly, partly because I promised Randy 20 minutes for his presentation and partly because there isn't really very much to say about this. Some of you probably know I'm one of the co-authors or co-responsible for the RIPE 229 RIPE flap damping recommendations and, at the last RIPE meeting, the authors presented that we should do something about the RIPE document as it stands because we didn't feel it was very much use any more or at least people feel differently about it and so forth. So this was my attempt to try and, I suppose, encourage that particular community to express an opinion and I think the timing of my talk right after the RIPE social event didn't particularly help matters. So I'm just going to highlight some of the things that I consider are issues here and hopefully get some useful feedback, either by e-mail or on the floor here now. So, just a bit of history, where it all came from. Early Internet was susceptible to what we called at the time "routing storms" and thinks were repeated withdrawal and re-announcements of /24 address blocks, mostly caused by dial-in users because, in those days, networks dialled up to the Internet rather than having the connectivity they have today. And that consumed significant CPU on a lot of the routers, certainly on the routers for the network that I ran. We could probably see anything from 10 to 50% consumed just dealing with prefixes coming and going. That caused a fair amount of instability in the Internet and therefore the flap damping proposal came in to try and do something about the effects of this particular instability. So that was introduced in BGP4. RFC2439 described the algorithm and the conditions under which flap damping were to be applied. The requirements basically were that you had to have fast convergence for normal route changes, suppress oscillating routes and announce stable ones so it was just sensible "announce the good stuff and try and keep the bad stuff out of the network". There were some issues. Implementations were highly configurable. That's not a problem in itself but it meant that people did the, "Ooh, what happens if I change this number?" and it ended up with people doing a lot of custom cooking of the route flap damping configuration for the network. It was a new toy to play with. They played with it. There was no operational experience to try and figure out what the optimum configuration might be so what happened was vendors came up with numbers that they thought were reasonable defaults and certainly the experience of the European operational community, many of the ISPs felt that the values that vendors had chosen were somewhat aggressive. A couple of flaps could cut you off the network for half an hour or even longer in some cases. So the first attempt at a solution was by the RIPE routing working group, who produced the RIPE 178 document co-authored by a few of the ISPs in Europe, after work at the routing working group. It was updated by RIPE 210 to include the concept of golden networks, which basically were address blocks of the 13 root servers so that those networks would not have flap damping applied to them and was further updated by RIPE 229 with router configuration examples and a website. And we all sat back at that stage and thought, "Right, OK, that solves the problem. If everybody just implements these values then the world will be a perfect place." But, of course, it wasn't. I suppose the most famous work was by Morley Mao and colleagues covering the issues that route flap damping exacerbated Internet routing convergence and there were fairly impressive demonstrations as to where things could go badly wrong. This of course isn't mentioning a lot of other work. Timothy Griffin has looked at this Craig Labovitz and Abha Ahuja have done work on this and not forgetting more recent work by Randy on happy packets and so forth. The main issue was on route changes caused by path exploration increments the flap penalty. A BGP attribute changes, then you've got flap damping as well and this was virtually undocumented little-known thing that seemed to catch a lot of people out. You changed an attribute, it caused flap damping and people were puzzled about this because, while the path was there, the prefix never disappeared, but just the path had changed. Yet, this prefix was being penalised. So the proposal was to do selective route flap damping, rather than just the dampen everything, we were going to try and do some selective route flap damping. This was back in 2002 this paper was proposed so we're looking about over three years ago now. So the proposal listed what is on the slide (refers to slide). I'm not going to go through the particulars but it was an attempt to penalise paths that changed if the paths got longer but, if the path length stayed the same, then we were not really going to do anything about it. Of course, that was the proposal. And I think that's pretty much where it ended. Getting back to what we were talking about earlier, nobody has really implemented any of this. Nobody has really gone knocking on doors to ask for it and I suppose a lot of people just said, "Oh, well, we use flap damping and we use the vendor defaults and they're enough for us" and they just assume that everything is just fine. So it really became, I suppose, a discussion point for the authors. We got a lot of individual feedback, certainly in the last couple of years saying, "Look, guys, this RIPE 229 document of yours is probably doing more harm than good." There was a big debate about the golden networks. Because our idea of golden networks was, "Well, OK, if the root name servers aren't around, the end users won't see much of the Internet" and we have lots of other people coming along saying, "You should add this or that to golden networks and what about Yahoo!, Google, Microsoft and all the rest?" So the golden network bit got out of hand as well. In my work and for my employer I spend a lot of time seeing a lot of ISP network configurations and people just switch on flap damping. When I ask why they're using the Cisco defaults or Juniper defaults they say, "That's what it says in the manual," and I say, "Don't you understand what it does to your network?" people pay no attention. They read it as a good thing to do: it would make the networks more stable. So we had a lot of feedback saying, "Either change the RIPE 229 document to say 'don't do flap damping unless you know what you're doing', 'don't do it at all' or fix it to come up with proper recommendations’.” Hence my presentation at the last RIPE meeting to say, "What should we be doing? Should we write something? Should we let the thing die? Declare it obsolete? Whatever?" I don't know. Is flap damping needed? Some people seem to feel it's needed more at the edge. Some people say it's not needed in the core at all but that's where a lot of problems are seen. By the Internet edge, I mean the ISPs who are not providing transit to any other Autonomous Systems. So these are all discussion points. I mean, I'm not standing up here trying to propose any solution and so forth. But I would really appreciate getting some kind of feedback so that, when I go to the RIPE meeting next month, I can at least a start a little bit more of a discussion about what we should do about the general flap damping thing itself. That's all I really have to say. Are there any thoughts - directly now on the floor? Would people like to put their thoughts on the mailing list? I'm really open to ideas. I'll be the chair as well - if you can use the microphone, name, affiliation, so forth. I think that microphone has died, actually. RAM NARULA: I’m Ram Narula from E-Znet in New Zealand. I have had experience with flapping at a local IX. It's funny - we were peering routes in Thailand. Sometimes we do some reconfiguration to see how it works and with asymmetric routing and things like that and it actually got damped. It was quite funny, I would say. That's just a comment from my side. PHILIP SMITH: A bit alarming I would say. RANDY BUSH: Something else broken about route servers? The list is very long. RAM NARULA: I called up and asked what was going on and they said, "Oh, you got damped." RANDY BUSH: Don't use route server. PHILIP SMITH: I think you are just reinforcing something I have seen all through the region is that people just switch it on because it's in the manual. People don't think about what it's actually doing on the network. Anyway, I welcome feedback on mailing list so people have ideas or experiences - please using the routing SIG mailing list or e-mail me directly if you prefer. OK, thank you. RANDY BUSH: Actually, I have a comment. You might remember this. (Refers to slide) A problem that the vendor implemented prefix damping, not path damping. I think that has caused more problems than anything else with route flap damping and I think that's what anything we produce needs to say. Right, so is that fair? OK, "The allocation and announcement of prefixes". OK, this work was done with the cooperation of ARIN Engineering. How much delay is there from when an RIR allocates space until it's actually announced in BGP? And what's the difference between allocations from an RIR and sub-allocations by the local registries, i.e. the ISPs and who is announcing? OK, the data sources from this were ARIN's Whois data processed by ARIN cooperatively. They made it easy for me to parse and they removed administrative oddities, i.e. people were suspended for one month for non-payment and then they were reinstituted. I didn't want to see that as a de-allocation and a re-allocation. The BGP data from routeviews - the ARIN data, we had the start IP, the end IP, so which address space and what's important is the start date and the end date. And I could tell whether it was a direct allocation from ARIN or a sub-allocation from an RIR. The routeviews data - you've seen it many times - here's the prefix, its length, its origin, when it first appeared - I scanned all the routeviews from 1979 forward to see when an announcement first appeared and when it last appeared, OK? - aAnd that's pinging it out of here and I picked up the prefix and the origin AS. The analysis - routeviews data is run at the University of Oregon on their machines. ARIN is run by ARIN. I combined and processed on my laptop. And the analysis was done using Python because I needed to learn a new computer language and Python rocks! Let me tell you - Python is so much better than Perl. Ah! OK, so, here we see this - the zero is when the prefix was allocated by ARIN. These are direct announcements by ARIN, direct allocations. In other words, ARIN gave it to an end site or an LIR and I see the BGP announcement for the entire prefix. Is when it was allocated, this is how many different prefixes fell into this category and these were weeks. Notice this is 10 years, 20 years (refers to slide). Some prefixes were not used for a long time. Interesting things to note - when they're allocated, there's a big spike. Some people announce very quickly. Notice that some were announced before they were ever allocated. Notice that there's the immediate spike and then there's a drop-off, then there's a long term - oh, one, two, three, five, six, seven years they're announced and then there's a long tail for when those were announced. This is 10 years out. OK. Now, these are direct announcements of full prefixes. Partial prefixes will be fragmentation. There's the curve we saw before, there's the fragmented stuff - we see more and it's distributed quickly. In other words, people are announcing partial prefixes and that number, you will notice, is much higher than the previous number, so people are announcing partial prefixes fairly quickly. These are partial announcements of something directly given. OK? This is the cumulated distribution. So the curves are very similar, you know? It's 10 years you're getting most of this stuff. One year, you're getting 10%, 20%. OK. This is direct versus sub-allocations. In other words, the reds are ARIN taking it to the LIR. The sub-allocations is the LIR gave it to a downstream. What's most interesting here is this part right here. This shows that the closer they get to needing more space, the more likely they are to put it in Whois. This is actually showing what we believe to be true, which is ISPs don't update the Whois data for their customers until they're getting near needing more space. OK. So some thoughts about it - the LIRs tend to announce pretty quickly once they get an allocation - some do. Don't announce for years. Some don't register sub-allocations until they need more space. Notice that this studies when allocations were announced, not when announced prefixes were allocated. In other words, this is not looking at hijacking. As I said, we're working on that as a separate issue. This is looking only at the allocations from ARIN and seeing when they're announced. OK? More study is needed. I'm especially trying to get data from other registries but it turns out the other registries don't have the same full data. In other words, I was discussing with George a few hours ago why I can't find for APNIC the LIRs data that they've - Whois data for what an LIR has allocated to its customers. That I'm calling sub-allocations here. OK. I have similar problems with RIPE data. So getting commensurate data from all the different RIRs is important so that things don't look so different between the RIRs when the problem is the data weren't the same. I just want to thank ARIN for their data and support, NSF for financing things, as usual, the University of Oregon for their help, Verio, Sprint and Juniper and Cisco and Procket for some of the routers that were used to gather data and the packets. And Geoff is thinking hard. Let's hear it, Geoff. Which slide? GEOFF HUSTON: This is Geoff Huston, APNIC. You used routeviews? And the earlier date is 11 November 1997. RANDY BUSH: Correct. GEOFF HUSTON: Which is approximately how many weeks ago from today? It's less than 10 years. RANDY BUSH: Right. GEOFF HUSTON: How can you get longer than 10 years there? RANDY BUSH: Because the first announcements in '97, the allocation record shows - the first announcement is for instance today and the allocation that is in the Whois data shows 1986. That's how you get 10 years. GEOFF HUSTON: There's a tail-end problem because you don't know if it was announced in '86 or not. Everything that's there in 1997 has an unknown date. So I'm wondering how you get the long tail? RANDY BUSH: That long tail is that artefact. It's the distribution of when things were allocated in Whois. GEOFF HUSTON: Do you ignore all the prefixes that were in the first routeviews that you saw? RANDY BUSH: No. GEOFF HUSTON: I think you might want to do that. RANDY BUSH: OK. I'll do it. I know I have to do something next but I'll check it for you tomorrow. GEOFF HUSTON: Do you have any comment, by the way, on the accuracy of the date in the ARIN data records? And why data records were you using to get those dates? The reason why I ask this is that, when I looked at the ARIN delegated files, when change is made to an existing allocation - in other words, they get some more - ARIN's data records that are published every day say, "Oh, this entry is now bigger than it was yesterday and the date is now today, so they get a - RANDY BUSH: These extractions were directly from the ARIN database and cleaned up those kinds of issues. That's why it says, "Thank you, ARIN". GEOFF HUSTON: So all those issues have been cleaned up? OK, thank you. RANDY BUSH: Whether I can say 'all' I don't know but every significant one that we could raise - and that was one of them - was handled. Maybe somebody from ARIN who knows well who is sitting hyped you would care to wander up to the microphone and answer your question. CATHY MURPHY: We did basically treat new delegations. I think what we ended up doing was treating it as a deletion - not a deletion but basically the chunks is new so it did reflect it over time. He's not looking at an allocation for, say, something that was made five years ago - a 17 that was made, then an additional 17 extension. He's seeing the different delegation sizes. RANDY BUSH: We did discuss these fairly extensively before the data extraction was done. Like the other registries, their data was not originally designed as you tragically know as well as I with such analyses in mind but, instead of scraping the raw Whois off a screen, I had the extreme cooperation and a fair amount of work done by actual ARIN Engineering, which is why you see them credited here, to extract the data as best we could to represent the allocations the way we aspirations think of it, not the way the registry as, you know, a business entity, thinks of it. I've been trying to get the stuff that would actually correspond and I would love that kind of relationship with some of the other RIRs. Hint, hint. No, you're not from Telstra any more, Geoff. PHILIP SMITH: Are there any other questions for Randy? No? OK, if not, thank you very much. So that brings us pretty much to the end of the Routing SIG for another APNIC meeting. I would like to thank all the speakers for their contributions. I checked a few minutes ago and I believe all the presentations are just about on the website so, if you would like to have your own copies of the presentations from this afternoon, please visit the meeting website for those. Otherwise, just reminders before we all leave - the social tonight is - it will start at seven o'clock. The first bus leaves at 6:30. The last bus is at 7:10. The two BoFs, which start as six o'clock - the APOPS BoF which is in Function Room 7 and there's also the PGP key signing BoF, I believe. You don't need to run out of either BoFs at 6:30 because there will be a bus waiting to take you to the social tonight at 7:10. Thanks very much to the speakers again, thank you all for attending, thank you to the stenographers for their hard work and see you in Perth. RANDY BUSH: Are the mosquitoes going to really be that bad? GERARD ROSS: Nah. RANDY BUSH: OK. You scared me!