Pakistan YouTube Block Breaks the World 343
Allen54 noted a followup to yesterday's story about Pakistan's decision to block YouTube. He notes that "The telecom company that carries most of Pakistan's traffic, PCCW, has found it necessary to shut Pakistan off from the Internet while they filter out the malicious routes that a Pakistani ISP, PieNet, announced earlier today. Evidently PieNet took this step to enforce a decree from the Pakistani government that ISP's must block access to YouTube because it was a source of blasphemous content. YouTube has announced more granular routes so that at least in the US they supercede the routes announced by PieNet. The rest of the world is still struggling."
Re:But how did they do it? (Score:3, Informative)
A more technical explanation/discussion is here (Score:5, Informative)
Re:How Does One ISP Poison Everything? (Score:2, Informative)
It goes some way to explaining why YouTube became unavailable, but doesn't go into detail.
Re:But how did they do it? (Score:4, Informative)
Re:But how did they do it? (Score:2, Informative)
http://arstechnica.com/news.ars/post/20080225-insecure-routing-redirects-youtube-to-pakistan.html [arstechnica.com]
Basically, pakistan telecom blackholed the network using BGP to advertise that all traffic to those destinations should go to a nonexistant router in their network. Then, instead of keeping that in their own network, they "accidentally" (never ascribe to incompetence what can be explained by rabid extremists) published their route publicly, where it was picked up by the rest of the internet, because the rest of the internet just assumes that nobody would ever fuck up something as arcane and complex as BGP, and makes no attempt to determine whether or not the origination of a given routing rule matches with who should actually be in charge of routing that network.
Orignal story is outdated (Score:3, Informative)
It's too bad that my comment from yesterday [slashdot.org], which links to detailed technical information [cydeweys.com], is still languishing buried.
A Better Technical Explanation (Score:5, Informative)
Re:But how did they do it? (Score:3, Informative)
Political, not religious reasons. (Score:5, Informative)
Re:But how did they do it? (Score:5, Informative)
The BGP [wikipedia.org] article on Wikipedia is as good a place to start as any. Beyond that you can do some Google searches for it.
Basically BGP is the protocol used by routers to exchange route information with each other. A real oversimplification would involve three networks/routers, A, B and C. C receives it's network connectivity through A. C announces the networks it's responsible for to A, whom aggregates them before announcing them (and it's own networks) to B.
In theory, A shouldn't accept any routes from C for IP addresses not owned by C. Apparently that wasn't the case here though, or Pakistan's little stunt wouldn't have impacted anybody outside of Pakistan.
Re:But how did they do it? (Score:2, Informative)
Arstechnica explains (Score:5, Informative)
Why it broke, in techie (Score:5, Informative)
I'll check back for related questions to fill in any blanks later :)
Re:How Does One ISP Poison Everything? (Score:1, Informative)
BGP is the core routing protocol of the Internet. It's used to connect really big blocks of IP address to the net, telling the rest of the Internet how to reach them. For BGP to work, all BGPs routers have to be connected to one another in a mesh network ( TCP connections ) and must have a unique number in this mesh. This number is unique in the world and is assigned by IANA. ISPs that have such number have the "power" to change global routes, and this nutball Pakistani ISP is one of them. What they did was to tell the internet, by updating a global route, that the route to Youtube is now through some router they had.
What they expected is obvious, they wanted toi redirect all request to Youtube to another site. But instead of propagating a new route to the INSIDE of their network, they updated a global route and screwed everything. Ok, they could have just changed an DNS entry and redirected all DNS queries to their DNS server, thus negating the workaround of using OpenDNS.
If they knew the consequences of changing the route is only relevant on the lawsuite Youtube should put on them. They should loose they AS number immediately. And someone should redraft RFC 4271 to give routes an "owner", so If you receive a update on your router from someone not the owner you just discard it. In this way, an router can only update their give block.
Re:But how did they do it? (Score:3, Informative)
Re:But how did they do it? (Score:5, Informative)
Because multi-homed networks may wish to have finer control over which links traffic comes in on then allowed for with simple static routes. Because my above example (A/B & C) was a drastic oversimplification and the actual internet involves tens of thousands (hundreds?) of different networks connected in different ways and trying to manage static routes for all of them would be virtually impossible.
Imagine if the process of connecting a new network to the internet involved having to update static routing tables in every core router on the internet and you'll start to understand why BGP exists. Hell, there are routing protocols meant for internal usage (OSPF [wikipedia.org]), because static routes become unmanageable if you have more then a few routers/netmasks to contend with.
Not strictly. The "honor system" should really be limited to the Tier 1 providers. Anybody else really should be filtering the routes that they will accept. There are already provisions in place to remove the "honor system" from consideration -- it seems that Pakistan's upstream provider choose not to use them.
This really isn't anything new. This kind of stuff has happened before. It's not even unique to the internet either -- the POTS network has a routing protocol used to setup calls/announce which switch is responsible for which number/range. One would suspect that SS7 can be abused by "bad" telcos as easily as BGP can be abused by "bad" ISPs.
Re:But how did they do it? (Score:3, Informative)
Try it without the slash:
http://en.wikipedia.org/wiki/Border_Gateway_Protocol [wikipedia.org]
But how did they do it? (Score:5, Informative)
which was found from Cydeweys [cydeweys.com] which is updating as the story progresses. Both of those sites seem to be running a bit slow, so hesitate before clicking.
Full text of Reneysys: Pakistan hijacks YouTube.
A few hours ago, Pakistan Telecom (AS 17557) began advertising a small part of YouTube's (AS 36561) assigned network. This story is almost as old as BGP. Old hands will recognize this as, fundamentally, the same problem as the http://merit.edu/mail.archives/nanog/1997-04/msg00380.html [slashdot.org]">infamous AS 7007 from 1997, a more recent ConEd mistake of early 2006 [renesys.com] and even TTNet's Christmas Eve gift 2005 [renesys.com].
Just before 18:48 UTC, Pakistan Telecom, in response to government order [renesys.com] to block access to YouTube (see news item [yahoo.com]) started advertising a route for 208.65.153.0/24 to its provider, PCCW (AS 3491). For those unfamiliar with BGP, this is a more specific route than the ones used by YouTube (208.65.152.0/22), and therefore most routers would choose to send traffic to Pakistan Telecom for this slice of YouTube's network.
I became interested in this immediately as I was concerned that I wouldn't be able to spend my evening watching imbecilic videos of cats doing foolish things (even for a cat). Then, I started to examine our mountains of BGP data and quickly noticed that the correct AS path ("Will the real YouTube please stand up?") was getting restored to most of our peers.
The data points identified below are culled from over 250 peering sessions with 170 unique ASNs. While it is hard to describe exactly how widely this hijacked prefix was seen, we estimate that it was seen by a bit more than two-thirds of the Internet.
This table shows the timing of the event and how quickly the route propagated (this is actually a fairly normal propagation pattern). The ASNs seeing the prefix were mostly transit ASNs below, so this means that these routes were distributed broadly across the Internet. Almost all of the default free zone (DFZ) carried the hijacked route at least briefly.
18:47:00uninterrupted videos of exploding jello [youtube.com]
18:47:45first evidence of hijacked route propagating in Asia, AS path 3491 17557
18:48:00several big trans-Pacific providers carrying hijacked route (9 ASNs)
18:48:30several DFZ providers now carrying the bad route (and 47 ASNs)
18:49:00most of the DFZ now carrying the bad route (and 93 ASNs)
18:49:30all providers who will carry the hijacked route have it (total 97 ASNs)
20:07:25YouTube, AS 36561 advertises the
20:07:30several DFZ providers stop carrying the erroneous route
20:08:00many downstream providers also drop the bad route
20:08:30and a total of 40 some-odd providers have stopped using the hijacked route
20:18:43and now, two more specific
20:19:3725 more providers prefer the
20:28:12peers of 36561 start seeing the routes that were advertised to transit at 20:07
20:50:59evidence of attempted prepending, AS path was 3491 17557 17557
20:59:39hijacked prefix is withdrawn by 3491,
BBC said outage was only 2 hours. (Score:4, Informative)
Re: Barack (Score:2, Informative)
I debated whether or not to even mention my involvement with him because half of the Democratic Party has yet to accept him, let alone the other half of the electorate.
FWIW, he was a civil rights attorney at one point in his life.
He's still going to need help. To quote him "Good ideas go to Washington to die". We'll never see any meaningful change come out of Washington until we decide to hold our Congressman to account for their actions. Nobody does though. How else do you explain that Congress (as a whole) has approval ratings in the 20s, yet people continue to send their existing Representatives back, year after year?
People are going to need to get involved in the process and speak louder and more forcefully then the special interests/lobbyists that have hijacked Congress. If that happens then I'm actually very hopeful that Obama can manage to unite this country. If it doesn't happen then I still feel that he will be a force for good -- but his grander ideas will probably fall off and die.
It's Pakistan's main telecom company (Score:3, Informative)
Just about any ISP is going to get themselves a BGP Autonomous System Number and use BGP to communicate with other ISPs.
A long long time ago, when the Internet was smaller and more trusting, long enough ago that I've forgotten the names of the guilty parties, some company in Virginia made a mistake in configuring their router, and announced that their T1 was a really really good route to MAE-East, and about 1/3 of the packets on the Internet decided to go use their T1, for a couple of seconds before it melted... Since then, it's become a Best Current Practice for ISPs to filter out routing announcements from their customers, and most ISPs also filter their peering links with other ISPs, though some are more aggressive about it than others (plus they tend to have limits on how specific a route can be announced, just to keep router table sizes from exploding.)
But even with that, occasional glitches can happen. A couple of years ago, an ISP in South America did a bad job of route summarization (probably using RIP internally, which uses the old Class A/B/C system instead of CIDR), and announced a route for the
It's highly unlikely that PCTL was trying to block YouTube access for the whole world, as opposed to just for their country. That doesn't mean what they did was competent, of course, but it's not too surprising that somebody exported a route to their peers that they really only intended for their customers. Their upstream provider probably should have filtered out the announcements as well. But things like this do happen, and if you're likely to be a major target, either of malice or of incompetence, you need to do the extra work to monitor route announcements that include your address space.
Re:But how did they do it? (Score:4, Informative)
If you're talking about reverend Åke Green he didn't just say "I think homosexuality is bad", he called homosexuality an "abnormal, a horrible cancerous tumor in the body of society". And he while he did stand trial he was not found guilty of any crime.
/Mikael
Re:CBG (Score:3, Informative)
Re:Blaspheme doubtful (Score:4, Informative)
Statistically, Pakistan has the one of the worst records of religious tolerance in the world, and is listed as a country of particular concern by the USCIRF (http://www.uscirf.gov/countries/countriesconcerns/index.html [uscirf.gov] http://en.wikipedia.org/wiki/1971_Bangladesh_atrocities [wikipedia.org]). Even middle-eastern countries are actually doing somewhat better.
Re:But how did they do it? (Score:3, Informative)
Indeed. And I've seen some really great stuff happen since number portability came out. Case in point:
Friend has Time Warner's "digital phone" VoIP offering. Switches to Vonage to save money and ports her number. Now Time Warner customers are unable to call her number -- they get a generic error message. Everybody else gets through just fine.
Vonage refuses to do anything about it (their customer service really sucks, doesn't it?) because it's "Time Warner's problem". Time Warner refuses to do anything about it because "You aren't a customer anymore". My friend gets screwed.
Worst part is, there doesn't really seem to be any appeals process for this type of thing, other then leaving Vonage and going to a provider that would actually enforce her number portability rights. At least with regulated POTS service you could likely file a complaint with the state regulatory agency (the PSC here in New York) and get something done that way. No such avenue for people relying on VoIP products.