TCP Vulnerability Published

Bob Slidell writes "According to Yahoo!, there is a critical flaw in TCP that affects everyone and everything. The article is scant on details and long on fear, hopefully someone will post more details on this." The advisory has more information, and is long on details but only moderate on fear.

OpenBSD is safe?

This just hit the misc@openbsd mail list:

Date: Tue, 20 Apr 2004 12:57:12 -0600
From: Theo de Raadt <[e-mail]>
[snip]

In the OpenBSD case, this is something not to worry about. For what
they discuss, OpenBSD handles this extremely well.

We'll explain more in a week or so.

It sounds (again) like proactive security auditting saves the day!

Re:OpenBSD is safe?

What about proactive spelling auditing?

NISCC slowing, here is the meat summary of article

The issue described in this advisory is the practicability of resetting an established TCP connection by sending suitable TCP packets with the RST (Reset) or SYN (Synchronise) flags set.

The packets need to have source and destination IP addresses that match the established connection as well as the same source and destination TCP ports.

The fact that TCP sessions can be reset by sending suitable RST and SYN packets is a design feature of TCP according to RFC 793, but a reset attack is only possible at all because the source IP address and TCP port can be forged or "spoofed".

Although denial of service using crafted TCP packets is a well known weakness of TCP, until recently it was believed that a successful denial of service attack was not achievable in practice. The reason for this is that the receiving TCP implementation checks the sequence number of the RST or SYN packet, which is a 32 bit number, giving a probability of 1/232 of guessing the sequence number correctly (assuming a random distribution).

The discoverer of the practicability of the RST attack was Paul A. Watson, who describes his research in his paper "Slipping In The Window: TCP Reset Attacks", presented at the CanSecWest 2004 conference. He noticed that the probability of guessing an acceptable sequence number is much higher than 1/232 because the receiving TCP implementation will accept any sequence number in a certain range (or "window") of the expected sequence number. The window makes TCP reset attacks practicable.

Any application protocol which relies on long term TCP connections and for which the source and destination IP addresses and TCP ports are known or can be easily guessed will be vulnerable to at least denial of service attacks

Re:NISCC slowing, here is the summary of article

BTW, I pasted this here mostly as damage control. I know how some people (and yahoo apparently) like to fly off the handle and claim the world is ending without bothering to even RTFA. You wonder why some people are Afraid to use a computer. If I wrote for the auto industry and intentionally tried to scare the shit out of people Detroit would sue me off the map.

There is a new vulnerability that will cause every GM vehicle and cause your children to cry. Vandals can place 1 domestic house cat into the fan and cause the fan to stop and under some cases, cause the vehicle to overheat. This was previously written off as house cats are usually soft ans squishy and have little effect on the powerful fan but Joe Shmoe PHD realised that many house cats have colars that are pretty tough for the fan to digest. Car experts say this is a serious problem and will be dealt with in a serious manner. Suggested work around is to keep your cat tied in the house, and to drive a bicycle instead.

Re:NISCC slowing, here is the summary of article

Besides the fact that their little kitty bones could get into the works and actually stop the fan.

I'd say this is a real threat. We need to protect our SUV's from the mobs of 1337 haxor kitten terrorists! I propose bombing __insert country here__, under the guise of giving them democracy and freedom, and simultaniously pass some laws at home which take away some of our freedom.

Re:NISCC slowing, here is the summary of article

Suicide terrorist kitties?

Al-Kitty?

Yes, that was corny, and no, I couldn't resist.

Re:NISCC slowing, here is the summary of article

"Al-Kitty?"

You're not mangling your Arabic-to-English transilteration enough. It would probably look more like "al Qiddy"

Re:NISCC slowing, here is the summary of article

And putting Al-Kitty through said fan will result in Al-Gore.

Re:NISCC slowing, here is the meat summary of arti

Note: that's 1/2^32 (substitute your own favorite exponential syntax), not one in two hundred thirty-two.

Re:NISCC slowing, here is the meat summary of arti

Actually, this is just a variation on an old attack with sequence number guessing. Some OSes have a very poor generator for these numbers (even though the vulnerability is known for ages) and it is possible to exploit it. Probably the most famous attack of this sort was Mitnick's break-in into Shimomura's machine ten years ago.

In practice, these attacks are quite tricky to perform, because they require good timing and quite a bit of skills, so they are quite rare. It is far easier to just exploit some common buffer overflow than this.

The impact could be not only DOS (by resetting the connection), but you can do also packet injection with potential for total system compromise. However, if the protocol used is encrypted (in Shimomura's case it wasn't, if I am not mistaken, Mitnick attacked rsh or rlogin), then the DOS is probably the worst thing that could happen to you. The encryption will reject the bogus packets, if properly implemented.

So, keep calm - this was here since at least 80-ties and there isn't much that you can do against it, except to use encryption. You can only hope, that your system vendor will decide to make their TCP stack less vulnerable (not likely to happen, since many systems still ship with the crappy sequence number generator!). Full fix is not possible anyway, unless you want to break the protocol.

Regards, Jan

Re:NISCC slowing, here is the meat summary of arti

Yep, sure, however, if you flood the network with your packets, it will be probably noticed soon. Also in order to flood the network, you need a fast connection to the attacked host, reducing the problem to the immediate neighborhood of the attacked host. That's why this attack is not very practical to perform and the impact on most normal mortals is low (except for the BGP issue, which could take out your upstream router)

The problem with BGP, which is mentioned in the original article, is that it has a particularly bad failure mode - when a BGP session goes down, it is very expensive to restart it. To top it off, BGP uses very long transmission windows (because it transfers lot of data and that is more efficient with longer windows) and that makes it easier to squeeze-in the spoofed packet. This is quite messy affair, when it happens.

However, if somebody attacks your favorite web site in this way, I doubt that you are even going to notice it, since http sessions are stateless and comparably cheap to establish.

So, I think this is just a big scare for nothing, it is mainly BGP issue for now. It affects just people who know how to fix it (and are fixing it right now) and the machines involved are relatively few. Unlike some Windows RPC exploit which unleashed a massive world-wide worm in few minutes.

Re:OpenBSD is safe?

Great, I guess Microsoft will just have to copy the BSD TCP/IP code again to ensure that their customers are safe ;-)

Re:OpenBSD is safe?

Yeah. The biggest problem here is the ease with which one could DoS the BGP-4 protocol.

The Internet BGP tables are ricketey enough these days - they don't need every other route to "flap"!

Re:OpenBSD is safe?

Some genius modded my post as a Troll. I guess it's because they know so much about this vulnerability, and how the exposure goes up as one increases TCP window-size. ;-)

Really, though. If you need to calculate a valid offset from the ISN, big TCP-window sizes are of advantage to the attacker.

To quote from the announcement:

In a TCP session, the endpoints can negotiate a TCP Window size. When this is taken into account, instead of attempting to send a spoofed packet with all potential sequence numbers, the attacker would only need to calculate an valid sequence number that falls within the next expected ISN plus or minus half the window size. Therefore, the larger the TCP Window size, the the larger the range of sequence numbers that will be accepted in the TCP stream.

BGP-4 relies on persistent connections, with huge window sizes.

Re:OpenBSD is safe?

As a side note, all the major sites with several BGP peering points have recently started using MD5 authentication. We have been updating all of our peering sessions over the last week or so.

Re:OpenBSD is safe?

If you are using MD5 encryption that is built into BGP4 that won't help you. That merely will stop route poisoning and such. As long as your underlying TCP/IP implementation isn't authenticating source/destination you can still get hosed.

Re:OpenBSD is safe?

Or it might be that you just read the article, and more or less parroted what a more experienced individual said. Especialy given that anyone who deals w/ BGP on a regular basis knows (or better know) how to secure peering links against this kind of vulnerability.

It's actually quite trivial to do. ebgp-multihop and using the remote host loopback address as the neighbor IP, along with a few small architectural configuration tricks are all that's needed to completely prevent this kind of attack, or, at the very least, minimize it significantly.

Are _you_ going to scan, not only the entire range of source ports, but all those ports times the entire RFC1918 address space? Good luck killing my systems, buddy, 'cause that's what you're in for.

Oh, even with the large window sizes, once you've found all the above, you've still got to find my sequence numbers.

RFC 2385

There is RFC 2385 [ietf.org] titled Protection of BGP Sessions via the TCP MD5 Signature Option. In the first paragraph of its Introduction section it says -

The primary motivation for this option is to allow BGP to protect itself against the introduction of spoofed TCP segments into the connection stream. Of particular concern are TCP resets.

The date of publishing - August 1998, which makes it about 6 years old.

However the proposed option was primarily meant as a quick BGP fixup, which should've got depricated as soon as IPsec got RFC'ed and deployed. It did went standard few months later, but IPsec implementations enjoyed full share of cross-vendor compatibility problems.

With time Authenticated Header (AH) IPsec protocol didn't see much use or acceptance and IPsec slowly evolved (and keeps evolving) into confidentiality rather than authentication layer.

Besides it's an IP security after all, while the RST spoofing is a TCP problem. And one can quite rightfully percieve authenticating TCP with IPsec as an overkill.

Anyhow, long story short - it's a known and rather old problem with handful of existing and equally unattractive solutions. Perhaps this time around it will be addressed better due to the amount of the publicity it's aimed to get.

Re:OpenBSD is safe?

>For what
they discuss, OpenBSD handles this extremely well. We'll explain more in a week or so.

Is the margin of the page too small to explain the wonderful reason why it handles this so well?

Re:OpenBSD is safe?

Good and funny :) but I think what they mean is that more details would give more info for exploits, so their sitting mum til things can get solidified a little more.

What details, the info is already out

The NISCC articles explains things clearly. Nobody needs any more information. Besides, this "new" attack was already known. People just didn't realize how easy it is.

Re:OpenBSD is safe?

Theo Wrote:
Let me be more clear.

This entire thing is being "sold" as `cross-vendor problem'. Sure.
Some vendors have a few small issues to solve in this area. Minor
issues. For us, those issues are 1/50000 smaller than they are for
other vendors. Post-3.5, we have fixes which make the problem even
smaller.

But one vendor -- Cisco -- has an *UTTERLY GIGANTIC HUGE* issue in
this regard, and as you can see, they have not yet made an
announcement see..

You are being told "lots of people have a problem". By not seperating
out the various problems combined in their notice, or the impact of
those problems, you are not being told the whole truth.

More Theo:
OpenBSD (and I am sure other systems too) have for some time contained
partial countermeasures against these things.

OpenBSD has one other thing. The target port numbers have been random
for quite some time. Instead of the Unix/Windows way of
1024,1025,1026,... adding 1 to the port number each time a new local
socket is established... we have been doing random for quite some
time. That means a random selection between 1024 and 49151. This
makes both these attacks 48,000 times harder; unless you already know
the remote port number in question, you must now send 48,000 more
packets to effect a change.

At least one other free operating system incorporated our random port
selection code today..

We've made a few post-3.5 changes of our own, since we are
uncomfortable with the ACK-storm potention of the solutions being
proposed by the UK and Cisco people; in-the window SYN or RST's cause
ACK replies which are rate limited.

At least one other free operating system today incorporated the same
changes......

Re:OpenBSD is safe?

It sounds (again) like proactive security auditting saves the day!

It doesn't save anything. When someone exploits this and takes out 90% of the Internet's routers, you're screwed no matter what.

It definitely makes a good argument for both OpenBSD and proactive security auditting. But it doesn't save the day.

Re:OpenBSD is safe?

It doesn't save anything. When someone exploits this and takes out 90% of the Internet's routers, you're screwed no matter what.

But it saves the day for my network of 3 linux boxen in my basement which are s0 K3wl, they r0x! While the Internet burns to the ground I can route packets back and forth with impunity between my 486 laptop and my Pentium II Server!! WooHoo!

Re:OpenBSD is safe?

LONG LIVE THE INTRANET!

Re:OpenBSD is safe?

You just wait until the stock market is driven crazy by all those dotlocals with impossible business plans.

Re:OpenBSD is safe?

...in my basement...

err um, don't you mean your parent's basement :)

Re:Server exploit, not router

If you read the article it'll explain it for you, but basically:

Routers from different AS's talk to each other using BGP4. This protocol uses TCP.

The "exploit" uses spoofed packets, so the router will process them as if it came from their neighbour.

So yes, while the router is mostly "shoveling packets", it learns the direction in which to shovel these packets via the exploitable method.

If you want to disagree, feel free to let all the major tier 1 ISPs who have been busily implementing MD5 authentication on their BGP sessions know their wrong.

Windows also safe

In a press release from Microsoft, Bill Gates states:

All Windows versions from 3.11 to 2003 are quite safe from this exploit, since Windows also supports the famously reliable NetBEUI protocol. In a proactive measure, Windows update will remove support for TCP/IP and ensure that all updated computers have support for NetBEUI only. NetBEUI will once again rule the earth! Take that, Steve! No, not you, Ballmer, the other Steve. The hippe. Woahahahahahaha!

In a quickly following press release, Bill Gates adds:

Woahahahahahaha! Hahahaha! Hahaha! Thank you.

Re:Windows also safe

Ah, come on! I was joking, not trolling for flames. And besides, how the hell was that going to attract flames? If that really was flamebait, it should be modded -1, ineffective.

(Was it the hippie part? Yeah, sure calling Steve Jobs a hippie is flamebait, but this was also clearly a joke. Some moderators are just in a dire need of a blow job.)

Re:Windows also safe

Some moderators are just in a dire need of a blow job.

Nice of you to volunteer, looks like their outlook has improved already :)

Re:Windows also safe

"Some moderators are just in a dire need of a blow job."

And that should be modded "-1, Redundant" but you don't hear me compl...oh..shit.

Re:Windows also safe

Security Update for Windows XP (KBTCPDRM-666)

This update addresses the vulnerability addressed in Microsoft Security Bulletin 666. Find out about more recent critical updates in the Overview section.

File Name:

WindowsXP-MSTCPDRM-x86-ENU.exe

Download Size:

1261 GB

Date Published:

4/20/2004

Version:

666

Overview

This patch fixes criticals security vulnerabilities present in Windows TCP stack.
This patch also add the new DRM TCP extension.
When is patch is applied, your computer will connect to drm.microsoft.com prior establishing any other connection to make sure the requested end point is an authorized Microsoft partner. All rogue packets are now rejected and reported by the Windows TCP-DRM firewall (TM).
This patch also upload the registry key HKEY_LOCAL_MACHINE and all subkeys and values to drm.microsoft.com so we can make sure all software is used according to their end user licence agreements.

System Requirements

Supported Operating Systems: Windows XP

Windows XP Professional
Windows XP Home Edition

More from Theo (was Re:OpenBSD is safe?)

From: Theo de Raadt <[e-mail]>
[snip]

Let me be more clear.

This entire thing is being "sold" as `cross-vendor problem'. Sure.
Some vendors have a few small issues to solve in this area. Minor
issues. For us, those issues are 1/50000 smaller than they are for
other vendors. Post-3.5, we have fixes which make the problem even
smaller.

But one vendor -- Cisco -- has an *UTTERLY GIGANTIC HUGE* issue in
this regard, and as you can see, they have not yet made an
announcement see..

You are being told "lots of people have a problem". By not seperating
out the various problems combined in their notice, or the impact of
those problems, you are not being told the whole truth.

Re:More from Theo (was Re:OpenBSD is safe?)

For us, those issues are 1/50000 smaller than they are for other vendors.

So, they are 50,000 times bigger ?

Re:More from Theo (was Re:OpenBSD is safe?)

>> For us, those issues are 1/50000 smaller than they are for other vendors.

> So, they are 50,000 times bigger ?

No, that would be 49999/50000 as big.

Re:OpenBSD is safe?

I guess they were smart enough to implement the new Evil Bit added to TCP last April. Those OpenBSD folks sure are forward thinking.

Re:OpenBSD SMP

Combine that with the fact that almost all of the programs used on a modern UNIX-like system arn't CPU bound and it's easy to see why SMP took a back seat to more "interesting" issues.

Wrong. Very wrong. Are you anaware of this field called "banking"? What about "financial trading"? These firms have huge portfolios and run and re-run various models on them. At night, the systems have to run various "end-of-day" scripts and reports, which take CPU-hours to complete. Most of this things run on Unix...

There is also on-the-fly image manipulation, and the scene-rendering done by fleets of Unix machines. The more CPUs each such Unix machine can fit, the better.

Then come databases -- depending on the queries (with joins and orderings), DB-servers can be CPU bound and appreciate multiple processors when available.

What about PVM [ornl.gov]? What was it -- and similar packages both free and commercial -- written for? What about this proverbial "beowulf clusters"? Of course, it is much better to have several CPUs inside the box, rather than in separate machines.

However there is a developer being paid to work full time writing SMP support for OpenBSD. He expects to have a working implementation by 3.6 or 3.7.

Until which time, the OpenBSD zealots will continue to deny the issue exists or is of any importance. I see...

Re:OpenBSD is safe?

Now I what read up on TCP/IP and my memory isn't that good, but isn't it the TCP part which a problem, mening that switching to a different IP protocol won't help you at all. IPv6 is as far as I understood it, only adressing, the TCP stuff is the same... maybe not.

Best security advice...

Just unplug your PC from the internet and wash your hands of it.. the whole thing feels holier than swiss cheese :(

Re:Best security advice...

Just unplug your PC from the internet

How would that keep you safe from DoS attacks?

Re:Best security advice...

Why would anyone put a computer running DOS on the internet in the first place?

He plans to show the exploit this Thursday!

This kind man responsible for finding this vulnerability is going to present this exploit at the security conference in Vancouver this Thursday. He then predicts "hackers will understand how to begin launching attacks 'within five minutes of walking out of that meeting.'" The article talks about how the government has been "fortifying" its networks against this, does that means they quickly rewrote the tcp protocol? I would love to know.

Re:He plans to show the exploit this Thursday!

Maybe the speed at which TCP was written is the problem. If they re-wrote it, I hope they did a slow re-write, because we will need the patches.

Really, I think the problem is that the flaw affected /some routers/ whose implementation of the TCP stack was flawed. That is what I gathered, anyway. If this is so, they just need to find non-flawed software.

Re:He plans to show the exploit this Thursday!

No, its not a problem that is software-specific. They are utilizing a 'flaw' in the design of TCP by spoofing the sender's IP address and port, *AND* guessing the recvwindow number!!

The reason it only affects long-term connections is because it takes awhile to probe for the magic number to reset the connection, and ALL this does is reset the connection, nothing more.

This whole thing is retarded, if you know enough about TCP this exploit was already on your mind and forgotten because its stupid. :P

-Bill

Neither forgotten nor stupid

The relevant parts of this vulnerability are 1) that RST attacks are much, much easier than formerly thought, making them possible for your average broadband sub, and 2) that BGP in particular is highly vulnerable, given the consequences of a terminated BGP session.

A recently published I-D (here [ietf.org]) claims 200 seconds is sufficient time for a broadband sub to successfully attack a TCP session, provided their ISP doesn't use egress filtering (and way too few do so).

This is rather serious. Whether you personally aren't susceptible is irrelevant if your upstreams are.

Re:Neither forgotten nor stupid

Of course, in a sanely designed backbone, ingress filtering should be in place to filter source ips from BGP peers that aren't specifically on the interface matching the peer (yes, there's multi-hop BGP, but ignoring that for the moment...).

I do realize this likely isn't the case on many networks, but perhaps this will push such sanity (and very simple) filtering to become more widespread.

Re:Neither forgotten nor stupid

> A recently published I-D (here) claims 200 seconds is sufficient time for a broadband sub to successfully attack a TCP session, provided their ISP doesn't use egress filtering (and way too few do so).

Maybe it's about fucking time ISPs started using egress filtering. At the very least, there'd be an order of magnitude less crap (smurfage, etc) if ISPs dropped spoofed source IP packets before they got to the backbone.

OK, so the ability of any skript kiddie to spoof and insert a BGP update is a pretty fucking huge mess. But "pretty fucking huge" it may be the only kind of mess that motivates the clueless fucktards pretending to be "ISPs" these days.

Re:He plans to show the exploit this Thursday!

From what I gathered in the two links and also my knowledge of TCP/IP, it would not necessarily require a flawed implementation of the stack in order to be vulnerable to attacks of these sort. In fact, it is the routers and/or software which doesn't implement the stack according to spec which are less likely to be affected.

In the mean time, there are a few workarounds which can be put in place, such as IPSec, and options which can be changed to reduce the liklihood of an attack, such as the window size. The smaller it is, the harder it is to guess a sequence number in the range quickly.

Not flawed, just unimaginative.

The problem isn't TCP stacks that are flawed because they don't implement TCP properly. It's TCP stacks that failed to imagine some of the creative ways to attack them. Sequence number guessing has been around for a while (see papers by Bellovin and others), but apparently the guy has figured a somewhat more efficient way to guess them on some popular platforms, apparently including Cisco routers.

Routers don't usually do a lot of TCP themselves, except SSH or telnet access for management, but the one big exception is the BGP protocol that routers use to connect to each other, primarily at the interfaces between carriers. The BGP sessions stay up for a long time, and killing them tells the router that it's no longer talking to its neighbor so it should go find a different route to get to that network, which is really really annoying. On the other hand, BGP has options to do more checking on BGP messages before accepting them, and carriers that do spoof-proofing to prevent their customers from forging packets have less risk, and carriers that block packets addressed to their routers accept from approved destinations are much safer.

Some customer connections to ISPs use BGP, especially if the customer uses multiple ISPs, though most are static - these could be subject to spoofing by somebody inside the customer's network, if they're not careful. (The customer could be an ISP, or they could be a regular customer with an employee who has next month's interesting virus.) So customers who don't want to get thrown off the net should probably be careful to do good firewalls to keep their own users from spoofing their connections.

Re:He plans to show the exploit this Thursday!

The article talks about how the government has been "fortifying" its networks against this, does that means they quickly rewrote the tcp protocol?

Nothing so drastic. Go back to the article and reread it, especially the "Mitigation" section. You will find:

• It mainly affects the Border Gateway Protocol (BGP) that occurs at a high level in the net. Few computers are involved.
• The issue was first publicized about a month ago at CanSecWest [cansecwest.com], so those in the know have had a month to work on this.
• The steps to mitigate the problem are a matter of tweaking settings (like window size) or setting up protocols (like encryption). This is not a matter of rewriting the entire protocol.

The article is interesting in that it shows how the bits and pieces of the Internet fit together, but I won't be losing any sleep over this one.

I'm sure this...

...will turn out to be nothi* [Carrier Lost]

Good

let's all just start again

TCP2
SMTP2
POP32 ...

Re:Good

I'm no expert, but wouldn't a security problem with TCP be completely unrelated to IP?

Re:Good

I don't think so.

Looks like the weakest point for net-wide effects in routers implementing BGP. A concerted attack could tie up critical routers rebuilding routes after losing connection to their peers. Since this could be globally critical, I suspect the major hardware vendors and service providers will be jumping through hoops to get the fix in before some blackhats get coordinated with an exploit. There would still be weaknesses, but IPv6 will get to sit idle a bit longer.

Re:Good

LOL OK I happen to setup BGP router as part of my living so I might be able to shed some light on the subject. Standard security settings allready in place will stop this attack. OK Lets go for starters, BGP packets unless multihop should have a TTL of exactly 1 and come through a point to point interface. Basic anti-spoofing on each side will stop any packets that cleam to be from the other end of that interface from comming in any other interface same for any packets with the routers own address from comming in any interface. Thats basic router security. BGP does support preshared keys as well though I'm not sure if that will stop this attack as it's more to prevent session hijacking. I dont see a 'fix' for this comming out besides normal security settings.

This exploit along with many others could be made significatly harder to do if more ISP would just turn on anti spoofing rules. I cant think of a router that dosent support at least manual anti spoofing setups. The only time antispoofing might be a problem would be people that are utilizing multiple connections and trying to agrigate them together for outgoing bandwith.

Re:Good

As frightening as this "vulnerability" sounds, this is nothing really new; other TCP weaknesses are syn floods [cs.hut.fi] (not quite the same thing, but somewhat similar -- in fact, this vulnerability might as well be called a "RST flood"), connection hijacking (by sniffing packets and sending spoofed packets with the correct sequence numbers), and so on. It's also an implementation issue that is largely caused by implementations having loose checking of TCP sequence and ack numbers, or accepting too large of a window of sequence numbers.

I wouldn't say TCP is broken or that some other solution would be much better; it would be tough to design a transport protocol that is still simple (and doesnt use CPU burning hashing/encryption techniques) that wouldn't have these sorts of vulnerabilities (especially since it's so easy to spoof IP packets); calling this vulnerability severe is like screaming that highways are fundamentally unsafe because someone could point their car the wrong way and start smashing into oncoming traffic.

-fren

Re:Good

other TCP weaknesses are syn floods (not quite the same thing, but somewhat similar -- in fact, this vulnerability might as well be called a "RST flood"),

1. We know what the other TCP vulerabilities were. Anyone who's still susceptible to them is a lunatic.
2. This is not at all the same thing and in no way similar other than the fact "it uses TCP".
3. This vulnerability should not be called an RST flood. That would confuse it with a SYN flood which works totally differently and is much simpler. This should be called a broken window attack or something.

I wouldn't say TCP is broken

After noting all the other kinds of irrelevant TCP attack and reading up on this rather serious one, you wouldn't say it was broken? Could it be that, like everyone else including me, you never realised it was broken because you never looked close enough?

it would be tough to design a transport protocol that is still simple (and doesnt use CPU burning hashing/encryption techniques) that wouldn't have these sorts of vulnerabilities

It's called IPSEC, it's secure on the IP level up so TCP is encrypted over it. The simpler way to increase security would be to maintain current window size but increase the sequence number field to 64 bits wide. This would make it nigh-impossible to find where the window is sitting.

calling this vulnerability severe is like screaming that highways are fundamentally unsafe because someone could point their car the wrong way and start smashing into oncoming traffic

Highways are fundamentally unsafe. They're full of retarded people shunting 3 ton hunks of metal around at speeds they're not comfortable with. But your point is void because people would not do that as they'd die as a result and kill a lot of people. I don't think a kid doing a TCP RST attack really needs to be that dedicated, and this could cost businesses millions of dollars if they don't wise up to it sharpish. Most people who'd took even a short course in networking would be wise to it already.

The best defence against this? Simply check for a stream of RST packets. They dont come in huge bundles with incrementing sequence numbers often. Detect that signature, block IP, sorted.

OSVDB

http://www.osvdb.org/displayvuln.php?osvdb_id=4030 [osvdb.org]

TCP Reset Spoofing

OSVDB ID: 4030
Rating: TBD
Disclosure Date: Apr 20, 2004

Description:

The TCP stack implementation of numerous vendors contains a flaw that may allow a remote denial of service. The issue is triggered when spoofed TCP Reset packets are received by the targeted TCP stack, and will result in loss of availability for the the attacked TCP services. ...

Re:OSVDB

So, it's simply another type of denial of service. A TCP Reset packet can be faked, which will result in a legitimate open TCP connection being closed by a third party.

It also assumes a few of things.. You would need to be in the network route, or have some other means of establishing the sequence numbers for the connection. You would also need to be in a network that does not filter out improper source addresses (which some networks do.. but more should do) to allow your spoofed packet to get through.

So, the problem boils down to a low bandwidth method to DoS TCP connections. It does not give a method to hijack the connections, or otherwise gain access to data.

Anyway.. Clear IP connections are very open, and completely based on trust. Any sensitive communication should be tunneled inside a VPN, where it would not be vulnerable to this type of attack. (A real IPSec tunnel would not be vulnerable - as it does not use TCP. "SSL VPNs" use TCP as a transport, so they might be open to DoS.. Stick with the security of IPSec for best results.

That's it!

I'm removing support for TCP right now. Give me UDP or give me death!

Re:That's it!

And what's ICMP, chopped liver?

I want a new internet based on morse code ping responses... 10 ms for a dah.

Re:That's it!

i think he meant ICMP when he said "...or give me death".

Re:That's it!

Connected to Internet

OSVDB ID: 4030
Rating: TBD
Disclosure Date: Apr 20, 2004

Description:

The Internet has been determined to be full of evil hax0rz. Any computers connected to the Internet are deemed vulnerable to this exploit.

Solution:

Unplug cable, power down WAP, close bomb shelter doors.

oops?

Looks like someone left ISEXPLOITABLEFLAG = TRUE in the code.

No problem

I'll just switch to UDP.

Re:No problem

more like:
UDP just I. switch ll'll to I just

S

yoda?

Is that you master?

L. Skywalker

Implementation issue

Neither of the linked articles helps understand the issue but this one does [osvdb.org],

Furthermore, RFC-793 allows a TCP implementation to verify both sequence and acknowledgement numbers prior to accepting a RST control flag as valid. No TCP stack implemention tested currently implements checking of both sequence and acknowledgement. All tested TCP stacks currently verify only the sequence number. This allows connections to be reset with dramatically less effort than previously believed.

Hence this is an implementation issue that can be patched in TCP stacks.

Move along, little to see here.

John.

Re:Mostly Related to BGP?

Yep, that's the issue. I submitted too, but :(.

Anyway, the way I read it you basically run the TCP attack against a BGP peering router, causing it to drop one or more of it's peering relationships. Do that enough and you can cause the routes being advertised by that router (and also TO that router from the peering connections you're breaking) to be 'dampened' - a protective mechanism in BGP to prevent a flapping route from making all the peers recalculate their routes nonstop.

It's kind of like one peer putting the other one's routes in "time-out" until he plays nice.

Re:Mostly Related to BGP?

Well, that means our home PCs aren't likely to get exploited by this. However, if our ISP's router gets exploited, we're knocked offline and our PC isn't as useful as it used to be.

The threat here is a DOS aimed at a few things that we don't want to see go down.

Work

As a web designer, taking advantage of this could get me off work faster than a snow storm. I don't know if I'm afraid or enthused. ;)

The time has come

to switch over to IPX

what about slow start?

you can exploit slow start too. was published last year for some conference. The paper is called "The Shrew vs. the Mice and the Elephants" and is by Aleksandar Kuzmanovic and Edward W. Knightly

Warning!

Your computer is broadcasting an IP address!

Seriously though, it doesn't look all that bad. (Nor does it look all that hard to do, but still..)

Impact moderate for users, serious for providers

The exploit apparently allows an attacker to disconnect TCP sessions, so really home users won't have much to fear except perhaps to get more trouble connecting to their various sites than usual, and that is in case they would be under active attack.

Service providers on the other hands, must protect their routers because the BGP protocol used to distribute Internet routes between them, massively uses TCP. And when routes go missing, it is hundreds if not thousands of routes to your favourite places that go unreacheable.

The problem in the case of BGP is made worse by dampening [cisco.com], i.e. keeping the flapping routes out of the routing table for a certain amount of time (up to several hours). BGP routes dampening is not always configured. A determined attacker with this knowlege would be able to knock large portions of the Internet offline for hours.

Simple BGP solution has been around since 1998

As they state, there is a simple solution: TCP MD5 Signature Option with BGP. Any ISP worth their salt will already be doing it. The rest will learn the hard way.

This has been supported in Cisco IOS way back since ~1998 in IOS 11.2 [cisco.com] .

Read the BGP "Bible": Internet Routing Architectures [bookpool.com] or look at any best-practices guides [google.com] which will state that TCP MD5 sigs should always be used with BGP.

Or search CCO [cisco.com]:

router bgp 109
neighbor 145.2.2.2 password v61ne0qkel33&

It's just a single line that has to be added to both peer sides.

BGP vulnerable

I happen to work for a large, nationwide backbone. We've known about this for about a week now. BGP, configured without an MD5 key (as is usually the case) is extremely vulnerable to this exploit. This is the reason for the top-secret effort in the past week to MD5 all peering sessions, both internal and external on most major networks worldwide. Without this, it's trivial to exploit, in fact we already have source code provided by the NCISS. Input a few IPs and BGP's TCP port number, and wham you take down a peering session. For those that don't understand what this means, prior to the security changes that have been implemented in the last week, the global internet was largely susceptible to this flaw in such a way that major portions could have been taken offline easily. A priority was put on this within the intra-NOC communications channels that exist that has never been seen before to lock this down before the public knew about it. We were embargoed by DHS to not release the information until tomorrow.

Re:BGP vulnerable

We were embargoed by DHS to not release the information until tomorrow.

And if anybody could determine the identity of an Anonymous Coward, it certainly wouldn't be an inside group of hardened NOC geeks.

Oh wait...

Good info, though. Thanks.

Re:BGP vulnerable

What I find most worrisome is this:

"BGP, configured without an MD5 key (as is usually the case)...."

So, critical infrastructure, worthy of "a priority...that has never been seen before", is routinely run unsecured? *shivers*

Empherial source ports

Funny, but it seems that empherial source ports for a TCP connection may be more secure in this case, since it increases the space that the attacker has to guess within.

Of course it is a pure "D'oh" that large TCP windows increase exposure to the older known weakness of TCP RST attacks (Steve Bellovin, wrote a paper [att.com] on it in 1989).

Known issue

Apparently this has been known about for a while. Here's an excerpt from an IETF draft on BGP vulnerabilities from June 2003. Section 3.2.1.4 specifically mentions the attack described by Watson: From http://mirrors.sunsite.dk/drafts/draft-ietf-idr-bg p-vuln-00.txt [sunsite.dk]

3.2.1.4. TCP RST/FIN/FIN-ACK

Event 18: If an attacker were able to spoof a RST, the BGP speaker would
bring down the connection, release all associated BGP resources, delete
all associated routes and run its decision process. If an attacker were
able to spoof a FIN, then data could still be transmitted, but any
attempt to receive would receive a notification that the connection is
closing. In most cases, this results in the connection being placed in
an Idle state, but if the connection is in the OpenSent state at the
time, the connection returns to an Active state. Spoofing a RST in this
situation requires an attacker to guess a sequence number that is in the
proper half of the sending window, generally an easier task than
guessing the exact sequence number so as to spoof a FIN. The use of [5]
will counter this attack.

Exactly.

It is indeed a problem with TCP but, it is in no way new. Many people have relied on this "vulnerabillity" for years. This is the technique that many firewalls implement in order to terminate undesirable sessions.

This vulnerabillity impacts the applications that are unable to handle session interruptions. BGP is such an appilication but, its impact on BGP is critical because this attack could seriously impeed BGP and since the internet backbone relies on BGP such an attack could covceivably shut down the internet. This is a known vulnerabillity in BGP, as you said, and it is surprising to me that BGP has not been modified to prevent this from happening.

By and large most applications will be able to deal with this and only minor annoyances such as DoS will occur.

As for earlier posts about OpenBSD being immune, I am very eager to hear how that is possible. No matter what the implementation, it is still possible to reset TCP sessions. This means that OpenBSD is still potentially vulnerable to DoS attacks.

Provided that BGP is modified to make it more fault tolerant, I can't see this problem with TCP being anything more than a short-lived niusance. I'm sure however that many people will try to use this as leverage to force IPv6 migration however, I believe that it too is vulnerable.

Re:Stupid

What is new here is that you don't have to know the completely specified sequence number, but only something close enough to fit within a window. This reduces the search space and makes the attack more pragmatic.

Joe User might not notice

... since TFA goes on to say the vulnerability explicitly affects "long-lived" TCP connections, not the POP3 / HTTP / SMTP that Joe User relies on. However, for businesses and security wonks this is a potentially big deal.

No problem...

i'm posting this over NetBEUI Protocol ;)

*sight*

RFC3360

For more information about what TCP resets are and why they can be harmful, see RFC3360 [faqs.org].

Here is what to do...

The article is being presented at CanSecWest, and is called "Slipping in the Window" by Paul A. Watson. I have two friends at CanSecWest, I've asked them to attend and report back what the feeling is.

NANOG members are talking about it, and several regional Tier-1 players have already issued customer notifications.

This exploit goes up against TCP connections that have been established for long periods of time. i.e. not web connections. The most prevalent would be BGP peer connections, which can be up for days on end easily. Without having read details, or the paper itself, by forging packets of BGP peers with adjusted window sizes, you can cause a router to reset (possibly hang, depending on IOS or JunoOS version, not sure about this) it's BGP peer connection. If you were doing eBGP and had your own AS, a directed attack against your gateway routers could force flapping, which would cause route dampening, and lead to denial of service.

What you need to do, is contact your ISP if you are an enterprise network admin, and establish MD5 authentication on your BGP sessions. Check with Cisco or Juniper and find out if your code will drop non-MD5 BGP packets directed at it. An ACL won't do, the attacker would forge the src-ip of a known peer.

This is a completely non-trivial attack to coordinate. You need to know the IP address of the BGP peer of a customer, or the route reflector, and then get the IP address right in an attempt to bypass ACLs and get the BGP session to hang. eBGP multihop means that IP could be any number of routers, and unless you have inside info, you don't know what it is.

Potentially, looking glasses could be used to mount attacks at NAPs or other peering points, but again, I think the major players will be ready for it very shortly, and will spend most of today (if they are any good) coordinating with legal teams to slam the shit out of any forged sessions they see, and start cooperating to run traces with other providers.

If I could editorialize one moment, none of this would be an issue if providers took better care to implement anti-spoofing techniques. Forged src-ip addresses are the bane of security. Most of these attacks don't care about 2-way communcations, they just want to reset connections. Spoofed src-ip lets them do that. Rant off.

SCREECH *BAM* *poof*

Wow. That uninterrupted block of text hit so hard it set off my browser's airbag!

Does the affect tcpip/cp?

I am a lonely man living on the Galapagos Island. I use TCP/IP over carrier pigeon to communicate with a Nigerian who has promised my great wealth in exchange for securing funds in the First Galapagos Bank, of which I am owner/ceo/clerk, and janitor.

I suspect someone is interupting my data stream and keeping the replies and account numbers he has been sending me in regards to my money. This vulnerability proves my theory. I am in desperate need!! How can I prevent this!!

Anyone willing to help I will share my wealth with.

/obscure humor (Does this make me a Galapagos Spammer?)

Link to US-CERT TA04-111A

Here's a link to US-CERT's TA04-111A [us-cert.gov] on this topic.

This is new??

Maybe I missed something in the advisory, but this sounds like a good old TCP reset attack.....which is neither a new or novel concept. People have been doing this for ages with a sniffer and a packet generator.

Re:This is new??

Maybe I missed something in the advisory, but this sounds like a good old TCP reset attack...

You did.

The news here is that you no longer require the sniffer, because you don't have to know the exact sequence number. Now you just have to be kind of close. How close apparently varies in size amongst vendor implementations.

Re:This is new??

I think the point is that it lets you perform this attack without being in the middle with a sniffer. The author figured out a way to significantly reduce the search space for the sequence number. Quoting from the article, "[Watson] noticed that the probability of guessing an acceptable sequence number is much higher than 1/2^32 because the receiving TCP implementation will accept any sequence number in a certain range (or 'window') of the expected sequence number. The window makes TCP reset attacks practicable."

Previously you had to have a packet sniffer in the middle to sniff the sequence and spoof it. Watson has pointed out a technique that allows you to guess the sequence in practical number of tries for connections that are long-lived (a few days).

bogus, depending on your OS.

I read the paper, which is nothing more than a rehash of what every security person already knows about: tcp sequence guessing.

This issue erupted on the freebsd irc chat, and I had to kill it by posting this linkage: http://lcamtuf.coredump.cx/newtcp/ [coredump.cx]

As you can see TCP sequncing is fairly random in most of the IP/TCP stacks out there in commercial use. The reasearch paper mentioned in the article only applies to the OS's in the above link that dont' have near true 32bit randomness.

The feasability of overcoming realtime 32 sequence guessing is insane, however non-zero. just my .02c

Re:bogus, depending on your OS.

The randomness of the TCP ISN generator doesn't matter. The fact is, you don't need to guess the current sequence number in the connection. If you guess *any* sequence number in the window ahead of the current sequence number, it's just as good. So all you need to do is count from 0x0-0xffffffff by increments of just under the TCP window size. You can hit the right number within a few minutes if you already know the source and destination ports and you have sufficient bandwidth. Longer if you have to guess the source port.

Visualizations of OS's TCP seq numbers

This is sort of on topic with your post. Here's a discussion and visualizations of various OS's TCP sequence numbers:

Strange Attractors and TCP/IP Sequence Number Analysis [bindview.com]

IETF TCP Security Considerations draft

There is a new Internet draft [ietf.org] addressing this issue.

The real problem?

The real problem is not in TCP (although sequence number windows are a bad idea), it's in allowing in spoofed IP datagrams. If network admins locked down their routers correctly then script kiddies wouldn't be able to send forget packets, and this wouldn't be an issue.

Note to netadmins and sysadmins: block packets with source IPs that are not valid for the interface they came from! Geesh!

Old news from 1998 and probably before

Mudge from the L0pht talked about taking down the internet in 30 minutes with a router DoS attack in front of the US Senate in May 1998. Privately the L0pht told NIPC that this could be done with a BGP TCP reset attack. L0pht said it could be mitigated by doing ingress/egress filtering but that ISPs were to lazy and cheap to do it.

In Aug 1998, RFC 2385 came out with protection of BGP with MD5 signatures. Using MD5 sigs will defeat this attack.

This is a well known issue with well known solutions. If the infrastructure is at risk it is because ISPs haven't been doing their job and following best practices.

-weld

That took long enough

This came up on the linux-kernel mailing list two years ago during a thread on TCP MD5 stuff to avoid this very problem. Why is it just now making a splash?

This post [google.com] from Alan Cox covers it nicely. Also see the rest of the thread.

This is an attack on sequence number, not ports/IP

Guessing a port and IP is fairly easy. Guessing the sequence number is not. This is why making sure that TCP initial sequence numbers are random [bindview.com] is important.

This is old news.

For the insanely paranoid use a hardware entropy generator (TRNG) [peertech.org] for choosing ISN's.

There are all sorts of attacks against network protocols when poor random number sources are used. This is but one example...

Juniper NetScreen Security Advisory 58784

Title: Juniper NetScreen Advisory 58784
Date: 21 April 2004
Version: 1

Impact:
A design flaw in the RFC specification of TCP may allow a blind attacker to successfully close a TCP connection.

Affected Products:
Juniper NetScreen Firewalls (all versions)

NISCC Reference: Vul/236929
http://www.uniras.gov.uk/vuls/2004/236929/tcp.htm

Max Risk: Medium

Summary:
A blind attacker with limited knowledge of a TCP connection may be able to successfully brute force the TCP Sequence number space and thereby cause a connection endpoint or firewall stateful filter to process a spoofed RST packet and close the connection.

Details:
The TCP Sequence number is one of the mechanisms that TCP uses to prevent a third party from inserting forged packets into the data-stream between two other hosts. While such an attack has always been known to be theoretically possible against TCP, it is was believed that the range of over four billion possible TCP Sequence numbers was large enough to prevent a successful attack. However, recently such an attack has been proven to be feasible in certain situations.

Specifically, if two hosts are known to talk to each other on a regular basis and/or for long periods of time over known port(s) then it may be possible for an attacker to brute force the TCP Sequence number space and successfully inject a forged packet into the connection and possibly disrupt communications. Certain connections, such as BGP4, which are long lived between two devices are especially vulnerable.

While all TCP connections are potentially vulnerable, NetScreen believes that NetScreen firewalls running BGP4 or with TCP Syn-Check enabled are likely to be vulnerable in practice. Other protocols such as SSH, HTTP and SMTP which usually have shorter connection times are less vulnerable.

Recommended Actions:
NetScreen firewall customers should do one or more of the following:
1) Configure Anti-Spoof protection as appropriate.
2) Use secure protocols such as ssh, HTTPS, BGP4 w/ MD5 Authentication
and IPSec which are more resistant to attack.
3) Limit management to dedicated and/or internal interfaces
4) Upgrade to ScreenOS 5.0r6 which enhances the stateful firewall
functionality to protect devices on the network.

Patch Availability:
ScreenOS version 5.0r6 is currently available for Juniper NetScreen firewalls.

How to get ScreenOS:

Customers with a valid product warranty or a support contract may download the software from the Juniper NetScreen CSO web portal:
http://www.juniper.net/support/

For all other customers, including those with expired support contracts, please call your regional Juniper NetScreen TAC center at one of the numbers listed in:
http://www.juniper.net/support/nscn_support/t ao/co ntact.html

Select option 2 from the telephone menu and be sure to select the correct product from the phone tree. Once connected with an engineer state that you are calling in regards to a Security Advisory and provide the title of this notice as evidence of your entitlement to the specified release.

As with any new software installation, NetScreen customers planning to upgrade to any version of ScreenOS should carefully read the release notes and other relevant documentation before beginning any upgrade.

NYT has an article too

Link here [nytimes.com] (soul reaving required).

In the article Watson is quoted as saying that any hacker can figure out this exploit in five minutes from the vaguest explanation. If that's true, why reveal the exploit until someone has figured out how to completely patch it? Or has the fix already been figured out? I couldn't tell from the article.

Peeve

From the advisory:

[...] In the absence of vendor patching of the TCP implementation, the following are general mitigating steps: Implement IP Security (IPSEC) which will encrypt traffic at the network layer, so TCP information will not be visible; [...]

We told you not to deploy NAT because (among other reasons) it would break IPsec authenticated header (AH) mode. You did it anyway and told us we were pedantic academic pinheads.

You deserve what you get.

--

Cisco's advisory, workaround & update informat

Here [cisco.com]

What about fragmentation?

From my daily reading it looked like the recent mailing regarding the Rose attack [securityfocus.com] Which could affect nearly as many systems as this TCP vulnerability. With two vulnerabilities of this extent, and their relative quiet reception by the security community I don't think there will be much of a push to fix this problem, while the k1dd13s go to work acquiring proof of concepts.

The Exploit

The following pseudo command will send an exploit packet:

linverse@KOS-MOS:~/blackhat/hping2-linv erse$ hping2 [victim.ipaddy] --spoof
[server.ipaddy] --rst --baseport [server.port] --destport [rand()%65536] --setseq
[rand()%((2^32)-1)] --sign "TCP RST Attack"

Each packet sent in this manor has a 1/2^32 chance of succeeding. You need to guess two (effectively) 16 bit numbers: the victim's unknown open port, and the tcp sequence number (within ~16 of its 32 bits, depending on the windowsize) This requires something on the order of 4 billion packets to have a reasonable expectation of closing the connection. There is really no way of knowing if or when a given connection is closed, unless you're taking down routers with it, or some othe observable scenario.

I modified hping2 to spam a victim with these packets. Attempting to reset a local connection with a remote machine takes approximately 20 hours to send the requisite ~4 billion packets. Were these packets to actually travel over a network, it should slow things down significantly.

If one had an army of zombies, then obviously the 20 hour figure can become a 20 minute, or even 20 second figure. But flooding a computer with 4 billion packets in 20 seconds will likely be at least as destructive as the actual payload, except perhaps in the case of major routers communicating over BGP.

Interestingly enough, however, one can still cause massive damage without flooding a single router. Instead, have each of your thousands of zombies try to take down arbitrary routers (perhaps each one sends packets randomly to each known major router). This algorithm allows one to make huge amounts of guesses without saturating the connection to any given router.

If my calculations are correct, then 10000 zombies armed with my exploit and a list of major routers could take them down at a rate of one per 7.2 seconds. At this point, we're talking about serious problems.

Has anybody else done any field testing / analysis?

OpenBSD is _not_ vulnerable

No, everyone is not vulnerable to the recently published vulnerability in the TCP protocol that allows to shut down BGP routers. Because Cisco hardware is, stupid writers yell that the whole internet is vulnerable. Come on, Cisco is not the internet.

As stated by Theo de Raadt and Henning Brauer, OpenBSD is not vulnerable because (quoting Henning) :

Even without TCP MD5, bgpd on OpenBSD is not affected, because: - we use random emphereal ports - we do not use insanely hughe window sizes as Cisco does - we require the RST sequence number to be right on the edge of the window

(quoting Theo) :

That is right. If you have a Cisco, you can tear down BGP sessions by
spoofing:

64K of
SYN's or RST's sent to #.#.#.#:179 -> #.#.#.#:{1024,+512,+512,...}

The SYN and RST methods are different, but the end effect is that
a tiny little burst of packets will cause a flap.

OpenBSD (and I am sure other systems too) have for some time contained
partial countermeasures against these things.

OpenBSD has one other thing. The target port numbers have been random
for quite some time. Instead of the Unix/Windows way of
1024,1025,1026,... adding 1 to the port number each time a new local
socket is established... we have been doing random for quite some
time. That means a random selection between 1024 and 49151. This
makes both these attacks 48,000 times harder; unless you already know
the remote port number in question, you must now send 48,000 more
packets to effect a change.

We've made a few post-3.5 changes of our own, since we are
uncomfortable with the ACK-storm potention of the solutions being
proposed by the UK and Cisco people; in-the window SYN or RST's cause
ACK replies which are rate limited.

It will have the most impact on vendors who do BGP over poor TCP
stacks. In particular, Cisco.

Cisco has not been teaching engineers to block SYN's coming in; they
have only been teaching them to block SYN-ACK's from going out in
return. And... well, you'll see.

Re:It's Al Gore's fault...

Vinton Cerf: Good evening, or whatever time zone you are in, hi!! While we're waiting for questions, I'd like to clear up one little item - about the Vice President ... He really does deserve some credit for his early recognition of the importance of the Internet and the technology that makes it work. He was certainly among the first if not the first in Congress to realize how powerful the information revolution would be and both as Senator and Vice President he has been enormously helpful in supporting legislation and programs to help further develop the Internet - for example the Next Generation Internet program. I get to see a lot of this stuff because I am a member of the President's Information Technology Advisory Committee and we regularly review the R&D programs of the US Government and many have relevance to the evolving Internet.

Real quote.