Cringely on P2P vs Streaming Data Centers

Anonymous Coward writes "Robert X Cringely is postulating today that as bandwidth applications grow, the data centers will never be ready to serve 30 million concurrent streams of data. Akamai, with its tens of thousands of servers spread in an intelligent topology, still can't serve more than 150,000 concurrent streams, which is never going to impress the TV network exec used to audiences in the millions. Cringely choruses that secure P2P is the solution to delivering not only high quality video but also to audiences that scale in the millions. BitTorrent seems to have worn out it's welcome with the MPAA recently, so maybe the future holds P2P networks owned and managed by Hollywood?"

Change the paradigm

Sure, currently 150,000 copies of data puts a large strain on the servers... what about one copy broadcast via multicast, much akin to airwaves?

The problem already has a solution

It's called multicasting. If anyone actually supported it, life would be great. We use it internally for live video and it's great.

BT and Hollywood

Hollywood hasn't soured on BitTorrent itself, only a bunch of w4r3z tracking sites.

What happened to all the...

... multicast and proxy technology that we have spent the last 10+ years working on to solve this problem?

Re:What happened to the MBONE?

Whatever happened to the MBONE? I see that a book on the subject is now posted to the web and freely copyable because it's gone out of print. The MBONE FAQ dates from 1993. That's like (/me whips out his HP-41C calculator) 13 years old. Apparently the IETF has a group for MBONE Deployment, but it hasn't been updated since last September, and even then it was a year late for its final milestone.

Multicast works....it's political

Multicast has been deployed on Internet2 for some time now. I've watched 720p streams multicasted from Europe with no problem.

The problem with deploying it on the commercial Internet is political. Backbone commercial Internet providers have had multicast on for a LONG time. ISP's that give you your home broadband connection which are mostly cable TV operators and companies like verizon don't want to provide a cost effective way for content providers on the net to deliver video. They would rather charge you for their "middleman" service. It's not like they don't know how to enable it, all they need to do is enable it on their switches and routers.

Most cable operators use multicast already to stream the channels through their set top boxes.

In Britain The BBC is working with ISP's to multicast to broadband connections. That would REALLY be nice if something similar happened here (In the U.S.)

http://www.bbc.co.uk/multicast/ [bbc.co.uk]

Figures

"Akamai, with its tens of thousands of servers spread in an intelligent topology, still can't serve more than 150,000 concurrent streams"

Assuming Akamai has only 10,000 servers, that's 15 streams per server. C'mon now, we're not that stupid.

Akamai embellishment

The Akamai figures are the embellishment of the submitter... Cringely doesn't mention Akamai anywhere in the article.

The future is peer.

Content creators and content consumers are becoming one and the same. You can see this every day on sites like Jamendo [jamendo.com] and Flickr [flickr.com].

CoDiO P2P Streaming

I recently attended a talk that was part of a PhD student's oral defense. He detailed a really nice streaming video system that is congestion optimized instead of rate optimized called CoDiO [stanford.edu]. I asked him how long he thinks it would take to market this, but I think he said that they're still working out the kinks in the practical application. So yeah, the technology is definitely there to stream video over P2P, but I don't know about DRM. Then again... regular terrestrial TV broadcasts aren't hampered with copy protection as far as I know, so maybe the DRM is unnecessary for broadcasts with commercials?

Predictions

Great. Another prediction on what technology will or will not be able to do in the near future.

We all know how accurate these are.

Also: There is a difference between serving the exact same fucking content, at the same time to 1 million people and generating custom pages on-demand for 1 million people.

Simple solution to this problem

I actually proposed a very simple solution to this problem recently in my blog post http://auzy.blogspot.com/2006/02/auzys-technology- predictionswishlist.html [blogspot.com]

If they add support for multiple href's in a a href tag, such as <a href="http://mirror1/..." a href="http://mirror2/...">Link</a> then it would open up the possibilities of doing P2P type webserving, as the users could run a program to announce the address of their computer to the webserver after they got the download, and the webserver could give each user a long list of a href's which the client could download simultaneously off a few for larger files. It would also make having a list of mirrors for a file easier to manage, instead of posting 30 links on a page.

It really is a simple problem to solve even now effortlessly, and its only a matter of time until browsers start adding support for such a mechanism.

Someone clearly didn't get the idea of P2P

so maybe the future holds P2P networks owned and managed by Hollywood?

No way. I'm gald to support the legal P2P community; I frequently leave Knoppix or other Linux distros running for weeks on end on a spare system here and make available my modest upstream bandwidth. And I can understand that some may want to use their bandwidth to share material that might anger the MPAA or RIAA (and particularly in the case of the RIAA I don't have very negative feelings about that). But that's a far cry from ever thinking that the RIAA or MPAA could ever get P2P working where others contribute their paid for bandwidth for these thugs to make a profit on. And for those few who do there will be plenty more like me who may go out of our way to poison the streams and keep the scheeme from working.

And before everyone gives feedback that it might work if the criminal organizations give a "discount" in return for leaving the feed up for so long, perhaps the public would indeed be stupid enough to fall for that, after all they buy songs and ringtones at insane prices in formats that lock them into DRM scheems and keep them from moving them to the next device they own. But in reality these groups will not be giving any discounts, they will just inflate the prices further and then tell people how much they "save" if they contribute bandwidth to support these rackets. Yes, maybe some people are stupid enough to fall for this, but it certainly should not be encouraged.

No free lunch

Why exactly would anyone want to donate their bandwidth to movie distributors? What benefit would you get out of it? Restricted viewing rights through DRM doesn't sound like a benefit to me. I don't see how they'd square this circle; it's not a reasonable trade-off.

Accelerating Returns

I wouldn't worry about that.

The computer and computing industry isn't standing still. Processor and signal transmission speeds increase exponentially. There will be quite enough bandwidth and processing power for everybody.

Who are Grid Networks?

Cringely talked about a company called Grid Networks and their killer P2P app that may change TV distribution. They seem to have an interesting idea, but I wanted to look into it further. Owing to the genericness of their name, however, I haven't been able to devise a Google search that finds their website.

Does anybody have any info on Grid Networks, or are they vaporware?

P2P is not "under control"

And thus I don't really think they will switch to this model. Simply put: Their "servers" would not be under their control. If we were to provide them with "servers", we could at least partly control what is shown.

Of course we would not get a say what we distribute. But that's not the point. You cannot rely on a P2P Server to provide real time content. Suddenly it's gone, because I switch the box off. Even if you have a few fallback "servers" on the list it's nothing you can build a reliable service on. And people do get angry if their favorite soap suddenly skips right after the words "I kept silent 'til now, but now I have to say it. I am..."

Not to mention the danger of tampering with the content. Yes, they will encrypt it, yes, they will make it near impossible to inject anything, but there is still the danger that in the middle of a Disney Movie you suddenly get to see ... use your imagination.

P2P sounds great but

they aren't paying for my cable modem, and my cable modem has a maximum upstream speed of about 45 kilobytes per second. That isn't going to help anyone really. Not to mention, I wouldn't be all that keen on maxing out my upstream just so I could watch American Idol.

Also, shouldn't they be paying ME to use MY bandwidth?

who in hollywood?

companies like sony? oh, i trust them with media/content, fuck you very much ... no thanks, i'll pass

Hollywood-run p2p? Unlikely

maybe the future holds P2P networks owned and managed by Hollywood?

That seems unlikely to me... people would have to be willing to trade away their spare bandwidth for... what, exactly? Being able to watch movies/TV on their computer? They can do that now if they want, without having to run any "industry-approved" p2p clients (and all that that implies).

Plenty of P2P CDN's

Chaincast
NetCableTV
Red Swoosh
Kontiki

Just to name a few.

Some of these have been in production for many years. Chaincast is/was the leader in radio streaming (at one time).

There are more advantages with P2P streaming/downloads than meet the eye. You also get better sharing of data in the local network. i.e. you're at Starbucks, you see someone watching somthing you want too - start the download an you get it at full speed from one laptop directly to the next. Also, from an infrastructure pespective, it's automatically fault tolerant.

It's big.

the changing nature of content

The way things are going, for the most part these "execs" won't have to worry about supplying millions of people with the same stream, because there will be too much competition. It has become much easier and cheaper to produce "content". Whereas in the past, for example, you might have had a "choice" of 4 networks on TV, and those few "execs" and shows, all very expensive to produce and broadcast, now you have a choice of hundreds of stations on cable or satellite, and soon, thousands or even millions on the net. Audience "viewership" numbers will fall for most programs, and rise for others, with the others being new and for the most part out of those 'concerned execs' hands or influence or "worry".

  "Consumers" aren't limited as much as they used to be, and those remaining limitations are dropping daily. A show in the near future might be extremely lucky to only have a thousand viewers (whatever, see "podcasts"), something that could be handled easily with todays "normal" streaming tech.

Congress should pass a multicast law

If ISP's were required to enable multicast all the way to the home all these video delivery problems would be MUCH easier.

You want to see cable and DSL operators go nutz with foaming mouths, get your congressman to introduce a bill requiring multicast to be enabled on all routers and switches, and add a provision punishing ISP's who knowingly degrate UDP.

Many people think that multicast is a failure and does not work, fact of the matter is, it's deployed WORLD WIDE on the backbones of both Internet and Internet2. I can tell you from my own experience that it works as advertized (on I2)

Hey Cringley. You missed one method.

Mentioned at Dslreports.com

http://www.dslreports.com/shownews/72293 [dslreports.com]

Revenue Streams

Cringeley doesn't mention Akamai. Where does this 150K max users figure come from? If "tens of thousands" of servers is only 10K servers, then 150K streams is only 15 streams:server.

But even a $2K P4/4.3GHz can serve over 1750 simultaneous 500Kbps video streams (from my own benchmarks), for 875Mbps. Since Gbps fiberoptics cost <$5000:mo, or under $3:stream:mo, 10K servers should serve at least 17 million simultaneous users; 58K servers serve over 100 million simultaneous streams.

Use more efficient servers, like SANs coupled more directly to routers, and you're talking about <$3:stream:mo for maybe 100K servers serving over 1 billion people, for a $100M investment that can be amortized over a few years. Years which can bring maybe $1-100:mo profit on 1-10 billion consumers, or 10-10,000x ROI.

Such a network is much more efficient and economical as P2P, or multicast. But even the raw numbers sound very profitable. That's why Akamai is making so much money, even though their market is still so small.

Protocol choice for massive video streaming

BitTorrent is peer to peer. Having said that, the BitTorrent as it stands, is drastically unsuitable for use in a streaming enviornment. It is designed to transfer files, not stream them in real time; we can start with the requirement that the server has the entire file to generate the .torrent file from (try that on a live video stream, for example), and continue with the lack of an guaranteed arrival order or time. Oh, and that .torrent file - still going to be hard for a few million users to grab at once.

A BitTorrent-like protocol could be used, something that sends the stream meta data as signed packets along with the stream itself, although actually ensuring in-order and on-time delivery is still going to be a massive headache. There are all sorts of interesting and complex trips that could be used, mostly focusing on a BitTorrent-like protocol to allow trivial proxying (so an ISP could buy a few computers, hook them up to their network, subscribe to the most popular streams, and their subscribers would automatically find and use them, as they would have high bandwidth to them).

Someone want to remind me what was wrong with multicast, though?

hits

Just gotta warm up the OC-3072 or get Hitachi to cram more lambdas into a single fiber.

P2P Video Broadcasting already exist

P2P video distribution is already here.

http://www.nft-tv.com/ [nft-tv.com]

They are already up and running.

Fine... Lets do it their way

BitTorrent seems to have worn out it's welcome with the MPAA recently, so maybe the future holds P2P networks owned and managed by Hollywood?"

I'll be happy to join their "P2P" network, buy the content for a reasponable price, and share pieces of files I download to other users that want the same thing. However, their litigious and moneygrubbing attitude makes me NOT want to share any of my bandwidth with them for free. They would have to offer me a monetary incentive to consider using my bandwidth to P2P it. If they want to be the type of association that is convicted of price fixing, and they want to sue everyone under the sun, I have no intention of helping them by sharing my bandwidth. They will get nothing free from me.

Others will take it much further, crack the DRM on the files, and re-share the files on the free networks right after they're released.

Does Apple sue people like this for stripping off the DRM or whatever? If they do, I guess they don't get hardly as much press, because I never hear about it. Even if they do sue, I doubt they send out blanket subpoenas to everyone and their grandmother. Apple has sold a billion legal song downloads because they make it easy, cheap, and fast to get what you want. The iTunes store doesn't treat their customers like criminals and enemies. Even if they don't say it, I think Apple's DRM is a placebo for the record companies -- easily circumvented. I think they understand the real basic truth to sales: Sell something at a reasonable price, treat your customer nicely, and they're more likely to buy the product instead of steal.

Be nice to your customers. Stop this HDCP, CSS, pricefixing, and lawsuit moneygrubbing and maybe things will work better for you.

Riight...

" BitTorrent seems to have worn out it's welcome with the MPAA recently, so maybe the future holds P2P networks owned and managed by Hollywood?"

That's assuming that Hollywood hasn't worn out its welcome with users, which it has in spades. I think the future holds P2P networks owned and managed by users, who will watch content owned and created by users, and BitTorrent is a great distribution method for it.

wow, Cringley discovers the head-end problem...

This problem has been well known forever. It was a key factor in the failure of internet set top boxes in the mid-nineties (when everyone was trying to make one, like Apple or Oracle/Liberate).

and everyone forgets about

multicasting

oh killler mbone apps where art though

I don't get it

Point 1: What is the point of streaming in the first place? The idea is STUPID and NOT what I as a (ab)user want. I want to be able to have the file, copy it to the device I want to view it on, pause when I feel like taking a break, start playing when I want to and so on. I want a +-700 MB avi divx file.

Point 2: BitTorrent allows you to add seeds to the torrent as you feel like. When I read "data centers will never be ready to serve 30 million concurrent streams of data." I ask the simple question: Really? Data centers can just fine serve 300 million concurrent streams of (bittorrent) data. However, I don't have 300 million users to stream to. Get me that and I'll prove my point. No problem.

Point 3: "Streaming" is about control: Controlling that you as a user have no freedom. Freedom is good for you. Being able to download the avi files is good for you. If Akamai can't serve 150,000 concurrent users then they really should ask themselves why they insist on "streaming" in the first place instead of distributing files like they should be doing and thus allowing their users more freedom.

What about...

A hybrid streaming/P2P application where bittorrent attempts to give you the bits in the correct order, but when it fails, a centralized server gives you what you need? Wouldn't be perfect, but it would take off a lot of the load from the server...

Cable Modems perfect for broadcasting?

Remember back in the day when you could sniff your neighbohr's packets because all of the local cable modems shared the same segment? After all, the cable network at the lowest level is just a bunch of houses sharing the same copper line.

Wouldn't this be then an ideal solution to distributing broadcast content? It would be exactly like today's premium channels, where everyone receives the data, but only some can decrypt it.

I know that today, you can't sniff on the segment anymore, but I think that is due to the modem blocking out all other packets except yours, since the physical infrastructure of the cable system has not changed. AFAIK, it is still not point-to-point like the phone network, but I might be wrong.

Cringely changes his column name..

Cringely States The Obvious.

The industry has been bleating about P2P for on-demand for years. It's the perfect solution for cable operators who have networks designed around INTERNAL traffic and pushing data around to subscribers. If the subscribers share the networking and you can have a city block feeding itself..

Gridnetwork.com suffers under the Slashdot Effect!

From the main page at Gridnetworks: "SLASHDOT ALERT!: A recent mention of GridNetworks, Inc. on Slashdot (via Robert Cringely's "Rules of the Road" article Saturday Morning), is affecting GridNetwork's overall performance at the present time. This "Slashdot Effect" should subside in the next eight-twelve hours. This effect has caused a high volume of player downloads and very quick expansion of our demo grid, so you may experience a significant slowdown in video performance during the period of unanticipated grid expansion. If you experience unsatisfactory performance today, we hope you'll visit us again in the coming week."

So much for those unexpected spikes in bandwidth that a company like Gridnetworks should be able to handle.

Oh, and props need to go out to Gridnetworks for one of the more amusing netbusinesspeak verbal cluster-fucks I've seen in quite a long while. From their "solutions" page: The monetization of media via the Internet requires the involvement of many specialties. Original content creators, distributors, aggregators, site designers and integrators, hardware and software vendors, hosting and bandwidth providers, user support specialists...the list goes on. For most, Internet content distribution has remained an expensive and complex proposition -- relatively few companies have married all of these moving parts to create a predictable, efficient 'content monitization engine.' The GridNetworks PowerGrid Platform(TM) was designed to address many key shortcomings of Internet content distribution and provide a means by which everyone in the value chain can easily participate in the success of each venture. PowerGrid is truly an end to end solution that is versatile at every level and every component."

Versatile until mentioned on Slashdot. I see.

In Other News - Downloading Vs. Streaming

Maybe it's old news, but I saw it first two days ago:

Democracy - Internet TV Platform [getdemocracy.com]

Out of curiosity (it's not like I watch much TV anymore, anyway) I'm giving it a try right now. Can't say much about the contents yet (about three hundred something channels right now, some look like video podcasts, some other apparently not working). On the strictly technical side, seems to work.

Should we tell Cringely? ;-)

Like this wasn't written by a PR person...

Isn't it great when a PR person's babble gets slashdotted by some "Anonymous Coward". Slashdot is the new voice of The Man.

editors?

BitTorrent seems to have worn out it's welcome with the MPAA recently,

It seems that even a non-native Finn would make a better editor than the paid natives here.

Multicasting?

Has anyone considered using multicast streams to distribute content?

Time to...

...setup a Tor [eff.org] node!

Shallowness is a virtue of those in a star network

Equally shallow: TV is too boring, so what.

Re:Sell the keys...

Bittorrent is about freedom of information, not scoring somebody cheap bandwidth. The whole idea of Bittorrent is to make freely available content more available without taxing the orginal host to death for bandwidth charges. if I put one cool video up on my website i don't want to get killed for a huge bill because a bunch of people liked my stuff. bittorrent helps relieve that by sharing the load among all the downloaders. They got a "free" download and "share" a little upload in return.

If somebody's trying to sell the next big blockbuster AND scam free bandwidth that's not how it works. Unfortunately, neither the hollywood suits nor the telco suits understand how the community works.. All they see is "$$$" and want their cut both ways. If they had it their way, we'll pay for BT "service" then pay for the DRM's movies while the movie providers "pay" for access to the telco BT network.

Re:Sell the keys...

I'm not going to give away my bandwidth for free for a PAID system... Regular P2P works because giving away bandwidth gets you something of value. Commercial P2P will never work, unless perhaps you get something in return for offering your bandwidth to others.