Researchers Suggest P2P As Solution To Video Domination of The Internet

JPawlak writes "NewScientistTech reports that big businesses may be realizing the benefits of P2P technologies. Blizzard uses it to distribute patches for World of Warcraft, and now researchers at Microsoft are indicating internet users may have to use it to help distribute online video clips. The growing cost associated with delivering such content may be becoming prohibitive for some companies. 'The team also suggest a way to prevent Internet Service Providers' costs jumping when their users start uploading much more data. The trick is to allow sharing only between people with the same provider, when data transactions are free. That restriction would cut the pool of sharers into smaller groups, meaning MSN's servers would have to do more to fill any gaps in the service. But costs could still fall by more than half, simulations showed.'"

haha oh wow

ISPs are gonna love this, since they're big fans of P2P as it is (Bittorrent and friends).

Re:haha oh wow

Actually they will.

Especially when someone points to the idiots from Redmondia (and other places) that they should stop reinventing multicast again and again. The technology to do what is needed is there, the ability of ISPs to control it so that it is not detrimental to other users is also there. It has been there since the dawn on the Internet. And it is Multicast. From the viewpoint of network design and network operation theory, P2P is nothing, but an extremely lame sorry and sad excuse for Multicast emulation.

Implementing it is solely a matter of minor network tidy-up for most ISPs along with some software updates for the CE devices (where not supplied by the ISP).

By the way, the same methods which are used to control multicast are also valid for P2P services. TTL adjustment down to under 8 will usually cut down the traffic to be solely within an ISP while cutting it down to under 4 will cut it down so it stays within the same RAS device (2 for non-NAT setups). It is also trivial to deliver a correct setting on a per-ISP basis and to autodetect the necessary setting adjustment.

There is no rocket science here and no research to be done. All the tech is already out there. The problem is that the suppliers of P2P services and developers of P2P software deliberately do not want to do this. In fact, they are doing everything they can to steal more service than the ISP is willing to allocate to them. As a result the ISPs have no other choice but to love this and use a big stick to provide the luving to the customer.

Re:haha oh wow

I probably did not express my thoughts clear enough. Let's give it another go.

There are two portions to a P2P network - discovery and data fetch. Discovery determines where do you get your data from and fetch is the actual data flow. An ISP can confine a P2P service to its own network by either limiting discovery or by limiting the actual fetches.

The discovery is where the P2P networks lamely emulate a multicast application. They try to determine if a piece of data A is present in any of the surrounding nodes B,C,D,E,F. In order to do that they in the trivial case transmit to each node. In the more modern networks they transmit to hypernodes and get info from there. In either case they try to emulate a multicast network via a tunnel mesh (just the way people try to emulate Multicast on ATM LANE).

Compared to that a discovery mechanism based on multicast with a unicast reply can give you the information on where exactly is the piece which you are interested with one request. There is usually no need for hypernodes either. It just works. Magically. Further to this, you can set your discovery scope to find nodes which are 1,2,3...n hops away by tweaking TTL. Further to this, it is a true P2P network - totally serverless. If you throw in PKI authentication you can also make it as secure as you wish.

ok but..

Sharing among people on the same network is only going to be effective for popular data. Not to mention I have a feeling Comcast would still send tell you you are using too much bandwidth even if it is all coming from within their network.

P2P != BitTorrent

Saying BitTorrent (and similar protocols, if such exist) is P2P is like saying the web is the internet.

It makes sense

Really some of us have been saying it for a long time. Some of the load can be taken off the internet especially bulk files such as video and bt by sharing them first within the network and then outside. I think that's it's a fine idea if people are willing to do it, that way you only have to seed some of the file to people on similar networks. The only place I see this falling short really is with very specific files, for instance I doubt that me or any of my neighbors are going to be watching the same clip on youtube at the exact same moment. However when it comes to system updates there's a good chance of that. For things such as digital cable it would be ideal for people with DVR. For some of the more popular programs you can seed some of the file to a bunch of different boxes and then have them share amongst eachother for the rest of the file. Anyway I haven't said anything that most people here don't already know or have thought about some way or other so I'm going to go back into my corner and keep playing with my rubberbands.

Hmmm...I don't think so

I have no problems at all with not for profit entities using some of my bandwidth to distribute their files.

I have serious problems with a for profit entity like Microsoft or Redhat doing the same.

The first one I call "charity" or "support". The second one I call "leaching", and its not far from "stealing".

If you're a for profit company and you can't afford bandwidth, then you need to find a new line of work. Don't expect your customers to give you freebies unless you're giving them something *good* in return, and something you're not also giving to those who don't share bandwidth.

Free upstream? That's rich....

Seriously, do these "researchers" even HAVE cable internet? Upstream is only user segregated to the head-end for bare copper technologies like DSL. Cable broadband is built on a tree network. Sure, you can build more nodes into the infrastructure to free up a bit more upstream within a single neighborhood, but eventually that upstream has to be combined with the upstream from all the other nodes. Eventually you just can't squeeze any more data into the upstream band and everything stalls. This is one of the reasons why you don't see SDSL equivalents in cable systems. They always give you more downstream than upstream because downstream is cheap for them. Massive upstream requires a lot more infrastructure investment, which heavily cuts into their profit margins. And yes, the data has to go all the way back to the head-end before it can be resent to an IP in the same cable system, even if they are on the same node.

P2P just isn't useful for cable systems. They're better served by caching technologies like transparent proxy servers.

That's sooo 1980..

Researcher rediscovers USENET.

What were the other solutions?

AFAIK p2p is the current solution to efficient video distribution. I think they are really trying to accomplish something else here, which is why the current solution won't do.

For instance, if you want to distribute that World of Warcraft patch, then make a torrent and post it to a tracker, done. If you're really paranoid then host it on your own tracker. No, because what they really want is to have an service running on your machine 24/7, so they can... I don't know, but whatever is I'm pretty sure I won't like it.

I've seen this happen here in italy

A friend of mine had an account with a provider called Fastweb, where he had a really fast connection but payed for traffic outside the fastweb network (which went through a nat i think, he had a local 10. something IP address).

He used file sharing software inside the network, and got very fast downloads (for content which is popular enough in italy).

Of course this is a rather rudimentary implementation but certainly one might be willing to configure his P2P file transfer client to only download from a certain network, if their provider does not offer real unlimited transfers at a reasonable rate. Of course it's only useful if the provider does not count local transfers to your bandwidth limit.

The idea is great, but...

...how do you implement it? Browsers currently have absolutely no support for implementing anything like this. I'm not sure whether it can be done in Flash. Java is so heavyweight that it would probably scare off most people. ActiveX is a no-go. You can't make people install client software either - 99% will never bother to do that. Unless you can make it work out-of-the-box on browsers, it'll not become popular.

And how do you implement P2P streaming? All P2P protocols until now allow peers to send file pieces in non-streamable order.

Re:The idea is great, but...

Browsers currently have absolutely no support for implementing anything like this.

Except for Opera, which is about the only program you actually need to interface with the entire web.

About the only thing it's not useful for is SSH and FTP.

Data within an ISPs network is not always free.

The article makes an assumption that data flow within an ISPs network is free. That is not always the case. Take for example an ADSL connection. The ADSL infrastructure (metallic path, DSLAM, etc.) is often (especially in the case of non-unbundled local loops) provided by a different company from the ISP. The ISP pays this provider per byte of data that flows over the connection to and from the end user.

Support Microsoft!

Download a P2P client and learn how to use it *today*. Help Apple! Share all of your files; learn how to become a seed. Lend the RIAA a hand--do their R&D for a new distribution model.

There is a term in Low German for the feeling I have right now--SchadenGoFuchyourselves.

Wow!

The trick is to allow sharing only between people with the same provider, when data transactions are free.

Sounds like multicasting . . . good things the ISPs have implemented this also . . . oh wait.

Multicast for realtime data

Other thing that will save loads of bandwidth and improve end user quality at the same time: multicast. For instance, the modern broadcast media companies that do TV / radio / concerts, could well set up streams that are relayed only once as to as many hops as there are subscribers, and copied at the final router to each subscriber.

Prior art

http://slashdot.org/~richie2000/journal/138354 [slashdot.org]

obligatory

Nerd: I've developed a program that downloads porn from the interet a million times faster than normal

Marge: Who would need that much porn

Homer: [drools]...oohhh..1 million times faster..

What does he mean by "Free"?

I'm sorry, but even if I send data to a neighbor I get that transfer charged on my 35GB monthly allowance. And that's 35GB for the upload+download total, too.

Companies using P2P to distribute THEIR files (i.e. WoW being a perfect example) are cutting into MY 35GB for the month. And if you try to block them out, you get ridiculously slow downloads, around 0.1KB/sec.

Screw'em all.

"No shit, Sherlock."

Unrestricted P2P across a true mesh topology is developmentally speaking, the ultimate logical destination for the Internet, in my own mind. If I was going to borrow an expression from someone the average Slashbot considers one of their patron deities, I'd even call it a "historical inevitability."

It's probably going to take a very long time. The telcos and big media can be counted on to fight it, kicking and screaming, every last milimeter of the way. Eventually however, if the net is to continue to exist at all, it will happen.

So what is being found out is that.....

Consumers do a lot that is good for business, that business doesn't have to pay for but have been complaining about.
P2P, genuinely fair use copyright (Some recent /. article on how fair use does a lot of good at stimulating teh economy....and even matters regarding the fraud of software patents (IBM the largest software Patent Holder have been releasing their patents to open source and others are beginning to follow.)

There was a time in this country (USA) where the people got together and created the country because it knew better how to do it. Perhaps we are starting to get back to the basics here. If this sort of direction continues what might the world look like tomorrow?

Clearly there is a lot the consumers do that helps business and considering that business is what provides the consumer with products and services and employment, why would it in sum, be any different?

Does business need to hold so tight to controlling property?

Not to sound communistic for the lack of individual incentive about the socialistic economic side of the word (the other side being a totalitarian government which is like business holding tight IP)....

But when you die, you can't take you money with you.... you can only enjoy the benefits of the value exchange it provides in improving the social and personal environment you live in. Your Living conditions!

For the consumer to be allowed to do what they do anyway (mass pirate production is not consumer acts but an illegal business usually for profit)... is to reduce business costs that really don't provide genuine benefit to the Living Conditions.

We are all in this together and the music industry RIAA just doesn't seem to get it as they have been the most in the news business attacking those who feed them.... They have lost my business.... I'll listen to free radio instead...

On a larger scale about people ... there are over six billion of us and a few (some fraction of one percent) that hog up resources that could be better spent making this world much better living conditions, a great deal better living conditions, an unimaginable better level of living conditions.....but they spend it on keeping that from happening.

The war mongers...

So perhaps in short time there will come enough solid evidence of the anti-benefit of such unnecessary overhead.
The evidence sure seems to be getting exposed....

Multicast

Since multicast keeps getting mentioned and as I imagine there are a few who are too lazy to Wikipedia it. Here is the gist of it:

IP Multicast is a technique for many-to-many communication over an IP infrastructure. It scales to a larger receiver population by not requiring prior knowledge of who or how many receivers there are. Multicast utilizes network infrastructure efficiently by requiring the source to send a packet only once, even if it needs to be delivered to a large number of receivers. The nodes in the network take care of replicating the packet to reach multiple receivers only where necessary.

http://en.wikipedia.org/wiki/Multicast [wikipedia.org]

oh, wow, like...

Microsoft invents Democracy Player and Joost, only a few years after they have been invented!

Market Solution for Video Distribution

The best solution for this problem is to provide for a true market solution for both the producers and the users. I've been researching and writing about peercasting for years now, and I do think this is a great solution to the problem.

First of all, if the content is free, then someone wants that content watched. If that original producer is willing to put a price on the cost of a complete download, those who are helping to provide bandwidth for that download should get offered a piece of the action. If it costs Microsoft $0.02 to transfer 100MB, they should offer $0.015 to anyone willing to provide 100MB of re-transfer bandwidth. The peercasting server would only handle peercasting to their top tier redistributors (based on recent history, bandwidth, stability, etc) who would then redistribute to others. Microsoft's costs drop, and the users have a market incentive for provide more bandwidth or stability. Top tier redistros don't necessarily even make more money than the guys at the bottom -- its all a numbers game.

Secondly, the opportunity for P2P to take over antiquated services such as TV, radio or any other broad-distribution medium is getting closer -- but it relies on advertisers, still. Let an end user become a re-distro and THEY can tap into the advertising proceed. Sure, this screws things up for the big monopolistic distribution companies (every TV and radio distributor, cable company, satellite company, etc), but it would quickly bring more stability to a market perverted by copyright rules and DMCA-style regulations.

P2P full bandwidth

Wouldn't it be great if ISP's could work it so that when you are doing P2P (Bittorrent, etc) that if somebody is in the same network local topology there is no cap on bandwidth?

This would work great for non P2P apps as well. Let people on comcast, cox, etc do full bandwidth videoconferencing between customers on the same ISP. For instance it costs comcast probably not much bandwidth wise to let my mother do a 5Mb/s video conference with my system when we are in the same local area (and the same cable ISP)

Combine with comcast limits and no users

If every users is doing 100gb of upload/download then comcast shuts them all off.

now no one has bandwidth.

the solution is to actually raise the bandwidth so that 100gb is trivial (like in korea and japan).

but...

Blizzard uses it to distribute patches for World of Warcraft, and now researchers at Microsoft are indicating internet users may have to use it to help distribute online video clips

But doesn't everyone already use it to distribute video clips (or films.. oops, was I not supposed to say that?)

Re:How is this better than just putting up a .torr

What's missing is a protocol to determine where your ISP's borders are and sticking to peers within those borders. Essentially what you have are P2P supernodes (after a fashion) at each major ISP (Akamai already does this for normal HTTP traffic, after all) and they're the only hosts that actually establish connections outside of the ISP's network.

You could do it with BitTorrent--in a controlled situation--by serving different .torrents to users of different ISPs, and making the supernode a tracker as well as a peer on the wider network.

Realistically, though, working along these lines is something that P2P protocols are going to have to do sooner or later, and kludges like serving different .torrents wouldn't scale well. If your BT (or other P2P) client can figure out a metric for a peer, it can prioritise the traffic appropriately: peers from the same ISP as you get prioritised highest, falling back to peers from other ISPs if they don't have what you need.

None if this is rocket science, conceptually, but figuring out whether host A is from the same ISP as host B is a bit trickier. I guess you could do it by fetching the AS number from WHOIS and then caching the result across the swarms (so WHOIS servers didn't get continually beaten). Alternatively, you could add TXT records to reverse DNS, though it suffers from inflexibility, and I'm not sure ISPs would find it sufficiently in their interests to add them (they'd rather just charge you extortionate fees for excess bandwidth).

The bottom line is that ISPs don't want to support P2P. They want P2P users to go away and use other providers. Persuading them to do anything that will make life easier for P2P users--even if it saves them money--is nigh-on impossible, so solutions must be sought at a protocol level.