Metcalfe's Law Refuted 225
pdp0x14 writes "Cnet News reports on a powerful refutation of Metcalfe's Law (that the value of a network goes up with n^2 in the number of members). The academic paper is available at Southwest Missouri State University. Basically, the thesis is that not all the links in a network are equally valuable, so Metcalfe's argument that everyone can connect to everyone (n(n-1)/2 links, roughly n^2) is irrelevant. The authors propose nlog(n) instead, a much smaller increase."
Use it for what (Score:3, Interesting)
Interesting. (Score:2, Interesting)
I don't know, since when has any computer-related "law" really been a law.
Smaller Networks Win Out (Score:5, Interesting)
It's common sense, of course, but worth taking note of.
finding nodes (Score:3, Interesting)
that's what makes google so valuable: the ability to provide a "meta" node-set.
does this debunk "six degrees of separation"? (Score:2, Interesting)
Odlyzko's Arguments are Good (Score:5, Interesting)
Where I think Metcalf's Laws does apply is in an information network where no proprietary secrets exist. For instance, searching for technical documentation or a movie star's biography. In these instances, the value of the network, as measured by the immediacy with which one could obtain useful information by asking a question, is proportional to something on the order of n*n for n nodes.
Consider the network the top 10 search results in Google for all possible queries. Let's pretend for a moment that Google wasn't polluted with Spam. In this case, each node (search result) is providing a substantial amount of value to the network, although no matter how small or targeted the group, Zipf's Law will be observable to a degree.
Or consider if you had personal tele-access to every person on the planet and could ask any one of them a question at any time. Clearly here the value of the network is something on the order of n*n.
Most or all of Odlyzko's examples presupposed economic interests or constraints.
Re:The real Metcalfe law (Score:2, Interesting)
Lesson of usenet--Value? What value? (Score:4, Interesting)
I think the decisive factor is that the fanatical propagators of misinformation must be aware (at some level) that they are fighting against reality--but their response is to shout louder and more frequently, simply repeating their misinformation. Are they hoping that lies repeated enough times will somehow become true? Or they just hope to bury the truth they hate?
Scarcely matters. The result is obvious, and the same phenomenon seems to be overtaking the WWW, too. Doesn't do a lot of good to connect to the network when all the sites are basically put on the same level by the constraints of HTML, but most of them are full of propaganda of various stripes.
Brook's Law (Score:3, Interesting)
I wonder if this means that Brook's formalization of the team-size problem is somewhat overstated as well.
Re:Andrew Odlyzko is godlike (Score:3, Interesting)
I always thought n log(n) sounded a little high neglecting the effects of noise and other costs of large networks. What's really the difference between being able to phone a million people compared to a billion people? I bet the jump from 100 to 1000 people is at least as big for most people.
You reach the point of diminishing returns even on the log scale. And if the value of the network to the average person doesn't even reach log(n), then the value of the network will be less than n log(n).
Re:Lesson of usenet--Value? What value? (Score:2, Interesting)
Perhaps the formula is to simplistic. It needs to take into consideration AOL users, Virus writers, spyware, and bad porn as detriments.....
Maybe
( (n + #linux users) - ( AOLusers + Kv * #viri + Ks * #spywareAps + #aviAmatuerPr0nVideos) ) ^ 2
Re:Andrew Odlyzko is godlike (Score:5, Interesting)
After all, it's the high end of that curve -- e.g. the anybody-to-anybody connection of the 'net -- that brings us things like wikipedia [wikipedia.org] and Linux [kernel.org]. IMO, when you start reaching scales "beyond mortal comprehension" (or at least everyday life) the growth isn't as much being able to connect to more individuals, but being able to have more specialized groupings and network those.
Even if the average person doesn't get very connected into the network, the value can still be quite high. Never forget the "Kevin Bacon" effect [virginia.edu].Re:Smaller Networks Win Out (Score:4, Interesting)
the total value of the larger network increases more.
Assuming a value of N log(N);
Value of 100,000 is 500,000
Value of 100 is 200
The value of 100,100 (the two together) 500,543
Increase in value per node for larger; 0.00543
Increase in value per node for smaller; 3.00543
Total increase across larger network 543
Total increase across smaller network 300
-- Should you believe authority without question?
A network's usefulness reaches a limit (Score:2, Interesting)
Furthermore, as you increase noise on the network (i.e. spam, popup ads wasting your time from what you intend to use the network for, random people bugging you about things unimportant to you, but nevertheless important to them for whatever reason), the network becomes less and less useful and the difficulty in sorting out useful nodes on the network from useless nodes becomes harder. So once this limit is reached, the more nodes you add, the more useless the network becomes.
For example, look at the web in 1990 versus what we have today. Back then, you could do a search on Lycos or Yahoo and most of the time you could find what you were looking for, but nowadays search engines are glorified phone books where the way documents/web sites get to the top of a search list has little to do with the usefulness of the content, but rather how much money they pay to web site portals to have their site ranked above others for any given topic. Furthermore, due to keyword stuffing from porn sites, blogs, and other irrelevant content web crawlers scan for web page indexing, sites like google are becoming less and less useful over time.
In the internet/cell phone/ANYONE CAN ANNOY ANYONE ELSE THEY FRIGGING WANT AT ANY GIVEN TIME culture we now live in, sometimes it is damned near impossible to get any real work done, or more importantly just be able to relax at the end of the day when a bunch of people who are addicted to communicating with others for no good reason feel the need to bother you just because they can.
I think Donald Knuth's solution of just pulling the plug on all communication devices is about the only option some of us have because like most things in science, if you give the average person a little bit of scientific rope, they will surely find a way to hang themselves.
And yah, yah, yah you can say you can just tell people who may be friends, family members, business associates, or whatever not to bug you for certain activities during certain hours (i.e. don't bug me at work about personal stuff and at home don't bug me about work, just because you know how to reach me on my cell phone or computer 24/7), but that is easier said than done without pissing a lot of people off you don't want to piss off for other reasons.
And when it comes to a network, the more nodes you add the more potential there is for people to waste your time and therefore less gets done and the network becomes useless.
Metcalf's law is good in theory, but in practice people sometimes don't realize how much of their life they waste getting interrupted by people who think that just because cell phone minutes are cheap nowadays, that it means your line is always intended to be open for any random topic of discussion as opposed to the good purpose of leaving it open for emergencies and truly important business.
In the old days, when someone wanted to discuss something with another person they physically had to make the effort to get off their lazy arse and meet that person somewhere. Nowadays, you just have to hit a number on speed dial or double click on someone in your buddy list to be able to "reach out and annoy someone".
If time is money, then the abuses many common folk make with the internet is costing the world trillions of dollars in lost productivity.
There are Diminishing Marginal Returns (Score:1, Interesting)
Re:Smaller Networks Win Out (Score:3, Interesting)
Structural Holes (Score:1, Interesting)
Re:"Refuted"? (Score:2, Interesting)
The usefulness goes DOWN [tt] with the number of possible connections, when more and more of those connections are low-quality.
I would argue that the utility does not go down with the number of possible connections. It is more likely that the extremely low learning curve for user-friendly aol-type connections has removed knowledge barriers that perhaps aught to have been left in place.
You get the dregs, like AO-Hell users who click on spam, windows lusers whose boxes breed viruses, etc
Wasn't AO-Hell an old AOL chat hacking tool from back in the dial-up days?
We may have passed the optimal size for the current breed of network users. Hopefully, the next generation won't be as clueless ..
It's all about the wizards first rule. The next generation will be clueless in new and exciting ways.
Re:Andrew Odlyzko is godlike (Score:4, Interesting)
The first is that a single user may be a user of multiple networks; obviously, little value is created on account of a user of both networks when they merge, since the user could already communicate with all of the users. This effect can mean that two networks combined can simply cause the two network owners to share the value each of them had before (for example, the advent of VoIP means that people no longer need POTS lines, so the amount of money that can be extracted from consumers drops).
The second is that the communication value of a network may not be the reason for having it. For example, in the US, cell phones often have SMS, but it's a fragmented network. The networks don't merge, however, because SMS isn't widely used in comparison to voice service. The companies derive the greatest benefit from people paying a bit extra to get a SMS-capable phone, but using voice instead. Merging the SMS networks would increase their value greatly, but still wouldn't compare to the value of the existing universal network.
Between these two effects, the dot-com bust is predictable, especially when you realize that it happened among a userbase who could already call each other on the phone. Even if the value of a global network is huge, the ability of companies to extract that value in revenue is very limited.
The effect of spam can be seen as changing the nature of the network to a broadcast network, generally acknowledged to be worth O(n). The change is value from adding users is negated if they communicate with the network as a whole rather than individually with each (or some) of the members.
The argument based on Zipf's Law makes sense as a general rule, because an individual gets 1/k value from the kth most valuable user. On the other hand, this misses two points.
The links which would be most valuable may not be in the network yet. Adding user k+1 doesn't give only value 1/(k+1) to each user, because the new user is probably not less valuable than all of the existing users to each of the existing users. If the network already included everyone, Zipf's law would apply directly. But the total value to a user of n users out of a world of m users is (n log m)/m. If we assume that there is a constant number of people in the world and that the users of a network are randomly chosen from that pool, then the total value to any given user of that user's links is O(n), and the value of the network is O(n^2). We just have to remember that we hit a wall at the point where practically everyone is connected, and growing the population is only worth O(m log m).
The basic insight is that, if your friends are split across two SMS networks, there is a large value to you in them joining. If your friends picked SMS networks at random (or based on some unrelated consideration), this is likely to happen.
On the other hand, a network constructed by value (that is, if new users are chosen to be of high value to the existing users) is going to extract the value of a larger network at a smaller size and then grow at the O(n log n) rate in a merger. This is why AOL was of high value by itself (lots of friends and family) and the internet was of high value by itself (lots of people who collaborated), but the connection did not add all that much to either (with the primary exception of AOL users going off to college). Opposed to this is the fact that a user may get a different set of high-value links by having new needs; picking up a new hobby will radically improve the values of a set of previously low-valued links, and, to a certain extent, reshuffle the selection of users on the network.
So my estimation is that there are several flaws with the essentially correct O(n^2) idea: separate networks get extra total value out of duplication, at the expense of the users; all networks, even with different properties, compete with each other; it's limited and
Re:Smaller Networks Win Out (Score:3, Interesting)
Well, it is mostly a question of who sits on the backbone, otherwise networks would merge to provide competitiveness. Imagine the small network peering with another 100 person network. Now that 200 person network enters peering with some 200 person network. Then 400, 800, 1600... soon they are a big network.
The situation describes the case pretty well when there's noone to merge with though. You have to go through the big network to talk to anyone else. That is why e.g. big apartment complexes often have much better networks than your house. The complex can "merge" and have market power, you are pretty much SOL with whatever the telco will give you.
Kjella