Comment The poster has misunderstood Zipf's law. (Score 1) 384
> In the 1930s, American social scientist George Kingsley Zipf discovered that if he ranked words in literary texts according to the number of times they appeared, a word's rank was roughly proportional to the inverse of the its frequency squared.
No, no! Looks like the poster hasn't understood Zipf's law to begin with. The law states that the number of occurences of a word (and not its rank) is roughly proportional to the inverse of its frequency squared.
A rank of r means that the word is the r-th most frequent word. Zipf's law then leads to the conclusion that for large ranks, the number of occurences of a word is roughly proportional to the inverse of its rank. For example, the 100-th most frequent word occurs 10 times more than the 1000-th most frequent word. This is not at all a trivial observation if you think carefully about the difference between the rank of a word and the number of its occurences.
So to truly understand Zipf's law, first you have to clearly grok the difference between the number of words which occur a given number of times, the number of times that a word occurs, and the rank of a word which is the number of other words that occur lesser times than this word. http://linkage.rockefeller.edu/wli/zipf/ hs links to articles for a clearer understanding of the law.
No, no! Looks like the poster hasn't understood Zipf's law to begin with. The law states that the number of occurences of a word (and not its rank) is roughly proportional to the inverse of its frequency squared.
A rank of r means that the word is the r-th most frequent word. Zipf's law then leads to the conclusion that for large ranks, the number of occurences of a word is roughly proportional to the inverse of its rank. For example, the 100-th most frequent word occurs 10 times more than the 1000-th most frequent word. This is not at all a trivial observation if you think carefully about the difference between the rank of a word and the number of its occurences.
So to truly understand Zipf's law, first you have to clearly grok the difference between the number of words which occur a given number of times, the number of times that a word occurs, and the rank of a word which is the number of other words that occur lesser times than this word. http://linkage.rockefeller.edu/wli/zipf/ hs links to articles for a clearer understanding of the law.
