I've always thought this happened by design. Although, I've never been sure exactly what the design was.
Perhaps it's an attempt to keep a conversation about a particular topic going through multiple headlines and stories. New stories are being written all of the time, usually with just a paragraph or two that are different from the old story.
Perhaps it was an attempt to cut down on the number of stupid posts. After all, who reads the end of a comment thread that contains
99,601 comments. Surely people realize that few people are going to read their comment after so many have been posted.
Perhaps it's an attempt to cut down on flame wars. It's not easy going through 100's of pages trying to find out if some moron picked up on your finely honed flame-bait.
Perhaps it's a database design issue, and they are simply trying to save disk space.