Tracking the Congressional Attention Span

Turismo writes "Ars Technica covers a new research project that uses computers to look at 70 million words from the Congressional Record. The project's goal was to track what our representatives were talking about at any given time, and researchers were able to do it without human training or intervention. From the article: '...researchers found, for instance, that "judicial nominations" have consumed steadily more Congressional attention between 1997 and 2004. In fact, the topic produced the most number of words published in a single "day" of the Congressional Record: 230,000 on November 12, 2003.' It looks like automated topic analysis has truly arrived."

Re:Pro-Gress vs Con-Gress

[AltGrendel]

I would think that both "follow the money" and this type of record analysis would be the best thing. Think of it as the money as the input and the speeches as the output.

Correlate the two and you'd really have something.

No, not that. What I meant was who outside of Congress is trying to push buttons, and who inside Congress is helping them. Also, you'd be able to watch for what you may consider important topics to see how they are dealt with.

Process Process Process

[LaughingCoder]

That disease that has so infected business - talking about process (how) rather than products (what) - is readily apparent in Congress as well. I added up the percentages of the "Procedural [HouseKeeping]" categories (egads, there were 6 different line items - not sure what the distinctions were), and it was 50%!!! So, for half the time Congress is talking about *how* they are going to talk about things. Ugggh. I suppose, as one who believes that the less the government does, the better, I should be happy. But oh, the global warming from all that hot air!

Garbage in, Garbage Out.

[GeneralCern]

The record isn't actually way they talked about...

...it's what they want you to THINK they talked about.

http://www.townhall.com/columnists/JohnStossel/200 6/05/31/myths_and_lies_on_the_record [townhall.com]

Congress Zeitgeist

[Speare]

So in web2.0 terms, this is Google Zeitgeist meets the Statistically Improbable Phrase analysis like you see on Amazon. Find pairs or sets of words which are out of the statistical norm for English, then start to track their rise and fall among the "marketplace of ideas" in Congress. Also, on the c|net news site, they have two graph views to visualize connections between similar-topic stories or often-viewed "hot" stories.

It would be interesting to see how many phrases are just a matter of the odd language that Congress uses. There's a stock metaphorical phrase for just about anything, and there are also a lot of phrases that are steeped in tradition which often get misunderstood by layfolk.

Re:Pro-Gress vs Con-Gress

[Whiney Mac Fanboy]

If Pro is the opposite of Con.... what'd Congress mean?

Just 'cause I was mildly interested (I've heard that wordplay before), I read the dictionary's entries for progress [reference.com], congress [reference.com] and con [reference.com].

And it appears con (when used in pros/cons of a decision) is different to con/com (the prefix).

The gress suffix is from indo-european ghredh (to go) and pro & con have root meanings of advance/forward & to meet respectively.

Progress = Forward Go.
Congress = Meet Go.

See also: Clustering senators by votes & topic

[Anonymous Coward]

You might also be interested in another topic model that not only automatically discovers topics, but also automatically discovers topic-specific groupings of the senators by their votes. http://www.cs.umass.edu/~mccallum/papers/grouptopi c_linkkdd05.pdf [umass.edu] "Group and Topic Discovery from Relations and Text."

It uses not only word data (from the text of 16 years worth of bills voted on in the U.S. Senate), but also the senator's voting records.

For example, you can see that Sen. Chafee (R-RI) (who was mentioned on this morning's NPR as a "liberal Republican") actually does fall into a cluster of Democrats, not fellow Republicans. When automatically discovering topics using word data alone (without the votes, as does the wustl.edu paper above) the topics on this Senate data are reasonably coherent, but the topics created by this "Group-Topic" new model are even more interesting because their discovery is driven by the need to predict the votes as well as the words. For example, "Social Security" doesn't appear in the old model, but pops out clearly in the new model because it has such a distinct voting pattern.

Some of the other results are also pretty interesting---on Education and Domestic policy the Republicans are more split than the Democrats (forming 3 groups, to the Democrats 1 group). On other topics, the split is the other way around.

Using the same technique, there is also an analysis of 60 years worth of voting records from the U.N. On the topic of "human rights", Nicaragua, Papua, Rwanda, Swaziland and Fiji all get clustered together---ouch!

Congressional Record *IS* false

[donutz]

If you want more proof, read this article by John Stossel [realclearpolitics.com], which takes a look at what the "Congressional Record" is really all about. Or like parent says, watch CSPAN.