You might want to reread my post. You're making the same points I did: namely, that the data on the website is poor due to problems with the underlying systems. This is not the fault of the website (though the GCharts mistake is a bit sloppy). However, the same agency is responsible for the site and for collecting the underlying data. It's time for them to correct the more fundamental problems with the system rather than continuing to repackage it.

actually, the contractor that maintains USASpending.gov is REI Systems, based in Herndon, VA. They're not faultless, but the real problems with the site have to do with the underlying data; unfortunately, REI can't do much about that.

I work at the Sunlight Foundation, where we're pretty familiar with the people and data systems powering USASpending.gov. I've seen a lot of comments here saying that the important thing is that the government is publishing something, and that it's understandable that their first pass might not be perfect.

But this isn't their first pass. The underlying data systems -- FAADS and FPDS -- have existed since the 90s, and have been riddled with errors throughout their existence. Instead of fixing the problems, OMB continues to slap new coats of paint on the same lousy data.

It's nice that we've got a new USASpending.gov, and I agree that it would be a mistake to put too much emphasis on a buggy visualization. But the underlying data is terrible, and so far no one is showing the will to fix it. Just look at USASpending's "data quality" tab -- it talks about the completeness of each row. Well, that's great, but it tells you nothing about the thousands upon thousands of missing rows, nor about the rows that massively under- or over-report their dollar amounts.

At Subsidyscope, the project on which I work, we've delved into these problems in more depth. Those who'd like to learn more about the shortcomings of the data systems powering USASpending can find a discussion of the relevant issues here.

I work at the Sunlight Foundation (though not on this project), and I feel I can safely say that we completely agree with you that the government *should* be issuing this data in a more easily usable format.

To be fair, though, it's not always as easy as all that: when you introduce such an infrastructure you need to make sure there are staff resources to handle the data entry, training available to help them do it, and somebody checking the overall data quality. My project's been looking at a lot of grant data, and we've consistently found that the central grant data directory -- a data set called FAADS -- is of lower quality than the reports issued on each program's website in excel, PDF, HTML tables or who knows what else. It doesn't make a lot of sense to people like you and me, but centralized systems really do introduce an added layer of difficulty for the data entry people. Just keeping track of the endless requirements imposed by legislation can be pretty daunting.

...none of which is to say that this shouldn't happen. It should! But it does explain why "publish earmarks" and "publish earmarks in a central location, in a machine-readable format" are two different things, and why the latter is more difficult to successfully ask for. We'll get there, though.

Earmarks make up about 1% of *discretionary* spending. Which is considerably smaller than the overall budget (which includes things like Medicare and Social Security). And if an earmark didn't exist, that doesn't mean the money wouldn't be spent. It just means that the person administering the program under which the earmark falls would be able to allocate that money more freely (presumably to a more efficient use than a senator's pet project).

Earmarks are very easy for journalists to write up as news stories, though, which accounts for them getting so much attention.

Mod parent down. He clearly doesn't understand that the idea of capitalism is to produce new wealth via the use of invested capital. As a sibling poster pointed out, the economy is not zero sum.

I'm hardly a market triumphalist -- personally, I think we ought to be working toward a socialist democracy modeled on Northern European nations like Denmark -- but you need to be at least a *little* informed before you start making radical critiques of the market system.