2 years ago I got an Athlon FX-2 6000+, a motherboard that supports 16 gigs of ram, a GeForce 8600 GTS (512MB), 4 gigs of ram, and a 256 GB SATAII hard drive for about $5-600 total and just threw it in an existing case with other peripherals. At today's prices I can upgrade the ram to 16 gigs and the vid card to something more recent pretty dang cheap.
The CPU is a bit dated but not much of what I do is CPU bound anyway. I run a lot of concurrent VMs that mostly just sit around and play a game here or there.
Just gotta shop around a bit when you buy. When I bought that stuff it was hardly high end - it all cost less than $600.