I have a linux server for a file server at our house, and I'm building a pretty decent system on the very cheap (not just because it's cheap, but because it's fun too) using a few different technologies..
My plan is to combine a few things into one system.. Security cameras, security sensors, and home automation including zoning. I'll list some of my resources below.. 90% of the battle is knowing what to look for..
zoning: search for 'diyzoning' and follow links from there to the real stuff.. diyzoning is actually being replaced..
automation:
1wire automation and sensors (cheap gadgets of all sorts): www.hobby-boards.com
you can find all sorts of other sites with 1wire and ibutton devices (ibuttons are a particular type of 1wire device, great contact based digital keys)
zigbee: easy to find when searching, it's more expensive, but makes a wireless mesh network which is easier than wires.. I like wires though, so I don't know much about these..
Security camera:
I currently use anything that works with linux and use "motion" for watching the cameras.. it takes videos only of events.. if you've got it tweaked right, it can also make an excellent intrusion sensor, you just have to be careful you don't get false positives.. it's very tweakable though)
Another security camera option that I've considered switching to is zoneminder.. It's got a web gui to make monitoring and configuration easier, but other than that it's very similar.
Another thing to do with the security cameras is to use something like rsync to get copies of events off site in case your server is stolen.. with any luck you'll still have the video of the person stealing your server..
I bought a card at a great price on ebay that takes 16 video inputs, and then I got balun's and cameras from www.dealextreme.com to hook to it.. easy as pie, and the $20 cameras from dealextreme have ir led arrays for pretty decent night vision..
Well, that's about all I can think of right off the top of my head.. Don't let people tell you a system like this is useless unless you pay a ton of $$ for it.. Even if you don't get broken in to, or you do and they still get the stuff and get away with it (happens with monitored systems too, I've seen it often at my work) you'll still have a lot of fun with the tech. I love blasting through the time lapse videos that motion puts together for me..