I haven't heard anyone mention laptops. Well-built laptops such as ThinkPads are probably top-to-bottom the most reliable hardware out there, especially if you get a low-power, preferably fanless model. Plug in a USB keyboard & mouse and probably an external monitor and go.
I'll defer to others on software and hard drive suggestions, but will add the anecdote that the only laptop hard drive i have had fail was a low-end SSD, which had complete data loss after only a few months. When i dropped my open, running ThinkPad T61 about 5 feet onto a wooden floor, the mechanical drive only lost about a 5GB region, which i have allocated around and had no problems since. Barely even cosmetic damage to the laptop.