I did as a lumberjack. This required about 10,000+ dietary calories per day though... Chopping down trees, moving wood, and burning doesn't require anywhere near the same order of mental strength as programming. At first I could only do the work for 2-3 hours a day, but my body became stronger and by the end of the summer I was able to do 15 hour days, and I liked working long and hard. My dad received a grant from the forest service to put in fire breaks the summer before I went to college. This was an effort to encourage private landowners to create firebreaks so that the Gila National forest could be left to burn during a fire, without as much pressure on the Forest Service to stop it because private property was threatened. His land borders the Gila on three sides.
For programming (and other mental jobs), I 100% agree with you based on my experience. I try and do about 20 hours a week of actual coding, 10-20 hours of business development and sales, and then about 10-20 hours per week of construction. That helps me keep my balance. Thankfully, I am self employed as a software developer and have rental property. The most I was ever actually able to code at a "job" was about 30 hours a week - the other 10-15 were usually spent in meetings, design, talking with customers and helping other people work through their problems (also meetings, but super informal.)
I think people are different. With the right amount of physical training, my body could handle 12 hour days at lumberjacking at 19 years old. Maybe I could do that again at 30, but I'm not sure. Maybe someone else would be very happy programming at their job for 60 hours a week. One of my buddies programs 40 for his job, and then does at least another 20 on his personal projects. He might take a 60 hour per week deal if his work offered him 1.5-2.0x his current salary, and be totally happy. I'm not sure, people are different.