In response to the original - simple answer, wrong question. The manager should stay if:
1. The manager can provide some sort of recognizable value to the effort (getting coffee, running interference, coding, whatever).
2. The manager is directly responsible for the late hours being incurred.
3. The manager can provide some sort of morale boost to the employees by physically being present.
On the flip side, should you remain in the office when the manager has to stay late? Perhaps he's staying late because your team didn't finish the code you promised at a particular time. What then? Why isn't the CEO staying late when you have to work overtime?
In response to parent:
Indeed, it may be time for the poster to stop looking at the individual "I'm putting in OT, why isn't my manager here" and start looking inwards, and at the macro level of the organisation. Recently, in an attempt to understand my own organization's psychotic management style (note here I'm not absolving myself of any wrongdoing), I've been reading Edward Yourdon's "Death March". Snippet:
"Companies both large and small are filled with politics and are staffed by managers and technical developers who suffer from hysterical optimism as well as the usual gamut of emotions such as fear, insecurity, arrogance, and naivete. And the combination of re-engineering, downsizing, outsourcing, and global competition - together with the opportunities provided by new technologies ... and the internet - suggest to me that death march projects are likely to be a common occurrence for years to come."
The rest of Yourdon's book is spent lending an understanding as to why these things happen from many standpoints, with the hope that you can empower yourself to make a *rational* decision about your involvement in the project, and even the company as a whole. At the end of the day, it's your time, and your life. But at least come to the conclusion that you're 50% of the problem - management isn't solely responsible for your unhappiness. With the frenetic pace of companies today, people have very little time to learn, develop, and get things right. By the time something settles into resembling a normal project, everything changes and the process of education, experience, mistakes, etc. starts all over again. The players, environment - everything changes every single day.
I'm not implying that everyone should throw up their hands. But if you're not actively attempting to change the environment around you through your own education and experience, then I submit that you're just part of the problem.