Living on Mars time is difficult when you're living on Earth and are subject to Earth's day/night cycle.
Exactly this. 24h40m days are exhausting when you're embedded in a 24h day/night cycle and have to mesh with others on that cycle. It's tough being out of phase with your surroundings. On Mars everyone else is on the same cycle you are, and the only contact you have with the 24h civilization has a significant time delay which makes real-time conversation impossible. Give it a few sols and you'll be right on track.
40 extra minutes of sleep per day... Not exhausting at all!