This is a solved problem in the security space. Being a security guard is typically boring and people quickly become inattentive. Depending on the situation, you need to have either counter-incentives, or random challenges, or both.
For an example of a counter-incentive, you could pay someone to catch safety drivers not paying attention. Give them video feeds into the driver's seats and let them flip between them at minimum wage with a bonus anytime they manage to catch a driver on camera not paying attention. Not the best system for the driver, but an effective one because they know at any time, someone could be checking on them. A driver (like this one) who doesn't pay attention regularly will be caught quickly.
In terms of random challenges, the autopilot could be programmed to randomly shut-off/fail once every half hour in a "safe" situation. Or insert a "fake" pedestrian or badly behaving car into the data feed, requiring a take-over. Something along those lines. The driver would then know they will be required to take over driving at any moment, the opposite of assuming they will rarely need to. In other words, they need to test the safety drivers on a regular basis as part of the safety testing.
Another is the one you mention, which is that every X amount of time, the safety drivers need to be doing something besides being a safety driver, to relieve the monotony. Build into the route for the car to auto-park and the driver to go for a walk, or swap drivers every X amount of time, etc...