Exactly. If you're ethical, you won't leave any access possible, so there's no doubt as to your integrity.
When I left my previous employer, a small business where I had full admin rights (I set most of it up), I made sure to wipe all my ssh keys, lock and delete my accounts so that the company directors could be sure I no longer had any access, remote or otherwise. No cron jobs, no source code, no customer information. A few months later they asked me if I could look into a problem that cropped up, and had to tell them it was impossible since I had no means to log in, but I could visit in person to briefly talk to their new staff. Mutual respect, and no possibility of any suspect practices due to being completely transparent about the leaving process. It's idiots like in TFA that give all of us a bad reputation, or at least cast a shadow of doubt upon our professionalism. Unfortunately, it's all too easy to do that if you don't want to act in good faith, particularly when you are entrusted with privileged access to a companies systems and processes.