Removing access immediately is important for 2 reasons. The first is obviously security. Then 2nd is figuring out what he does & making sure somebody else has that access & knowledge.
If he's still in the office & gets a call or something to fix an issue it will have to get bounced to somebody else. You'll have him available to do knowledge transfer on what he use to have access to do. If he's not in the office, but still getting paid he's still available for knowledge transfer. If he's past his 2 weeks notice, he has 0 obligation to assist you guys or provide any knowledge & training to his former employers.
Whenever I give notice I expect to loose my administrative access pretty much immediately. I've already backed up anything personal. I feel no disrespect when it happens. Seriously? Boho, you are giving me 2 weeks of paid vacation time, cry me a river. It's slightly annoying if I'm still around for those 2 weeks with no privileges to do anything, but I know exactly why they have been removed. Being ostracized is one thing (and not really kosher), but merely having admin credentials revoked should be expected.
As far as a security issue goes, any competent disgruntled sysadmin has already done the done the damage or set the logic bomb before they have given their notice. Still, better safe than sorry.