One should also remember that government employees with privileged access are people, and people can misuse the access they have.
We should recognize that the Fourth Amendment of the US Constitution was created to prevent this exact scenario. Law abiding people encrypt sensitive information to protect it from misuse by criminals, but the information can be misused by ANYONE with access.
Dividing a backdoor key between multiple parties simply creates a requirement that all parties agree to access the information before it can be accessed. It doesn't guarantee that the access will be lawful.