It is if we are permitted to keep our own information secret from law enforcement except when compelled to deliver it by warrant.
That's an interesting statement, because some US courts have ruled that we cannot be so compelled because it violates the fifth amendment protection against self-incrimination.
I see three options:
1. Makers of devices are required to provide back doors for law enforcement access. This was part of the idea of the Clipper chip... which was a total flop because no one wanted to buy it, and Congress didn't get around to (or didn't dare to) compel usage.
2. Makers of devices don't have to provide back doors, but people can be held in contempt for refusing to provide access to officials with a warrant. Some US courts have taken this position.
3. Makers of devices don't have to provide back doors, and fifth amendment protection prevents requiring people to provide law enforcement access. Some US courts have taken this position.
So, which should we aim for? I think 1 is clearly not a good idea, not least because providing a LE backdoor that can't be exploited by malicious actors is far easier said than done. 2 is what you suggested. 3 is what many on slashdot believe they prefer.
Personally, I lean towards 3, though I can see arguments for 2.