The currently-in-development Neo900 project (which hit a few snags because PayPal are scumbags and withheld large chunks of project funding but seems to have recovered now that PayPal has released the money) has some features that will help with this.
It contains physical hardware level off switches for phone radios/antennas (including a separate disconnect for the GPS antenna to prevent it from being able to do GPS location unless you enable it) but more importantly the design of the hardware means the baseband radios (including the cellular baseband module) have NO access to the main application processor or its RAM or Flash storage. All audio goes through the main application processor as well (meaning the baseband has no access to the microphone in the phone at all)
As far as I am aware it will be possible to run the Neo900 and use its features (make calls, access the internet etc etc) using only open source software on the main application processor (so no blobs that could contain backdoors).
No its not an iPhone or an Android device, no its not super-thin and super-light, no its not packing the latest super-powerful CPU and no it wont run Pokemon Go or Netflix but it will prevent bad actors (whoever they may be) from remotely activating your phones microphone and recording everything you say without your knowledge.