And for compiling something like a basic C compiler, one could feasibly write their own using ASM from a base of something like CC500 (a 600ish line C compiler). Use said custom compiler to build something like PintOS (full code review possible by one person, I had to do so in collegiate OS courses) on a micro that is running nothing but your compiler from a RS232 port that you are monitoring with a logic analyzer (to watch out for stray data from the 'host' computer at this point). This gets you up to OS and compiler on your chip and board of choice, though you may need a bootloader. From there, you could compile the rest of a known tool chain, like GCC and all it's accompanying tools; if you've reviewed them satisfactorily.
As for trusting your hardware: good luck, you'll need it. Even if you can get a copy of the lithos used in producing your chip, you will have just a statistical analysis of the chance of a spy in your chip. Since you can't just decap and dissolve the layers to make sure. Perhaps with the lithos in hand you could get custom made chips, but that's not going to be any 'big iron' like an x86-64. So you've shifted the needed trust down to just the silicon (and microcode if needed) that are comparatively harder for an individual to make on their own. I suppose you could mimic the CPU on an FPGA or PLC, but you are back to "trusting trust" that the compiler didn't recognize something and stuff it in the binary.
It still amuses me that the shift from analog devices to digital shifted where the specialization was required so drastically. A 555 could be built from a handful of discrete components (resistors were just long lengths of wire, capacitors were just two plates with a gap, and diodes were whiskers; transistors were the exception), but programming analog devices was considered a black magic art. Now, with IDEs and reference books most people can write some code if they sit down and follow a book (like building a crystal radio from a kit back in the "before my time" days) but building the hardware at the most basic level (logic gates on silicon) is magic beyond all but a few.