No-one, apart from maybe Dan Bernstein, is good enough to reliably write bug-free code. The hubris that says "I am!" is the core reason why we have such enormous security problems.
> How many lines of code is in the Rust compiler & library. How much of that must be flawed? That'll get passed on to every program that uses it.
No, that's not how it works.
A bug in the compiler only matters if it was triggered during the build that produced your binary *and* the build succeeds *and* the results pass your test suite. Unless your code is quite unlike anyone else's code, you will hit this a lot less than bugs in your own code.
A bug in Rust's standard library can affect a lot of programs, but much of Rust's standard library is written in safe Rust so gets the same safety guarantees as regular Rust code. And of course the standard library gets a lot more testing and inspection than your own Rust program.
The bigger picture is that formal verification technology is advancing so that in time, we'll be able to verify that a build worked correctly (i.e. the generated code preserves the safety properties of the Rust code), and we'll be able to write proofs for most of the unsafe parts of the Rust standard library that they also preserve safety properties.