If you're really concerned about security, you are likely running OpenBSD or a heavily-modified linux kernel by now.
Linus Torvalds was asked during a LinuxWorld keynote two years back if he was told by government agents to put hardware backdoors in linux. he said no, while nodding yes. His father, Nils Torvalds, a member of EU parliament, put it on the record that his son was approached by government agents requesting backdoors.
There is a known issue with the random number generator being _forced_ to do hardware-based (known to be broken on Intel/AMD chipsets) random number generation. under Open/Net/FreeBSD, there's an intermediary (software) random number generator that ensures actual randomness. Linus uncharacteristically led this charge to keep the RDRAND weakened, even resorting to calling others stupid for thinking otherwise. a prominent developer resigned due to it.
There is at least one recent Intel Management Engine talk at last year's Chaos Communication Congress. There was a similar talk the year before about AMD chipsets and their secret undocumented internal firmware. If you enjoy strong encryption, you would be wise to apply the proposed RDRAND patches that Linus rejected.
Now that all the major distributions have adopted systemd, there's now a full RPC backdoor to not only the GPL's linking requirements, but a backdoor to run "Approved" (by whom? we'll get to that) code automatically. Many people have pubatlicly posited that systemd will be the cause of "The Big One" vulnerability that eventually comes out of Linux and ruins its reputation.
Now, for the Ubuntu side: Canonical is incorporated in City of London, which means they are under the jurisdiction of GCHQ. Anyone who has watched/read a talk by Moxie Marlinspike will know that SSL/TLS is easily-spoofable by nation states. They will probably also know how exploitable SSL/TLS is today. All the draconian crap the GCHQ has jurisdiction over can easily be extended to a corporation registered under their governance. If Canonical refuses, they will be forced to, the way Google is forced to comply in the United States under similar framework. End result is that you cannot trust anything beyond your initial install CD, if you can even trust that.
You will likely never look through the custom patches compiled into your binaries, let alone think about Ken Thompson's "Trusting Trust" essay. You will just download your updates, and assume everything is A-OK. You are an end-user, and that's okay. Just don't think Shuttleworth's words are anything but a big fat placebo to keep his stock value afloat.