A major flaw in your argument here: Uranium isn't very radioactive. Fuel pellets for nuclear reactors, that have not yet been used, you can handle with your hands without much in the way of problems (heavy metal poisoning, like with a block of lead, would be far more significant than any radiation here). On the other hand, once they've been through a reactor, you don't want to be anywhere near them.
The major source of radioactivity comes from the fission products. Heavy atoms need to be more neutron-rich to be stable, and lighter atoms are stable with a more even ratio of protons and neutrons. So when you split a heavy atom, you generally end up with a couple of lighter atoms which are far too neutron-rich to be stable, and which fairly rapidly decay into other things, producing and radiating away energy in the process (i.e. radiation).
That's where all of the lasting radioactivity comes from, after the reactor has been turned off. Uranium has far too long a half-life, billions of years, or hundreds of millions of years, for the main couple of isotopes, to have much effect on the amount of radioactivity. (generally, the longer the half-life, the less radiation, since a long half-life means less decay events per period of time. The candle that burns half as long burns twice as bright, or something like that. Isotopes that decay over a period of years for example, would produce approximately 8-9 orders of magnitude more radiation.)