Seriously, the stuff that's radioactive enough to make a dirty bomb is radioactive enough to kill you before you get offsite. New fuel (less than 5% enriched uranium) is not particularly radioactive. It's perfectly safe to stand next to it; to inspect it before you put it in the nuclear reactor.
On the other hand, spent fuel is incredibly radioactive, and when it's being handled it's kept under 30' of water so it doesn't kill everyone in the building.
Now, let's assume you had access to the fuel long enough to get it out of the pool. You would receive a lethal dose of radiation in 36 seconds; enough to kill you within a month. Even if death doesn't come for weeks, you would be rapidly debilitated- which of course would leave you immobile next to something giving off massive amounts of radiation, so I imagine you'd be dead-dead within a half hour. Probably much less.
Now, there is spent fuel that's had several years to decay sitting in dry storage on most nuclear sites, but they're kept in casks and bunkers which are so robust, you're not going to be able to steal or breach them in less time than it takes for three states worth of Law Enforcement and FBI to come crashing down on your party.
That fuel in dry storage would still kill you, but it would take longer.