The US has a history of grabbing people and then realizing later it wasn't in its own best interests, but it doesn't stop the US doing it. There are British citizens who were rendered to countries like Egypt for a few months of torture before the US realized it had the wrong person and just dumped them back in the UK. Some of the people in Gitmo have been waiting over a decade for release.
Given what the US has done and continues to do, I wouldn't want to risk my life on the off chance that it had somehow suddenly got a lot smarter. The US isn't one entity, there are many different groups involved and it's a huge risk to assume that none of them will grab someone like Assange.
Plus, if he wasn't a valuable target then why would the UK government spend $12,000,000+ to keep him in that embassy? It's odd that the Swedish won't come to interview him either, despite having interviewed nearly 50 other people in the UK since making their first request.
It's not paranoia when there is evidence to back it up.