This is exactly the attitude I encounter very often when I talk to other physicists and astrophysicists (I'm the latter). But I think this actually is partly a self-fulfilling prophecy. While it's true that many of the codes we write will never be re-examined by anyone, a few of them will. But many scientists write totally inscrutable code, so no one bothers trying to re-use code in that way, because it's easier to just start over from scratch. Thus, no wonder everyone thinks no one is going to look at their code - if they wrote more readable code, perhaps someone would!
That being said, I certainly have had to take code from someone and build upon it or make it work properly for a slightly different instrument or something like that, and I can definitely say that I'm biased, because those experiences are invariably the most frustrating work I find myself doing! Part of it is that a lot of scientists seem to say "if I add comments everywhere, that's all that's needed to make it easy/readable," but they then don't follow indentation rules, or consider their code structure before they implement, or consider possible extensions of their code when they are writing it. So it wouldn't take that much more work, but some scientists just don't realize that in fact it probably is worth the effort for the sake of future work.