I'm not sure that I completely agree - I listen to music on the way into work, and I've never even once found myself in a position when lack of audio clues put me in any danger.
As long as you use regular earbuds, and not the in-ear-silicon things that block out noise, then you'll be fine.
I do agree about audiobooks though, albeit for a different reason. You'll never be able to concentrate on them properly because you'll regularly have to concentrate too much on avoiding the idiot drivers. You'll just end up missing whole pages.
[Also rode to work for 15 years...]