There was a paper at ASPLOS two years ago that showed that you get a 30% delta on most programs from randomised code layouts just from different cache interaction
What's funny is when someone micro-benchmarks the heck out of something and shows me empirical "evidence" that their code is faster than mine. Then they put the code into production and their code is suddenly running really slow under heavy load. Throw in my code for S&Gs and suddenly the service is running much faster, and I don't even put much effort into the design like they do. Many people fail to understand how processes within the system interact with each other when it comes to cache and memory bandwidth. Even when I try to explain my theory as to why my code runs faster, their eyes glaze over. And that's why many of our services run like crap.
You have to be able to manage both high level and low level details *in the same context*
I code in C#, while I don't code in ASM I have respect for it and have done a lot of reading on it when I was around 7 and have read about modern CPU architectures and cycle latencies and cycle throughputs of different instructions. I have also read about how C#'s GC works, how objects work(More than 128 bytes for an empty object), how interfaces are work(method indirection can be O(1) when you have one implementing class, but O(N) when you have many and they're being used), casting for inheritance (casting child to parent is pretty much free but casting parent to child is expensive, like hundreds of cycles), method parameters work (large structures or too many parameters cause an object to be allocated on the heap that holes the parameters, same with method return types if they're too large).
I can many times jump into some other programmer's code and make it a few factors faster. It some cases, orders of magnitude. My pet peeve is that most people think of performance as an afterthought because "don't preemptively optimize", but they apply this to their architecture. A high performing architecture needs to be designed from the beginning, and this also requires being able to mix high and low level details, even before the low level details are fleshed out.
Beware of all enterprises that require new clothes, and not rather a new wearer of clothes. -- Henry David Thoreau