Honestly, when I think back of the last 100 times I typed in ALL CAPS, as I did just now, I did so holding down the SHIFT key with my left pinkie. That's probably because I learned how to type back in high school on an old manual typewriter. At this point, using the SHIFT key is completely automatic.
Now, when I think about the last 100 times I used the CAPS LOCK key, all 100 of them were an accident. I used a slightly different keyboard, and the location of the SHIFT key was, well...shifted, from where my automatic pinkie was used to finding it and BOOM, ALL CAPS. Grrr.
That being said, if you're going to get rid of CAPS LOCK, you have to fill the space with something. What? I have no idea. I like where the CTRL and ALT keys are located. TAB too. Maybe a second DELETE or BACKSPACE key? Or make the left SHIFT key larger. How about a button that dispenses M&Ms?