That certainly happens - a college student who has basically sailed through elementary and high school suddenly runs into difficult material for the first time ever and it's hard to cope. That can happen in fields other than science, math, and engineering, of course.
Another side of this might just be that there is a difference between liking a subject academically (in high school) and deciding you want to do it for the rest of your life. I had an amazing high school chemistry teacher who set me on the path to AP chemistry and eventually a BS in chemistry. But right around junior year (first semester of physical chemistry lecture and second semester of organic lab) I realized I didn't want to have a career in chemistry. I still really enjoyed the intellectual challenge of chemistry (otherwise I wouldn't have finished the degree), but at that point it was a path to grad school. I ended up doing my graduate studies in communication, and I've ended up with a career I love. I enjoy teaching and research in communication is just as intellectually challenging (but in a different way).
I guess my point is that we shouldn't underestimate the impact of students realizing the difference between "I like science subject x" and "I want a career in subject x."