I think your conclusion is essentially correct. The problem is that spotting good science in realtime is hard work. The scientists reviewing the paper can only put it in the perspective of their experience. If it too far outside and they are too far inside, then the paper gets rejected. It frequently requires a fair amount of time to pass before the results of a paper can be properly analyzed, and that's if the paper hasn't been so buried that no one recalls it ever being written.
To make matters worse, there are a fair number of whack jobs out there who act as though they were serious scientists when they are not, or are regurgitating something that may not even be their work and of which they have a dim understanding. And then there are charlatans who think science is some sort of dodge (e.g., those indulging their fantasies in scientific creationism).
Another complication is that interdisciplinary science gets rejected by the disciplines it spans because the reviewers inhabit a single discipline and see the interdisciplinary work as some other discipline infringing on their Universal Right to define their discipline.