It seems to me that part of the issue here is that you're trying to form a single ranking of all the papers/journals, and there might not be one. Netflix doesn't try to form a single ranking of all movies, they try to find the ones that a particular individual will like - a personalised definition of good.

This allows the crackpots to have their own definition of 'good', and there is nothing wrong with that.

For individual researchers this approach would probably work very well. Funding bodies would need to specify more constraints than just that they want "good research" to get a useful answer. Figuring out what those extra constraints 'should' be is an interesting question.