I have had the pleasure of being the project manager on several fairly large software projects during my career. These projects were finished on time and to spec. Everything the customer asked for in the agreed-upon scope worked. Everyone was happy...
Good news. But finishing projects on time and precisely to spec does actually not always make the customer happy. And making the customer happy, profitably, is essentially the purpose of all business. On-time and to-spec deliverables are means to that end, but in the end the payer should be happy and the company should make a profit or the deal shouldn't be done.
In the case at hand, it sounds like the PM is trying to make the buyer happy, but that he himself can't be made happy (profitably) by any contractors, in which case his business model is flawed. Either he needs to set expectations better with his customers such that he can pass on an easier set of expectations to the contractors, or he needs to price things such that the contractors can still make a profit while delivering him what he needs to make his customers happy. An example of the former might be setting uptime metrics, and example of the latter is a defined test-fix-test period at a fixed cost.
"I don't pay for bugs" is just a negotiating tactic.