The test was good. It was Safari that had the flaw.
I disagree, at least in part. The setting which was modified for testing purposes was intended to disable local caching of webpages, correct? Theoretically this was done in order to simulate a more accurate representation of real world performance, by pulling static test pages across the network instead of from the local cache. I don't know about you, but my own browsing habits do not have me frequently reloading static pages; rather, I'm loading various dynamic pages throughout the day, such as Google search results, Slashdot, Ars Technica, a webmail interface... etc. Certain of these pages will sometimes -- but not every time -- require a network fetch. That is to say, now and then the browser determines that a page has not been updated since the last fetch, and so that page can reasonably be pulled from the cache instead of over the network. Very obviously, that's the whole point of the cache. Therefore, an accurate testing scenario would by necessity have to include some mixture of dynamic pages alongside the static pages within the test suite, so that toggling hidden settings not commonly used by the general public would not be required in order to facilitate a "more accurate" simulated testing environment.
That said: It's a peculiar side effect that they happened upon a bug in that hidden setting, and the existence of that bug was indeed Apple's fault. However, the fact that Consumer Reports based their recommendation on tests which have proven to be poor representations of the systems performance, and as of yet they have failed to update their review to account for that... that firmly leaves us in a mixed responsibility scenario, with Consumer Reports sharing some of the fault as well. Apple has performed their due diligence by fixing their bug and patching it for the next beta build; Consumer Reports now needs to likewise complete their due diligence, by updating the article accordingly upon receipt of that fixed build. (Or, you know... I suppose they could fix their tests in a more permanent fashion, by simply adding a few artificially generated dynamic pages to their test server, and not using hidden and/or non-default features during testing in the first place. Creating those dynamic pages wouldn't even remotely be difficult; a basic random number generator and just about any server-side scripting environment would handily do the trick.)
... As for actual user, you do know that Macs are popular among web designers?
The original poster should have said "typical user" rather than "actual user" -- but nonetheless, I would still have to call this a red herring; it's largely immaterial to the testing scenario under discussion. But if it is pertinent at all, then I would suggest that it's probably still quite common for manufacturers to (rightly) cry foul when a supposedly impartial reviewer changes the default settings of a unit under testing, generating adverse test results... just as it's common for reviewers and users alike to cry foul, when manufacturers attempt to pull the same stunt to inflate test results.