Commissioning Misleading Core i9-9900K Benchmarks (techspot.com) 124
On Monday, Intel unveiled the 9th Gen Core i9-9900K, which will rival AMD's Ryzen 2700X when it goes on sale in two weeks. We will soon be reading reviews of the 9th Gen Core i9-9900K, which Intel claims is the "world's best gaming processor," to see how exactly it fares against its AMD counterpart. But as reviewers test the new CPU and comply with an NDA/embargo (non-disclosure agreement) with Intel, which requires them to not share performance data of Intel's new CPU for another few days, surprisingly, one publication has already made a bold claim. In a story published this week, news outlet PCGamesN said, "Intel's Core i9 9900K is up to 50% faster than AMD's Ryzen 7 2700X in games." The publication cites data from an Intel-commissioned report [PDF] by third-party firm Principle Technologies to make the claim. TechSpot explains the issues with this: So Intel can go and publish their own "testing" done suspiciously through a third party ten days before reviews, while reviewers are prohibited from refuting the claims due to the NDA. First bad sign. Scrolling down PCGamesN says the following when looking over Intel's commissioned benchmarks. "But the real point of all this is for Intel to be able to hold out the 9900K as hands down the best gaming processor compared with the AMD competition, and in that it seems to have excelled. On some games, such as Civ 6 and PUBG, the performance delta isn't necessarily that great, but for the most part you're looking at between 30 and 50% higher frame rates from the 9900K versus the 2700X."
Right away many of the results looked very suspect to me, having spent countless hours benchmarking both the 2700X and 8700K, I have a good idea of how they compare in a wide range of titles and these results looked very off. Having spotted a few dodgy looking results my next thought was, why is PCGamesN publishing this misleading data and why aren't they not tearing the paid benchmark report apart? Do they simply not know better?
Over at the Principled Technologies website you can find the full report which states how they tested and the hardware used. Official memory speeds were used which isn't a particularly big deal, though they have gone out of their way to handicap Ryzen, or at the very least expose its weaknesses. Ryzen doesn't perform that well with fully populated memory DIMMs, two modules is optimal. However timings are also important and they used Corsair Vengeance memory without loading the extreme memory profile or XMP setting, instead they just set the memory frequency to 2933 and left the ridiculously loose default memory timings in place. These loose timings ensure compatibility so systems will boot up, but after that point you need to enable the memory profile. It's misleading to conduct benchmarks without executing this crucial step.
Right away many of the results looked very suspect to me, having spent countless hours benchmarking both the 2700X and 8700K, I have a good idea of how they compare in a wide range of titles and these results looked very off. Having spotted a few dodgy looking results my next thought was, why is PCGamesN publishing this misleading data and why aren't they not tearing the paid benchmark report apart? Do they simply not know better?
Over at the Principled Technologies website you can find the full report which states how they tested and the hardware used. Official memory speeds were used which isn't a particularly big deal, though they have gone out of their way to handicap Ryzen, or at the very least expose its weaknesses. Ryzen doesn't perform that well with fully populated memory DIMMs, two modules is optimal. However timings are also important and they used Corsair Vengeance memory without loading the extreme memory profile or XMP setting, instead they just set the memory frequency to 2933 and left the ridiculously loose default memory timings in place. These loose timings ensure compatibility so systems will boot up, but after that point you need to enable the memory profile. It's misleading to conduct benchmarks without executing this crucial step.
Intel any thing to win other then more pci-e or (Score:4, Insightful)
Intel any thing to win other then more pci-e lanes or no raid keys
Re: (Score:3, Insightful)
Re:Intel any thing to win other then more pci-e or (Score:4, Insightful)
Re:Intel any thing to win other then more pci-e or (Score:5, Interesting)
>>> will you discard it and build a new one to manufacture a new chip?
Generally, yes you will. A large part of the cost of the factory is the machines in it - and generally they all need to be replaced when you move from one process node to the next.
Let's say you spent $1B to build a 40 nm fab. You start building state-of-the-art wafers in it, and (being a good businessman) after a year are running it near 100% capacity. Next year, the bleeding edge wafers need to be at 32 nm. Your choices are:
1. Shut down production, spend 6 months retooling for 32 nm, then re-open. Cost: Six months of production plus new machines. End result: One fab producing 32nm wafers.
2. Keep the plant running, and build a new 32 nm fab. Cost: New fab with new machines. End Result: you still have the old fab cranking out 40nm wafers which everyone who doesn't need the bleeding edge will be buying for the next several years. In fact, you might have 55 nm, 40 nm, and 32 nm fabs all running in parallel. And those state-of-the-art 250 nm fabs from 20 years ago? Some are still running, putting out dirt-cheap wafers for people whose needs are met by low-performance, dirt-cheap ASICs.
Re: (Score:2)
The silicon needed for a typical CPU only costs about $5
While I'm no fan of Intel and their underhanded tactics vis-a-vis these benchmarks, this statement is so horrifically misleading can't let it stand.
The cost of a typical CPU is not just the cost of the silicon it's printed on. Literally thousands of engineers labor for years -- sometimes decades -- to develop the CPU design, the lithographic technologies, the fab designs, the materials design, and countless other tasks required to produce a modern CPU. The aggregate cost for such endeavors runs into the b
Re: (Score:2)
I like how you think all the employees, cost of the fab, and the cost of running the fab is all free for Intel.
Re: (Score:2)
It's not just transistor count or silicon area which determines how much your production process actually costs on a marginal basis. Volumes, layers, process steps, interconnects (pin counts), yields, testing and a lot more can dominate the cost equation.
Two cars with a V8 engine: sometimes one of them costs a lot more than the other.
Two chips with 3 billion tran
Desperation... (Score:5, Insightful)
Re: (Score:2)
Re:Desperation... (Score:5, Informative)
Re: Desperation... (Score:4, Informative)
Game mode is designed for threadripper cpus. It disables half the cores so that it will perform better in gaming as games aren't extremely multithreaded and won't scale up that well past 8 cores.
But... (Score:2)
Very few PC games perform worse with more cores, mostly recent console ports.
Additional cores allow other programs to run on them, so your game doesn't get paused/swapped out while some windows housekeeping thing needs a few cycles.
The more modern games I'm playing love having 12 cores; Doom, for one.
Yes, the new one, lol.
Go try Arcade mode, then turn off half the cores, and see how that works out.
Re: (Score:2)
Ryzens are NUMA parts. That means memory access is non-uniform. If your application is not NUMA aware, you can see actual performance degradation on single-threaded applications (when the OS scheduler moves them to a core on the other side of the NUMA barrier) or multithreaded applications that have significant IPC across that barrier, but not enough parallelization to overcome that cost.
This is why game mode exists. Because the AMD, the guys who designed the chip
Re: (Score:2)
This includes the CPUs tested.
"Game Mode" improved the performance of every game they tested, minus Ashes of the Singularity.
It has nothing to do with games scaling past 8 cores. It has to do with preventing high-performance low-thread-count applications (usually games) from being moved across a NUMA barrier and hurting the performance of that application.
Re: (Score:3)
I'll say. I was able to pick up a new 2700X for $297.68 after tax a few weeks ago from a local retailer. Most online stores list the 2700X at $320 without any specials or discounts. The summary is saying the i9-9900K is supposed to compete with that, but the 9900K's MSRP is a whopping $488 [anandtech.com]. At those prices, the 9900K needs to stomp all over the 2700X. Anything less would be a disaster. Simply trading blows with the 2700X wouldn't be anywhere close to good enough.
But that's probably because it's a silly comp
Comparison to Apple (Score:3)
I saw the prices for the new core, and I feel like Intel is trying to be the Apple of CPUs. Inflated price just because "Intel" rather than those other guys "AMD".
Re: (Score:2)
Except that Intel has what, 80% of the market and Apple 10%?
Re: (Score:2)
480 for the technology isn't a bad price.
Comparing it to apple is pretty foolish.
Corporate shills (Score:1)
Re: (Score:3, Insightful)
I'm not sure why this is news. Have manufacturer supplied benchmarks EVER been a reliable measure of real world performance?
Re: (Score:2)
Yep. Never pan out. Zen being 40% faster than their previous chip was totally false. Oh wait, that's because it was over 50% faster.
Re: (Score:1)
Re: (Score:1)
Re: (Score:1)
Yeah maths does really work that way. If that chip does 2 MIPS and you add 50% more speed then it will end up with 3 MIPS. That's a one third increase, not a half.
Re: (Score:2)
Except for the minor fact that both were 8 core chips, you're totally right.
Re: (Score:2)
Of course this doesn't make what Intel did here any better. The way I see it, they showed either some pretty bad incompetence by claiming the numbers of such a questionable testing method or knew that they were mostly worthless and didn't care.
In any case we should wait for benchmarks fr
Re: Ryzen doesn't like fully populated DIMMs? (Score:1)
Official memory speed with 4 Dimms is lower than with two. However it overclocks quite well and the 4 dimms give you extra performance due to additional memory interleaving. Whoever wrote that sentence has never tried it themselves
Re: (Score:2)
I'm a gamer; but i just want to game; not fill out memory banks; change clock speed; & do other tinker-stuff with it.
https://www.dell.com/en-us/shop/desktops/area-51-threadripper/spd/alienware-area51-r6
https://www.dell.com/en-us/shop/desktops/area-51/spd/alienware-area51-r5
Tell me; if i buy that AMD processor; & fill the memory banks; WHERE do i get the warning that that is not a good idea?
https://www.hardwarecanucks.com/forum/hardware-canucks-reviews/74814-amd-ryzen-7-1800x-performance-review-5.html
From the 2 minutes I've researched, it seems like these issues have largely been fixed by BIOS updates.
Go. Game. Have fun.
Some more speculation (Score:5, Informative)
Speculation on Reddit about this seems to suggest they may have enabled the gaming profile* in Ryzen Master for games that don't benefit it (threadded / multi-core friendly games), and disabled it for those (single threadded dependent games).
For a multi-threadded suddenly loosing access to 4 cores, and for a single threadded game suddenly losing access to an additional 200MHz will give you some of those gimped benchmarks.
*For those who don't know, Gaming profile disables half the cores on a Ryzen 2, specifically targetting the poorest performing cores, and then raises the boost frequency thanks to the additional thermal / power headroom available. This is of great benefit to games that don't take advantage of multi-core processors.
Are tuned benchmarks really applicable (Score:2)
Instead they just set the memory frequency to 2933 and left the ridiculously loose default memory timings in place.
For the purpose of a review, is it more or less representative to tune every aspect of the system like this? When a reviewer tunes and tweaks every possible setting, the results really are only applicable to that motherboard + RAM combination. I would rather have apples-to-apples comparisons.
Only if you tune the one that pays you only. (Score:4, Interesting)
An even benchmark would have used stock timings on both boards, not XMP on intel, and no optimization on AMD.
That's an obvious Fail on their part; to me, that means this is the only way they can compete now.
And these chips still have ALL the flaws, and require software mitigations that drop performance 20-40%.
When they fix those, and stop being lying douchebags, I may buy intel again.
I'm still using a 17-3930k at 4.8GHz; it's been running that on ALL cores since about 2011 or so.
A chip that's only 10 or 20% faster really doesn't impress me enough to upgrade; it still plays Quake2 just fine, and Crysis works great. :)
Re: (Score:2)
This is true, but the overall memory bandwidth to the actual processor hasn't increased a lot over the years.
The system I'm using is about 55GBps, same as the x5670 I'm using on another system.
I haven't seen DDR4 systems running faster than that, but I don't own any yet.
Adding memory channels does help, but at some point, the internal processor architecture runs out of bandwidth.
Re:Are tuned benchmarks really applicable (Score:4, Insightful)
is it more or less representative to tune every aspect of the system like this?
To tune it like this? Less representative. They've effectively overclocked one system while underclocked the other. Not to mention disabled half the cores on AMD chip.
Re: (Score:1)
The Ryzen performs better in most games with half of its cores disabled, so that's not a bad thing for gaming benchmarks.
However, heavily multithreaded games like AoS will clearly be hurt by this, and it should be disabled for them, as you have also pointed out elsewhere.
This benchmark may not have been great, but I'm not sold that it was intentionally misleading. They seemed to s
Re: (Score:2)
The Ryzen performs better in most games with half of its cores disabled, so that's not a bad thing for gaming benchmarks.
Not quite. It is highly dependent on the game. Quite specifically it is a feature which should be used selectively to gain the best performance. For example in Ashes of the Singularity you will get an approximately 40% performance hit when enabling Game mode. Likewise for any other game that makes liberal use of multi-threadding, of which there are a few in that list.
This benchmark may not have been great, but I'm not sold that it was intentionally misleading.
Regardless of the specifics of *how* they performed the test, the important part is the results, and as plenty of others have said the actual
Re: (Score:2)
Not quite. It is highly dependent on the game. Quite specifically it is a feature which should be used selectively to gain the best performance. For example in Ashes of the Singularity you will get an approximately 40% performance hit when enabling Game mode. Likewise for any other game that makes liberal use of multi-threadding, of which there are a few in that list.
I quite literally said exactly that. AoS is actually one of a limited amount of games that will perform better. The idea is that IPC between cores on different CCX modules is *expensive*. Game Mode essentially allows a non-NUMA aware application perform better, which is most games.
Regardless of the specifics of *how* they performed the test, the important part is the results, and as plenty of others have said the actual results for the AMD chips are not remotely representative of AMD chips even if simply left in their "Optimised defaults" configuration in the BIOS, and when the actual AMD based features are properly used the gaps between the slightly faster performing Intel chips and the AMD ones are just that, slight.
No, sorry. The configuration of the benchmarks did nothing but hurt the top-end multicore side of the benchmarks. Single-core performance was about on par with most other benchmarks for these two chip families. Throw away the AoS
Re: (Score:2)
No, sorry. The configuration of the benchmarks did nothing but hurt the top-end multicore side of the benchmarks. Single-core performance was about on par with most other benchmarks for these two chip families.
Yeah because purposefully gimping memory latency and bandwidth really affects multi-core gaming. Look you can say what you want. The results however speak for themselves. Go online and compare how these benchmarks *ACROSS ALL THE GAMES TESTED* are not representative of the chip.
And regardless what you think about game mode, it's a specific mode created for Threadripper CPUs, and Ryzen Master will actually mention that it shouldn't be used unless you have a Threaddripper when you hover over the button.
Sorry, but you're a shill.
Shill:
Re: (Score:2)
Yeah because purposefully gimping memory latency and bandwidth really affects multi-core gaming. Look you can say what you want. The results however speak for themselves. Go online and compare how these benchmarks *ACROSS ALL THE GAMES TESTED* are not representative of the chip.
Purposefully gimping is a stretch, but I think everyone agrees it was uneven. However, amateur retests of the subject test have been done, and more aggressive but supported memory profiles are predictably barely a rounding error in benchmark performance improvement.
And regardless what you think about game mode, it's a specific mode created for Threadripper CPUs, and Ryzen Master will actually mention that it shouldn't be used unless you have a Threaddripper when you hover over the button.
Every processor with more than 4 cores supports Game Mode, as of the Feb 2018 "AMD Ryzen ThreadRipper and AMD Ryzen" whitepaper. This would make sense, because again, any Ryzen with multiple CCXs can show better performance with lower-thread-coun
Re: (Score:2)
This includes the Ryzen 7.
I'm sorry your mom dropped you when you were a baby.
Re: (Score:1)
For the purpose of a review, is it more or less representative to tune every aspect of the system like this? When a reviewer tunes and tweaks every possible setting, the results really are only applicable to that motherboard + RAM combination. I would rather have apples-to-apples comparisons.
Apples-to-apples is the obvious ideal but that's not what happened here. The Intel platform was tuned; the AMD platform was left at the defaults.
TL;DR (Score:5, Insightful)
They had to know they'd be called out by the benchmarking community. That and Youtubers hungry for video content.
Re:TL;DR (Score:4, Informative)
Also "Gaming mode" doesn't benefit all games. Effectively it disables half the cores on a Ryzen chip in favour of a small MHz boost on the remainder. E.g. This could account for a close to 40% performance drop in Ashes of the Singularity during conditions just perfect to be CPU bound.
This has been going on for 35+ years (Score:4, Interesting)
Seriously. I remember in the 1981/1982 time frame when the Motorola 68000 was starting to make some inroads in desktops and Intel released their own performance reviews showing how the 8086/8088 was better at user (Intel specified benchmark) tasks. Motorola's response was to fight fire with fire showing that the 68k was better in a highly subjective benchmark. This has been going on between Intel and whomever is their current main competition since then.
It sounds like actual hardware will be available in a week or so with actual standard benchmarks being available a couple of weeks after that.
Avoid the hype and just wait for tests on actual hardware.
Re: (Score:2)
However the 68k series was a pleasure to program in assembler. And for high level languages it was easy to create efficient code!
Re: (Score:1)
It truly was the betamax of CPU architectures. Imagine what it would be like now with heaps of nice wide registers. Those were the days. :-(
Re: (Score:2)
Unfortunately the better technology is not always winning at the market :(
But many architectures are remotely similar to 68k ... ARM, SPARC, PowerPC ...
I guess even modern x86 copied much of it :D (never dug into the current ISA)
Game performance increase is a bad benchmark. (Score:2)
The thing is - some games are GPU-bound and others are CPU-bound.
If it's the former - then you can replace your CPU with DeepThought, Holly or HAL and your frame rate won't move an inch.
So whether this contraption does you any good depends sensitively on the games you play and the performance of your GPU.
Even the performance of your GPU will depend on your screen resolution.
The ONLY reliable benchmark is the actual application you're running on the actual Before and After hardware setups.
Re:Game performance increase is a bad benchmark. (Score:4, Informative)
The thing is - some games are GPU-bound
With a 1080TI playing at 1080p there's not a game out there that is GPU bound.
However games are incredibly variable in how they utilise their CPU. Ashes of the Singularity is a good example. It's a very well threaded game that happily smashes all cores on a typical Ryzen process for benchmark purposes, but by enabling "Game mode" in Ryzen master they successfully disabled half the CPU. Youtube videos aplenty show that this incurs a huge performance hit in this particular game, as well as any other game that relies heavily on multi-threading.
Who could have foreseen this? (Score:2)
If only someone had known Intel was going to do this, maybe we could have stopped them. [slashdot.org]
If you are not detecting the sarcasm then it's probably because you've died.
Dell also likes these guys (Score:5, Informative)
It's the second time I hear this company's name in less than a month. Some days ago I read a press release by Dell [dell.com] about a panoplia of new products, and the entire list, ranging from laptops to server computers was full of performance improvements (vs competitors) claims. all of them referring to paid-for reviews by this same company.
I personally find their motto - "win the attention war" - amusing. Also of interest is the fact (pun setup) they interchange links with their main domain and with a redirect from my country's TLD subdomain "facts.pt" (pun successful..?), as a subtle way to include their initials as something factual, and for the unsuspecting eye to believe it's a different company or to provide credit to their reviews with such a "reputable" subdomain. Genius stuff.
These companies are the audit companies of tangible products. Usually, you have Big Four conducting external audits for finantial institutions, country elections and whatnot, gathering data only these auditors are given access. The process is usually compulsory, but still paid by the targets of the audit, and there's always the sense the best auditors are usually the more positive. Now we get these paid product reviewers acting exactly the same way, getting paid to review products before they come out so companies can make bold claims. Then just NDA every other actually independant party interested in reviewing the product. See a pattern?
Re: (Score:2)
Also of interest is the fact (pun setup) they interchange links with their main domain and with a redirect from my country's TLD subdomain "facts.pt" (pun successful..?)
Nope.
Re: (Score:2)
:'(
Re: (Score:2)
I'll be fair and correct myself: apparently the only PT's-backed claims were about laptops, from 3 separate PDF files (all separately commissioned from the looks of the them). All other product text appears either unsubstantiated or based on internal testing.
Is that with or without Meltdown & TLBleed? (Score:3, Insightful)
That would be;
Neither Meltdown nor TLBleed affect AMD and AMD's Ryzen processors, as far as we know now...
So any benchmark of an Intel CPU without those security mitigations, (if needed), would be showing that they still want to abuse security for the performance gain. Something AMD appears not to want to do.
Kiddy ram does NOT improve performance (Score:2)
One hard learned lesson I've learned over the years is never ever think about buying kiddy ram.
Kiddy ram can be identified by aggressive timings, "XMP" profiles and "crazy looking heat spreaders" often marketed to "gamers". Basically companies that produce these things scrape chips they didn't produce from the bottom of the bin and do insufficient integration testing of the final result. Some vendors have previously allowed chips with a threshold of detected bit errors to pass QA and make their way into s
Gamers Nexus covered this pretty well (Score:1)
Check out the videos, first with Steve going through the test report and saying how shoddy it seems and the second he actually rocked up to the company who did the benchmark tests for Intel and interviews them.
Intel's Gross Incompetence & Principled Technologies (Intel Responds) https://www.youtube.com/watch?... [youtube.com]
Exclusive: Interview w/ Principled Technologies on Intel Testing (9900K) https://www.youtube.com/watch?... [youtube.com]
Not much else needs saying, other than what a ridiculous move from Intel. How dumb do the
Re: (Score:2)
By on average 3% IPC not counting AVX and AVX 512. Their biggest advantage at the consumer level is the ring bus, not IPC or even clock speed, which becomes significantly less useful as they add more cores. Meanwhile clock speeds above 4.6 GHz get diminishing returns.
Re: (Score:1)
amd fanbois can try to argue their case all they want, but they'll still be w.r.o.n.g. intel's architecture is more efficient, more speed per core and per megahertz (even a fucking pentium 'gold' has faster single core speed than the fastest ryzen), and has been for over a decade. the new chips are no different, even with the recent strides amd has made, intel still kicks amd's ass, especially where cost is no object. period. end of thread.
If all that were true, why the NDA/embargo and this early "cooked books" review?
PS - There's this concept in the English language called "capitalization" [wikipedia.org]. Learn how to use them, and you won't come across as an ignorant backwoods doofus. Communication skills matter...
Re:it's a no brainer. (Score:4, Insightful)
I switched from intel to some Amd box with 1800X CPU and it works very fine. I could have never gotten the bang for bucks with another Intel machine, and then there were also the serious security flaws of Intel chips. My machine handles everything including flight simulation and every game very well, so getting even more speed would be pointless at this time.
It's a pragmatic decision and it's stupid to get emotionally attached to companies. A PC is nothing but a tool (or a fun toy, when we speak about gaming). If Intel produces something better in 5 years from now, maybe I'll switch back to them.
Re: (Score:1)
"It's a pragmatic decision and it's stupid to get emotionally attached to companies."
Yes it is stupid to get emotionally attached but considering the historical track record of the company isn't necessarily emotional. AMD has always kept price/performance as an important metric. They've also been very focused on supporting open platforms. And yes, right now they've got the best line-up for most use cases, even in the server market which is definitely something new.
There is a point where the gap grows too la
Re: (Score:2)
Well said.
Re: (Score:1)
Nice to see that the guy who gets most of his annual salary by blowing Intel's board of directors has found part-time work as an AC on Slashdot.
Way to climb that career ladder!
Re: (Score:2)
even a fucking pentium 'gold' has faster single core speed than the fastest ryzen
With a statement like this I'm sure "fanbois" could argue all they won't. It's not like you would listen, your paycheck won't allow you to. Or braindamage. I dare not call you a shill without proof. Mental illness is a very real problem these days.