AMD EPYC Milan Review Part 2: Testing 8 to 64 Cores in a Production Platform
by Andrei Frumusanu on June 25, 2021 9:30 AM ESTAMD Platform vs GIGABYTE: IO Power Overhead Gone
Starting off with the big change for toady’s review: the new production-grade GIGABYTE Milan compatible test platform.
In our original review of Milan, we had initially discovered that AMD’s newest generation chips had one large glass jaw: the platform’s extremely high idle package power behaviour exceeding 100W. This was a notable regression compared to what we saw on Rome, and we deemed it as a core cause as of why Milan was seeing some performance regressions in certain workloads compared to the predecessor Rome SKUs.
We had communicated our findings and worries to AMD prior to the review publishing, but never root-caused the issue, and never were able to confirm whether this was the intended behaviour of the new Milan chips or not. We theorized that it was a side-effect of the new sIOD which had the infinity fabric running at a higher frequency, which this generation runs in 1:1 mode with the memory controller clocks.
To our surprise, when setting up the new GIGABYTE system, we found out that this behaviour of extremely high idle power was not being exhibited on the new test platform.
Indeed, instead of the 100W idle figures as we had tested on the Daytona system, we’re now seeing figures that are pretty much in line with AMD’s Rome system, at around 65-72W. The biggest discrepancy was found in the 75F3 part, which now idles 39W less than on the Daytona system.
Milan Power Efficiency | |||||||||||||
SKU | EPYC 7763 (Milan) |
||||||||||||
Motherboard/ Platform |
Daytona | GIGABYTE | |||||||||||
TDP Setting | 280W |
||||||||||||
Perf |
PKG (W) |
Core (W) |
Perf | PKG (W) |
Core (W) |
||||||||
500.perlbench_r | 281 | 274 | 166 | 317 | 282 | 195 | |||||||
502.gcc_r | 262 | 262 | 131 | 271 | 265 | 150 | |||||||
505.mcf_r | 155 | 252 | 115 | 158 | 252 | 132 | |||||||
520.omnetpp_r | 142 | 249 | 120 | 144 | 244 | 133 | |||||||
523.xalancbmk_r | 181 | 261 | 131 | 195 | 266 | 152 | |||||||
525.x264_r | 602 | 279 | 172 | 641 | 283 | 196 | |||||||
531.deepsjeng_r | 262 | 267 | 161 | 296 | 283 | 196 | |||||||
541.leela_r | 267 | 249 | 148 | 303 | 274 | 199 | |||||||
548.exchange2_r | 487 | 274 | 176 | 543 | 262 | 202 | |||||||
557.xz_r | 190 | 260 | 141 | 206 | 272 | 171 | |||||||
SPECint2017 | 255 | 260 | 141 | 275 | 265 | 164 | |||||||
kJ Total | 2029 | 1932 | |||||||||||
Score / W | 0.980 | 1.037 | |||||||||||
503.bwaves_r | 354 | 226 | 90 | 362 | 218 | 99 | |||||||
507.cactuBSSN_r | 222 | 278 | 150 | 229 | 285 | 174 | |||||||
508.namd_r | 282 | 279 | 176 | 280 | 260 | 193 | |||||||
510.parest_r | 153 | 256 | 119 | 162 | 259 | 138 | |||||||
511.povray_r | 348 | 275 | 176 | 387 | 255 | 193 | |||||||
519.lbm_r | 39 | 219 | 84 | 40 | 210 | 92 | |||||||
526.blender_r | 372 | 276 | 165 | 396 | 282 | 188 | |||||||
527.cam4_r | 399 | 278 | 147 | 417 | 285 | 170 | |||||||
538.imagick_r | 446 | 278 | 178 | 471 | 268 | 200 | |||||||
544.nab_r | 259 | 278 | 175 | 275 | 282 | 198 | |||||||
549.fotonik3d_r | 110 | 220 | 86 | 113 | 215 | 95 | |||||||
554.roms_r | 88 | 243 | 106 | 89 | 241 | 119 | |||||||
SPECfp2017 | 211 | 240 | 110 | 220 | 235 | 123 | |||||||
kJ Total | 4980 | 4716 | |||||||||||
Score / W | 0.879 | 0.9361 |
A more detailed power analysis of the EPYC 7763 during our SPEC2017 runs confirms the change in the power behaviour. Although the total average package power hasn’t changed much between the systems, in the integer suite now 5W higher at 265W vs 260W, and in the FP suite now 5W lower at 235W vs 240W, what more significantly changes is the core power allocation which is now much higher on the GIGABYTE system.
In core-bound workloads with little memory pressure, such as 541.leela_r, the core power of the EPYC 7763 went up from 148W to 199W, a +51W increase or +34%. Naturally because of this core power increase, there’s also a corresponding large performance increase of +13.3%.
The behaviour change doesn’t apply to every workload, memory-heavy workloads such as 519.lbm don’t see much of a change in power behaviour, and only showcase a small performance boost.
Reviewing the performance differences between the original Daytona system tested figures and the new GIGABYTE motherboard test-runs, we’re seeing some significant performance boosts across the board, with many 10-13% increases in compute bound and core-power bound workloads.
These figures are significant enough that they do change the overall verdict of those SKUs, and they also change the tone of our final review verdict on Milan, as evidently the one weakness the new generation had was actually not a design mishap, but actually was an issue with the Daytona system. It explains a lot of the more lacklustre performance increases of Milan vs Rome, and we’re happy that this was ultimately not an issue for production-grade platforms.
As a note, because we also have the 4-chiplet EPYC 7443 and EPYC 7343 SKUs in-house now, we also measured the platform idle power of those units, which came in at 50 and 52W. This is actually quite a bit below the 65-75W of the 8-chiplet 7763, 75F3 and 72F3 parts, which indicates that this power behaviour isn’t solely internal to the sIOD chiplet, but actually part of the sIOD and CCD interfaces, or as well the CCD L3 cache power.
58 Comments
View All Comments
DannyH246 - Thursday, July 1, 2021 - link
lol - no need to be subtle about it. www.IntelTech.com has been doing this for years.Qasar - Thursday, July 1, 2021 - link
hilarious, go back to wccftech then dannywhatthe123 - Friday, June 25, 2021 - link
good god man, the review quite literally posts hard numbers of epyc simply thrashing xeon in performance even on a per core basis, and you think they're worried that intel will retaliate against them if they don't say something nice about one corner of a segment of performance?what is it about technology that attracts cultists?
Threska - Saturday, June 26, 2021 - link
There's a reason the PCMasterRace forum exists.msroadkill612 - Sunday, June 27, 2021 - link
"Cultists" - I like it :)So true, & on many levels. I see strangely neurotic behaviour from such an allegedly smart & rational demographic.
As a group, they are prone to be great at rattling off streams of presumably accurate numbers and jargon, but arrive at childishly naive conclusions, & ask the wrong questions based plain wrong premises.
devione - Sunday, June 27, 2021 - link
Jesus Christ man. Grow some fucking balls. If you're going to call out Anandtech for being biased at least be straightforward and frank. No need to write an essay trying to couch and justify and be obtuse about itMakste - Monday, July 5, 2021 - link
LmaoOxford Guy - Friday, June 25, 2021 - link
What are the RAM timings? I don’t see that information in the charts.Andrei Frumusanu - Saturday, June 26, 2021 - link
These are standardised PC4-3200AA-RB2-12 sticks, running at JEDEC timings.https://www.micron.com/products/dram-modules/rdimm...
Oxford Guy - Monday, June 28, 2021 - link
Thank you. And the other systems tested?