Our Benchmark Methods and Choices
As is traditional now with AMD CPU launches, we got very little time to perform our benchmarks. By the time we were running with the right BIOS and figured out that our Adaptec RAID cards absolutely refused to work with this new BIOS, we had less than a week left to do our server benchmarks which take at least a few hours per setup. So we had to make some choices. Without our Adaptec card, we had to cancel the most disk intensive test we used so far: the transactional DVD Store test. For all other tests, our four local SLC SSD’s kept disk queues more than low enough.
Despite timing constraints, we tried to stay as faithful as we can to our new benchmark methodology. Remember that instead of throwing every software box we happen to have on the shelf, we decided that the “buyers” should dictate our benchmark mix. Basically, every software type that is really important should have at least one and preferably two representatives in the benchmark suite. In the table below you can find an overview of the software types servers are bought for and the benchmarks you may expect in this review. We add the “relevance” column, as “Istanbul” only targets a part of this market. Very few people will buy a hex-core for print, domain controller or mailservers.
| Server Software Market | Importance | Benchmarks Used | Relevance (Six-Core) |
| ERP, OLTP | 10-14% | SAP SD 2-tier (Industry Standard benchmark) Oracle Charbench (Free available benchmark) | High, but not yet published High |
| Reporting, OLAP | 10-17% | MS SQL Server (Real world + vApus) | Very high |
| Collaborative | 14-18% | MS Exchange Loadgen (TBD) | Medium |
| Software Dev. | 7% | Not yet | Medium |
| e-mail, DC, file/print | 32-37% | MS Exchange Loadgen (TBD) | Very Low (not CPU intensive) |
| Web | 10-14% | MCS eFMS (Real World + vApus) | Low |
| HPC | 4-6% | TBD | Only specific dense matrix apps are relevant |
| Other | 2%? | 3dsmax (Our own bench) | Medium |
| Virtualization | 33-50% | VMMark (Industry standard), vApus Mark I | Very High |
Due to time constraints, we decided to postpone the Exchange and Linpack benchmarking. Their relevance for evaluating “Istanbul” is low anyway. SAP benchmarks were not available at the time that we wrote this.
Benchmark Configuration
None of our benchmarks required more than 20 GB. Database files were placed on a 3 drive RAID-0 Intel X25-E SLC 32 GB SSD, log files on one Intel X25-E SLC 32 GB.
Xeon Server 1: ASUS RS700-E6/RS4 barebone
Dual Intel Xeon "Gainestown" X5570 2.93GHz
ASUS Z8PS-D12-1U
6x4GB (24GB) ECC Registered DDR3-1333
NIC: Intel 82574L PCI-E Gbit LAN
Xeon Server 2: Intel "Stoakley platform" server
Dual Intel Xeon E5450 "Harpertown" at 3GHz
Supermicro X7DWE+/X7DWN+
24GB (12x2GB) Crucial Registered FB-DIMM DDR2-667 CL5 ECC
NIC: Dual Intel PRO/1000 Server NIC
Xeon Server 3: Intel "Bensley platform" server
Dual Intel Xeon X5365 "Clovertown" 3GHz
Dual Intel Xeon L5320 at 1.86GHz
Dual Intel Xeon 5080 "Dempsey" at 3.73GHz
Supermicro X7DBE+
24GB (12x2GB) Crucial Registered FB-DIMM DDR2-667 CL5 ECC
NIC: Dual Intel PRO/1000 Server NIC
Opteron Server: Supermicro SC828TQ-R1200LPB 2U Chassis
Dual AMD Opteron 2435 at 2.6GHz
Dual AMD Opteron 8384 at 2.7GHz
Dual AMD Opteron 2222 at 3.0GHz
Dual AMD Opteron 8356 at 2.3GHz
Supermicro H8QMi-2+
24GB (12x2GB) DDR2-800
NIC: Dual Intel PRO/1000 Server NIC
vApus/Oracle Calling Circle Client Configuration
Intel Core 2 Quad Q6600 2.4GHz
Foxconn P35AX-S
4GB (2x2GB) Kingston DDR2-667
NIC: Intel PRO/1000
Facebook
Twitter
RSS
40 Comments
View All Comments
iocedmyself - Wednesday, June 17, 2009 - link
Well something that was failed to be mentioned was that the 2P opteron machine costs about $6700, where as the nehalem 2p machine is very near to $16,000.as for power consumption a straight up comparison would be HP380 Xeon and HP 385 Opteron. At idle, both are 140W. With 100% CPU / Ram, 385 is around 300W, 380 (Xeon) is about 450W.
another thing not discussed here - 4P Istanbul is 70-80% faster than 2P Nehalem, and there is no 4P Nehalem. 8P Istanbul is over 3 times as fast as 2P Nehalem. so until next gen Nehalem, there is no competition in the high end which probably has something to do with istanbul orders being through the roof.
I also have to wonder if these benchmarks were conducted using one of Intel's little helpful optimized compilers.
yasbane - Wednesday, June 10, 2009 - link
would be nice to see some unix or linux benchmarks...riskyburden - Thursday, June 4, 2009 - link
I might be naive here but surely the majority of these applications are favouring clock speed and no more than two cores, should there not be a bench for those companies that run multiple apps such as SQL and AD or IPFX etc all from one server and make a comparison there. I don't suggest it to be good network practice but that would interest me more.mino - Friday, June 5, 2009 - link
For this part of SMB market pretty much any dual core CPU will do.Their bottleneck is almost allways on the storage side, sometimes with insufficient memory.
And most also run default install where basic SW tweaks would make 100's percents in performance.
befair - Wednesday, June 3, 2009 - link
Johan never proves me wrong. Even an article meant to talk about AMD Opteron starts with a good deal of "Intel is the king!" stuff, as usual.alpha754293 - Wednesday, June 3, 2009 - link
What happened to them?I would have to loved to have seen what the new 6-core AMDs would be able to do in this arena since it is (presumably) a much more competitive offering than the fastest Xeons all around.
lopri - Tuesday, June 2, 2009 - link
A Question: Is the 'snoop-filter' a hardware-based? I read that it can be enabled/disabled via BIOS, and since the cores are same as Shanghai cores.. But my question is, whether it's hardware-based or software-based (BIOS), shouldn't this work for inter-core communication as well if AMD decides to implement it?JohanAnandtech - Tuesday, June 2, 2009 - link
I have to check, but I am pretty sure it is both. The "uncore" part has changed somewhat on Istanbul."shouldn't this work for inter-core communication as well if AMD decides to implement it"
Since the L3-cache keeps copies of shared L2-cachelines, I don't think that will help. There is already a very fast way of communicating with little overhead.
tygrus - Monday, June 1, 2009 - link
I would like to know the performance difference when using a cell size of 3 not 6 on the 6-core units or of 8 not 4 on Xeon 4Core8Thread ?Will have to wait for latter for more raw performance numbers (eg. memory local/system, SPEC CPU, task switching, OS/IO task servicing).
How long before they update the boards for DDR3 based memory and better IO onboard ?
It's a pity the ESX 4.0 update hasn't helped AMD .. are the improvements only available for Intel or was it to correct a previous Intel only problem ? What can AMD/partners do to improve performance ?
JohanAnandtech - Tuesday, June 2, 2009 - link
"I would like to know the performance difference when using a cell size of 3 not 6 on the 6-core units?"A cell size of 3 will not do any good if your VMs are MP. Eventhough ESX features "relaxed co-scheduling", there might quite a few cases where the Scheduler is not able to use all "slots" as some of vCPUs of the VMs might be behind. From the momemt you use more than 2 vCPUs, you will get situations where only one VM with 2 CPUs is scheduled on a cell of 3 CPUs. 8-cell: I have to try it.
"How long before they update the boards for DDR3 based memory and better IO onboard ? "
The AMD's Fiorano platform that will be available in a few weeks should have better I/O (PCIe gen 2) but will still be DDR-2 based.
DDR-3 CPUs are scheduled for 2010.
"It's a pity the ESX 4.0 update hasn't helped AMD .. are the improvements only available for Intel or was it to correct a previous Intel only problem ? "
VMware's docs tell us they that CPU locking goes more quickly and that the scheduler is "cache aware", but most of the biggest improvements are EPT and better support for Hyperthreading.