AMD's Thuban Six-Core Phenom II X6 1090T & 1055T Reviewed

by: Anand Lal Shimpi
A very smart man once told me that absolute performance doesn’t matter, it’s performance at a given price point that makes a product successful. While AMD hasn’t held the absolute performance crown for several years now, that doesn’t mean the company’s products haven’t been successful. During the days of the original Phenom, AMD started the trend of offering more cores than Intel at a given price point. Intel had the Core 2 Duo, AMD responded with the triple core Phenom X3.

As AMD’s products got more competitive, the more-for-less approach didn’t change. Today AMD will sell you three or four cores for the price of two from Intel.In some situations, this works to AMD’s benefit. The Athlon II X3 and X4 deliver better performance in highly threaded applications than the Intel alternatives. While Intel has better performance per clock, you can’t argue with more cores/threads for applications that can use them.

When Intel announced its first 6-core desktop processor, the Core i7 980X at $999, we knew a cheaper AMD alternative was coming. Today we get that alternative, this is the Phenom II X6 based on AMD’s new Thuban core.

It’s still a 45nm chip but thanks to architecture and process tweaks, the new Phenom II X6 still fits in the same power envelope as last year’s Phenom II X4 processors: 125W.

Update: AMD tells us that it gave us the wrong pricing on the 1090T. The part sells for $295, not $285, in 1000 unit quantities.

CPU Specification Comparison
Processor	Clock Speed	Max Turbo	L2 Cache	L3 Cache	TDP	Price
AMD Phenom II X6 1090T	3.2GHz	3.6GHz	3MB	6MB	125W	$295
AMD Phenom II X6 1055T	2.8GHz	3.3GHz	3MB	6MB	125W	$199
AMD Phenom II X4 965 BE	3.4GHz	N/A	2MB	6MB	125W/140W	$185
AMD Phenom II X4 955 BE	3.2GHz	N/A	2MB	6MB	125W	$165
AMD Phenom II X4 945	3.0GHz	N/A	2MB	6MB	95W	$155
AMD Phenom II X4 925	2.8GHz	N/A	2MB	6MB	95W	$145

You also don’t give up much clock speed. The fastest Phenom II X6 runs at 3.2GHz, just 200MHz shy of the fastest X4.

When Intel added two cores to Nehalem it also increased the L3 cache of the chip by 50%. The Phenom II X6 does no such thing. The 6 cores have to share the same 6MB L3 cache as the quad-core version.

The Phenom II X6 die. Monolithic, hexa-core

There’s also the issue of memory bandwidth. Intel’s Core i7 980X is paired with a triple channel DDR3 memory controller, more than enough for four cores under normal use and enough for a six core beast. In order to maintain backwards compatibility, the Phenom II X6 is still limited to the same dual channel memory controller as its quad-core predecessor.

CPU Specification Comparison
CPU	Codename	Manufacturing Process	Cores	Transistor Count	Die Size
AMD Phenom II X6 1090T	Thuban	45nm	6	904M	346mm2
AMD Phenom II X4 965	Deneb	45nm	4	758M	258mm2
Intel Core i7 980X	Gulftown	32nm	6	1.17B	240mm2
Intel Core i7 975	Bloomfield	45nm	4	731M	263mm2
Intel Core i7 870	Lynnfield	45nm	4	774M	296mm2
Intel Core i5 670	Clarkdale	32nm	2	384M	81mm2
AMD Phenom II X4 965	Deneb	45nm	4	758M	258mm2

The limitations are nitpicks in the grand scheme of things. While the 980X retails for $999, AMD’s most expensive 6-core processor will only set you back $285 and you can use them in all existing AM2+ and AM3 motherboards with a BIOS update. You're getting nearly 1 billion transistors for $200 - $300. Like I said earlier, it’s not about absolute performance, but performance at a given price point.

AMD 2010 Roadmap
CPU	Clock Speed	Max Turbo (<= 3 cores)	L3 Cache	TDP	Release
AMD Phenom II X6 1090T	3.2GHz	3.6GHz	6MB	125W	Q2
AMD Phenom II X6 1075T	3.0GHz	3.5GHz	6MB	125W	Q3
AMD Phenom II X6 1055T	2.8GHz	3.3GHz	6MB	125W/95W	Q2
AMD Phenom II X6 1035T	2.6GHz	3.1GHz	6MB	95W	Q2
AMD Phenom II X4 960T	3.0GHz	3.4GHz	6MB	95W	Q2

We'll soon see more flavors of the Phenom II X6 as well as a quad-core derivative with 2 of these cores disabled. As a result, motherboard manufacturers are already talking about Phenom II X4 to X6 unlocking tools.

The new Phenom II X6 processors are aimed squarely at Intel’s 45nm Lynnfield CPUs. Both based on a 45nm process, AMD simply offers you more cores for roughly the same price. Instead of a quad-core Core i7 860, AMD will sell you a six-core 1090T. Oh and the T stands for AMD’s Turbo Core technology.

AMD’s Turbo: It Works

In the Pentium 4 days Intel quickly discovered that there was a ceiling in terms of how much heat you could realistically dissipate in a standard desktop PC without resorting to more exotic cooling methods. Prior to the Pentium 4, desktop PCs saw generally rising TDPs for both CPUs and GPUs with little regard to maximum power consumption. It wasn’t until we started hitting physical limits of power consumption and heat dissipation that Intel (and AMD) imposed some limits.

High end desktop CPUs now spend their days bumping up against 125 - 140W limits. While mainstream CPUs are down at 65W. Mobile CPUs are generally below 35W. These TDP limits become a problem as you scale up clock speed or core count.

In homogenous multicore CPUs you’ve got a number of identical processor cores that together have to share the maximum TDP of the processor. If a single hypothetical 4GHz processor core hits 125W, then fitting two of them into the same TDP you have to run the cores at a lower clock speed. Say 3.6GHz. Want a quad-core version? Drop the clock speed again. Six cores? Now you’re probably down to 3.2GHz.

Single Core	Dual Core	Quad Core	Hex Core

This is fine if all of your applications are multithreaded and can use all available cores, but life is rarely so perfect. Instead you’ve got a mix of applications and workloads that’ll use anywhere from one to six cores. Browsing the web may only task one or two cores, gaming might use two or four and encoding a video can use all six. If you opt for a six core processor you get great encoding performance, but worse gaming and web browsing performance. Go for a dual core chip and you’ll run the simple things quickly, but suffer in encoding and gaming performance. There’s no winning.

With Nehalem, Intel introduced power gate transistors. Stick one of these in front of a supply voltage line to a core, turn it off and the entire core shuts off. In the past AMD and Intel only put gates in front of the clock signal going to a core (or blocks of a core), this would make sure the core remained inactive but it could still leak power - a problem that got worse with smaller transistor geometries. These power gate transistors however addressed both active and leakage power, an idle core could be almost completely shut off.

If you can take a single core out of the TDP equation, then with some extra logic (around 1M transistors on Nehalem) you can increase the frequency of the remaining cores until you run into TDP or other physical limitations. This is how Intel’s Turbo Boost technology works. Depending on how many cores are active and the amount of power they’re consuming a CPU with Intel’s Turbo Boost can run at up to some predefined frequency above its stock speed.

With Thuban, AMD introduces its own alternative called Turbo Core. The original Phenom processor had the ability to adjust the clock speed of each individual core. AMD disabled this functionality with the Phenom II to avoid some performance problems we ran into, but it’s back with Thuban.

If half (or more) of the CPU cores on a Thuban die are idle, Turbo Core does the following:

1) Decreases the clock speed of the idle cores down to as low as 800MHz.
2) Increases the voltage of all of the cores.
3) Increases the clock speed of the active cores up to 500MHz above their default clock speed.

The end result is the same as Intel’s Turbo Boost from a performance standpoint. Lightly threaded apps see a performance increase. Even heavily threaded workloads might have periods of time that are bound by the performance of a single thread - they benefit from AMD’s Turbo Core as well. In practice, Turbo Core appears to work. While I rarely saw the Phenom II X6 1090T hit 3.6GHz, I would see the occasional jump to 3.4GHz. As you can tell from the screenshot above, there's very little consistency between the cores and their operating frequencies - they all run as fast or as slow as they possibly can it seems.

AMD's Turbo Core Benefit
AMD Phenom II X6 1090T	Turbo Core Disabled	Turbo Core Enabled	Performance Increase
x264-HD 3.03 1st Pass	71.4 fps	74.5 fps	4.3%
x264-HD 3.03 2nd Pass	29.4 fps	30.3 fps	3.1%
Left 4 Dead	117.3 fps	127.2 fps	8.4%
7-zip Compression Test	3069 KB/s	3197 KB/s	4.2%

Turbo Core generally increased performance between 2 and 10% in our standard suite of tests. Given that the max clock speed increase on a Phenom II X6 1090T is 12.5%, that’s not a bad range of performance improvement. Intel’s CPUs stand to gain a bit more (and use less power) from turbo thanks to the fact that Lynnfield, Clarkdale, et al. will physically shut off idle cores rather than just underclock them.

I have noticed a few situations where performance in a benchmark was unexpectedly low with Turbo Core enabled. This could be an artifact of independent core clocking similar to what we saw in the Phenom days, however I saw no consistent issues in my time with the chip thus far.

The Performance Summary

At $199 and $285 the obvious comparison points are Intel’s Core i5 750 and Core i7 860. We’ll dive into the complete performance tests in a bit, but if you’re looking for some quick analysis here’s what we’ve got.

Single threaded performance is squarely a Lynnfield advantage. Intel’s quad-cores can turbo up more and Intel does have the advantage of higher IPC.

Phenom II X6 vs. Intel's Lynnfield Processors
	Cinebench R10 (Single Threaded)	Cinebench R10 (Multithreaded)	3dsmax r9	x264 HD - 2nd Pass	Left 4 Dead
AMD Phenom II X6 1090T	3951	18526	13.7	28.5 fps	127.2 fps
AMD Phenom II X6 1055T	3547	16268	12.7	25.1 fps	111.5 fps
Intel Core i7 860	4490	16598	15.0	26.8 fps	131.0 fps
Intel Core i5 750	4238	14142	13.4	21.0 fps	130.0 fps

Highly threaded encoding and 3D rendering performance are obviously right at home on the Phenom II X6. The 6MB L3 cache and lower IPC does appear to hamper the Phenom II X6 in a couple of tests but for the most part if you need threads, the X6 is the way to go.

Applications in between generally favor Intel’s quad-cores over the Phenom II X6. This includes CPU-bound games.

None of this should be terribly surprising as it’s largely the same conclusion we came to with the Athlon II X3 and X4. If you run specific heavily threaded applications, you can’t beat the offer AMD is giving you. It’s the lighter or mixed use workloads that tend to favor Intel’s offerings at the same price points.

AMD’s 890FX Chipset

The Phenom II X6 will work in all existing Socket-AM2+ and AM3 motherboards that can 1) support the 125W TDP of the processors, and 2) have BIOS support (apparently over 160 boards at launch). Despite this impressive showing of backwards compatibility, we also get a new chipset today for those of you looking to build a new system instead of upgrade.

The 890FX is a mildly updated version of AMD’s 790FX chipset, mostly adding AMD’s SB850 South Bridge with 6Gbps SATA support. The number of PCIe 2.0 lanes and other major features remains unchanged.

	AMD 890FX	AMD 890GX	AMD 790FX
CPU	AMD Socket-AM3	AMD Socket-AM3	AMD Socket-AM3/AM2+
Manufacturing Process	65nm	55nm	65nm
PCI Express	44 PCIe 2.0 lanes	24 PCIe 2.0 lanes	44 PCIe 2.0 lanes
Graphics	N/A	Radeon HD 4290 (DirectX 10.1)	N/A
South Bridge	SB850	SB850	SB750
USB	14 USB 2.0 ports	14 USB 2.0 ports	12 USB 2.0 ports
SATA	6 SATA 6Gbps ports	6 SATA 6Gbps ports	6 SATA 3Gbps ports
IOMMU	1.2	N/A	N/A
Max TDP	19.6W	25W	19.6W

You get IOMMU support (an advantage over 790FX) and despite the chipset being built on TSMC's 65nm process, it pulls less power than the 890GX as it lacks any integrated graphics.

The Test

To keep the review length manageable we're presenting a subset of our results here. For all benchmark results and even more comparisons be sure to use our performance comparison tool: Bench.

Motherboard:	ASUS P7H57DV- EVO (Intel H57) Intel DP55KG (Intel P55) Intel DX58SO (Intel X58) Intel DX48BT2 (Intel X48) Gigabyte GA-MA790FX-UD5P (AMD 790FX) MSI 890FXA-GD70 (AMD 890FX)
Chipset Drivers:	Intel 9.1.1.1015 (Intel) AMD Catalyst 8.12
Hard Disk:	Intel X25-M SSD (80GB)
Memory:	Corsair DDR3-1333 4 x 1GB (7-7-7-20) Corsair DDR3-1333 2 x 2GB (7-7-7-20)
Video Card:	eVGA GeForce GTX 280 (Vista 64) ATI Radeon HD 5870 (Windows 7)
Video Drivers:	ATI Catalyst 9.12 (Windows 7) NVIDIA ForceWare 180.43 (Vista64) NVIDIA ForceWare 178.24 (Vista32)
Desktop Resolution:	1920 x 1200
OS:	Windows Vista Ultimate 32-bit (for SYSMark) Windows Vista Ultimate 64-bit Windows 7 x64

SysMark 2007 Performance

Our journey starts with SYSMark 2007, the only all-encompassing performance suite in our review today. The idea here is simple: one benchmark to indicate the overall performance of your machine.

SYSMark really taxes two cores most of the time, giving the edge to Lynnfield and its aggressive turbo modes. Lightly threaded or mixed workloads won't do so well on the Phenom II X6.

Adobe Photoshop CS4 Performance

To measure performance under Photoshop CS4 we turn to the Retouch Artists’ Speed Test. The test does basic photo editing; there are a couple of color space conversions, many layer creations, color curve adjustment, image and canvas size adjustment, unsharp mask, and finally a gaussian blur performed on the entire image.

The whole process is timed and thanks to the use of Intel's X25-M SSD as our test bed hard drive, performance is far more predictable than back when we used to test on mechanical disks.

Time is reported in seconds and the lower numbers mean better performance. The test is multithreaded and can hit all four cores in a quad-core machine.

Performance here is good, but even Photoshop doesn't make consistent enough use of all six cores to really give the Phenom II X6 the edge it needs here. It's faster than the Phenom II X4, but not faster than the Core i5 750.

DivX 6.8.5 with Xmpeg 5.0.3

Our DivX test is the same DivX / XMpeg 5.03 test we've run for the past few years now, the 1080p source file is encoded using the unconstrained DivX profile, quality/performance is set balanced at 5 and enhanced multithreading is enabled.

Thanks to AMD's Turbo Core the Phenom II X6 is pretty close here, but still not able to topple Intel's Core i5 and i7.

x264 HD Video Encoding Performance

Graysky's x264 HD test uses x264 to encode a 4Mbps 720p MPEG-2 source. The focus here is on quality rather than speed, thus the benchmark uses a 2-pass encode and reports the average frame rate in each pass.

And we finally see the Phenom II X6 flex its muscle, even the 1055T is faster than the Core i7 860:

In the actual encoding pass the 1055T falls behind the 860 but it's still a good 19% faster than the Core i5 750.

3dsmax 9 - SPECapc 3dsmax CPU Rendering Test

Today's desktop processors are more than fast enough to do professional level 3D rendering at home. To look at performance under 3dsmax we ran the SPECapc 3dsmax 8 benchmark (only the CPU rendering tests) under 3dsmax 9 SP1. The results reported are the rendering composite scores.

Not all heavily threaded workloads will show the Phenom II X6 in a good light. Here Intel maintains the advantage:

Cinebench R10

Created by the Cinema 4D folks we have Cinebench, a popular 3D rendering benchmark that gives us both single and multi-threaded 3D rendering results.

Single threaded performance is obviously an Intel advantage, but crank up the thread count and there's no match for the Phenom II X6. As we pointed out earlier, if you've got a lot of CPU intensive threads there's no replacement for more cores.

POV-Ray 3.73 beta 23 Ray Tracing Performance

POV-Ray is a popular, open-source raytracing application that also doubles as a great tool to measure CPU floating point performance.

I ran the SMP benchmark in beta 23 of POV-Ray 3.73. The numbers reported are the final score in pixels per second.

Once again, the Phenom II X6 does very well here.

WinRAR - Archive Creation

Our WinRAR test simply takes 300MB of files and compresses them into a single RAR archive using the application's default settings. We're not doing anything exotic here, just looking at the impact of CPU performance on creating an archive.

The i7 860 wins against the 1090T, but the lack of Hyper threading keeps the 750 behind the Phenom II X6 1055T.

7-Zip Performance

Gaming Performance

None of the games here can take advantage of more than 4 cores. The Phenom II X6 ends up performing no different than a Phenom II X4. Thankfully due to Turbo Core you rarely see a drop in performance compared to the Phenom II X4 965.

If you want the better gaming chip, you want Lynnfield.

Power Consumption

Most impressive is AMD's ability to run six 45nm cores at the same power consumption as four 45nm cores. The Phenom II architecture in general does reasonably well at idle, but without power gating AMD can't compete with Intel's idle power levels.

Under load Intel also has the clear advantage.

Overclocking

The Phenom II X6 1090T is a Black Edition part, meaning it has a fully unlocked clock multiplier. With very little effort our 3.2GHz sample was up and running at 3.80GHz without any additional cooling beyond the stock heatsink/fan.

With a little extra effort, 3.9GHz should be possible, but the fact that we can even run at 3.8GHz with six 45nm cores is very impressive. Update: You asked, and we pushed harder. Our 1090T sample can hit 4GHz at 1.45V and even reach 4.1GHz but not with great stability. The even more important takeaway is that AMD's 64-bit/4GHz limits appear to be gone with Thuban.

Final Words

Today's conclusion is no different than what we've been saying about AMD's CPU lineup for several months now. If you're running applications that are well threaded and you're looking to improve performance in them, AMD generally offers you better performance for the same money as Intel. It all boils down to AMD selling you more cores than Intel at the same price point.

Applications like video encoding and offline 3D rendering show the real strengths of the Phenom II X6. And thanks to Turbo Core, you don't give up any performance in less threaded applications compared to a Phenom II X4. The 1090T can easily trump the Core i7 860 and the 1055T can do even better against the Core i5 750.

You start running into problems when you look at lightly threaded applications or mixed workloads that aren't always stressing all six cores. In these situations Intel's quad-core Lynnfield processors (Core i5 700 series and Core i7 800 series) are better buys. They give you better performance in these light or mixed workload scenarios, not to mention lower overall power consumption.

The better way to look at it is to ask yourself what sort of machine you're building. If you're building a task specific box that will mostly run heavily threaded applications, AMD will sell you nearly a billion transistors for under $300 and you can't go wrong. If it's a more general purpose machine that you're assembling, Lynnfield seems like a better option.

Read More..

AMD's Opteron 6174 12 cores "Magny-Cours"

source: anandtech.com
If the Westmere Xeon EP were a car engine, it would've been made by Porsche. With "only" six cores, each core in the new Xeon offers almost twice the performance of the competition. A 32nm CPU that only occupies 248 mm2 the Westmere Xeon EP embodies pure refinement and intelligent performance, both Porsche traits. It's just made in Portland, not Zuffenhausen.

AMD's offering today is very different. Magny-cours is the CPU version of the American muscle car. It's a brutally large 12-core CPU: two dies, each measuring 346mm2 connected by a massive 24 link Hyper Transport pipe. AMD's Magny-cours Opteron has almost two billion transistors and 19.6MB of cache on-die.

12 cores, 692 mm² die, 19.6MB of cache on-die

It's not all raw horsepower though. At 2.2GHz this 12-core monster is supposed to be content with only 80 precious watts, and 115W at most. HT assist also makes an appearance to keep CPU-CPU accesses to a necessary minimum, a problem that could get out of hand with 12 cores otherwise. AMD originally added HT assist with its first 6-core Opterons. So Magny-Cours is a like hybrid V12 Dodge Viper with traction control. Will this cocktail of raw core muscle and energy savings be enough to beat the competitor from Portland?

For once we could not resist the temptations of car analogies. As interesting as we found the Xeon Westmere EP, something was missing: a challenger, a competitor to make things more exiting. In the last review, we just knew that the Xeon X5670 would crush the competition. This time is going to be close. AMD still won’t have a chance if your application does not scale well with extra cores. In that case you are better off with the higher clocked and better per-core performance of the Intel CPUs. But it is unclear if Intel will prevail in truly multi-threaded software now that a grim and determined AMD is willing to offer two CPUs for the price of one just to win the race.

Magny-Cours

You probably heard by now that the new Opteron 6100 is in fact two 6-core Istanbul CPUs bolted together. That is not too far from the truth if you look at the micro architecture: little has changed inside the core. It is the “uncore” that has changed significantly: the memory controller now supports DDR-1333, and a lot of time has been invested in keeping cache coherency traffic under control. The 1944-pin (!) organic Land Grid Array (LGA) Multi Chip Module (MCM) is pictured below.

The red lines are memory channels, blue lines internal HT cache coherent connects. The gray lines are external cache HT connections, while the green line is a simple non coherent I/O HT connect.

Each CPU has two DDR-3 channels (red lines). That is exactly the strongest point of this MCM: four fast memory channels that can use DDR-1333, good for a theoretical bandwidth peak of 42.7 GB/s. But that kind of bandwidth is not attainable, not even in theory bBecause the next link in the chain, the Northbridge, only runs at 1.8GHz. We have two 64-bit Northbridges both working at 1.8 GHz, limiting the maximum bandwidth to 28.8 GB/s. That is price AMD’s engineers had to pay to keep the maximum power consumption of a 45nm 2.2 GHz below 115W (TDP).

Adding more cores makes the amount of snoop traffic explode, which can easily result in very poor scaling. It can get worse to the point where extra cores reduce performance. The key technology is HT assist, which we described here. By eliminating unnecessary probes, local memory latency is significantly reduced and bandwidth is saved. It cost Magny-cours 1MB of L3-cache per core (2MB total), but the amount of bandwidth increases by 100% (!) and the latency is reduced to 60% of it would be without HT-assist.

Even with HT-assist, a lot of probe activity is going on. As HT-assist allows the cores to perform directed snoops, it is good to reach each core quickly. Ideally each Magny-cours MCM would have six HT3 ports. One for I/O with a chipset, 2 per CPU node to communicate with the nodes that are off-package and 2 to communicate very quickly between the CPU nodes inside the package. But at 1944 pins Magny-Cours probably already blew the pin budget, so AMD's engineers limited themselves to 4 HT links.

One of the links is reserved for non coherent communication with a possible x16 GPU. One x16 coherent port communicates with the CPU that is the closest, but not on the same package. One port is split in two x8 ports. The first x8 port communicates with the CPU that is the farthest away: for example between CPU node 0 and CPU node 3. The remaing x16 and x8 port are used to make communication on the MCM as fast as possible. Those 24 links connect the two CPU nodes on the package.

The end result is that a 2P configuration allows fast communication between the four CPU nodes. Each CPU node is connected directly (one hop) with the other one. Bandwidth between CPU node 0 and 2 is twice than that of P0 to P3 however.

Whilte it looks like two Istanbuls bolted together, what we're looking at is the hard work of AMD's engineers. They invested quite a bit of time to make sure that this 12 piston muscle car does not spin it’s wheels all the time. Of course if the underground is wet (badly threaded software), that will still be the case. And that'll be the end of our car analogies...we promise :)

The SKUs

Let's see what Intel and AMD are offering.

Intel Xeon Model	Cores	TDP	Clock Speed	Price	AMD Opteron Model	Cores	TDP	GHz	Price
Intel Xeon W5680	6	130W	3.3GHz	$1663	AMD Opteron 6176 SE	12	105/137W	2.3 GHz	$1386
Intel Xeon X5670	6	95W	2.93GHz	$1440
Intel Xeon X5660	6	95W	2.80GHz	$1219	AMD Opteron 6174	12	80/115W	2.2 GHz	$1165
Intel Xeon X5650	6	95W	2.66GHz	$996	AMD Opteron 6172	12	80/115W	2.1 GHz	$989
Intel Xeon X5677	4	130W	3.46GHz	$1663	AMD Opteron 2439SE	6	105/137W	2.8 GHz	?
Intel Xeon X5667	4	95W	3.06GHz	$1440
					AMD Opteron 6168	12	80/115W	1.9 GHz	$744
Intel Xeon E5640	4	80W	2.66GHz	$744	AMD Opteron 6136	8	80/115W	2.4 GHz	$744
Intel Xeon E5630	4	80W	2.53GHz	$551	AMD Opteron 6134	8	80/115W	2.3 GHz	$523
Intel Xeon E5620	4	80W	2.40GHz	$387	AMD Opteron 6128	8	80/115W	2.0 GHz	$266
Intel Xeon L5640	6	60W	2.26GHz	$996	AMD Opteron 6164 HE	12	65/? W	1.7 GHz	$744
Intel Xeon L5630	4	40W	2.13GHz	$551	AMD Opteron 6128 HE	8	65/? W	2.0 GHz	$523
Intel Xeon L5620	4	40W	1.86GHz	$440	AMD Opteron 6124 HE	8	65/? W	1.8 GHz	$455

The 6176 looks a bit ridiculous as it delivers only 4% more performance at 30% higher power and 20% higher prices. The real reason behind this CPU is to battle another tanker, the Nehalem EX that Intel is going to launch tomorrow. The TDP and clockspeeds of that huge chip are very similar. If your application scales poorly and you don't care about power consumption, the X5677 is your champion; it is probably the fastest chip on the market for applications with low thread counts.

The most interesting parts that AMD offers are the dodeca-core 6174 (2.2GHz), the octal-core 6136 (2.4GHz) and the octal-core low power 6128 (2.0GHz). The 6174 targets those with well scaling multi-threaded applications such as huge databases and virtualized loads. The 8-core 6136 might even be better as most schedulers find it easier to distribute threads and process over a power of 2 cores. Lots of applications also don't scale beyond 16 cores and the chip comes with a 200MHz clockspeed bonus and a very reasonable price.

The 6128 HE is also an interesting one. The 6128 HE might be a good way to reconcile low response times with low power, but we'll have to find that out later.

Benchmark Methods and Systems

First of all, I like to offer my thanks to my colleague Tijl Deneut who helped me out with the complex virtualization benchmarks.

None of our benchmarks required more than 20 GB of RAM. Database files were placed on a two drive RAID-0 Intel X25-E SLC 32GB SSD, with log files on one Intel X25-E SLC 32GB. Adding more drives improved performance by only 1%, so we are confident that storage is not our bottleneck.

Xeon Server 1: ASUS RS700-E6/RS4 barebone
Dual Intel Xeon "Gainestown" X5570 2.93GHz, Dual Intel Xeon “Westmere” X5670 2.93 GHz
ASUS Z8PS-D12-1U
6x4GB (24GB) ECC Registered DDR3-1333
NIC: Intel 82574L PCI-EGBit LAN
PSU: Delta Electronics DPS-770 AB 770W

Opteron Server 1 (Dual CPU): AMD Magny-Cours Reference system (desktop case)

Dual AMD Opteron 6174 2.2 GHz
AMD Dinar motherboard with AMD SR5690 Chipset & SB750 Southbridge
8x 4 GB (32 GB) ECC Registered DDR3-1333
NIC: Broadcom Corporation NetXtreme II BCM5709 Gigabit
PSU: 1200W PSU

Opteron Server 2 (Dual CPU): Supermicro A+ Server 1021M-UR+V
Dual Opteron 2435 "Istanbul" 2.6GHz
Dual Opteron 2389 2.9GHz
Supermicro H8DMU+
32GB (8x4GB) DDR2-800
PSU: 650W Cold Watt HE Power Solutions CWA2-0650-10-SM01-1

vApus/Oracle Calling Circle Client Configuration

First client (Tile one)
Intel Core 2 Quad Q9550 2.83 GHz
Foxconn P35AX-S
GB (2x2GB) Kingston DDR2-667
NIC: Intel PRO/1000

Second client (Tile two)
Single Xeon X3470 2.93GHz
S3420GPLC
Intel 3420 chipset
8GB (4 x 2GB) 1066MHz DDR3

Understanding the Performance Numbers

As Intel and AMD are adding more and more cores to their CPUs, we encounter two main challenges to keep these CPUs scaling. Cache coherency messages can add a lot of latency and absorb a lot of bandwidth, and at the same time all those cores require more and more bandwidth. So the memory subsystem plays an important role. We still use our older stream binary. This binary was compiled by Alf Birger Rustad using v2.4 of Pathscale's C-compiler. It is a multi-threaded, 64-bit Linux Stream binary. The following compiler switches were used:

-Ofast -lm -static -mp

We ran the stream benchmark on SUSE SLES 11. The stream benchmark produces 4 numbers: copy, scale, add, triad. Triad is the most relevant in our opinion, it is a mix of the other three.

The new DDR3 memory controller gives the Opteron 6100 series wings. Compared to the Opteron 2435 which uses DDR-2 800, bandwidth has increased by 130%. Each core gets more bandwidth, which should help a lot of HPC applications. It is a pity of course that the 1.8 GHz Northbridge is limiting the memory subsystem. It would be interesting to see 8-core versions with higher clocked northbridges for the HPC market.

Also notice that the new Xeon 5600 handles DDR3-1333 a lot more efficiently. We measured 15% higher bandwidth from exactly the same DDR3-1333 DIMMs compared to the older Xeon 5570.

The other important metric for the memory subsystem is latency. Most of our older latency benchmarks (such as the latency test of CPUID) are no longer valid. So we turned to the latency test of Sisoft Sandra 2010.

	Speed (GHz)	L1 (Clocks)	L2 (Clocks)	L3 (Clocks)	Memory (ns)
Intel Xeon X5670	2.93GHz	4	10	56	87
Intel Xeon X5570	2.80GHz	4	9	47	81
AMD Opteron 6174	2.20GHz	3	16	57	98
AMD Opteron 2435	2.60GHz	3	16	56	113

With Nehalem, Intel increased the latency of the L1 cache from 3 cycles to 4. The tradeoff was meant to allow for future scaling as the basic architecture evolves. The Xeons have the smallest (256 KB) but the fastest L2-cache. The L3-cache of the Xeon 5570 is the fastest, but the latency advantage has disappeared on the Xeon X5670 as the cache size increased from 8 to 12 MB.

Interesting is also the fact that the move from DDR2-800 to DDR3-1333 has also decreased the latency to the memory system by about 15%. There's nothing but good news for the 12-core Opteron here: more bandwith and lower latency access per core.

Rendering: Cinebench 11.5

The old Cinebench 10 benchmark was limited to 16 threads. Luckily, the new Cinebench 11.5 does not have that limitation. Cinebench only represents a very small part of the 3D Animation market, but the advantage is that this a benchmark that you can perform at home too.

Although the Opteron 6174 manages to stay close to the newest Xeon, the Xeon is the CPU to get. The reason is that the performance difference will grow as you are rendering smaller and less complex scenes. In those cases, the percentage of the serial code will increase. And Amdahl’s law is unrelenting: in that case the CPU with the highest single threaded performance will win. You also get the benefit of higher single threaded performance when you are modeling.

Rendering: Blender 2.5 Alpha 2

Blender 2.5 Alpha 2
Operating System	Windows 2008 Enterprise R2 (64-bit)
Software	Blender 2.5 Alpha 2
Benchmark software	Built-in render engine

3dsmax 2010 crashed on almost all our servers. Granted, it is not meant to be run on a server but on a workstation. We’ll try some tests with Backburner later when the 2011 version is available. In the meantime, it is time for something less bloated and especially less expensive: Blender.

Blender has been getting a lot of positive attention and judging by its very fast growing community it is on its way to become one of the most popular 3D animation packages out there. The current stable version 2.49 can only render up to 8 threads. Blender 2.5 alpha 2 can go up to 64. To our surprise, the software was pretty stable, so we went ahead and started testing.

If you like, you can perform this benchmark very easily too. We used the “metallic robot”, a scene with rather complex lighting (reflections!) and raytracing. To make the benchmark more repetitive, we changed the following parameters:

1. The resolution was set to 2560 x 1600
2. Anti-alias was set to 16
3. We disabled compositing in post processing
4. Tiles were set to 8x8 (X=8, Y=8)
5. Threads was set to auto (one thread per CPU is set).

Let us first check out the results on Windows 2008 R2:

At first the Opteron 6174 results were simply horrible: 44.6 seconds, slower than the dual Opteron six-core!

Ivan Paulos Tomé, the official maintainer of the Brazilian Blender 3D Wiki, gave us some interesting advice. The default number of tiles is apparently set of 5x5. This result in a short period of 100% CPU load on the Opteron 6174 and a long period where the CPU load drops below 30%. We first assumed that 8x6, two times as many tiles as the number of CPUs would be best. After some experimenting, we found that 8x8 is the best for all machines. The Xeons and six-core Opterons gained 10%, while the 12-core Opteron became 40% (!) faster. This underlines that the more cores you have, the harder they are to make good use of.

Blender can be run on several operating systems, so let us see what happens under 64 bit Linux (Suse SLES 11).

Rendering: Blender 2.5 Alpha 2 on SLES 11

Blender 2.5 Alpha 2
Operating System	SUSE SLES 11, Linux Kernel 2.6.27.19-5-default SMP
Software	Blender 2.5 Alpha 2
Benchmark software	Built-in render engine

What happened here? Not only is Blender 50 to 70% faster on Linux, the tables have turned. As the software is still in Alpha 2 phase, it is good to take the results with a grain of salt, but still. For some reason, the Linux version is capable of keeping the cores fed much longer. On Windows, the first half of the benchmark is spent at 100% CPU load, and then it quickly goes down to 75, 50 and even 25% CPU load. In Linux, the CPU load, especially on the Opteron 6174 stays at 99-100% for much longer.

So is the Opteron 6174 the one to get? We are not sure. If these benchmarks are still accurate when we test with the final 2.5 version, there is a good chance that the octal-core 6136 2.4 GHz will be the Blender champion. It has a much lower price and slightly higher performance per core for less complex rendering work. We hope to follow up with new benchmarks. It is pretty amazing what Blender does with a massive number of cores. At the same time, we imagine Intel's engineers will quickly find out why the blender engine fails to make good use of the the dual Xeon X5670's 24 logical cores. This is far from over yet…

OLTP benchmark Oracle Charbench “Calling Circle”

Oracle Charbench Calling Circle
Operating System	Windows 2008 Enterprise Edition (64-bit)
Software	Oracle 10g Release 2 (10.2) for 64-bit Windows
Benchmark software	Swingbench/Charbench 2.2
Database Size:	9GB

Calling Circle is an Oracle OLTP benchmark. We test with a database size of 9 GB. To reduce the pressure on our storage system, we increased the SGA size (Oracle buffer in RAM) to 10 GB and the PGA size was set at 1.6 GB. A calling circle tests consists of 83% selects, 7% inserts and 10% updates. The “calling circle” test is run for 10 minutes. A run is repeated 6 times and the results of the first run are discarded. The reason is that the disk queue length is sometimes close to 1, while the subsequent runs have a DQL (Disk Queue Length) of 0.2 or lower. In this case it was rather easy to run the CPUs at 99% load. Since DQLs were very similar, we could keep our results from the Nehalem article.

As we noted in our previous article, we work with a relatively small database. The result is that the benchmark doesn't scale well beyond 16 cores. The Opteron 6174 has a 10MB L3 cache for 12 cores, while the Opteron 2435 has 6MB L3 for 6 cores. The amount of cache might explain why the Intel Xeons scale a lot better in this benchmark. For this kind of OLTP workload is the Opteron 6174 not the right choice. To go back to the car analogy earlier: the muscle car is burning rubber while spinning its wheels, but is not making much progress.

SAP S&D 2-Tier

SAP S&D 2-Tier
Operating System	Windows 2008 Enterprise Edition
Software	SAP ERP 6.0 Enhancement package 4
Benchmark software	Industry Standard benchmark version 2009
Typical error margin	Very low

The SAP SD (sales and distribution, 2-tier internet configuration) benchmark is an interesting benchmark as it is a real world client-server application. We decided to take a look at SAP's benchmark database. The results below all run on Windows 2003 Enterprise Edition and MS SQL Server 2005 database (both 64-bit). Every 2-tier Sales & Distribution benchmark was performed with SAP's latest ERP 6 enhancement package 4. These results are NOT comparable with any benchmark performed before 2009. The new 2009 version of the benchmark produces scores that are 25% lower. We analyzed the SAP Benchmark in-depth in one of our earlier articles. The profile of the benchmark has remained the same:

• Very parallel resulting in excellent scaling
• Low to medium IPC, mostly due to "branchy" code
• Somewhat limited by memory bandwidth
• Likes large caches (memory latency!)
• Very sensitive to sync ("cache coherency") latency

(*) Estimate

The last time we discussed the SAP S&D 2-tier benchmark, we had to estimate the Xeon X5670 results. Since then HP has benchmarked its latest G6 servers, giving us results for the X5670. The performance is nothing short of astonishing. The dual Xeon X5670 outperforms a quad Opteron 8345 at 2.6 GHz. The Magny-Cours Opteron can only compete based on its somewhat lower price. We doubt that the SAP buyers care about a few hundred dollars though. A quad Opteron 6174 might have a chance against the Nehalem EX performance wise, but the SAP market will probably prefer the extensive RAS list of the Xeon Nehalem EX. The ERP market is most likely going to be dominated by Intel based servers.

Decision Support benchmark: Nieuws.be

Decision Support benchmark Nieuws.be
Operating System	Windows 2008 Enterprise RTM (64 bit)
Software	SQL Server 2008 Enterprise x64 (64 bit)
Benchmark software	vApus + real-world "Nieuws.be" Database
Database Size	> 100GB
Typical error margin	1-2%

The Flemish/Dutch Nieuws.be site is one of the newest web 2.0 websites, launched in 2008. It gathers news from many different sources and allows the reader to completely personalize his view on all this news. Needles to say, the Nieuws.be site is sitting of top of a pretty large database, more than 100 GB and growing. This database consists of a few hundred separate tables, which have been carefully optimized by our lab (the Sizing Server Lab).

Almost all of the load on the database are selects (99%), about 5% of them are stored procedures. Network traffic averages 6.5MB/s and peaks at 14MB/s. So our Gigabit network connection has still a lot of headroom. Disk Queue Length (DQL) is at 2 in the first round of tests, but we only report the results of the subsequent rounds where the database is in a steady state. We measured a DQL close to 0 during these tests, so there is no tangible intervention of the harddisks.

We now use an new even heavier log. As the Nieuws.be application became more popular and more complex, the database has grown and queries have become more complex too. The results are no longer comparable to previous results. They are similar, but much lower.

Pretty amazing performance here. And while AMD gets a pat on the back, it is the hard working people of Microsoft SQL Server team we should send our kudos to. Our calculations show that SQL Server adds about 80% of performance when adding an extra 12 cores, which is simply awesome scaling. The result of this scaling is that for once, you can notice which CPUs have real cores vs. ones that have virtual (Hyper Threading) cores: the 12-core Opteron 6174 outperforms the best Xeon by 20%. The people with transaction databases should go for the Intel CPUs, while the data miners should consider the latest Opteron. The architectures that AMD and Intel have chosen are complete opposites, and the result is that the differences between the different software categories are very dramatic. Profile your software before you make a choice! It has never been so important.

Virtualization & Consolidation

VMmark - which we discussed in great detail here - tries to measure typical consolidation workloads: a combination of a light mail server, database, fileserver, and website with a somewhat heavier java application. One VM is just sitting idle, representative of workloads that have to be online but which perform very little work (for example, a domain controller). In short, VMmark goes for the scenario where you want to consolidate lots and lots of smaller apps on one physical server.

(*) preliminary benchmark data

Cisco has produced the first VMmark score for the Xeon X5600 series. The Cisco server with two X5680s at 3.3GHz achieved an impressive 35.83 score with 26 VMmark tiles. Twenty-six tiles, that is good for 156 VMs! Based on this number we can estimate where the Xeon X5670 will land. The 6174 numbers are based on AMD’s own preliminary data.

VMmark is a clear victory for the Intel CPUs. Contrary to the SAP market, AMD can play the pricing card here. As long as you do not require dynamic resource scheduling, the software licences costs are nowhere like those of typical ERP projects. So the pricing of the hardware matters more. Also, contrary to other applications, there is no bonus for single threaded performance. The usage models of Databases, 3D Animation software and other all include scenarios where a number of cores will be idling while the others are working very hard. In a virtualization scenario where you are running tons of VMs, single threaded performance does not matter. So while Intel is clearly winning here, servers based on the newest Opteron might still be on the shortlist of those looking for good performance per dollar.

vApus Mark I: Performance-Critical Applications Virtualized

Our vApus Mark I benchmark is not a VMmark replacement. It is meant to be complimentary: while VMmark uses runs 60 to 120 light loads, vApus Mark I runs 8 heavy VMs on 24 virtual CPUs (vCPUs). Our current vApus Stressclient is being improved to scale to much higher amount of vCPUs, but currently we limit the benchmark to 24 virtual CPUs.

A vApus Mark I tile consists of one OLTP, one OLAP and two heavy websites are combined in one tile. These are the kind of demanding applications that still got their own dedicated and natively running machine a year ago. vApus Mark I shows what will happen if you virtualize them. If you want to fully understand our benchmark methodology: vApus Mark I has been described in great detail here. We have changed only one thing compared to our original benchmarking: we used large pages as it is generally considered as a best practice (with RVI, EPT).

The current vApus Mark I uses two tiles. Per tile we have 4 VMs with 4 server applications:

• A SQL Server 2008 x64 database running on Windows 2008 64-bit, stress tested by our in-house developed vApus test (4 vCPUs).
• Two heavy duty MCS eFMS portals running PHP, IIS on Windows 2003 R2, stress tested by our in house developed vApus test (each 2 vCPUs).
• One OLTP database, based on Oracle 10G Calling Circle benchmark of Dominic Giles (4 vCPUs).

The beauty is that vApus (stress testing software developed by the Sizing Servers Lab) uses actions made by real people (as can be seen in logs) to stress test the VMs, not some benchmarking algorithm.

Update: we have noticed that the CPU load of Magny-cours is at 70-85%, while the Six-core "Istanbul" is running at 80-95%". As we have noted before, 24 cores is at the limit of our current benchmark until we launch vApus Mark 2. We have reason to believe that the opteron 6174 has quite a bit of headroom left. The results above are not wrong, but do not show the full potential of the 6174. We are checking the CPU load numbers of the six-core Xeon X5670 as we speak. Expect an update in the coming days.

The AMD Opteron 6174 performs well here, but disappoints a bit at the same time. vApus Mark I does not scale as well as VMmark. The reason is simple: as we used 4 virtual CPUs for both the OLTP as the OLAP virtual machine, scaling depends more on the individual applications. One VM with 4 virtual CPUs will not scale as well as 16 VMs sharing the same 4 virtual CPUs. Also, we use heavy database applications that typically like a decent amount of cache. The difference with the Xeon X5670 is small though. Servers based on both CPUs will make excellent virtualization platforms.

Next, the same test with Hyper-V, the hypervisor beneath Windows 2008 R2. We are testing with Hyper-V R2 6.1.7600.16385 (21^st of July 2009).

Based on the excellent results of the Dual Opteron 2435 we expected AMD to take the crown in this benchmark, but that did not happen. We only had one week to get all of the Opteron testing done (AMD didn't have any hardware until the last minute), so we could not analyze this in depth. For some reason, the Opteron 6174 does not scale very well in our vApus benchmark. Compared to a 2.2GHz six-core, we only see a 30% increase in performance, about the same as Intel gets out of adding 2 extra cores to their Xeon. Part of the reason might be our benchmark: at the moment we are limited to 24 CPUs. We’ll investigate this in more detail in the coming quarter when vApus v2 is available.

The difference with the Xeon X5670 is small though, and the slightly lower price of the Opteron makes up for the slightly lower performance.

HPC and Encryption Benchmarks

Just a few days prior to today's launch, we were able to get access to the benchmark numbers that Intel and AMD produced for LSDyna’s (Crash simulation) and Fluent (fluid dynamics) from Ansys. The first benchmark is the Ansys Fluent Truck_14 m benchmark.

The next one is LS Dyna “Neon refined revised”.

In both cases, the four memory channels and 12 core mix per CPU seem to pay off: AMD can beat Intel again in the HPC benchmarks, although the advantage is small.

Next we ran Sisoft Sandra 2010's encryption benchmark. Do remember that this is a completely synthetic benchmark. A 100% encryption performance advantage might translate in a very small performance advantage in a real world application. For example the code run on a website might only include a small part of encryption code.

Once the Xeon X5670's AES instructions can do their work, encryption is lightening fast. Here the new Xeon is 19 times faster than its older brother and 9 times faster than the best Opteron. Encryption can be broken up easily in smaller parts, it scales extremely well. The result is that the CPU with the most threads, the Xeon 5670 and Opteron 6174 easily outperform their older brothers in cryptographic hash functions.

Power Consumption

The Magny-Cours Opteron arrived one week ago, which is barely enough time to do virtualization benchmarking. So we have to postpone extensive power testing to a later date. The Opteron 6174 came in a desktop reference system which is in no way comparable to our Xeon X5670 1U server. We do have an six-core Opteron based system which is very similar to the Opteron 6174 reference system: the motherboard is also equipped with the new AMD SR5670 chipset and housed in the same desktop system. We can tell you that the idle power of the Opteron 6174 is a few watts lower than the six-core Opteron 2435. Both throttle back to 800 MHz, but the Opteron 6100 series gets a real C1E mode.

C1E mode can only be entered if all CPUs are idle. In a dual socket system, both CPUs enter C1E or they don’t. C1E mode is entered only after longer periods of inactivity. All cores flush their L1 and L2 caches to the L3-cache. Then all cores are clockgated (C1). Once that happens, the Hyper Transport links are put in a lower power state. This allows the chipset to enter a lower power state as well. Only when all these previous steps are done, both sockets are in C1E. DMA events will make the sockets go out of the C1E state. So C1E probably won't happen much on server systems. The C1E state is only entered if absolutely no processing is happening at all.

The C1E mode can reduce power quite a bit:

• Core clocks are turned off (Clockgate C1 state)
• L3, North Bridge, and memory controller all divide their clock frequencies (but are not clockgated!)
• All HyperTranspor links transition to LS2 low power state (LDT_STOP_L)
• DRAM DLL’s disabled
• Memory Transitions from precharge power down mode to self refresh mode (low power)

According to AMD, at full load a 1.7GHz 65W ACP Opteron 6164 HEwould consume about 4% more power than a 2.1 GHz 55W ACP 6-core Opteron 2425 HE. AMD measured 225W for the former, 215W for the latter. We measured 263W on the same system at full load with an Opteron 6174. That's 48W more, or about 24W per CPU. Assuming that the low power CPUs were running at their ACP (65W), we can conclude that the 2.2 GHz Magny-Cours needs about 89W. While the new twelve-core Opteron clearly needs a bit more power than the six-core Opteron, it's not a dramatic increase.

Final Words

The beancounters will probably point out that AMD’s strategy of bolting two CPU dies at 346 mm² together is quite costly. But this is the server CPU market, margins are quite a bit higher. Let AMD worry about the issue of margins. If AMD is willing to sell us - IT professionals - two CPUs for the price of one, we will not complain. It means that the fierce competitive market is favoring the customer. The bottom line is: is this twelve-core Opteron a good deal? For users waiting to use it in a workstation we have our doubts. You’ll benefit from the extra cores when rendering complex scenes, but in all other scenarios (quick simple rendering, modeling) the higher clocked and higher IPC Xeon X5600 series is simply the better choice.

Applications based on transactional databases (OLTP and ERP) are also better off with new Xeon. The SAP and our own Oracle Calling Circle benchmark all point in the same direction. Intel has a tangible performance advantage in both benchmarks.

Data mining applications clearly benefit from having “real” instead of “logical” cores. For datamining, we believe the 12-core Opteron is the clear winner. It offers 20% better performance at 20% lower prices, a good deal if you ask us. Intel’s relatively high prices for its six-core are challenged. The increased competition turns this into a buyers market again.

And then there is the most important segment: the virtualization market. We estimate that the new Opteron 6174 is about 20% slower than the Xeon 5670 in virtualized servers with very high VM counts. The difference is a lot smaller in the opposite scenario: a virtualized server with a few very heavy VMs. Here the choice is less clear. At this point, we believe both server CPUs consume about the same power, so that does not help either to make up our minds. It will depend on how the OEMs price their servers. The Opteron 6100 series offers up to 24 DIMMs slots, the Xeon is “limited” to 18. In many cases this allows the server buyer to achieve higher amount of memory with lower costs. You can go for 96 GB of memory with affordable 4 GB DIMMs, while the Intel server is limited to 72 GB there. That is a small bonus for the AMD server.

The HPC market seems to favor AMD once again. AMD holds only a small performance advantage, and this market is very cost sensitive. The lower price will probably convince the HPC people to go for the AMD based servers.

All in all, this is good news for the IT professional that is a hardware enthusiast. Profiling your application and matching it to the right server CPU pays off and that is exactly what set us apart from the average IT professional.

Read More..