Quickbench: Linux and Solaris on the Sun T2000


Some people say Sun's shiny new UltraSPARC T1 performs better with Linux than with Solaris. That's what I'm trying to find out in the first episode of our quickbench series.

Why quickbench? Because, I would like to make sure about that everybody understands these tests have nothing to do with reality, they are just quick (synthetic) benchmarks, lacking any preliminary tuning and the detailed analysis of the test results. Also, there were no feedbacks from the results to the benchmarks itself.

Let's take a look on the machine: it was a Sun T2000 with 8 GB of RAM, two 72 GB 2,5" SAS disks and a 1 GHz, eight core UltraSPARC T1 CPU.

This CPU is Sun's new top gun, which contains eight CPU cores in one package, can handle 32 threads with hardware assistance. According to Sun we can count this CPU as a product of frequency and the number of cores, so this one would be equal to a 8 GHz chip.

Of course this is not true, or at least not that way. The recipe is simple: you can only fully use this CPU if you can give it enough work via many active threads.
Unfortunately, there are many applications which can't behave on this way, so don't wonder if you sit on the front of this shiny new 9.6 GHz (ours is 8 GHz) machine and feel like it is an old PIII 450.

Let's see the software environment: one of the operating systems I tried is the world's most advanced Unix, Solaris 10. Sadly, due to the lack of time, I could not try Nevada (will be Solaris 11). The other operating system is the recently neglected bugware, Linux.
The latter is just ported to this platform, but thanks to the Ubuntu team and Dave Miller's hard work, the Dapper Drake can be installed flawlessly. Kudos to them (and Sun) for this.

The Ubuntu distribution arrives with a heavily patched 2.6.15 Linux kernel, only the kernel.org version can't be run on this machine. The upcoming 2.6.17 will be usable out of the box.

For me, Linux could only recognize 24 out of the 32 (virtual) processors, but I could not look into this issue, because I had only limited time.

I would like to try FreeBSD either, but according to the developer, there are some issues with its usability (namely, currently it's not possible to use programs, which need more than 16 MB of RAM), so I had to give up and leave FreeBSD out of the list.

Let's strike into the letcho. (silly, isn't it? But we say very often. :)

The first test is the omniscient sysbench's memory benchmark. The test ran by different block sizes. The first run is with the factory default (by Ubuntu) 2.6.15 kernel:

Apparent, in spite of that Linux could only recognize 24 processors, the memory bandwidth has grown up to 32 threads, reaching its maximum.

The next one is with 2.6.17 (from kernel.org):

There are no huge differences. Let's see what can Solaris do:

As we can see the endevour has run out somewere at 1kB blocksize, therefore I decided by 2kB blocksize-based comparation, because it is paralell with the others on both systems and we can observe the beginning of the characteristic curve easily:

The curve is interesting in many different considerations. On the one hand, it seems 2.6.17 kernel has significant differs from 2.6.15, so not to be wondered that many people struggle to compare the throughput at several kernel versions.
On the other hand, whoever measures its e-penis by sysbench, gets more bang for the buck with Solaris. On the pitch of the curve at 32 threads double deflaction comes up, that may mean Solaris has been already regularly optimised on this platform, and Linux has not yet, nevertheless the same deviation exists on its x86 variant. Whoever curious its detailed explanation go read the source code.

The next test is about MySQL, still with sysbench. To avoid that I measure sysbench's performance, instead of MySQL's I've used another machine, which was connected to the T2000 with a direct Gigabit link. Therefore the tests ran over the network, while the other machine was a 32 bit Linux (Ubuntu). The machine had two 2.6 GHz Opterons, so sysbench had plenty resources to burn.

The first test is with MySQL 4.1 and I've tried to figure out which is better on Solaris: 32 or 64 bit:

I think the question has been answered. Sadly, due to the lack of time and other things, I could not make this test on Linux, so the following tests are with a 32 bit binary.

This test measures SELECT performance on a 1M rows HEAP table, so no disk IO was involved:

Discernable, there are multiple versions of MySQL. 4.1 and 5.0 on Linux and an additional 5.1 on Solaris. The configuration of MySQL didn't change between the runs.

Solaris seems to be the clear winner of this test, which not too suprising. It is highly possible that Linux are not that optimized on T1 as Solaris. The latter at 512 threads hit Linux by a factor of four, running MySQL 4.1.
It's interesting to see how MySQL gets slow down when going from 4.1 to 5.1, especially with a lot of clients.

Conclusion: the Sun T1 is struggling with tasks which has low concurrency and flies when we can use it at least with 32 active threads. Sun promotes this processor mainly for web and database servers. We really should take its advice.
The disadvantage of this architecture (many, slow cores) that in contract to its enemies (like the dual core Opteron or Xeon) we can't get high performance if we can only run few threads. In the case of a dual core Opteron, sometimes we can reach the performance maximum and the lowest response times with only two, or four threads (but with 8 we are mostly OK). To reach that with T1, you need around 32 threads, which is not suitable for everybody.

The final word: if you want to make your money out of this machine, you should give it a lot of work.


Ha gondolod, legkozelebb publikalas elott kuldd el a szoveget privatban, szivesen kijavitom az angolt, ha van ra igeny. Vagy menjen a wikibe, es akkor kozvetlenul is szerkesztheto.


You can also flame this article in English. :)))

Sokan nincsenek tudatában annak, / hogy egyszer mindenki meghal. / Akik ráébrednek erre, / azonnal abbahagyják az ellenségeskedést.

bra, if you allow to receive private messages, I send you a corrected one. :-)

Ригидус а бетегадьбол