[opensuse] OpenSUSE 10.3 benchmarking revisited
Hi! Some of you may recall Ian Smith's post about low benchmark scores for openSuse 10.3 [1]. Now that the Suse team has fixed one of the main issues, it is time to benchmark again. This time, openSuse 10.3 scores _much_ better than the last time. Further investigation of the bad performance revealed that the main cause for the extremely low performance was very high overhead for system calls (syscalls), so I filed bug #333739. Tony Jones had a look at it, fixing both kernel and auditd. The bug is closed now, new packages were released a while ago. Another source of bad benchmark results is grep. The version of grep included in Suse 10.3 is awfully slow when processing UTF-8 data, see bug #308698 [3]. On my system, grep on Suse 10.3 is ~30 times slower than on 10.0. Exact results may vary depending on setup, but it is slooow. Grep was also a source of inconsistent benchmark results with unixbench 5.1, because that version of unixbench did not set the LANG environment variable consistently. Unixbench 5.1.1 fixes that. I decided to run another benchmark, comparing openSuse 10.3 with new kernel+auditd and a fast grep against one with the old kernel+auditd and the default grep (as originally benchmarked by Ian Smith). As a fast grep, I simply took the rpm for Suse 10.2. As a reference, I have included results for Suse Linux 10.0. All tests were run in runlevel 1 with unixbench 5.1.1 (identical unixbench binaries compiled under Suse 10.0) on a Pentium M 1.3Ghz. 10.3 new 10.3 old 10.0 ========== ======== ========== Dhrystone 247 247 242 Whetstone 177 177 177 Execl 642 603 714 File Copy 1024 560 484 538 File Copy 256 432 366 454 File Copy 4096 809 742 597 Pipe Throughput 374 295 479 Context Switch 473 394 569 Process Creat 821 793 1032 Shell Scripts1 524 358 556 Shell Scripts8 522 349 537 System Call 772 336 882 ------ ------ ------ Index Score: 485 393 513 The current version of openSuse 10.3 easily outscores the GM version. The modified version of grep is responsible for higher scores in the shell script tests since they use grep as one of their commands. It should not affect the other test results at all. Maybe Suse should consider to downgrade grep if the performance problems cannot be resolved, because the new grep is slow to a point where it is simply unusable. Fixes in kernel+auditd significantly helped all tests that interact a lot with the kernel (basically all but the first two). Syscall performance is measured directly in the last test, where the score has more than doubled! However, comparing results to 10.0 shows that overall, the performance of the base system is still not as good as it could be, at least in this benchmark. Suse 10.0 still scores higher in most areas. Since the syscall test scores are ~12% lower than in 10.0, it seems logical that the slowdown is due to the kernel. It would be interesting to see if the vanilla kernel suffers from the same slowdown or if this is specific to Suse. Regards nordi -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
|-----Original Message----- |From: nordi [mailto:nordi@addcom.de] |Sent: 3. desember 2007 19:36 |To: suse |Subject: [opensuse] OpenSUSE 10.3 benchmarking revisited | |The current version of openSuse 10.3 easily outscores the GM version. Tanks for the update,Nordi, maye it is time to try 10.3 then. Is there any plan to refresh the GM, so we do not need to patch up so much. Redhat updates their distro CDs/DVDs from time to time. |However, comparing results to 10.0 shows that overall, the |performance of the base system is still not as good as it |could be, at least in this benchmark. Suse 10.0 still scores |higher in most areas. Since the syscall test scores are ~12% |lower than in 10.0, it seems logical that the slowdown is due |to the kernel. It would be interesting to see if the vanilla |kernel suffers from the same slowdown or if this is specific to Suse. If you look at the repositories you can find nightly build kernels you can test, They are mostly vanilla + default suse kernel settings http://download.opensuse.org/repositories/home:/dipe/openSUSE_10.3/ (just remember the relevant kernel-source :-) I've not yet updated to 10.3 but for 10.2 they all have worked fine when I've upgraded apart from standard kernel related gotchas, like nvidia-drivers, mb-chipsets, but all those problems have been easily resolved searching the net. -- MortenB -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
nordi wrote:
However, comparing results to 10.0 shows that overall, the performance of the base system is still not as good as it could be, at least in this benchmark. Suse 10.0 still scores higher in most areas. Since the syscall test scores are ~12% lower than in 10.0, it seems logical that the slowdown is due to the kernel. It would be interesting to see if the vanilla kernel suffers from the same slowdown or if this is specific to Suse.
I suppose it will vary depending on what options you configure in at build time. Most people believe that auditing will degrade performance to some extent. Examples to the contrary seem to be generally ill remembered (or maybe just not well known). Auditing doesn't have to slow things down so noticeably though -- especially when turned off! :-) Concepts like limiting auditing to those system calls that change the security state (vs. a blanket auditing of every system call) and using a block-device for kernel->user space implementation are probably basic. Beyond that, depends on what gets audited... -linda -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (3)
-
Linda Walsh -
Morten Bjørnsvik -
nordi