[Bug 387347] New: pipe uses only 2 CPU on dual-core ("zcat | wc")
https://bugzilla.novell.com/show_bug.cgi?id=387347 User koenig@linux.de added comment https://bugzilla.novell.com/show_bug.cgi?id=387347#c381873 Summary: pipe uses only 2 CPU on dual-core ("zcat | wc") Product: openSUSE 11.0 Version: Factory Platform: x86-64 OS/Version: Other Status: NEW Severity: Normal Priority: P5 - None Component: Kernel AssignedTo: bnc-team-screening@forge.provo.novell.com ReportedBy: koenig@linux.de QAContact: kernel-maintainers@forge.provo.novell.com Found By: --- as a side result from bug #381873 I got some bad performace for a small pipe on my IBM T60 using intel T7200 core2duo cpu, only one cpu core gets used: harald > zcat /var/lib/pin/ARCHIVES.gz | LC_ALL=C time wc 3331696 30052221 535205599 0:09.43 real, 4.50 user, 0.13 sys, 49% cpu top shows that gzip and wc take about the same about of cpu-time, both around 50%. running xosview shows that CPU0 and CPU1 frequently get changed/used, both show more or less random usage pattern. if I use "zcat | bzip2 > /dev/null" then bzip2 uses 90+ % cpu because it's much more cpu bound... why does "zcat | wc" run "so slow" (doesn't use both CPUs for this pipe) ??? here is the last data for this report for the pipe issue: using "cat" instead of "wc" harald > time zcat /var/lib/pin/ARCHIVES.gz{,,,,,,,} | LC_ALL=C time cat > /dev/null 0:41.11 real, 0.16 user, 1.61 sys, 4% cpu 41.128 0m41.128s real, 37.850 user, 3.148 sys, 99.68 cpu harald > time zcat /var/lib/pin/ARCHIVES.gz{,,,,,,,} | LC_ALL=C time wc > /dev/null 1:16.89 real, 36.23 user, 1.24 sys, 48% cpu 76.902 1m16.902s real, 73.297 user, 2.608 sys, 98.70 cpu NOTE that the 2nd line of time output shows the cpu time sums of the whole pipe (zcat + wc), so wc got 48% cpu and zcat the other 50% harald > uname -a Linux harald 2.6.25-26-default #1 SMP 2008-04-30 07:56:05 +0200 x86_64 x86_64 x86_64 GNU/Linux what's wrong here, why not use both cores ? ;-( -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=387347
Harald Koenig
https://bugzilla.novell.com/show_bug.cgi?id=387347
User gregkh@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=387347#c1
Greg Kroah-Hartman
https://bugzilla.novell.com/show_bug.cgi?id=387347
User koenig@linux.de added comment
https://bugzilla.novell.com/show_bug.cgi?id=387347#c2
Harald Koenig
This is only a single process running, how could both cores be used for it?
please read again (or maybe try yourself?)! there are _two_ processes running for this test: gzip and wc, both using almost exactly the same amount of cpu bandwidth. actually, it's pretty hard using a pipe with only a single process, isn't it ;-) top shows _two_ processes, both only getting 50% cpu instead of 100% each, so I'm loosing a factor of 2 in real time (wall clock) performace. "procinfo -dn1" shows 50% idle -- that's the unused 2nd core!
This shows the scheduler is working quite well, and properly :)
NOPE. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=387347
User gregkh@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=387347#c3
Greg Kroah-Hartman
https://bugzilla.novell.com/show_bug.cgi?id=387347
User koenig@linux.de added comment
https://bugzilla.novell.com/show_bug.cgi?id=387347#c4
Harald Koenig
On x86-64 box, you are right, it does seem to want to favor one processor, with almost one pegged at 100%, and the other around 10%.
But that doesn't mean the scheduler isn't working properly. As the workload seems about 50% for one cpu, putting both tasks on 1 cpu makes sense.
So again, I really don't see this being a problem.
from a user point of view (what's what I do the suse beta-tests for!) there _is_ a problem: for very simple and frequent operations (e.g. "pin") opensuse 11.0 will be 60% to 100% slower on dual-core compared to 10.3 ! here is the same timing running 10.3 with kernel 2.6.22.17-0.1-default: harald > time zcat /11.0/var/lib/pin/ARCHIVES.gz{,,,,,,,} | LC_ALL=C time wc > /dev/null 0:41.56 real, 27.32 user, 1.19 sys, 68% cpu 41.568 0m41.568s real, 63.952 user, 2.448 sys, 100.00 cpu (btw. that "100.00 cpu" [%] in the 10.3 output are bogus too -- bug bug;) unfortuneately that's not the only performace issue: there is this utf-8 issue for grep/wc/sed/... which gives a factor of 10+ of slowness (see bugs #308698 and #381873 -- thats an open issue since september 2007 found in 10.3-beta and never fixed so far :-(( I wonder why novell/suse does not handle these more reasonable way. I understand that kernel scheduler problems are very sensible issues, but at least this should be discussed and not closed as WONTFIX within hours :-( not handling the utf-8 problems for a full distribution cycle is just insane IMHO. so I wonder where is novell/suse going with their products (both opensuse and SLES) ???? I'm ver disapointed to see this development, being a happy suse user and active beta-tester and developer for far more than 10 years...
If you disagree, please create a bug at bugzilla.kernel.org as this is an upstream issue, we do not change the scheduler at all in our opensuse kernels.
IMHO _you_ (suse) should be interested to get this thing work better -- if possible before releasing 11.0-GM or at least with a kernel update real soon after 11.0-release.... marking this WONTFIX so fast won't help neither your users nor suse. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=387347
User aj@novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=387347#c5
--- Comment #5 from Andreas Jaeger
participants (1)
-
bugzilla_noreply@novell.com