multi-processors, threads and top
Please help, We have a machine that has 4 dual-core CPUs. Now we have a Java application that has multi-thread support. The problem is, when I use Top to see what is going on, I usually see a single process on a single CPU using 99%. Rarely do I ever see more than one Java process using CPU even though there are 5 running. So, can Top show a single process on more than one CPU? Is there some other way I can look at this to get a better picture of how many of these CPUs are being used and how much? As always, any help would be appreciated. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
Will, On Monday 23 October 2006 20:19, Will Yonker wrote:
Please help,
We have a machine that has 4 dual-core CPUs. Now we have a Java application that has multi-thread support. The problem is, when I use Top to see what is going on, I usually see a single process on a single CPU using 99%. Rarely do I ever see more than one Java process using CPU even though there are 5 running.
So, can Top show a single process on more than one CPU? Is there some other way I can look at this to get a better picture of how many of these CPUs are being used and how much?
As always, any help would be appreciated.
Are you certain that your Java code is written to exploit SMP? If you're running on a 2.4.x Linux kernel, each thread appears in ps and top output as a separate process. On 2.6.x kernels, threads do not appear as separate threads. For that matter, are you running a "-smp" or "-bigsmp" kernel? If not, you're only using a single core of a single CPU. If you use the SuSE installer on this hardware (and its a version of SuSE that's modern enough to support SMP), then a suitable kernel should have been chosen. On the other hand, if you installed on a single-processor system then upgraded the mainboard, you may not have an smp-enabled kernel. It's also possible, at least on some systems, that BIOS settings may be interfering with multi-processor and / or multi-core systems. If that's the case, you may need to fix those settings and then install a suitable SMP kernel. You can tell whether your system (kernel) supports SMP and everything else necessary is set correctly by typing "1" to top. This will toggle its separate CPU display mode. By default when you launch top, it merges all the CPUs utilization into a single summary. When you toggle this mode, you'll see a line in the top summary section for each CPU / core. You also didn't say which version of Java you're using. Some older Sun JVMs use the so-called "green" threads, which don't exploit system-level multi-threading and so cannot use multiple CPUs or cores. I don't know if GCJ supports system-level threads or not, but I cannot recommend it since its simply not mature enough for most uses. Randall Schulz
Will,
On Monday 23 October 2006 20:19, Will Yonker wrote:
Please help, We have a machine that has 4 dual-core CPUs. Now we have a Java application that has multi-thread support. The problem is, when I use Top to see what is going on, I usually see a single process on a single CPU using 99%. Rarely do I ever see more than one Java process using CPU even though there are 5 running. So, can Top show a single process on more than one CPU? Is there some other way I can look at this to get a better picture of how many of
<quote who="Randall R Schulz"> these CPUs are being used and how much?
As always, any help would be appreciated.
Are you certain that your Java code is written to exploit SMP?
No. I'm not in direct contact with the developers. Here is what I found:
java -version java version "1.4.2_08" Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_08-b03) Java HotSpot(TM) Client VM (build 1.4.2_08-b03, mixed mode)
Is there an easy way to tell if this version is threaded or not?
If you're running on a 2.4.x Linux kernel, each thread appears in ps and top output as a separate process. On 2.6.x kernels, threads do not appear as separate threads.
For that matter, are you running a "-smp" or "-bigsmp" kernel?
/proc> cat version Linux version 2.6.5-7.244-smp (geeko@buildhost) (gcc version 3.3.3 (SuSE Linux)) #1 SMP Mon Dec 12 18:32:25 UTC 2005 Do I need the -bigsmp? It's 4 CPUs with 32GB of memory.
You can tell whether your system (kernel) supports SMP and everything else necessary is set correctly by typing "1" to top. This will toggle its separate CPU display mode. By default when you launch top, it merges all the CPUs utilization into a single summary. When you toggle this mode, you'll see a line in the top summary section for each CPU / core.
I've also toggeled the Irix mode but didn't notice a big difference. I've attached a sample of my top output. This is not a good sample because of the wait state but you get the idea. top - 16:34:23 up 6 days, 4:05, 3 users, load average: 2.97, 3.02, 3.18 Tasks: 228 total, 1 running, 226 sleeping, 0 stopped, 1 zombie Cpu0 : 5.0% us, 3.0% sy, 0.0% ni, 88.7% id, 2.0% wa, 0.0% hi, 1.3% si Cpu1 : 2.3% us, 6.7% sy, 0.0% ni, 89.0% id, 2.0% wa, 0.0% hi, 0.0% si Cpu2 : 4.0% us, 3.7% sy, 0.0% ni, 73.8% id, 18.6% wa, 0.0% hi, 0.0% si Cpu3 : 2.3% us, 5.6% sy, 0.0% ni, 90.7% id, 1.3% wa, 0.0% hi, 0.0% si Cpu4 : 3.3% us, 7.3% sy, 0.0% ni, 12.0% id, 77.4% wa, 0.0% hi, 0.0% si Cpu5 : 1.0% us, 1.0% sy, 0.0% ni, 94.7% id, 2.7% wa, 0.0% hi, 0.7% si Cpu6 : 1.7% us, 1.0% sy, 0.0% ni, 94.0% id, 3.3% wa, 0.0% hi, 0.0% si Cpu7 : 2.0% us, 1.0% sy, 0.0% ni, 94.4% id, 2.3% wa, 0.0% hi, 0.3% si Mem: 32394876k total, 32027300k used, 367576k free, 6744236k buffers Swap: 10490436k total, 389260k used, 10101176k free, 22365696k cached PID USER NI VIRT RES SHR S %CPU %MEM TIME+ P COMMAND 30863 sdm 0 632m 194m 19m S 99.9 0.6 13:31.22 6 java 28448 sdm 0 1020m 632m 20m S 0.5 2.0 21:12.51 0 java 8641 root 0 211m 9740 1484 S 0.1 0.0 56:38.44 3 java 20114 root 0 17000 1784 1276 S 0.1 0.0 21:30.28 5 perl 19182 root 0 0 0 0 S 0.1 0.0 0:58.92 2 kjournald 28573 dbadmin 0 1796m 122m 120m S 0.1 0.4 0:23.38 5 oracle 28665 dbadmin 0 1796m 84m 82m S 0.1 0.3 0:29.64 4 oracle 30347 dbadmin 0 1797m 308m 306m S 0.1 1.0 0:41.87 4 oracle -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
Will, On Tuesday 24 October 2006 13:42, Will Yonker wrote:
...
Are you certain that your Java code is written to exploit SMP?
No. I'm not in direct contact with the developers. Here is what I found:
Can you point us to the software you're using?
java -version
java version "1.4.2_08" Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_08-b03) Java HotSpot(TM) Client VM (build 1.4.2_08-b03, mixed mode)
Is there an easy way to tell if this version is threaded or not?
Don't worry. That version, while not the latest, definitely will exploit system-level threads and hence multi-core multi-processor hardware whenever available.
...
/proc> cat version Linux version 2.6.5-7.244-smp (geeko@buildhost) (gcc version 3.3.3 (SuSE Linux)) #1 SMP Mon Dec 12 18:32:25 UTC 2005
Do I need the -bigsmp? It's 4 CPUs with 32GB of memory.
If you want to use more than about 3GB of that physical RAM, you need a "big" kernel. If you want to exploit more than one CPU / core, you need "smp," so yes, you need a -bigsmp kernel, or you're wasting a great deal of that very high-end machine you've got there.
...
I've also toggeled the Irix mode but didn't notice a big difference. I've attached a sample of my top output. This is not a good sample because of the wait state but you get the idea.
top - 16:34:23 up 6 days, 4:05, 3 users, load average: 2.97, 3.02, 3.18 Tasks: 228 total, 1 running, 226 sleeping, 0 stopped, 1 zombie Cpu0 : 5.0% us, 3.0% sy, 0.0% ni, 88.7% id, 2.0% wa, 0.0% hi, 1.3% si Cpu1 : 2.3% us, 6.7% sy, 0.0% ni, 89.0% id, 2.0% wa, 0.0% hi, 0.0% si Cpu2 : 4.0% us, 3.7% sy, 0.0% ni, 73.8% id, 18.6% wa, 0.0% hi, 0.0% si Cpu3 : 2.3% us, 5.6% sy, 0.0% ni, 90.7% id, 1.3% wa, 0.0% hi, 0.0% si Cpu4 : 3.3% us, 7.3% sy, 0.0% ni, 12.0% id, 77.4% wa, 0.0% hi, 0.0% si Cpu5 : 1.0% us, 1.0% sy, 0.0% ni, 94.7% id, 2.7% wa, 0.0% hi, 0.7% si Cpu6 : 1.7% us, 1.0% sy, 0.0% ni, 94.0% id, 3.3% wa, 0.0% hi, 0.0% si Cpu7 : 2.0% us, 1.0% sy, 0.0% ni, 94.4% id, 2.3% wa, 0.0% hi, 0.3% si Mem: 32394876k total, 32027300k used, 367576k free, 6744236k buffers Swap: 10490436k total, 389260k used, 10101176k free, 22365696k cached
From this blob, I deduce you have an SMP kernel installed (which is consistent with the contents of /proc/version, of course). Note that "uname -a" works well for displaying this information, too: % uname -a Linux smiley 2.6.18-9-bigsmp #1 SMP Mon Oct 2 23:27:41 UTC 2006 i686 i686 i386 GNU/Linux (By the way, Please consider excluding this kind of content from line wrapping.)
PID USER NI VIRT RES SHR S %CPU %MEM TIME+ P COMMAND 30863 sdm 0 632m 194m 19m S 99.9 0.6 13:31.22 6 java 28448 sdm 0 1020m 632m 20m S 0.5 2.0 21:12.51 0 java 8641 root 0 211m 9740 1484 S 0.1 0.0 56:38.44 3 java 20114 root 0 17000 1784 1276 S 0.1 0.0 21:30.28 5 perl 19182 root 0 0 0 0 S 0.1 0.0 0:58.92 2 kjournald 28573 dbadmin 0 1796m 122m 120m S 0.1 0.4 0:23.38 5 oracle 28665 dbadmin 0 1796m 84m 82m S 0.1 0.3 0:29.64 4 oracle 30347 dbadmin 0 1797m 308m 306m S 0.1 1.0 0:41.87 4 oracle
Well, this tells me that the first java process shown is currently using process number 6, and pretty much all of it... I don't think anything is wrong (beyond the need for a "big" memory model kernel). In all likelihood that Java application is not written in a manner that exploits real, physical hardware concurrency. Some Java developers are under the mistaken belief that Java's support for multi-threading is automatic in the sense that it will itself parallelize your code. It most certainly will not, and the task of writing proper concurrent Java code is not trivial, though it's not as bad as doing so in C or C++. Randall Schulz
<quote who="Randall R Schulz">
Well, this tells me that the first java process shown is currently using process number 6, and pretty much all of it...
Thank you very much for the help. So since I'm running a 2.6 kernel, top will only show the process on one CPU. Will it go about 99% utilization. Is the thread glitch a limitation of top or the 2.6 kernel? Ugh, that just made my life much harder. :( -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
Will, On Tuesday 24 October 2006 14:23, Will Yonker wrote:
<quote who="Randall R Schulz">
Well, this tells me that the first java process shown is currently using process number 6, and pretty much all of it...
There was a small typo there: I meant "... using processor number 6 ...".
Thank you very much for the help. So since I'm running a 2.6 kernel, top will only show the process on one CPU. Will it go about 99% utilization. Is the thread glitch a limitation of top or the 2.6 kernel?
No, top is not limited in this way and there aren't any "glitches" afoot (that I know of). That display just shows the assignment of process to processor at the moment. It will probably change throughout the execution of the process, possibly quite frequently.
Ugh, that just made my life much harder. :(
I don't think so, really. The only things left for you to do are: 1) Install the "bigsmp" kernel. 2) Find out from the author / vendor of that Java program whether it actually takes advantage of multiple processors (or cores, which are indistinguishable to Java code). There could be a (1a) task. On my mainboard you must enable the large memory model in the BIOS. Otherwise there's an approximately 1 GB region (from about 3GB to 4GB) of address space reserved for I/O mapping. This I/O device range puts the ceiling on the amount of accessible physical RAM. If your BIOS works like this, you need to switch it to large memory mode. After that, the SUSE installer will choose a bigsmp kernel for you. Randall Schulz
On Tuesday 24 October 2006 11:03, Randall R Schulz wrote:
/proc> cat version Linux version 2.6.5-7.244-smp (geeko@buildhost) (gcc version 3.3.3 (SuSE Linux)) #1 SMP Mon Dec 12 18:32:25 UTC 2005
Do I need the -bigsmp? It's 4 CPUs with 32GB of memory.
If you want to use more than about 3GB of that physical RAM, you need a "big" kernel. If you want to exploit more than one CPU / core, you need "smp," so yes, you need a -bigsmp kernel, or you're wasting a great deal of that very high-end machine you've got there.
I have four GB of memory. Yast has always picked the kernel, in this case: :/proc> cat version Linux version 2.6.13-15.12-smp (geeko@buildhost) (gcc version 4.0.2 20050901 (prerelease) (SUSE Linux)) #1 SMP Thu Aug 24 11:23:58 UTC 2006 should I be using the 'big' kernel? Thanks, Jerome
On Tuesday 24 October 2006 16:31, Susemail wrote:
...
If you want to use more than about 3GB of that physical RAM, you need a "big" kernel. If you want to exploit more than one CPU / core, you need "smp," so yes, you need a -bigsmp kernel, or you're wasting a great deal of that very high-end machine you've got there.
I have four GB of memory. Yast has always picked the kernel, in this case: :/proc> cat version Linux version 2.6.13-15.12-smp (geeko@buildhost) (gcc version 4.0.2 20050901 (prerelease) (SUSE Linux)) #1 SMP Thu Aug 24 11:23:58 UTC 2006
How much memory is actually available to your kernel? To wit: % head -1 /proc/meminfo MemTotal: 4140144 kB % uname -a Linux smiley 2.6.18-9-bigsmp #1 SMP Mon Oct 2 23:27:41 UTC 2006 i686 i686 i386 GNU/Linux Vs.: % head -1 /proc/meminfo MemTotal: 2073084 kB % uname -a Linux twain 2.6.13-15.11-smp #1 SMP Mon Jul 17 09:43:01 UTC 2006 i686 i686 i386 GNU/Linux If I were to reboot the first of these two systems (the 4GB one) with the BIOS switched to small memory mode, then it would report only 3 GB available, even though I did not remove any RAM from the mainboard.
should I be using the 'big' kernel?
I believe that if you have more than ~3GB of physical RAM (on an Intel system) you do. And you'll need a CPU and chipset that supports Physical Address Extensions (PAE) to exploit more than 3GB.
Thanks, Jerome
Randall Schulz
Today all CPUs & chipsets support PAE.
On Tuesday 24 October 2006 14:51, Randall R Schulz wrote:
On Tuesday 24 October 2006 16:31, Susemail wrote:
...
If you want to use more than about 3GB of that physical RAM, you need a "big" kernel. If you want to exploit more than one CPU / core, you need "smp," so yes, you need a -bigsmp kernel, or you're wasting a great deal of that very high-end machine you've got there.
I have four GB of memory. Yast has always picked the kernel, in this case: :/proc> cat version Linux version 2.6.13-15.12-smp (geeko@buildhost) (gcc version 4.0.2 20050901 (prerelease) (SUSE Linux)) #1 SMP Thu Aug 24 11:23:58 UTC 2006
How much memory is actually available to your kernel?
:~> head -1 /proc/meminfo MemTotal: 4041804 kB :~> uname -a Linux Orion 2.6.13-15.12-smp #1 SMP Thu Aug 24 11:23:58 UTC 2006 x86_64 x86_64 x86_64 GNU/Linux
To wit:
% head -1 /proc/meminfo MemTotal: 4140144 kB
% uname -a Linux smiley 2.6.18-9-bigsmp #1 SMP Mon Oct 2 23:27:41 UTC 2006 i686 i686 i386 GNU/Linux
Vs.:
% head -1 /proc/meminfo MemTotal: 2073084 kB
% uname -a Linux twain 2.6.13-15.11-smp #1 SMP Mon Jul 17 09:43:01 UTC 2006 i686 i686 i386 GNU/Linux
If I were to reboot the first of these two systems (the 4GB one) with the BIOS switched to small memory mode, then it would report only 3 GB available, even though I did not remove any RAM from the mainboard.
should I be using the 'big' kernel?
I believe that if you have more than ~3GB of physical RAM (on an Intel system) you do. And you'll need a CPU and chipset that supports Physical Address Extensions (PAE) to exploit more than 3GB.
Thanks, Jerome
Randall Schulz
On Mon, 2006-10-23 at 23:19 -0400, Will Yonker wrote:
Please help,
We have a machine that has 4 dual-core CPUs. Now we have a Java application that has multi-thread support. The problem is, when I use Top to see what is going on, I usually see a single process on a single CPU using 99%. Rarely do I ever see more than one Java process using CPU even though there are 5 running.
So, can Top show a single process on more than one CPU? Is there some other way I can look at this to get a better picture of how many of these CPUs are being used and how much?
As always, any help would be appreciated.
Me too. The best I can see is with 'ps -eLf' to see the threads. But that does not show how much each thread uses. It is all attributed to the parent. I also have a multiple threaded app that I would hope the kernel would run in various cores on a dual core processor. I know how to set processor affinity for a process. I am not sure it works for a thread. And I would first like to see what the kernel does when left to it's own devices. Can you see the various CPUs in top? -- Roger Oberholtzer OPQ Systems AB Ramböll Sverige AB Kapellgränd 7 P.O. Box 4205 SE-102 65 Stockholm, Sweden Tel: Int +46 8-615 60 20 Fax: Int +46 8-31 42 23
On Monday 23 October 2006 22:18, Roger Oberholtzer wrote:
Can you see the various CPUs in top?
Yes, the answer was posted previously in this thread. type the digit 1 in the top display and it will break out the processors for you. Of course there is always: man top (but then you knew that, didn't you ;-) -- _____________________________________ John Andersen
On Tuesday 24 October 2006 05:19, Will Yonker wrote:
Please help,
We have a machine that has 4 dual-core CPUs. Now we have a Java application that has multi-thread support. The problem is, when I use Top to see what is going on, I usually see a single process on a single CPU using 99%. Rarely do I ever see more than one Java process using CPU even though there are 5 running.
So, can Top show a single process on more than one CPU? Is there some other way I can look at this to get a better picture of how many of these CPUs are being used and how much?
As always, any help would be appreciated.
-- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
-- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com
Sorry for barging in and slipping of subject. But what system do you use? (Vendor/model) I am looking for a good quad SMP system. -- /Rikard ----------------------------------------------------------------------------- email : rikard.j@rikjoh.com web : http://www.rikjoh.com mob: : +46 (0)763 19 76 25 ------------------------ Public PGP fingerprint ---------------------------- < 15 28 DF 78 67 98 B2 16 1F D3 FD C5 59 D4 B6 78 46 1C EE 56 >
participants (7)
-
Alexey Eremenko -
John Andersen -
Randall R Schulz -
Rikard Johnels -
Roger Oberholtzer -
Susemail -
Will Yonker