I have two Opteron clusters - one SLES8 based - the other Suse9. Both are running a 2.4.21 kernel with numa enabled. Are there any commands available to launch a process onto a given cpu (or set of cpus) within an SMP ? and likewise to check: which cpus a process in on? (cf /proc/12345/cpu) where it's memory is coming from? (cf say /proc/buddyinfo) Yours, Daniel. -------------------------------------------------------------- Dr. Dan Kidger, Quadrics Ltd. daniel.kidger@quadrics.com One Bridewell St., Bristol, BS1 2AA, UK 0117 915 5505 ----------------------- www.quadrics.com --------------------
Hi Daniel,
Have a look at
http://www.kernel.org/pub/linux/kernel/people/rml/cpu-affinity/ ...
There's tons of information here that should answer most of your
questions...
To see what cpu a process is running, you can use top... By default, the
"Last Used CPU" field may not be showing - simply hit "f" then "y" to
enable that field and voila! (note that the commands may be slightly
different depending on your version of top)...
As for the where a process' memory is coming from, I don't know how to do
that, but am certainly interested in hearing the answer... :)
----------
Dan Behman
DB2 for Linux Platform Development
IBM Software Development Lab, Office C4-432
Phone: (905) 413-4416 Email: dbehman@ca.ibm.com
Please send DB2 for Linux questions to our DB2LinuxDev-CAN distribution
list.
----------
daniel.kidger@qua
drics.com
To
03/16/2004 08:06
daniel.kidger@quadrics.com writes:
I have two Opteron clusters - one SLES8 based - the other Suse9. Both are running a 2.4.21 kernel with numa enabled.
Are there any commands available to launch a process onto a given cpu (or set of cpus) within an SMP ?
There's a numactl package that offers some options - but also some options that are not implemented in the kernel... Andreas -- Andreas Jaeger, aj@suse.de, http://www.suse.de/~aj SuSE Linux AG, Maxfeldstr. 5, 90409 Nürnberg, Germany GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
On Tue, 16 Mar 2004 20:34:45 +0100
Andreas Jaeger
daniel.kidger@quadrics.com writes:
I have two Opteron clusters - one SLES8 based - the other Suse9. Both are running a 2.4.21 kernel with numa enabled.
Are there any commands available to launch a process onto a given cpu (or set of cpus) within an SMP ?
There's a numactl package that offers some options - but also some options that are not implemented in the kernel...
numactl won't work on the shipped kernels (it was for an experimental kernel patch was never production ready) In SL 9.0 you can use the commands in the scheduler-utilities rpm (not installed by default, but on the DVD) SLES8 currently doesn't have a tool, but there may be an update with a scheduler-utilities rpm. If you cannot wait for that you could compile the SL9 rpm yourself there. -Andi
Andi / Andreas, thanks for the information. We patch and compile our our own kernel - enabling the NUMA options amiongst other things. Under sles8 I installed numactl (from SP3) - it worked perfectly. :-) I could not only see the lastCPUused under top - but I coudl detect the performance change when I use a CPU close/far from the Golum PCI-X chips. Under suse9 - I installed the suse9 numactl - but it seem to be silently ignored - even though the the kernel binary is eactly the same as the sles8 cluster (2.4.21-171numa). I am investigating. Finally I tried 'scheduler-utilities' under suse9 - this only contains one file: $ rpm -qlp \ /home/packages/Linux_x86_64/suse9.0/scheduler-utilities-0.1-150.x86_64.rpm /usr/sbin/resched and this doesn't seem to have any NUMA options - just the usual nice-value et al. Am I missing something ? Daniel. -------------------------------------------------------------- Dr. Dan Kidger, Quadrics Ltd. daniel.kidger@quadrics.com One Bridewell St., Bristol, BS1 2AA, UK 0117 915 5505 ----------------------- www.quadrics.com -------------------- On Tuesday 16 March 2004 8:46 pm, Andi Kleen wrote:
On Tue, 16 Mar 2004 20:34:45 +0100
Andreas Jaeger
wrote: daniel.kidger@quadrics.com writes:
I have two Opteron clusters - one SLES8 based - the other Suse9. Both are running a 2.4.21 kernel with numa enabled.
Are there any commands available to launch a process onto a given cpu (or set of cpus) within an SMP ?
There's a numactl package that offers some options - but also some options that are not implemented in the kernel...
numactl won't work on the shipped kernels (it was for an experimental kernel patch was never production ready)
In SL 9.0 you can use the commands in the scheduler-utilities rpm (not installed by default, but on the DVD)
SLES8 currently doesn't have a tool, but there may be an update with a scheduler-utilities rpm. If you cannot wait for that you could compile the SL9 rpm yourself there.
-Andi
A quick update on this. It seem that the numactl shipped with suse9.0 doesn't work for --cpubind= - it simply ignores this option. However if I remove this and install the one shipped with SLES8-SP3 it does work. This is for the shipped '2.4.21-193-smp' kernel In all cases though /usr/bin/numademo reports: "your system does not support the numa API." I am looking through the source now. Also all the other options ( --membind, --localalloc, --interleave, --homenode) fail with: "numactl: this system does not support NUMA policy" presumably after check_numa() calls numa_available() which in turn calls numa_homenode() Daniel. On Wednesday 17 March 2004 10:27 am, Dan Kidger wrote:
Andi / Andreas, thanks for the information.
We patch and compile our our own kernel - enabling the NUMA options amiongst other things.
Under sles8 I installed numactl (from SP3) - it worked perfectly. :-) I could not only see the lastCPUused under top - but I coudl detect the performance change when I use a CPU close/far from the Golum PCI-X chips.
Under suse9 - I installed the suse9 numactl - but it seem to be silently ignored - even though the the kernel binary is eactly the same as the sles8 cluster (2.4.21-171numa). I am investigating.
Finally I tried 'scheduler-utilities' under suse9 - this only contains one file: $ rpm -qlp \ /home/packages/Linux_x86_64/suse9.0/scheduler-utilities-0.1-150.x86_64.rpm /usr/sbin/resched
and this doesn't seem to have any NUMA options - just the usual nice-value et al. Am I missing something ?
Daniel.
-------------------------------------------------------------- Dr. Dan Kidger, Quadrics Ltd. daniel.kidger@quadrics.com One Bridewell St., Bristol, BS1 2AA, UK 0117 915 5505 ----------------------- www.quadrics.com --------------------
On Tuesday 16 March 2004 8:46 pm, Andi Kleen wrote:
On Tue, 16 Mar 2004 20:34:45 +0100
Andreas Jaeger
wrote: daniel.kidger@quadrics.com writes:
I have two Opteron clusters - one SLES8 based - the other Suse9. Both are running a 2.4.21 kernel with numa enabled.
Are there any commands available to launch a process onto a given cpu (or set of cpus) within an SMP ?
There's a numactl package that offers some options - but also some options that are not implemented in the kernel...
numactl won't work on the shipped kernels (it was for an experimental kernel patch was never production ready)
In SL 9.0 you can use the commands in the scheduler-utilities rpm (not installed by default, but on the DVD)
SLES8 currently doesn't have a tool, but there may be an update with a scheduler-utilities rpm. If you cannot wait for that you could compile the SL9 rpm yourself there.
-Andi
On Wed, 17 Mar 2004 12:36:52 +0000
Dan Kidger
A quick update on this.
It seem that the numactl shipped with suse9.0 doesn't work for --cpubind= - it simply ignores this option. However if I remove this and install the one shipped with SLES8-SP3 it does work. This is for the shipped '2.4.21-193-smp' kernel
In all cases though /usr/bin/numademo reports: "your system does not support the numa API." I am looking through the source now.
Also all the other options ( --membind, --localalloc, --interleave, --homenode) fail with: "numactl: this system does not support NUMA policy" presumably after check_numa() calls numa_available() which in turn calls numa_homenode()
Yes, that is what I wrote in my first mail: the 2.4 NUMA API was an experimental kernel patch that was never included in any production kernel. -Andi
participants (5)
-
Andi Kleen
-
Andreas Jaeger
-
Dan Behman
-
Dan Kidger
-
daniel.kidger@quadrics.com