Linda Walsh <suse@tlinx.org> wrote:
I can easily have missed something -- and if I have, please tell me -- but it seems to me that user and system times from /proc/stat and /proc/pid/stat should have matching units. My next best guess is that there are things attributed to the utime/stime of a process which are attributed to different activities on the CPU level.
Double and triple sigh! The docs out of date w/r/t/ the code? Unheard of!! ;^) But the jiffies would seem to be a questionable indicator of cpu time, since they don't seem to be based on an absolute time, but an amount of cpu time. The amount of that would vary based on what speed each of the cores are going.
I don't know about your specific cpu model, but things like 'turbo mode', where the clock runs at 1-4, more, added multiples of 133MHz when some or all of the other cores are idle.
I am testing on an Intel Core i7-4770, a quadcore with hyperthreading and turbo mode.
Maybe if you disabled power control and forced your cpu into 1 speed the numbers might get closer?
I tried a number of approaches today: using the cpufreq "performance" governor to disable frequency scaling, disabling turbo mode in the firmware, disabling hyperthreading, taking cores and hardware threads offline, pinning the process to specific cores and/or hardware threads using taskset, and combinations of these things. The only time the CPU and process times matched was when I was taking all but one virtual CPU (= hardware thread) offline, or when I was pinning the thread to exactly one specific virtual CPU. I didn't see any significant improvement with anything else I tried.
Erratum depressing: The worst news to me is how accurate perf events and timer events really are. In the processor erratum on the 5600 series, it includes dropped interrupts for the timer among others. [...]
That does sound bad. But while this could skew the results, I think missed timer interrupts should result in lost jiffies for both the process and CPU counts.
Python's multiprocessing package should offer better utilization since it avoids the GIL by spawning processes instead of threads, which in this case would defeat the purpose of the script.
But mapping it to procs would be a better mapping on linux to reality, as linux threads are built on procs with varying amounts of memory shared.
[...]
Perl, BTW, went the way of linux -- building it's threads on procs, which is a likely contributing cause to their efficiency on linux.
That is true, however the application(s) that I want to observe this way already exist and are making extensive use of threads. However, on the matter of processes vs threads -- it might be that the continuous summation of per-task (i.e. thread) counts to the per-process counts is inexact somehow. I'll test if there is a significant deviance between the sum of per-thread utimes and the process utime tomorrow.
Oh, another "gotcha", is how timing is done. During boot, on my system, I see: [...]
So the jiffies calculated via the processors's speed, are counted in terms of the HPET clocksource, which is accurate to about .014ms. @ 5586 jiffies, that equates to a maximum accuracy of ~80 jiffies (79.98) / tick.
That, alone could account for quite a skew.
That is significant. However, I would expect that it also skews both the per-process and CPU-wide jiffy counts.
Given that skewing, and the erratum problems, I wouldn't expect too much in the way of accurate timing info on one of these processors... *cough*...(sigh)
Actually, I'm not really after highly accurate timing or resource usage data. What I actually want is to estimate how much particular processes or threads in a guest VM are affected by time stolen by the hypervisor. My idea is to see what fractions of the entire guest's user and system time over short periods are consumed by which tasks, and then attribute the steal time recorded in the same period to them, at the same ratios. The emphasis here being strongly on the word "estimate" ;) -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org