Mailinglist Archive: opensuse-kernel (75 mails)

< Previous Next >
Re: [opensuse-kernel] Re: user/sys ticks for process exceed overall user/sys ticks
Linda Walsh <suse@xxxxxxxxx> wrote:
I can easily have missed something -- and if I have, please tell me
-- but it seems to me that user and system times from /proc/stat and
/proc/pid/stat should have matching units. My next best guess is
that there are things attributed to the utime/stime of a process
which are attributed to different activities on the CPU level.
---
Double and triple sigh! The docs out of date w/r/t/ the code?
Unheard of!! ;^) But the jiffies would seem to be a questionable
indicator of cpu time, since they don't seem to be based on an
absolute time, but an amount of cpu time. The amount of that would
vary based on what speed each of the cores are going.

I don't know about your specific cpu model, but things like 'turbo
mode', where the clock runs at 1-4, more, added multiples of 133MHz
when some or all of the other cores are idle.

I am testing on an Intel Core i7-4770, a quadcore with hyperthreading
and turbo mode.

Maybe if you disabled power control and forced your cpu into 1
speed the numbers might get closer?

I tried a number of approaches today: using the cpufreq "performance"
governor to disable frequency scaling, disabling turbo mode in the
firmware, disabling hyperthreading, taking cores and hardware threads
offline, pinning the process to specific cores and/or hardware threads
using taskset, and combinations of these things.

The only time the CPU and process times matched was when I was taking
all but one virtual CPU (= hardware thread) offline, or when I was
pinning the thread to exactly one specific virtual CPU. I didn't see any
significant improvement with anything else I tried.

Erratum depressing: The worst news to me is how accurate perf events
and timer events really are. In the processor erratum on the 5600
series, it includes dropped interrupts for the timer among others.
[...]

That does sound bad. But while this could skew the results, I think
missed timer interrupts should result in lost jiffies for both the
process and CPU counts.

Python's multiprocessing package should offer better utilization
since it avoids the GIL by spawning processes instead of threads,
which in this case would defeat the purpose of the script.

But mapping it to procs would be a better mapping on linux to
reality, as linux threads are built on procs with varying amounts of
memory shared.

[...]

Perl, BTW, went the way of linux -- building it's threads on procs,
which is a likely contributing cause to their efficiency on linux.

That is true, however the application(s) that I want to observe this
way already exist and are making extensive use of threads.

However, on the matter of processes vs threads -- it might be that the
continuous summation of per-task (i.e. thread) counts to the
per-process counts is inexact somehow. I'll test if there is a
significant deviance between the sum of per-thread utimes and the
process utime tomorrow.

Oh, another "gotcha", is how timing is done. During boot, on my
system, I see: [...]

So the jiffies calculated via the processors's speed, are counted
in terms of the HPET clocksource, which is accurate to about
.014ms. @ 5586 jiffies, that equates to a maximum accuracy of
~80 jiffies (79.98) / tick.

That, alone could account for quite a skew.

That is significant. However, I would expect that it also skews both the
per-process and CPU-wide jiffy counts.

Given that skewing, and the erratum problems, I wouldn't expect too
much in the way of accurate timing info on one of these processors...
*cough*...(sigh)

Actually, I'm not really after highly accurate timing or resource usage
data. What I actually want is to estimate how much particular processes
or threads in a guest VM are affected by time stolen by the hypervisor.
My idea is to see what fractions of the entire guest's user and system
time over short periods are consumed by which tasks, and then attribute
the steal time recorded in the same period to them, at the same ratios.
The emphasis here being strongly on the word "estimate" ;)
--
To unsubscribe, e-mail: opensuse-kernel+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-kernel+owner@xxxxxxxxxxxx

< Previous Next >
List Navigation
Follow Ups