[opensuse-virtual] full cstate/cpufreq/cpupower support withOUT Xen 4.13 on kernel 5.4.14; WITH xen, none at all. bug or config?
( I'd already posted this at xen-users; no traction to date ) I'm running linux kernel lsb_release -rd Description: openSUSE Leap 15.1 Release: 15.1 uname -rm 5.4.14-24.gfc4ea7a-default x86_64 dmesg | grep DMI: [ 0.000000] DMI: Supermicro X10SAT/X10SAT, BIOS 3.0 05/26/2015 cat /proc/cpuinfo | grep "model name" | head -n 1 model name : Intel(R) Xeon(R) CPU E3-1220 v3 @ 3.10GHz kernel & xen are pkg-installed from my KernelStable and Virtualization-Xen repos @ OBS. BIOS *is* setup for max cstate support. Xeon E3-1220 does support intel_pstate driver. Testing first, (1) boot, NO XEN pstate driver's init'd dmesg | egrep -i "intel_pstate" [ 6.132964] intel_pstate: Intel P-state driver initializing pstate/cstate info cat /sys/module/intel_idle/parameters/max_cstate 9 cd /sys/devices/system/cpu/cpu0/cpuidle for state in state{0..9} do echo c-$state `cat $state/name` `cat $state/latency` done c-state0 POLL 0 c-state1 C1 2 c-state2 C1E 10 c-state3 C3 33 c-state4 C6 133 c-state5 C7s 166 cat: state6/name: No such file or directory cat: state6/latency: No such file or directory c-state6 cat: state7/name: No such file or directory cat: state7/latency: No such file or directory c-state7 cat: state8/name: No such file or directory cat: state8/latency: No such file or directory c-state8 cat: state9/name: No such file or directory cat: state9/latency: No such file or directory c-state9 cpufreq scaling info's available, cpupower frequency-info analyzing CPU 0: driver: intel_pstate CPUs which run at the same hardware frequency: 0 CPUs which need to have their frequency coordinated by software: 0 maximum transition latency: Cannot determine or is not supported. hardware limits: 800 MHz - 3.50 GHz available cpufreq governors: performance powersave current policy: frequency should be within 800 MHz and 3.50 GHz. The governor "powersave" may decide which speed to use within this range. current CPU frequency: Unable to call hardware current CPU frequency: 799 MHz (asserted by call to kernel) boost state support: Supported: yes Active: yes & scaling is in effect, cat /proc/cpuinfo | grep MHz cpu MHz : 798.106 cpu MHz : 798.129 cpu MHz : 798.964 cpu MHz : 798.154 (2) boot, WITH Xen 4.13 rpm -qa | grep -i xen | sort grub2-x86_64-xen-2.04-lp151.6.5.noarch xen-4.13.0_04-lp151.688.2.x86_64 xen-libs-4.13.0_04-lp151.688.2.x86_64 xen-tools-4.13.0_04-lp151.688.2.x86_64 Xen cmd line includes, grep options= /boot/grub2/xen-4.13.0_04-lp151.688.cfg [config.1] options=dom0=pvh dom0-iommu=map-reserved dom0_mem=4016M,max:4096M dom0_max_vcpus=4 cpufreq=xen cpuidle ucode=scan ... intel_pstate support is now DISABLED for this cpu xl dmesg | grep pstate [ 6.851121] intel_pstate: CPU model not supported c-states report, xenpm get-cpuidle-states 0 All C-states allowed cpu id : 0 total C-states : 6 idle time(ms) : 45780911 C0 : transition [ 3204855] residency [ 160769 ms] C1 : transition [ 9204] residency [ 1018 ms] C2 : transition [ 10181] residency [ 2848 ms] C3 : transition [ 22784] residency [ 17236 ms] C4 : transition [ 7181] residency [ 11793 ms] C5 : transition [ 3155504] residency [ 45668846 ms] pc2 : [ 1685 ms] pc3 : [ 30695 ms] cc3 : [ 16858 ms] cc6 : [ 11640 ms] cc7 : [ 45602872 ms] NO cpupower frequency-info is available cpupower frequency-info analyzing CPU 0: no or unknown cpufreq driver is active on this CPU CPUs which run at the same hardware frequency: Not Available CPUs which need to have their frequency coordinated by software: Not Available maximum transition latency: Cannot determine or is not supported. Not Available available cpufreq governors: Not Available Unable to determine current policy current CPU frequency: Unable to call hardware current CPU frequency: Unable to call to kernel boost state support: Supported: no Active: no and scaling is NOT in effect cat /proc/cpuinfo | grep MHz cpu MHz : 3092.828 cpu MHz : 3092.828 cpu MHz : 3092.828 cpu MHz : 3092.828 attempt to add acpi-cpufreq module fails lsmod | grep acpi-cpufreq (empty) find /lib/modules/ | grep acpi-cpu /lib/modules/5.4.14-24.gfc4ea7a-default/kernel/drivers/cpufreq/acpi-cpufreq.ko modprobe acpi-cpufreq modprobe: ERROR: could not insert 'acpi_cpufreq': No such device insmod /lib/modules/5.4.14-24.gfc4ea7a-default/kernel/drivers/cpufreq/acpi-cpufreq.ko insmod: ERROR: could not insert module /lib/modules/5.4.14-24.gfc4ea7a-default/kernel/drivers/cpufreq/acpi-cpufreq.ko: No such device Is this bug, or config? -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
On 28.01.2020 16:56, PGNet Dev wrote:
( I'd already posted this at xen-users; no traction to date )
I'm running linux kernel
lsb_release -rd Description: openSUSE Leap 15.1 Release: 15.1
uname -rm 5.4.14-24.gfc4ea7a-default x86_64
dmesg | grep DMI: [ 0.000000] DMI: Supermicro X10SAT/X10SAT, BIOS 3.0 05/26/2015
cat /proc/cpuinfo | grep "model name" | head -n 1 model name : Intel(R) Xeon(R) CPU E3-1220 v3 @ 3.10GHz
kernel & xen are pkg-installed from my KernelStable and Virtualization-Xen repos @ OBS.
BIOS *is* setup for max cstate support. Xeon E3-1220 does support intel_pstate driver.
Testing first,
(1) boot, NO XEN
pstate driver's init'd
dmesg | egrep -i "intel_pstate" [ 6.132964] intel_pstate: Intel P-state driver initializing
pstate/cstate info
cat /sys/module/intel_idle/parameters/max_cstate 9
cd /sys/devices/system/cpu/cpu0/cpuidle for state in state{0..9} do echo c-$state `cat $state/name` `cat $state/latency` done c-state0 POLL 0 c-state1 C1 2 c-state2 C1E 10 c-state3 C3 33 c-state4 C6 133 c-state5 C7s 166 cat: state6/name: No such file or directory cat: state6/latency: No such file or directory c-state6 cat: state7/name: No such file or directory cat: state7/latency: No such file or directory c-state7 cat: state8/name: No such file or directory cat: state8/latency: No such file or directory c-state8 cat: state9/name: No such file or directory cat: state9/latency: No such file or directory c-state9
cpufreq scaling info's available,
cpupower frequency-info analyzing CPU 0: driver: intel_pstate CPUs which run at the same hardware frequency: 0 CPUs which need to have their frequency coordinated by software: 0 maximum transition latency: Cannot determine or is not supported. hardware limits: 800 MHz - 3.50 GHz available cpufreq governors: performance powersave current policy: frequency should be within 800 MHz and 3.50 GHz. The governor "powersave" may decide which speed to use within this range. current CPU frequency: Unable to call hardware current CPU frequency: 799 MHz (asserted by call to kernel) boost state support: Supported: yes Active: yes
& scaling is in effect,
cat /proc/cpuinfo | grep MHz cpu MHz : 798.106 cpu MHz : 798.129 cpu MHz : 798.964 cpu MHz : 798.154
(2) boot, WITH Xen 4.13
rpm -qa | grep -i xen | sort grub2-x86_64-xen-2.04-lp151.6.5.noarch xen-4.13.0_04-lp151.688.2.x86_64 xen-libs-4.13.0_04-lp151.688.2.x86_64 xen-tools-4.13.0_04-lp151.688.2.x86_64
Xen cmd line includes,
grep options= /boot/grub2/xen-4.13.0_04-lp151.688.cfg [config.1] options=dom0=pvh dom0-iommu=map-reserved dom0_mem=4016M,max:4096M dom0_max_vcpus=4 cpufreq=xen cpuidle ucode=scan ...
intel_pstate support is now DISABLED for this cpu
xl dmesg | grep pstate [ 6.851121] intel_pstate: CPU model not supported
c-states report,
xenpm get-cpuidle-states 0
All C-states allowed
cpu id : 0 total C-states : 6 idle time(ms) : 45780911 C0 : transition [ 3204855] residency [ 160769 ms] C1 : transition [ 9204] residency [ 1018 ms] C2 : transition [ 10181] residency [ 2848 ms] C3 : transition [ 22784] residency [ 17236 ms] C4 : transition [ 7181] residency [ 11793 ms] C5 : transition [ 3155504] residency [ 45668846 ms] pc2 : [ 1685 ms] pc3 : [ 30695 ms] cc3 : [ 16858 ms] cc6 : [ 11640 ms] cc7 : [ 45602872 ms]
NO cpupower frequency-info is available
cpupower frequency-info analyzing CPU 0: no or unknown cpufreq driver is active on this CPU CPUs which run at the same hardware frequency: Not Available CPUs which need to have their frequency coordinated by software: Not Available maximum transition latency: Cannot determine or is not supported. Not Available available cpufreq governors: Not Available Unable to determine current policy current CPU frequency: Unable to call hardware current CPU frequency: Unable to call to kernel boost state support: Supported: no Active: no
and scaling is NOT in effect
cat /proc/cpuinfo | grep MHz cpu MHz : 3092.828 cpu MHz : 3092.828 cpu MHz : 3092.828 cpu MHz : 3092.828
attempt to add acpi-cpufreq module fails
lsmod | grep acpi-cpufreq (empty) find /lib/modules/ | grep acpi-cpu /lib/modules/5.4.14-24.gfc4ea7a-default/kernel/drivers/cpufreq/acpi-cpufreq.ko modprobe acpi-cpufreq modprobe: ERROR: could not insert 'acpi_cpufreq': No such device insmod /lib/modules/5.4.14-24.gfc4ea7a-default/kernel/drivers/cpufreq/acpi-cpufreq.ko insmod: ERROR: could not insert module /lib/modules/5.4.14-24.gfc4ea7a-default/kernel/drivers/cpufreq/acpi-cpufreq.ko: No such device
Is this bug, or config?
Neither (at least as far as Xen and the kernel are concerned). The management of this lives in Xen, and hence the kernel doesn't know (and kernel drivers don't get involved). The various user space tools would of course be nice if they knew how to get at the relevant data, but that's a separate issue. Until then xenpm is what shows you some of the relevant data (and if more is needed, such could be discussed on xen-devel). Just like you've used "xenpm get-cpuidle-states 0" to get C-state information, you should be able to obtain P-state one via "xenpm get-cpufreq-states 0". Jan -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
On 1/28/20 8:06 AM, Jan Beulich wrote:
Just like you've used "xenpm get-cpuidle-states 0" to get C-state information, you should be able to obtain P-state one via "xenpm get-cpufreq-states 0".
i agree with the _should_ part. unfortunately, it's not currently consistenly responsive, xenpm start 5 Timeout set to 5 seconds Start sampling, waiting for CTRL-C or SIGINT or SIGALARM signal ... Elapsed time (ms): 5000 CPU0: Residency(ms) Avg Res(ms) C0 19 ( 0.40%) 0.03 C1 0 ( 0.00%) 0.00 C2 0 ( 0.00%) 0.00 C3 1 ( 0.02%) 0.22 C4 1 ( 0.02%) 0.59 C5 4978 (99.56%) 8.07 Avg freq -302378336 KHz CPU1: Residency(ms) Avg Res(ms) C0 3 ( 0.08%) 0.03 C1 0 ( 0.00%) 0.00 C2 0 ( 0.00%) 0.00 C3 0 ( 0.00%) 0.00 C4 0 ( 0.00%) 0.00 C5 4996 (99.92%) 35.94 Avg freq -302378336 KHz CPU2: Residency(ms) Avg Res(ms) C0 5000 (100.00%) 5000.27 C1 0 ( 0.00%) 0.00 C2 0 ( 0.00%) 0.00 C3 0 ( 0.00%) 0.00 C4 0 ( 0.00%) 0.00 C5 0 ( 0.00%) 0.00 Avg freq -302378336 KHz CPU3: Residency(ms) Avg Res(ms) C0 5 ( 0.11%) 0.03 C1 0 ( 0.00%) 0.00 C2 0 ( 0.00%) 0.00 C3 0 ( 0.01%) 0.32 C4 0 ( 0.00%) 0.00 C5 4994 (99.88%) 25.48 Avg freq -302378336 KHz Socket 0 PC1 0 ms 0.00% PC2 0 ms 0.00% PC3 0 ms 0.00% Core 0 CPU 0 CC1 0 ms 0.00% CC2 0 ms 0.00% CC3 1 ms 0.02% CC4 0 ms 0.00% CC5 0 ms 0.00% CC6 1 ms 0.02% CC7 4965 ms 99.30% Core 1 CPU 1 CC1 0 ms 0.00% CC2 0 ms 0.00% CC3 0 ms 0.00% CC4 0 ms 0.00% CC5 0 ms 0.00% CC6 0 ms 0.00% CC7 4993 ms 99.86% Core 2 CPU 2 CC1 0 ms 0.00% CC2 0 ms 0.00% CC3 0 ms 0.00% CC4 0 ms 0.00% CC5 0 ms 0.00% CC6 0 ms 0.00% CC7 0 ms 0.00% Core 3 CPU 3 CC1 0 ms 0.00% CC2 0 ms 0.00% CC3 0 ms 0.01% CC4 0 ms 0.00% CC5 0 ms 0.00% CC6 0 ms 0.00% CC7 4989 ms 99.79% xenpm get-cpuidle-states 0 All C-states allowed cpu id : 0 total C-states : 6 idle time(ms) : 46635271 C0 : transition [ 3242686] residency [ 162089 ms] C1 : transition [ 9214] residency [ 1018 ms] C2 : transition [ 10254] residency [ 2871 ms] C3 : transition [ 23259] residency [ 17544 ms] C4 : transition [ 7252] residency [ 11896 ms] C5 : transition [ 3192706] residency [ 46522535 ms] pc2 : [ 1685 ms] pc3 : [ 30695 ms] cc3 : [ 17158 ms] cc6 : [ 11742 ms] cc7 : [ 46455767 ms] but, freq states not so much atm, xenpm get-cpufreq-states (empty) xenpm get-cpufreq-para [CPU0] failed to get cpufreq parameter [CPU1] failed to get cpufreq parameter [CPU2] failed to get cpufreq parameter [CPU3] failed to get cpufreq parameter xenpm enable-turbo-mode [CPU0] failed to enable turbo mode (13 - Permission denied) [CPU1] failed to enable turbo mode (13 - Permission denied) [CPU2] failed to enable turbo mode (13 - Permission denied) [CPU3] failed to enable turbo mode (13 - Permission denied) i did find this _old_ (2013) post, Linux 3.4 dom0 kernel error loading xen-acpi-processor: Input/output error https://lists.gt.net/xen/devel/274864 which similarly manifests no cpufreq/turbomode in Dom0, and a fail to load 'xen_acpi_processor' mod unless/until > I found this: http://en.community.dell.com/techcenter/power-cooling/w/wiki/best-practices-... > > So I had to enable "OS Control" for "Power Management" in the Dell server BIOS, > and after that the CPU P-states are available in the ACPI tables, > and xen-acpi-processor driver loads and works OK in the dom0 kernel! Here, on my SuperMicro server, lsmod | grep xen xen_pciback 81920 0 xen_netback 73728 0 xen_blkback 53248 0 xen_gntalloc 20480 0 xen_gntdev 45056 1 xen_evtchn 16384 0 xenfs 16384 1 xen_privcmd 28672 17 xenfs as in that post modprobe -v xen-acpi-processor insmod /lib/modules/5.4.14-25.g170524c-default/kernel/drivers/xen/xen-acpi-processor.ko modprobe: ERROR: could not insert 'xen_acpi_processor': No such device checking the man https://www.supermicro.com/manuals/motherboard/C226/MNL-1544.pdf SM's BIOS doesn't have an identical "OS Control" option. don't recognize what'd be similar ... -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
On 28.01.2020 17:14, PGNet Dev wrote:
Here, on my SuperMicro server,
lsmod | grep xen xen_pciback 81920 0 xen_netback 73728 0 xen_blkback 53248 0 xen_gntalloc 20480 0 xen_gntdev 45056 1 xen_evtchn 16384 0 xenfs 16384 1 xen_privcmd 28672 17 xenfs
as in that post
modprobe -v xen-acpi-processor insmod /lib/modules/5.4.14-25.g170524c-default/kernel/drivers/xen/xen-acpi-processor.ko modprobe: ERROR: could not insert 'xen_acpi_processor': No such device
Well, this is what wants investigating then. I'd suggest to report to xen-devel, with suitable hypervisor log (complete, maximum log level) and at least the respective kernel log fragment. I recall fixing on instance this when having encountered it here. I also vaguely recall thinking that there likely are more quirks there. Jan -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
On 1/28/20 8:22 AM, Jan Beulich wrote:
Well, this is what wants investigating then. I'd suggest to report to xen-devel, with suitable hypervisor log (complete, maximum log level) and at least the respective kernel log fragment. I recall fixing on instance this when having encountered it here. I also vaguely recall thinking that there likely are more quirks there.
will do. my serial cable just fell apart -- and I've no solder handy. sigh. so no serial console output until i can pick up a spare, later :-/ is there _any_ useful logging -- even with debug level flags -- from 'just' `dmesg` output? i suspect not, and will likely need to wait for that cable ... -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
On 28.01.2020 18:03, PGNet Dev wrote:
On 1/28/20 8:22 AM, Jan Beulich wrote:
Well, this is what wants investigating then. I'd suggest to report to xen-devel, with suitable hypervisor log (complete, maximum log level) and at least the respective kernel log fragment. I recall fixing on instance this when having encountered it here. I also vaguely recall thinking that there likely are more quirks there.
will do.
my serial cable just fell apart -- and I've no solder handy. sigh. so no serial console output until i can pick up a spare, later :-/
is there _any_ useful logging -- even with debug level flags -- from 'just' `dmesg` output? i suspect not, and will likely need to wait for that cable ...
"xl dmesg" will be enough for all cases where the hoist doesn't die in some way. "dmesg" is just for kernel messages. Jan -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
On 1/28/20 9:49 AM, PGNet Dev wrote:
should appear there shortly.
fyi, https://lists.xenproject.org/archives/html/xen-devel/2020-01/msg02404.html -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
participants (2)
-
Jan Beulich -
PGNet Dev