[Bug 1229085] New: irq/9-acpi high cpu usage with kernel 6.10.3-1-default
https://bugzilla.suse.com/show_bug.cgi?id=1229085 Bug ID: 1229085 Summary: irq/9-acpi high cpu usage with kernel 6.10.3-1-default Classification: openSUSE Product: openSUSE Tumbleweed Version: Slowroll Hardware: x86-64 OS: openSUSE Tumbleweed Status: NEW Severity: Normal Priority: P5 - None Component: Kernel Assignee: kernel-bugs@opensuse.org Reporter: petr@jevklidu.cz QA Contact: qa-bugs@suse.de Target Milestone: --- Found By: --- Blocker: --- Hello, after upgrading to kernel 6.10.3-1-default I have noticed 85% cpu usage of single core by irq/9-acpi. System load with no usage is about 2.6 ...
cat /proc/interrupts | awk ' /^\s+9:/ {print}' 9: 20220768 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-IO-APIC 9-fasteoi acpi
I haven't seen this with kernel 6.9.9 but I will boot to older kernel to be sure. Reboot doesn't help. Thank you. Petr -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c1 Petr Valenta <petr@jevklidu.cz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |petr@jevklidu.cz --- Comment #1 from Petr Valenta <petr@jevklidu.cz> --- There is no such problem with kernel 6.9.9-1-default. Petr -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c3 --- Comment #3 from Petr Valenta <petr@jevklidu.cz> --- Created attachment 876642 --> https://bugzilla.suse.com/attachment.cgi?id=876642&action=edit hwinfo output with kernel-6.10.3 -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c4 --- Comment #4 from Petr Valenta <petr@jevklidu.cz> --- Created attachment 876643 --> https://bugzilla.suse.com/attachment.cgi?id=876643&action=edit dmesg with kernel 6.10.3 -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c5 --- Comment #5 from Petr Valenta <petr@jevklidu.cz> --- Created attachment 876644 --> https://bugzilla.suse.com/attachment.cgi?id=876644&action=edit hwinfo output with kernel-6.9.9 -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c6 --- Comment #6 from Petr Valenta <petr@jevklidu.cz> --- Created attachment 876645 --> https://bugzilla.suse.com/attachment.cgi?id=876645&action=edit dmesg with kernel 6.9.9 -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c7 --- Comment #7 from Petr Valenta <petr@jevklidu.cz> --- I have tries all kernels you mentioned: 6.10.{2,3,4} - same problem 6.11-rc - same problem too So only working right now is 6.9.9 I am attaching hwinfo a dmesg output for kernel 6.9.9 and 6.10.3 as you wanted. -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c10 --- Comment #10 from Petr Valenta <petr@jevklidu.cz> --- (In reply to Takashi Iwai from comment #8)
Thanks. Then it's worth to report to the upstream as a regression:
https://www.kernel.org/doc/html/latest/admin-guide/reporting-regressions.htm... Care to report there?
I have never reported bug or regression to kernel mailing list so first I have to study all the things on link above. However, I will hope that the problem will disappear in the meantime. -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c11 --- Comment #11 from Petr Valenta <petr@jevklidu.cz> --- Created attachment 876664 --> https://bugzilla.suse.com/attachment.cgi?id=876664&action=edit dmesg with kernel 6.10.3 with additional boot paramaters: acpi.debug_level=0x08000000 acpi.debug_layer=0xffffffff -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c12 --- Comment #12 from Petr Valenta <petr@jevklidu.cz> --- (In reply to Petr Valenta from comment #11)
Created attachment 876664 [details] dmesg with kernel 6.10.3 with additional boot paramaters: acpi.debug_level=0x08000000 acpi.debug_layer=0xffffffff
I have attached the required dmesg output with acpi.debug_level=0x08000000 acpi.debug_layer=0xffffffff. There is a interesting consequence of these kernel parameters because after adding these, problem with high cpu usage is gone. With booting without these parameters 1 cpu core is fully utilized again. After adding them back again problem is gone. Thank you for this workaround :) -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c15 --- Comment #15 from Petr Valenta <petr@jevklidu.cz> --- Created attachment 876675 --> https://bugzilla.suse.com/attachment.cgi?id=876675&action=edit journalctl -b -t kernel output for 6.9.9 and 6.10.3 kernel with: acpi.debug_level=0x08000000 acpi.debug_layer=0xffffffff -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c16 --- Comment #16 from Petr Valenta <petr@jevklidu.cz> --- Created attachment 876676 --> https://bugzilla.suse.com/attachment.cgi?id=876676&action=edit requested interrupts output for 6.10.3 with and without acpi debug -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c19 --- Comment #19 from Petr Valenta <petr@jevklidu.cz> --- Rafael J. Wysocki suggested this:
echo mask > /sys/firmware/acpi/interrupts/gpe6D
and high CPU usage by irq/9-acpi is immediately gone. -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c21 --- Comment #21 from Petr Valenta <petr@jevklidu.cz> --- (In reply to Jiri Slaby from comment #20)
Could you try a kernel with:
commit b2c289415b2b2ef112b78d5e73b4acecf5db409e Author: Bjorn Helgaas <bhelgaas@google.com> Date: Mon Mar 25 17:29:49 2024 -0500
e1000e: Remove redundant runtime resume for ethtool_ops
reverted in: https://build.opensuse.org/project/monitor/home:jirislaby:stable-bsc1229085
uname -r 6.10.5-4.g30fd964-default echo 'auto' > /sys/bus/pci/devices/0000:00:1f.6/power/control Command above causes irq storm again same as with official distribution kernel. During testing and typing this reply (uptime few minutes) high cpu usage is gone. I will check the same with distribution kernel. -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c22 --- Comment #22 from Petr Valenta <petr@jevklidu.cz> --- (In reply to Petr Valenta from comment #21)
(In reply to Jiri Slaby from comment #20)
Could you try a kernel with:
commit b2c289415b2b2ef112b78d5e73b4acecf5db409e Author: Bjorn Helgaas <bhelgaas@google.com> Date: Mon Mar 25 17:29:49 2024 -0500
e1000e: Remove redundant runtime resume for ethtool_ops
reverted in: https://build.opensuse.org/project/monitor/home:jirislaby:stable-bsc1229085
uname -r 6.10.5-4.g30fd964-default
echo 'auto' > /sys/bus/pci/devices/0000:00:1f.6/power/control
Command above causes irq storm again same as with official distribution kernel. During testing and typing this reply (uptime few minutes) high cpu usage is gone. I will check the same with distribution kernel.
uname -r 6.10.5-1-default uptime 6 minutes and irq storm is gone -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c24 --- Comment #24 from Petr Valenta <petr@jevklidu.cz> --- (In reply to Jiri Slaby from comment #23)
So I assume this was fixed somewhere betwenn 6.10.3 and 6.10.5?
Not exactly. When I reboot a run this command: echo 'auto' > /sys/bus/pci/devices/0000:00:1f.6/power/control then one CPU core is fully utilized for about 6 minutes by irq/9-acpi and after that time IRQ storm disappears. This does not happen with kernel 6.9.9. I have tried self compiled kernel 6.9.12 and it was ok too. So IRQ storm starts to happen with 6.10. This finding is quite enough for me because I will avoid using power control of ethernet controller in the future. But some sort of problem is still in 6.{10,11}.x Hope it's more clear now. -- You are receiving this mail because: You are the assignee for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229085 https://bugzilla.suse.com/show_bug.cgi?id=1229085#c26 --- Comment #26 from Petr Valenta <petr@jevklidu.cz> --- (In reply to Jiri Slaby from comment #25)
(In reply to Petr Valenta from comment #24)
This does not happen with kernel 6.9.9. I have tried self compiled kernel 6.9.12 and it was ok too. So IRQ storm starts to happen with 6.10.
So this happens the same with the revert in my repo, right? I will try to revert also the second patch if so.
exactly, this happens the same with kernel 6.10.5-4.g30fd964-default from your repo -- You are receiving this mail because: You are the assignee for the bug.
participants (1)
-
bugzilla_noreply@suse.com