Runaway kworker process after last update
Hi, I just upgraded my laptop (Lenovo T470p) to latest TW 20210820, including kernel 5.13.12. Now I have a permanent kworker process running at full load: 27615 root 20 0 0 0 0 R 99.00 0.000 5:49.23 kworker/u16:3+events_unbound Is there a way to find out what exactly it is related with?
Peter Suetterlin wrote:
Hi,
I just upgraded my laptop (Lenovo T470p) to latest TW 20210820, including kernel 5.13.12.
Now I have a permanent kworker process running at full load:
27615 root 20 0 0 0 0 R 99.00 0.000 5:49.23 kworker/u16:3+events_unbound
Is there a way to find out what exactly it is related with?
So it's not running permanently, for a minute or so, and then stops again. Absolutely zero info in the syslog :(
W dniu 24.08.2021 o 20:06, Peter Suetterlin pisze:
Peter Suetterlin wrote:
Hi,
I just upgraded my laptop (Lenovo T470p) to latest TW 20210820, including kernel 5.13.12.
Now I have a permanent kworker process running at full load:
27615 root 20 0 0 0 0 R 99.00 0.000 5:49.23 kworker/u16:3+events_unbound
Is there a way to find out what exactly it is related with?
So it's not running permanently, for a minute or so, and then stops again. Absolutely zero info in the syslog :(
Sorry for not being helpful, but I just wanted to add that I also noticed this.
For systems using btrfs it may be related to this: https://bugzilla.kernel.org/show_bug.cgi?id=212185 One of the notebooks I work on had this issue a while ago and I solved it by: 1) adjusting snapper config to be easier on my small (19Gb) root partition 2) running btrfs balance # snapper set-config SPACE_LIMIT=0.6 # snapper set-config NUMBER_LIMIT_IMPORTANT=4-8 # snapper set-config NUMBER_LIMIT=2-8 # btrfs balance start -dusage=80 /
On 24. 08. 21, 17:42, Peter Suetterlin wrote:
Hi,
I just upgraded my laptop (Lenovo T470p) to latest TW 20210820, including kernel 5.13.12.
Now I have a permanent kworker process running at full load:
27615 root 20 0 0 0 0 R 99.00 0.000 5:49.23 kworker/u16:3+events_unbound
Is there a way to find out what exactly it is related with?
Hi, what's the output of: cat /proc/27615/stack ? (Change 27615 to whatever PID the kworker currently has.) regards, -- js suse labs
Bruno Damasceno Freire wrote:
For systems using btrfs it may be related to this:
https://bugzilla.kernel.org/show_bug.cgi?id=212185
One of the notebooks I work on had this issue a while ago and I solved it by:
1) adjusting snapper config to be easier on my small (19Gb) root partition 2) running btrfs balance
# snapper set-config SPACE_LIMIT=0.6 # snapper set-config NUMBER_LIMIT_IMPORTANT=4-8 # snapper set-config NUMBER_LIMIT=2-8 # btrfs balance start -dusage=80 /
Yes, I had the suspicion it might be related to some btrfs balance or similar, but would have expected some hint in the syslog that balancing is actually started - which is not the case. Anyhow, good to see it's not only me seeing this. So I'm not paranoid ;^> Jiri Slaby had asked me to check /proc/PID/stack of that process. Unfortunately, I haven't seen it again since yesterday - which could be a hint to some maintenence thing going on...
On Wednesday 2021-08-25 07:53, Jiri Slaby wrote:
On 24. 08. 21, 17:42, Peter Suetterlin wrote:
I just upgraded my laptop (Lenovo T470p) to latest TW 20210820, including kernel 5.13.12.
Now I have a permanent kworker process running at full load:
27615 root 20 0 0 0 0 R 99.00 0.000 5:49.23 kworker/u16:3+events_unbound
Is there a way to find out what exactly it is related with?
Hi,
what's the output of: cat /proc/27615/stack ? (Change 27615 to whatever PID the kworker currently has.)
As an aside, under 5.13.8, when I do some I/O with btrfs, these processes shoot to the top with /usr/bin/top: 32110 root 20 0 0 0 0 I 24.75 0.000 0:03.32 kworker/u16:7-btrfs-worker 4736 root 20 0 0 0 0 I 22.77 0.000 0:00.31 kworker/u16:5-btrfs-worker 5097 root 20 0 0 0 0 I 6.931 0.000 0:00.07 kworker/u16:15-flush-btrfs-1 So it's already obvious they are associated with btrfs. But perhaps that is not the case with rebalance (I don't have a suitably large fs at hand ATM); if not, can it be made so? Or is the events_unbound queue kind of a problem here..
On 8/24/21 11:53 PM, Jiri Slaby wrote:
On 24. 08. 21, 17:42, Peter Suetterlin wrote:
Hi,
I just upgraded my laptop (Lenovo T470p) to latest TW 20210820, including kernel 5.13.12.
Now I have a permanent kworker process running at full load:
27615 root 20 0 0 0 0 R 99.00 0.000 5:49.23 kworker/u16:3+events_unbound
I encountered this issue last week and again today. 11663 root 20 0 0 0 0 R 100.0 0.000 20:41.19 kworker/u16:0+events_unbound
Is there a way to find out what exactly it is related with?
Hi,
what's the output of: cat /proc/27615/stack
Nothing # cat /proc/11663/stack # Maybe some other info in procfs will be helpful # cat /proc/11663/stat 11663 (kworker/u16:0+events_unbound) R 2 0 0 0 -1 69238880 0 0 0 0 0 98281 0 0 20 0 1 0 11727122 0 0 18446744073709551615 0 0 0 0 0 0 0 2147483647 0 0 0 0 17 6 0 0 175 0 0 0 0 0 0 0 0 0 0 # cat /proc/11663/statm 0 0 0 0 0 0 0 # cat /proc/11663/syscall running # cat /proc/11663/status Name: kworker/u16:0+events_unbound Umask: 0000 State: R (running) Tgid: 11663 Ngid: 0 Pid: 11663 PPid: 2 TracerPid: 0 Uid: 0 0 0 0 Gid: 0 0 0 0 FDSize: 64 Groups: NStgid: 11663 NSpid: 11663 NSpgid: 0 NSsid: 0 Threads: 1 SigQ: 0/127677 SigPnd: 0000000000000000 ShdPnd: 0000000000000000 SigBlk: 0000000000000000 SigIgn: ffffffffffffffff SigCgt: 0000000000000000 CapInh: 0000000000000000 CapPrm: 000001ffffffffff CapEff: 000001ffffffffff CapBnd: 000001ffffffffff CapAmb: 0000000000000000 NoNewPrivs: 0 Seccomp: 0 Seccomp_filters: 0 Speculation_Store_Bypass: thread vulnerable SpeculationIndirectBranch: conditional enabled Cpus_allowed: ff Cpus_allowed_list: 0-7 Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000001 Mems_allowed_list: 0 voluntary_ctxt_switches: 12690 nonvoluntary_ctxt_switches: 11059 # cat /proc/11663/wchan 0 Time to reboot before something becomes toast... Jim
G'day Tumblers, On 31/8/21 09:04, Jim Fehlig wrote:
On 8/24/21 11:53 PM, Jiri Slaby wrote:
On 24. 08. 21, 17:42, Peter Suetterlin wrote:
Hi,
I just upgraded my laptop (Lenovo T470p) to latest TW 20210820, including kernel 5.13.12.
Now I have a permanent kworker process running at full load:
27615 root 20 0 0 0 0 R 99.00 0.000 5:49.23 kworker/u16:3+events_unbound
I encountered this issue last week and again today.
11663 root 20 0 0 0 0 R 100.0 0.000 20:41.19 kworker/u16:0+events_unbound
Is there a way to find out what exactly it is related with?
Time to reboot before something becomes toast...
Jim
This happened to me about a two months ago, then again a few weeks ago. For me, this was btrfs doing strange things. My laptop is also a Lenovo (Thinkpad T580) A reboot fixed it for an hour or so, but a rebalance seemed to fix it completely each time. I seemed to coincide with when I run my firefox container under docker (old Firefox with Flash and Java), so I thought it may be an issue with the docker plugin causing a lot of BTRFS activity and the kernel doing.. something. My filesystem is not super full, but it is on a Samsung 970 PRO NVMe so the CPU will get a little smashed with the data throughput when background 'stuff' happens which is only I/O limited. It does not happen every time, but it never seems to happen any other time for me. Either way, a rebalnce stopped it and it went away. The fan on my laptop gets very annoying otherwise. Not sure if this correlates with anything anyone else is seeing. Unfortunately, I do not have anywhere near enough information to actually file a quarter decent bug report, but I can confirm this has been happening since at least 5.12.x and it seems to also be plastered across the Fedora mailing list as well (but once again, nobody gathering enough information for a real bug report). Always that same events_unbound. -- Ben
participants (7)
-
Adam Mizerski -
Ben Holmes -
Bruno Damasceno Freire -
Jan Engelhardt -
Jim Fehlig -
Jiri Slaby -
Peter Suetterlin