After Fedora 33 update; kworker/u32:2+events _unbound killing my cpu

Discussions in English on installing or upgrading Fedora
#1

Fedora 33 install has machine slowing down to a crawl and hanging.

Had been using Fedora 32 kernel 5.8.13.200 without any issues.
Did a system update to kernel 5.9.10.100.fc2 and had many issues.
Main issue was os would slow to a crawl and become unresponsive.

The culprit seemed to be a process owned by root:
kworker/u32:2+events _unbound
taking up 36% of the cpu.

I started booting back to kernel 5.8.13.200 and all was stable again.

Did a full OS upgrade to 33. Booted into 33 and the same issue persists.
The same process kworker/u32:2+events _unbound now takes up 55% of the cpu and the system remain unresponsive.
I have had to reboot multiple times today with the same issue.
And, it doesn’t really take much time upon bootup for this issue to start happening.
To the point where my system is now unusable.

Any ideas on what this process is and whats causing it take so much of cpu usage?

One observation: It might have started when I entered an invalid keystroke at the terminal?
The terminal freezes cause the whole system freezes.
Not sure about this, but just an observation.

1 Like
Fedora 33 freezes frequently and randomly
#2

IDK if this helps but kworker is kernel worker and you should look up it’s PID in /proc to get info

#3

I can get the pid, but cant do much else since the system is very unresponsive and at a crawl.
Because keystrokes or mouse movements dont exactly register, It takes me minutes to just get the system to power off using the power menu.

I was hoping someone would know what that process does and what activates it to run as hot as it does.

#4

… same issue for me …

Top shows “root 20376 2 3 13:54 ? 00:00:48 [kworker/u8:1-events_unbound]” with heavy load. The system is nearly unusable.

With fedora 32 no such behavior could be observed.

My suspicion falls on the graphics driver nouveau because the problem often appears while viewing movies.
My onbord graphic card is a: 01:00.0 VGA compatible controller: NVIDIA Corporation GT216M [GeForce GT 330M] (rev a2)

with best regards, hu

1 Like
#5

… I’ll try “acpi=off” as kernel-parameter in the grub config and wait if the issue repeats anymore …

1 Like
#6

… “acpi=off” leads to a boot-process without boot-console and a wrong resolution with wayland / gnome …

In relation to my suspicion that the display card driver nouveau causes the problems, I turned off hyper threading in the BIOS and added “pci=biosirq” to the kernel parameters in grub configuration …

The laptop boots fine now again (with boot console and right resolution). The nouveau related interrupts looks like this:
grep nvkm /proc/interrupts
29: 0 (CPU 0) 103958 (CPU1) PCI-MSI 524288-edge nvkm

and not anymore like this before disabling hyper threading in the BIOS:
grep nvkm /proc/interrupts
29: 0 (CPU 0) 0 (CPU1) 1504756 (CPU 2) 0 (CPU 3) PCI-MSI 524288-edge nvkm

… I’ll wait and see what happens … I’ll report here …

… it was a try … no more … but the problem still exists …

I did some performance analyzing with perf while the error situation appears and get the following dump:

Samples: 57K of event 'cycles', Event count (approx.): 39393633680
  Children      Self  Command          Shared Object                             Symbol
+   77,49%     0,00%  kworker/u4:0+ev  [kernel.kallsyms]                         [k] ret_from_fork                       â—†
+   77,49%     0,00%  kworker/u4:0+ev  [kernel.kallsyms]                         [k] kthread                             â–’
+   77,49%     0,00%  kworker/u4:0+ev  [kernel.kallsyms]                         [k] worker_thread                       â–’
+   77,49%     0,00%  kworker/u4:0+ev  [kernel.kallsyms]                         [k] process_one_work                    â–’
+   77,49%     0,00%  kworker/u4:0+ev  [nouveau]                                 [k] nv50_disp_atomic_commit_tail        â–’
-   73,26%    73,10%  kworker/u4:0+ev  [kernel.kallsyms]                         [k] ioread32                            â–’
     73,10% ret_from_fork                                                                                                â–’
        kthread                                                                                                          â–’
        worker_thread                                                                                                    â–’
        process_one_work                                                                                                 â–’
      - nv50_disp_atomic_commit_tail                                                                                     â–’
         - 44,75% nv50_wndw_flush_set                                                                                    â–’
            - 29,07% base507c_ntfy_set                                                                                   â–’
               - nv50_dmac_wait                                                                                          â–’
                  - 22,31% nvif_timer_wait_test                                                                          â–’
                       nvif_device_time                                                                                  â–’
                       nvif_object_mthd                                                                                  â–’
                     - nvkm_ioctl                                                                                        â–’
                            - 22,31% nvkm_udevice_mthd                                                                       â–’
                           - nv04_timer_read                                                                             â–’
                                ioread32                                                                                 â–’
                    6,71% ioread32                                                                                       â–’
            - 15,67% base827c_image_set                                                                                  â–’
               - nv50_dmac_wait                                                                                          â–’
                  - 12,13% nvif_timer_wait_test                                                                          â–’
                       nvif_device_time                                                                                  â–’
                       nvif_object_mthd                                                                                  â–’
                     - nvkm_ioctl                                                                                        â–’
                        - 12,13% nvkm_udevice_mthd                                                                       â–’
                           - 12,12% nv04_timer_read                                                                      â–’
                                ioread32                                                                                 â–’
                    3,51% ioread32                                                                                       â–’
         - 15,77% nv50_disp_atomic_commit_wndw                                                                           â–’
              base507c_update                                                                                            â–’
            - nv50_dmac_wait
            - nv50_dmac_wait                                                                                             â–’
               - 12,29% nvif_timer_wait_test                                                                             â–’
                        nvif_device_time                                                                                     â–’
                    nvif_object_mthd                                                                                     â–’
                    nvkm_ioctl                                                                                           â–’
                  - nvkm_udevice_mthd                                                                                    â–’
                     - 12,29% nv04_timer_read                                                                            â–’
                          ioread32                                                                                       â–’
                 3,44% ioread32                                                                                          â–’
         - 12,59% base507c_ntfy_wait_begun                                                                               â–’
            - 8,10% nvif_timer_wait_test                                                                                 â–’
                 nvif_device_time                                                                                        â–’
                 nvif_object_mthd                                                                                        â–’
                 nvkm_ioctl                                                                                              â–’
               - nvkm_udevice_mthd                                                                                       â–’
                  - 8,10% nv04_timer_read                                                                                â–’
                       ioread32                                                                                          â–’
              4,47% ioread32                                                                                             â–’
+   62,36%     0,04%  kworker/u4:0+ev  [nouveau]                                 [k] nv50_dmac_wait                      â–’
+   56,97%     0,04%  kworker/u4:0+ev  [nouveau]                                 [k] nvif_timer_wait_test                â–’
+   56,91%     0,04%  kworker/u4:0+ev  [nouveau]                                 [k] nvif_device_time                    â–’
+   56,87%     0,15%  kworker/u4:0+ev  [nouveau]                                 [k] nvif_object_mthd                    â–’
+   56,24%     0,23%  kworker/u4:0+ev  [nouveau]                                 [k] nvkm_ioctl                          â–’
+   55,49%     0,23%  kworker/u4:0+ev  [nouveau]                                 [k] nvkm_udevice_mthd                   â–’
+   55,20%     0,25%  kworker/u4:0+ev  [nouveau]                                 [k] nv04_timer_read                     â–’
+   46,19%     0,00%  kworker/u4:0+ev  [nouveau]                                 [k] nv50_wndw_flush_set                 â–’
+   29,98%     0,00%  kworker/u4:0+ev  [nouveau]                                 [k] base507c_ntfy_set                   â–’
+   16,21%     0,00%  kworker/u4:0+ev  [nouveau]                                 [k] base827c_image_set                  â–’
+   16,17%     0,00%  kworker/u4:0+ev  [nouveau]                                 [k] nv50_disp_atomic_commit_wndw        â–’
+   16,17%     0,00%  kworker/u4:0+ev  [nouveau]                                 [k] base507c_update                     â–’
+   15,13%     0,03%  kworker/u4:0+ev  [nouveau]

It seems, that I’m not so wrong with my assessment. Since I don’t like NVIDA propriatory display drivers, I have to wait for updated nouveau kernel modules … you are welcome with any other suggestions …

This morning (12.12.2020) there was a kernel update in the fedora 33 update repository, so I switched from vmlinuz-5.9.12-200.fc33.x86_64 to 5.9.13-200.fc33.x86_64.
With this update there are many graphic related dependancies (like mesa-dri-drivers-20.2.4-1, mesa-libEGL-20.2.4-1, mesa-libGL-20.2.4-1, mesa-libOSMesa-20.2.4-1, mesa-libOpenCL-20.2.4-1, mesa-libgbm-20.2.4-1, mesa-libglapi-20.2.4-1, mesa-libxatracker-20.2.4-1, mesa-vulkan-drivers-20.2.3-1, …) included. I’ll cross my fingers …

#7

Unfortunately the problem persists.

#8

… I did the same as technog as workaround … downgrade the kernel to a previous well functioning version …

# uptime
11:21:28 up 14:52,  1 user,  load average: 0,69, 1,07, 0,74

# dnf list kernel.x68*
kernel.x86_64                  5.8.15-301.fc33                      @fedora 
kernel.x86_64                  5.9.11-200.fc33                      @updates
kernel.x86_64                  5.9.13-200.fc33                      @updates

# vi /etc/dnf/dnf.conf
[main]
gpgcheck=1
#installonly_limit=3
installonly_limit=0
clean_requirements_on_remove=True
#exclude=bluez*

# dnf -v downgrade kernel
# grubby --set-default /boot/vmlinuz-5.8.15-301.fc33.x86_64
# reboot

# grubby --info /boot/vmlinuz-5.8.15-301.fc33.x86_64
index=2
kernel="/boot/vmlinuz-5.8.15-301.fc33.x86_64"
args="ro vconsole.font=latarcyrheb-sun16 rhgb quiet root=/dev/sda3"
root="/dev/sda3"
initrd="/boot/initramfs-5.8.15-301.fc33.x86_64.img"
title="Fedora (5.8.15-301.fc33.x86_64) 33 (Thirty Three)"
id="9053981f084f4fd9842092076b1fe146-5.8.15-301.fc33.x86_64"

… solved (for me) … as soon as new kernels arrives, I can boot now into the new kernel and try if the problem still exists. Because of “installonly_limit=0” old kernels never will be removed by dnf and I have always a running fallback, until a new linux kernel will be released which worked fine again with my setup …

#9

Like the previous comments, I was working under a 5.8 kernel. After the kernel was updated to anything 5.9 (or later) it started locking up with kworker events_unbound issues after anywhere from 15 minutes to an hour. And like the others, the only way out was to hit the reset hardware switch on my machine. Booting from the 5.8 kernel works without any issues. I used to run the NVIDIA drivers, but dropped those to try and clear this problem. Still happens without using anything NVIDIA (except the card, of course). Not doing custom builds of kernels. Simply running firefox alone, or virtualbox alone, or even just a terminal session alone with top running in it gives me the same issue of CPU being hogged by the events_unbound process. Sticking with the 5.8 kernel for now.

#10

vmlinuz-5.8.15-301.fc33.x86_64: anything works fine, no kworker issue, no suspend to disk issue (AllowHibernation=no, AllowSuspend=yes)

vmlinuz-5.9.11-200.fc33.x86_64: kworker issue, suspend to disk issue unconfirmed

vmlinuz-5.9.12-200.fc33.x86_64: kworker issue, suspend to disk issue unconfirmed

vmlinuz-5.9.13-200.fc33.x86_64: kworker issue, suspend to disk issue unconfirmed

vmlinuz-5.9.14-200.fc33.x86_64: anything works fine, no kworker issue, no suspend to disk issue (AllowHibernation=no, AllowSuspend=yes)

vmlinuz-5.9.15-200.fc33.x86_64: kworker issue unconfirmed, computer unusable after suspend to disk (AllowHibernation=no, AllowSuspend=yes)

#11

Any thing new on this matter. I have had this problem since I upgraded in december, and every time a new upgrade is availble I hope it will disappear - but it still persist.
I have managed to survive by close the lid - wait for system to suspend - and then open the lidd again.
Then the kworker problem is gone for a while - until 5, 10 og 15min later - and then same procedure.

I agree with hu, that this is connected with the screen-device driver - I use noveau and havn’t tried the native NVIDIA driver. It makes sense because the screen it shutdown during suspend, and is re-initialized during wake up. Maybe this issue is only happening on Geforce graphics card.

I have used Linux for 30 years, and it is first time have had a serious problem, that wasn’t fix with in a month.

If any one have suggestion to avoid this problem please reply - it might help others too.
or ideas to debug it ?

BTW - I’m currently running vmlinuz 5.10.7 and still having the kworker issue.

#12

… sorry for the late update …
After 4 days running vmlinuz-5.9.14-200.fc33.x86_64 without shutdown the kworker issue appears regrettably.

With vmlinuz-5.10.6-200.fc33.x86_64 and vmlinuz-5.10.7-200.fc33.x86_64 there was no kworker issue until now.

#13

@Tom: my only suggestion for you is (as long as the kworker issue appers for you under the 5.10 kernels) like I described above (downgrade the kernel to 5.8):

# vi /etc/dnf/dnf.conf
[main]
gpgcheck=1
#installonly_limit=3
installonly_limit=0
clean_requirements_on_remove=True
#exclude=bluez*

# dnf -v downgrade kernel
# grubby --set-default /boot/vmlinuz-5.8.15-301.fc33.x86_64
# reboot

Old kernels wont be deleted in future. You have to rerun grubby --set-default /boot/vmlinuz-5.8.15-301.fc33.x86_64 as often as new kernels appear in system updates.

#14

Thanks @huscape for the update.

I have avoided using the old kernel, because I have hoped the error was found and fixed. But since it seems difficult it might be the way to go.

I just find it strange that there are so few others experiencing this problem. Actually I started to believe that by good old ThinkPad was slowly dying.

#15

… don’t worry about your hardware Tom … like described above, I debugged the CPU eating kworker process with perf. The dump shows clearly the call of nv50_disp_atomic_commit_tail consuming al the CPU power, which is part of the nouveau kernel module.

And the above mentioned method of downgrading the kernel includes, that new kernels farther are installed and can be tested. If a new kernel appears, install it and reboot. If the error returns reboot again with the downgrade kernel which is saved and selectable in the bootloader. Make it permanent again with: grubby --set-default /boot/vmlinuz-5.8.15-301.fc33.x86_64

1 Like
New install 33 workstation Firefox unusable slow, Chromium fine?
#16

I have never seen the issue in this thread on my machines using the nvidia drivers. To me what you say here implies a problem between the nouveau driver and the kernel. I wonder if a good workaround may be to install the nvidia drivers instead. Worth a try?

I suspect most use the nvidia drivers which could explain why few see this problem.

#17

You might be on the save side if you use the proprietary NVIDIA driver. But consider, that you have to recompile the proprietary NVIDIA kernel module by hand, as often new kernels appear in your regularly system updates.

#18

For those using Ubuntu style systems dkms takes care of the automatic recompile and for Fedora/RedHat style systems akmods takes care of that. Hardly a problem.

For those that don’t set it up to automatically recompile the driver it is easy to rerun the .run file and do the update that way.

#19

… I know … maybe the procedure runs smoother today. In the days I decided to switch from the NVIDIA to the nouveau driver, there were often problems with building the new kernel module. I avoided these problems since using nouveau and this is the first sequence of problems with my graphics since years.
But for me the most important thing is to avoid proprietary software. That’s why I use linux.

#20

kworker persist in new kernel 5.10.8 ;-(

Found another thead that could indicate that the problem was found in tne nuveau driver last week.