Issues with watchdog BUG soft lockup, possibly relateted to NFS

20 Nov 2024


      Since years my tumbleweed runs perfectly But now, since ~3 weeks or so I get really This happens most of the time when transferring

Kernel is: openSUSE Tumbleweed 20241118 Hardware: Gigabyte Mainboard Z77-DS3H, no nvidia or amd graphics (intel onchip)

Messages file shows: 2024-11-20T12:51:55.252558+01:00 bach kernel: [ 2024-11-20T12:51:55.252572+01:00 bach kernel: [ 7 vfat fat pktcdvd intel_rapl_msr snd_hda_codec_hdmi 2_memops i2c_mux efi_pstore snd_ice1712 _ich fan thermal button nfsd auth_rpcgss 2024-11-20T12:51:55.252574+01:00 bach kernel: [ _display_helper scsi_dh_emc scsi_dh_rdac cm snd_timer mc snd_rawmidi snd_seq_device 2024-11-20T12:51:55.252575+01:00 bach kernel: [ 2024-11-20T12:51:55.252576+01:00 bach kernel: [ 2024-11-20T12:51:55.252577+01:00 bach kernel: [ 2024-11-20T12:51:55.252578+01:00 bach kernel: [ 2024-11-20T12:51:55.252579+01:00 bach kernel: [ 2024-11-20T12:51:55.252580+01:00 bach kernel: [ 2024-11-20T12:51:55.252581+01:00 bach kernel: [ 2024-11-20T12:51:55.252581+01:00 bach kernel: [ 2024-11-20T12:51:55.252582+01:00 bach kernel: [ 2024-11-20T12:51:55.252583+01:00 bach kernel: [ 2024-11-20T12:51:55.252584+01:00 bach kernel: [ 2024-11-20T12:51:55.252585+01:00 bach kernel: [ 2024-11-20T12:51:55.252585+01:00 bach kernel: [ 2024-11-20T12:51:55.252586+01:00 bach kernel: [ 2024-11-20T12:51:55.252587+01:00 bach kernel: [ 2024-11-20T12:51:55.252588+01:00 bach kernel: [ 2024-11-20T12:51:55.252588+01:00 bach kernel: [ 2024-11-20T12:51:55.252589+01:00 bach kernel: [ 2024-11-20T12:51:55.252590+01:00 bach kernel: [ 2024-11-20T12:51:55.252590+01:00 bach kernel: [ 2024-11-20T12:51:55.252591+01:00 bach kernel: [ 2024-11-20T12:51:55.252593+01:00 bach kernel: [ 2024-11-20T12:51:55.252601+01:00 bach kernel: [ 2024-11-20T12:51:55.252602+01:00 bach kernel: [ 2024-11-20T12:51:55.252603+01:00 bach kernel: [ 2024-11-20T12:51:55.252604+01:00 bach kernel: [ 2024-11-20T12:51:55.252605+01:00 bach kernel: [ 2024-11-20T12:51:55.252606+01:00 bach kernel: [ 2024-11-20T12:51:55.252606+01:00 bach kernel: [ 2024-11-20T12:51:55.252607+01:00 bach kernel: [ 2024-11-20T12:51:55.252608+01:00 bach kernel: [ 2024-11-20T12:51:55.252777+01:00 bach kernel: [ 2024-11-20T12:51:55.252780+01:00 bach kernel: [ 2024-11-20T12:51:55.252781+01:00 bach kernel: [ 2024-11-20T12:51:55.252782+01:00 bach kernel: [ 2024-11-20T12:51:55.252782+01:00 bach kernel: [ 2024-11-20T12:51:55.252783+01:00 bach kernel: [ 2024-11-20T12:51:55.252784+01:00 bach kernel: [ 2024-11-20T12:51:55.252785+01:00 bach kernel: [ 2024-11-20T12:51:55.252786+01:00 bach kernel: [ 2024-11-20T12:51:55.252786+01:00 bach kernel: [ 2024-11-20T12:51:55.252787+01:00 bach kernel: [ 2024-11-20T12:51:55.252788+01:00 bach kernel: [ 2024-11-20T12:51:55.252788+01:00 bach kernel: [ 2024-11-20T12:51:55.252938+01:00 bach kernel: [ 2024-11-20T12:51:55.252942+01:00 bach kernel: [ 2024-11-20T12:51:55.252943+01:00 bach kernel: [ 2024-11-20T12:51:55.252943+01:00 bach kernel: [ 2024-11-20T12:51:55.252944+01:00 bach kernel: [ 2024-11-20T12:51:55.252944+01:00 bach kernel: [ --- nfs-client (and nfs-kernel-server) is the NAS ist mounted via autofs with the type nfs (rw,nosuid,nodev,noatime,vers=3,rsize= --- In this state the system gets unusable Thanks in advance

well, with only some small gltches solved fast. often this soft lockup errors and then the system gets mostly unusable, even a normal shutdown hangs, systesm must be resetted. some bigger files via nfs to a NAS (Synology) with dirvish (Backup-tool), sometimes also with small files. Linux 6.11.8-1-default x86_64 CPU Intel i7-3770 3.4GHz ("older" system) C6] watchdog: BUG: soft lockup - CPU#6 stuck for 52s! [kworker/u32:3:43418] C6] Modules linked in: nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs netfs snd_seq_dummy snd_hrtimer af_packet vboxnetadp(O) vboxnetflt(O) qrtr rfkill vboxdrv(O) snd_seq_midi snd_seq_midi_event snd_seq nf_tables binfmt_misc ext4 mbcache jbd2 nls_iso8859_1 nls_cp43 intel_rapl_common x86_pkg_temp_thermal intel_powerclamp coretemp uvcvideo snd_hda_codec_realtek iTCO_wdt kvm_intel videobuf2_vmalloc snd_hda_codec_generic intel_pmc_bxt at24 uvc mei_pxp mei_hdcp iTCO_vendor_support snd_hda_scodec_component kvm i2c_i801 videobuf snd_hda_intel videobuf2_v4l2 pcspkr i2c_smbus videodev snd_cs8427 snd_i2c snd_intel_dspcfg mei_me snd_intel_sdw_acpi snd_ice17xx_ak4xxx snd_ak4xxx_adda snd_hda_codec videobuf2_common snd_ac97_codec ac97_bus mei snd_hda_core atl1c snd_mpu401_uart joydev tiny_power_button lpc nfs_acl lockd grace sunrpc loop fuse dm_mod configfs nfnetlink dmi_sysfs ip_tables x_tables C6]  hid_logitech_hidpp hid_logitech_dj hid_jabra hid_generic usbhid sr_mod cdrom i915 crct10dif_pclmul crc32_pclmul polyval_clmulni polyval_generic ahci ghash_clmulni_intel sha512_ssse3 libahci sha256_ssse3 sha1_ssse3 libata i2c_algo_bit drm_buddy ttm sd_mod drm xhci_pci scsi_dh_alua xhci_pci_renesas sg serio_raw scsi_mod aesni_intel xhci_hcd gf128mul crypto_simd cec cryptd ehci_pci rc_core ehci_hcd scsi_common video wmi btrfs blake2b_generic libcrc32c crc32c_intel xor raid6_pq snd_usb_audio snd_hwdep snd_usbmidi_lib snd_ump snd_p snd soundcore usbcore msr i2c_dev efivarfs C6] CPU: 6 UID: 0 PID: 43418 Comm: kworker/u32:3 Tainted: G           O L     6.11.8-1-default #1 openSUSE Tumbleweed 1400000003000000474e55001983fa4dd342f6a5 C6] Tainted: [O]=OOT_MODULE, [L]=SOFTLOCKUP C6] Hardware name: Gigabyte Technology Co., Ltd. To be filled by O.E.M./Z77-DS3H, BIOS F9 09/19/2012 C6] Workqueue: writeback wb_workfn (flush-0:197) C6] RIP: 0010:nfs_lock_and_join_requests+0x58/0x240 [nfs] C6] Code: 24 4c 89 ef e8 a9 f4 ff ff 48 89 c3 48 85 c0 74 2e 48 8d 68 38 eb 0c 48 89 df e8 d3 ea ff ff 85 c0 78 19 f0 48 0f ba 6d 00 00 <72> eb 49 39 5d 28 74 37 48 89 df e8 e8 ba ff ff eb c5 48 63 d8 48 C6] RSP: 0018:ffffb700a451f850 EFLAGS: 00000282 C6] RAX: ffff974b1fa6d380 RBX: ffff974b1fa6d380 RCX: 0000000000000000 C6] RDX: 0000000000000003 RSI: 0000000000000000 RDI: ffff974a58de62d4 C6] RBP: ffff974b1fa6d3b8 R08: 0000000000000020 R09: 0000000000000000 C6] R10: 0000000000000006 R11: ffffffffffffffc0 R12: ffff974b1fa6d380 C6] R13: ffffd75a48a41000 R14: 0000000000000001 R15: 0000000000000000 C6] FS:  0000000000000000(0000) GS:ffff974d4fb00000(0000) knlGS:0000000000000000 C6] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 C6] CR2: 00007f0094087000 CR3: 00000003b4638003 CR4: 00000000001706f0 C6] Call Trace: C6]  <IRQ> C6]  ? watchdog_timer_fn.cold+0x3d/0xa1 C6]  ? __pfx_watchdog_timer_fn+0x10/0x10 C6]  ? __hrtimer_run_queues+0x132/0x2a0 C6]  ? hrtimer_interrupt+0xfa/0x210 C6]  ? __sysvec_apic_timer_interrupt+0x55/0x100 C6]  ? sysvec_apic_timer_interrupt+0x6c/0x90 C6]  </IRQ> C6]  <TASK> C6]  ? asm_sysvec_apic_timer_interrupt+0x1a/0x20 C6]  ? nfs_lock_and_join_requests+0x58/0x240 [nfs 1400000003000000474e5500c95675f39c80242d] C6]  ? __lruvec_stat_mod_folio+0x83/0xd0 C6]  nfs_page_async_flush+0x1b/0x1f0 [nfs 1400000003000000474e5500c95675f39c80242d] C6]  ? __pfx_nfs_writepages_callback+0x10/0x10 [nfs 1400000003000000474e5500c95675f39c80242d] C6]  nfs_writepages_callback+0x30/0x50 [nfs 1400000003000000474e5500c95675f39c80242d] C6]  write_cache_pages+0x60/0xb0 C6]  nfs_writepages+0x122/0x270 [nfs 1400000003000000474e5500c95675f39c80242d] C6]  do_writepages+0xd2/0x240 C6]  ? __pfx_simple_copy_to_iter+0x10/0x10 C6]  ? __skb_datagram_iter+0x75/0x2e0 C6]  __writeback_single_inode+0x41/0x340 C6]  writeback_sb_inodes+0x21c/0x4e0 C6]  __writeback_inodes_wb+0x4c/0xe0 C6]  wb_writeback+0x184/0x300 C6]  wb_workfn+0x32f/0x420 C6]  process_one_work+0x16b/0x320 C6]  worker_thread+0x2ea/0x420 C6]  ? __pfx_worker_thread+0x10/0x10 C6]  kthread+0xd2/0x100 C6]  ? __pfx_kthread+0x10/0x10 C6]  ret_from_fork+0x34/0x50 C6]  ? __pfx_kthread+0x10/0x10 C6]  ret_from_fork_asm+0x1a/0x30 C6]  </TASK> installed in version 2.8.1-44.1, the most actual version I think. parameters: 131072,wsize=131072,namlen=255,soft,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.1.11,mountvers=3,mountport=892,mountproto=udp,local_lock=none,addr=192.168.1.11) and I hope you can give me some hints, what to try and check

Markus Naaf

Oliver Schwabedissen

tags

participants (2)