[Bug 1188589] New: libbpf: failed to find valid kernel BTF
https://bugzilla.suse.com/show_bug.cgi?id=1188589 Bug ID: 1188589 Summary: libbpf: failed to find valid kernel BTF Classification: openSUSE Product: openSUSE Tumbleweed Version: Current Hardware: Other OS: Other Status: NEW Severity: Normal Priority: P5 - None Component: Kernel Assignee: kernel-bugs@opensuse.org Reporter: mrostecki@suse.com QA Contact: qa-bugs@suse.de Found By: --- Blocker: --- Since the update to the TW snapshot with the kernel 5.13.2-1, I see the following errors when trying to load BPF CO-RE skeletons with libbpf: libbpf: failed to find valid kernel BTF libbpf: Error loading vmlinux BTF: -3 The full example - I'm trying to load BPF programs which I have in this project: https://github.com/rancher-sandbox/lockc and the full output is: ��� sudo ./out/lockcd libbpf: elf: skipping unrecognized data section(9) .rodata.str1.1 libbpf: failed to find valid kernel BTF libbpf: Error loading vmlinux BTF: -3 libbpf: failed to load object 'lockc_bpf' libbpf: failed to load BPF skeleton 'lockc_bpf': -3 Error: libbpf error Caused by: System error, errno: 3 Most likely, the solution for them would be setting the following kernel parameters to "y": CONFIG_DEBUG_INFO_REDUCED CONFIG_DEBUG_INFO_SPLIT But I'm not entirely sure yet. trying to test that... -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1188589 https://bugzilla.suse.com/show_bug.cgi?id=1188589#c1 --- Comment #1 from Micha�� Rostecki <mrostecki@suse.com> --- This is the strace output: access("/boot/vmlinux-5.13.2-1-default", R_OK) = -1 ENOENT (No such file or directory) access("/lib/modules/5.13.2-1-default/vmlinux-5.13.2-1-default", R_OK) = -1 ENOENT (No such file or directory) access("/lib/modules/5.13.2-1-default/build/vmlinux", R_OK) = -1 ENOENT (No such file or directory) access("/usr/lib/modules/5.13.2-1-default/kernel/vmlinux", R_OK) = -1 ENOENT (No such file or directory) access("/usr/lib/debug/boot/vmlinux-5.13.2-1-default", R_OK) = -1 ENOENT (No such file or directory) access("/usr/lib/debug/boot/vmlinux-5.13.2-1-default.debug", R_OK) = -1 ENOENT (No such file or directory) access("/usr/lib/debug/lib/modules/5.13.2-1-default/vmlinux", R_OK) = -1 ENOENT (No such file or directory) write(2, "libbpf: failed to find valid ker"..., 40libbpf: failed to find valid kernel BTF Seems like we are not exposing BTF info in any directory that libbpf expects it to be in. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1188589 https://bugzilla.suse.com/show_bug.cgi?id=1188589#c2 --- Comment #2 from Micha�� Rostecki <mrostecki@suse.com> --- We have the /sys/kernel/btf/vmlinux file though. Wondering why libbpf doesn't honor it... -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1188589 https://bugzilla.suse.com/show_bug.cgi?id=1188589#c3 --- Comment #3 from Micha�� Rostecki <mrostecki@suse.com> --- full strace output: https://paste.opensuse.org/3215684 -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1188589 Micha�� Rostecki <mrostecki@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |glin@suse.com -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1188589 Gary Ching-Pang Lin <glin@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |shung-hsi.yu@suse.com -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1188589 https://bugzilla.suse.com/show_bug.cgi?id=1188589#c4 --- Comment #4 from Micha�� Rostecki <mrostecki@suse.com> --- My bad, I didn't notice before that the /sys/kernel/btf/vmlinux file was actually used, but then libbpf started checking the other files anyway: access("/sys/kernel/btf/vmlinux", R_OK) = 0 openat(AT_FDCWD, "/sys/kernel/btf/vmlinux", O_RDONLY) = 3 newfstatat(3, "", {st_mode=S_IFREG|0444, st_size=4511209, ...}, AT_EMPTY_PATH) = 0 read(3, "\237\353\1\0\30\0\0\0\0\0\0\0\350\223(\0\350\223(\0\351A\34\0\1\0\0\0\0\0\0\1"..., 4096) = 4096 newfstatat(3, "", {st_mode=S_IFREG|0444, st_size=4511209, ...}, AT_EMPTY_PATH) = 0 lseek(3, 4509696, SEEK_SET) = 4509696 read(3, "t\0pirq_vlsi_get\0pirq_sis_set\0pir"..., 4096) = 1513 lseek(3, 0, SEEK_SET) = 0 mmap(NULL, 4513792, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3945622000 read(3, "\237\353\1\0\30\0\0\0\0\0\0\0\350\223(\0\350\223(\0\351A\34\0\1\0\0\0\0\0\0\1"..., 4509696) = 4096 read(3, "\224\n\0\0Y\2\0\0\0/\0\0\236\n\0\0I\0\0\0\0001\0\0\253\n\0\0I\0\0\0"..., 4505600) = 4096 read(3, "_pm_parse_addr\0mptcp_pm_nl_data_"..., 4096) = 4096 read(3, "t\0pirq_vlsi_get\0pirq_sis_set\0pir"..., 4096) = 1513 mmap(NULL, 4513792, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f39451d4000 close(0) = 0 munmap(0x7f39451d4000, 4513792) = 0 munmap(0x7f3945622000, 4513792) = 0 close(3) = 0 access("/boot/vmlinux-5.13.2-1-default", R_OK) = -1 ENOENT (No such file or directory) access("/lib/modules/5.13.2-1-default/vmlinux-5.13.2-1-default", R_OK) = -1 ENOENT (No such file or directory) access("/lib/modules/5.13.2-1-default/build/vmlinux", R_OK) = -1 ENOENT (No such file or directory) access("/usr/lib/modules/5.13.2-1-default/kernel/vmlinux", R_OK) = -1 ENOENT (No such file or directory) access("/usr/lib/debug/boot/vmlinux-5.13.2-1-default", R_OK) = -1 ENOENT (No such file or directory) access("/usr/lib/debug/boot/vmlinux-5.13.2-1-default.debug", R_OK) = -1 ENOENT (No such file or directory) access("/usr/lib/debug/lib/modules/5.13.2-1-default/vmlinux", R_OK) = -1 ENOENT (No such file or directory) write(2, "libbpf: failed to find valid ker"..., 40libbpf: failed to find valid kernel BTF ) = 40 write(2, "libbpf: Error loading vmlinux BT"..., 38libbpf: Error loading vmlinux BTF: -3 ) = 38 write(2, "libbpf: failed to load object 'l"..., 42libbpf: failed to load object 'lockc_bpf' ) = 42 write(2, "libbpf: failed to load BPF skele"..., 52libbpf: failed to load BPF skeleton 'lockc_bpf': -3 My theory so far is that processing the first vmlinux file fails somewhere here: https://github.com/torvalds/linux/blob/master/tools/lib/bpf/btf.c#L4431-L443... But the `pr_debug` message doesn't show up. And what's more, the debug mode can't be enabled in BPF skeletons... it can be set only when executing `bpftool` commands. I will try to dig in more and see what's happening in that file. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1188589 https://bugzilla.suse.com/show_bug.cgi?id=1188589#c5 --- Comment #5 from Micha�� Rostecki <mrostecki@suse.com> --- I noticed that dumping BTF with bpftool also doesn't work on >=5.13 openSUSE kernels: ��� sudo bpftool btf dump file /sys/kernel/btf/vmlinux Error: failed to load BTF from /sys/kernel/btf/vmlinux: Invalid argumen -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1188589 https://bugzilla.suse.com/show_bug.cgi?id=1188589#c6 Shung-Hsi Yu <shung-hsi.yu@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mrostecki@suse.com Flags| |needinfo?(mrostecki@suse.co | |m) --- Comment #6 from Shung-Hsi Yu <shung-hsi.yu@suse.com> --- Hi Micha��, is this still failing? FWIW I submitted a service request on build.opensuse.org for updating libbpf v0.5, perhaps that will fix it. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1188589 https://bugzilla.suse.com/show_bug.cgi?id=1188589#c7 Jiri Slaby <jslaby@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED CC| |jslaby@suse.com Resolution|--- |NORESPONSE --- Comment #7 from Jiri Slaby <jslaby@suse.com> --- Feel free to reopen when providing the requested info. -- You are receiving this mail because: You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠suse.com