[Bug 1229396] New: Old kernel file and initrd not deleted when removing kernels with systemd boot
https://bugzilla.suse.com/show_bug.cgi?id=1229396 Bug ID: 1229396 Summary: Old kernel file and initrd not deleted when removing kernels with systemd boot Classification: openSUSE Product: openSUSE Tumbleweed Version: Current Hardware: x86-64 OS: openSUSE Tumbleweed Status: NEW Severity: Normal Priority: P5 - None Component: Bootloader Assignee: screening-team-bugs@suse.de Reporter: rosuna@suse.com QA Contact: qa-bugs@suse.de Target Milestone: --- Found By: --- Blocker: --- Problem as explained in description. I reported it as a zypper problem, so this issue will probably be closed as invalid: https://github.com/openSUSE/zypper/issues/564 OS: openSUSE Tumbleweed Bootloader: systemd boot Other info: encrypted filesystem Does this always happened? At least I can confirm it for the last two kernel updates. Which files are the ones I'm reporting as not having been deleted as expected? They're under /boot/efi/bad01a2647cb4fef8827293b9d389245 (obviously the last directory has some random string related to systemd boot). There's a kernel directory for each installed version there, containing the kernel itself and at least one initrd (more if dracut has been called several times). Why am I reporting this? The default size for /boot/efi defaults to half a GB, which seems to be enough... Well, I'm definitely going to double that in my next installation, but even with the default 2 kernels, this is easy to fill if calling dracut for whatever reasons (and sometimes, scripts are the ones calling it). If one of the kernels to be purged is not actually purged there, the partition gets filled very easily. I'm attaching some output to illustrate it. Let me know if you need anything else. Please also let me know if this is not zypper purge-kernels's fault (and if possible, point me to where to report this). mordor:~ # rpm -qa | grep -i kernel-default kernel-default-6.10.3-1.1.x86_64 kernel-default-6.10.4-1.1.x86_64 mordor:~ # uname -a Linux mordor.arkayate.org 6.10.4-1-default #1 SMP PREEMPT_DYNAMIC Mon Aug 12 05:49:20 UTC 2024 (0363a35) x86_64 x86_64 x86_64 GNU/Linux mordor:~ # ls -l /boot/efi/bad01a2647cb4fef8827293b9d389245/ total 12 drwxr-x--- 2 root root 4096 Aug 16 22:55 6.10.2-1-default drwxr-x--- 2 root root 4096 Aug 16 22:55 6.10.3-1-default drwxr-x--- 2 root root 4096 Aug 16 22:55 6.10.4-1-default mordor:~ # du -sh /boot/efi/bad01a2647cb4fef8827293b9d389245/* 175M /boot/efi/bad01a2647cb4fef8827293b9d389245/6.10.2-1-default 95M /boot/efi/bad01a2647cb4fef8827293b9d389245/6.10.3-1-default 175M /boot/efi/bad01a2647cb4fef8827293b9d389245/6.10.4-1-default mordor:~ # df -h /boot/efi/ Filesystem Size Used Avail Use% Mounted on /dev/nvme0n1p1 511M 449M 63M 88% /boot/efi mordor:~ # zypper purge-kernels -D Reading installed packages... Preparing to purge obsolete kernels... Configuration: latest,latest-1,running Running kernel release: 6.10.4-1-default Running kernel arch: x86_64 Resolving package dependencies... Nothing to do. In the output above, the directory /boot/efi/bad01a2647cb4fef8827293b9d389245/6.10.2-1-default should get deleted (I believe that by zypper purge-kernels, but not 100% sure). P.S.: Completely unrelated, but I'm happy to have seen that my FDE encryption with TPM chip as decryption key now survives kernel updates, this was also broken and got fixed recently, even before I could report this. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229396 https://bugzilla.suse.com/show_bug.cgi?id=1229396#c1 --- Comment #1 from Raúl Osuna <rosuna@suse.com> --- I've reported this as "bootloader" believing the systemd boot is responsible for this. If the component should be the kernel, feel free to modify the component as needed. Also available for any troubleshooting, even in Slack. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229396 Raúl Osuna <rosuna@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |rosuna@suse.com -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229396 https://bugzilla.suse.com/show_bug.cgi?id=1229396#c3 Raúl Osuna <rosuna@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(rosuna@suse.com) | --- Comment #3 from Raúl Osuna <rosuna@suse.com> --- (In reply to Thorsten Kukuk from comment #2)
Are this kernels referenced by a snapshot? If yes, it's correct that this kernels don't get deleted, else your snapshot would become unbootable and useless.
And since you only removed kernels from the current snapshot, but not old snapshots, I'm pretty sure this is the reason why this kernels don't get removed.
The thing is that I did not remove the kernels myself (at least not manually). I would suppose this is "zypper purge-kernels" taking care of it, with the default configuration of openSUSE (and SLES) to just keep the running kernel and only one previous version. raul@mordor:~$ sudo snapper ls # | Type | Pre # | Date | User | Used Space | Cleanup | Description | Userdata -----+--------+-------+--------------------------+------+------------+---------+-----------------------+-------------- 0 | single | | | root | | | current | 740* | single | | Thu Jun 27 22:07:11 2024 | root | 23.59 MiB | | writable copy of #706 | 839 | pre | | Mon Aug 12 23:51:07 2024 | root | 193.74 MiB | number | zypp(zypper) | important=yes 840 | post | 839 | Mon Aug 12 23:51:30 2024 | root | 84.31 MiB | number | | important=yes 843 | pre | | Fri Aug 16 22:52:27 2024 | root | 311.79 MiB | number | zypp(packagekitd) | important=yes 844 | post | 843 | Fri Aug 16 22:55:59 2024 | root | 1.36 MiB | number | | important=yes 845 | pre | | Fri Aug 16 22:57:07 2024 | root | 460.00 KiB | number | zypp(zypper) | important=yes 846 | post | 845 | Fri Aug 16 22:57:29 2024 | root | 956.00 KiB | number | | important=yes 847 | pre | | Sun Aug 18 00:35:09 2024 | root | 14.90 MiB | number | zypp(zypper) | important=no 848 | post | 847 | Sun Aug 18 00:36:20 2024 | root | 4.50 MiB | number | | important=no 849 | pre | | Mon Aug 19 00:07:05 2024 | root | 1.66 MiB | number | zypp(packagekitd) | important=yes 850 | post | 849 | Mon Aug 19 00:08:46 2024 | root | 2.08 MiB | number | | important=yes 851 | pre | | Mon Aug 19 00:28:56 2024 | root | 3.17 MiB | number | zypp(zypper) | important=yes 852 | post | 851 | Mon Aug 19 00:29:19 2024 | root | 5.21 MiB | number | | important=yes raul@mordor:~$ sudo zypper purge-kernels Reading installed packages... Preparing to purge obsolete kernels... Configuration: latest,latest-1,running Running kernel release: 6.10.5-1-default Running kernel arch: x86_64 Resolving package dependencies... Nothing to do. raul@mordor:~$ rpm -qa | grep -i kernel-default kernel-default-6.10.4-1.1.x86_64 kernel-default-6.10.5-1.1.x86_64 raul@mordor:~$ sudo ls -la /boot/efi/bad01a2647cb4fef8827293b9d389245 total 20 drwxr-x--- 5 root root 4096 Aug 19 00:07 . drwxr-x--- 5 root root 4096 Jan 1 1970 .. drwxr-x--- 2 root root 4096 Aug 19 00:08 6.10.3-1-default drwxr-x--- 2 root root 4096 Aug 19 00:08 6.10.4-1-default drwxr-x--- 2 root root 4096 Aug 19 00:08 6.10.5-1-default P.S: Just before these commands, I had removed snapshot 706, which looked like this: 706 | single | | Wed Jun 12 12:55:30 2024 | root | 13.16 GiB | | Before removing nvidia packages and manually installing 555 version (I'm not using nvidia drivers any more but nouveau). -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229396 https://bugzilla.suse.com/show_bug.cgi?id=1229396#c5 --- Comment #5 from Raúl Osuna <rosuna@suse.com> --- OK, with Torsten's and Alberto's input I did some cleanup, anyway that 740 default snapshot was very old and it was the default one, I'm a bit confused. So I have left things like this: raul@mordor:~$ sudo snapper ls # | Type | Pre # | Date | User | Used Space | Cleanup | Description | Userdata -----+--------+-------+--------------------------+------+------------+---------+-------------------+-------------- 0 | single | | | root | | | current | 843 | pre | | Fri Aug 16 22:52:27 2024 | root | 532.82 MiB | number | zypp(packagekitd) | important=yes 844 | post | 843 | Fri Aug 16 22:55:59 2024 | root | 1.36 MiB | number | | important=yes 845 | pre | | Fri Aug 16 22:57:07 2024 | root | 460.00 KiB | number | zypp(zypper) | important=yes 846 | post | 845 | Fri Aug 16 22:57:29 2024 | root | 956.00 KiB | number | | important=yes 847 | pre | | Sun Aug 18 00:35:09 2024 | root | 14.90 MiB | number | zypp(zypper) | important=no 848 | post | 847 | Sun Aug 18 00:36:20 2024 | root | 4.50 MiB | number | | important=no 849 | pre | | Mon Aug 19 00:07:05 2024 | root | 1.66 MiB | number | zypp(packagekitd) | important=yes 850 | post | 849 | Mon Aug 19 00:08:46 2024 | root | 2.08 MiB | number | | important=yes 851 | pre | | Mon Aug 19 00:28:56 2024 | root | 3.40 MiB | number | zypp(zypper) | important=yes 852* | post | 851 | Mon Aug 19 00:29:19 2024 | root | 16.34 MiB | number | | important=yes 853 | pre | | Mon Aug 19 14:58:05 2024 | root | 72.98 MiB | number | zypp(zypper) | important=yes 854 | post | 853 | Mon Aug 19 14:58:44 2024 | root | 3.88 MiB | number | | important=yes Eventually the snapshots 843-851 will age and will get autodeleted, 852 will be my default one. If there's something I should really correct, help is appreciated. I did run "sudo sdbootutil remove-kernel 6.10.3-1-default", however they directory still remained. If this part is under development, I understand that I was too fast to report this then. I removed the files manually for the time being, it looks like this now: mordor:~ # ls /boot/efi/bad01a2647cb4fef8827293b9d389245/ 6.10.4-1-default 6.10.5-1-default mordor:~ # rpm -qa | grep -i kernel-default kernel-default-6.10.4-1.1.x86_64 kernel-default-6.10.5-1.1.x86_64 So either you close this as invalid, or if you want to see what happens with the next kernel update, I can report here (in case my snapshots make sense, otherwise let me know how to fix them to look more useful). Thanks! -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1229396 https://bugzilla.suse.com/show_bug.cgi?id=1229396#c6 --- Comment #6 from Raúl Osuna <rosuna@suse.com> --- There was a kernel update, to 6.10.7. According to the default rules to keep the running kernel and only one previous, 6.10.4 was deleted, and 6.10.5 was kept. I can still see the initrd's and kernel files of 6.10.4, with the boot partition close to full with the default size of half a GB. According to Torsten's comment, I might have some snapshot with the 6.10.4 kernel and hence the files do not get deleted, because of this. I would understand and accept that then "it works as expected". But we have a problem, in my opinion, with the defaults. So my suggestion would be to fine tune one of these possible values: - /boot/efi default size - Number/date of snapshots kept - Number of initrd files kept - Any other thing that can come to your mind to improve the situation. It is not that easy to resize /boot/efi partition once installed (I guess it can be done, but it is at least disturbing), so let's try that this is only seldom needed. Attaching all the relevant info I could think of, I won't delete the directory this time till I don't get any disk full error. Let me know if any other information from my system would be useful and if you find my thinking with enough base to tune the defaults or not. Maybe this is something new to fight with following the change from grub2 to systemd boot. mordor:/boot/efi/bad01a2647cb4fef8827293b9d389245 # ls 6.10.4-1-default 6.10.5-1-default 6.10.7-1-default mordor:/boot/efi/bad01a2647cb4fef8827293b9d389245 # df -h . Filesystem Size Used Avail Use% Mounted on /dev/nvme0n1p1 511M 448M 64M 88% /boot/efi mordor:/boot/efi/bad01a2647cb4fef8827293b9d389245 # uname -a Linux mordor.arkayate.org 6.10.7-1-default #1 SMP PREEMPT_DYNAMIC Fri Aug 30 04:50:21 UTC 2024 (54e6893) x86_64 x86_64 x86_64 GNU/Linux mordor:/boot/efi/bad01a2647cb4fef8827293b9d389245 # snapper ls # │ Type │ Pre # │ Date │ User │ Used Space │ Cleanup │ Description │ Userdata ─────┼────────┼───────┼──────────────────────────┼──────┼────────────┼─────────┼───────────────────┼────────────── 0 │ single │ │ │ root │ │ │ current │ 851 │ pre │ │ Mon Aug 19 00:28:56 2024 │ root │ 610.23 MiB │ number │ zypp(zypper) │ important=yes 852* │ post │ 851 │ Mon Aug 19 00:29:19 2024 │ root │ 2.80 MiB │ number │ │ important=yes 861 │ pre │ │ Thu Aug 22 14:31:04 2024 │ root │ 357.21 MiB │ number │ zypp(packagekitd) │ important=yes 862 │ post │ 861 │ Thu Aug 22 14:33:24 2024 │ root │ 314.97 MiB │ number │ │ important=yes 867 │ pre │ │ Mon Aug 26 20:58:36 2024 │ root │ 236.17 MiB │ number │ zypp(packagekitd) │ important=yes 868 │ post │ 867 │ Mon Aug 26 21:00:00 2024 │ root │ 60.01 MiB │ number │ │ important=yes 871 │ pre │ │ Tue Aug 27 11:06:49 2024 │ root │ 73.35 MiB │ number │ zypp(zypper) │ important=no 872 │ post │ 871 │ Tue Aug 27 11:07:17 2024 │ root │ 9.41 MiB │ number │ │ important=no 873 │ pre │ │ Thu Aug 29 11:31:01 2024 │ root │ 15.46 MiB │ number │ zypp(zypper) │ important=no 874 │ post │ 873 │ Thu Aug 29 11:36:30 2024 │ root │ 17.23 MiB │ number │ │ important=no 875 │ pre │ │ Sun Sep 1 23:33:53 2024 │ root │ 15.54 MiB │ number │ zypp(zypper) │ important=yes 876 │ post │ 875 │ Sun Sep 1 23:35:08 2024 │ root │ 5.80 MiB │ number │ │ important=yes 877 │ pre │ │ Mon Sep 2 09:32:40 2024 │ root │ 2.10 MiB │ number │ zypp(packagekitd) │ important=yes 878 │ post │ 877 │ Mon Sep 2 09:35:47 2024 │ root │ 10.58 MiB │ number │ │ important=yes 879 │ pre │ │ Tue Sep 3 08:04:00 2024 │ root │ 11.96 MiB │ number │ zypp(packagekitd) │ important=yes 880 │ post │ 879 │ Tue Sep 3 08:05:30 2024 │ root │ 1.82 MiB │ number │ │ important=yes 881 │ pre │ │ Tue Sep 3 08:14:50 2024 │ root │ 284.00 KiB │ number │ zypp(zypper) │ important=yes 882 │ post │ 881 │ Tue Sep 3 08:15:12 2024 │ root │ 268.00 KiB │ number │ │ important=yes mordor:/boot/efi/bad01a2647cb4fef8827293b9d389245 # ls -lR .: total 12 drwxr-x--- 2 root root 4096 Sep 2 09:35 6.10.4-1-default drwxr-x--- 2 root root 4096 Sep 3 08:04 6.10.5-1-default drwxr-x--- 2 root root 4096 Sep 3 08:04 6.10.7-1-default ./6.10.4-1-default: total 178736 -rw-r----- 1 root root 84140536 Sep 2 09:35 initrd-3529c816dea29a4603d945a03164ec13317a415f -rw-r----- 1 root root 84111195 Aug 26 20:59 initrd-8d197e6be32f09f5e1fb0b0320e27efa71d49ad3 -rw-r----- 1 root root 14768496 Aug 12 07:45 linux-7d5327c507b7c87023417fa732be1f3abbc82c44 ./6.10.5-1-default: total 178732 -rw-r----- 1 root root 84136746 Sep 2 09:35 initrd-2714b7f360f4a92ec702dfa17981f3e4206adef2 -rw-r----- 1 root root 84107132 Aug 26 20:59 initrd-419e437d7832c5308ca64a8e9e67b143982edba2 -rw-r----- 1 root root 14772592 Aug 16 12:16 linux-33dc501f1107ebd291ad87f56baf407335b03061 ./6.10.7-1-default: total 96808 -rw-r----- 1 root root 84244524 Sep 3 08:04 initrd-9df433deef45c2e45a0321618404144e2439867c -rw-r----- 1 root root 14883184 Aug 30 06:48 linux-1be192dfa22187b3101d832c0f4a5562be886833 mordor:/boot/efi/bad01a2647cb4fef8827293b9d389245 # rpm -qa|grep kernel-default kernel-default-6.10.7-1.1.x86_64 kernel-default-6.10.5-1.1.x86_64 mordor:/boot/efi/bad01a2647cb4fef8827293b9d389245 # zypper purge-kernels -D Reading installed packages... Preparing to purge obsolete kernels... Configuration: latest,latest-1,running Running kernel release: 6.10.7-1-default Running kernel arch: x86_64 Resolving package dependencies... Nothing to do. mordor:/boot/efi/bad01a2647cb4fef8827293b9d389245 # rpm -qa --last|grep -i kernel-default kernel-default-6.10.7-1.1.x86_64 Tue Sep 3 08:04:49 2024 kernel-default-6.10.5-1.1.x86_64 Mon Aug 19 14:58:32 2024 mordor:/boot/efi/bad01a2647cb4fef8827293b9d389245 # du -sh * 175M 6.10.4-1-default 175M 6.10.5-1-default 95M 6.10.7-1-default -- You are receiving this mail because: You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠suse.com