Hi Andrei, On 5/4/23 02:57, Andrei Borzenkov wrote:
Also, do you know if SUSE has modified grub to mount the default btrfs
subvolume?
Yes and no (yes, I know, and no, it did not). It modified grub to resolve paths on btrfs relative to default subvolume if snapshots are enabled.
Thank you so much for your responses! After writing that message, I realized that fstab specifies the fully qualified paths to the subvolumes and therefore my assumption was wrong about grub mounting the default subvolume. Your answer seems to be the "magic" that the other distros don't have which is why their grubs all have the full path to subvolumes coded in grub and which results in snapper rollback not working. Looking through grub I found the following items which I believe are what you are referring to /etc/default/grub SUSE_BTRFS_SNAPSHOT_BOOTING="true" /usr/share/grub2/grub-mkconfig_lib make_system_path_relative_to_its_root () { if [ "x${SUSE_BTRFS_SNAPSHOT_BOOTING}" = "xtrue" ] && [ "x${GRUB_FS}" = "xbtrfs" ] ; then "${grub_mkrelpath}" -r "$1" else "${grub_mkrelpath}" "$1" fi } /usr/bin/grub2-mkrelpath Which supports -r option Other distros are using grub-mkrelpath which does NOT have the -r option Are there additional pieces which are also related to the grub changes that I have missed? I am planning to try and patch those into one of the other distros I have been testing with to see if I can get it working.
If yes, are there any plans to submit that patch upstream so that other distros will pick it up?
Upstream grub used default subvolume in the past. It was changed to always access full btrfs exactly because it was impossible to access anything outside of the default subvolume.
Right, that all makes sense now, but are there any plans to submit the grub modifications to resolve paths on btrfs relative to the default subvolume if snapshots are enabled ? That would allow snapper to work exactly the same on all distros and then snapper rollback would work too. Since it is a config option, they would not have to enable it or change their subvolume layout, but if they did or the installer chose to enable and use the TW subvolume layout then it would all just work.
Snapshot (rollback) is simply incompatible with how grub is using /boot by default. The way SUSE does it remains compatible with "grub at boot time is located below /boot" and "kernel is installed in /boot" and allows to still use grub-install. This breaks os-prober, but os-prober as implemented is the wrong way to multiboot Linux anyway.
Snapshot rollback works on TW ( or it did at least last time I tested it ) so I assume you mean incompatible with other distros because they don't have the TW grub patch. I was not aware that os-prober was broken, I tend to use VMs for other distros so no need for it, but I know it was working a year or so ago, because I had a VM which had TW as well as several other distros installed into the same vm.
It is quite possible to
- install grub in separate directory (subvolume) - make grub to query for current default subvolume and load its configuration from there
grub.cfg in each default subvolume would use full paths that would point to this subvolume, so each grub.cfg would always remain valid and would also work from within other distributions with os-prober. Default subvolume here would be used just as an indication of "current root". Solaris (which implemented this long ago) is using a separate zfs property to indicate currently active root (or boot environment), but this is implementation detail.
This is (almost) exactly what happens if Secure Boot is enabled, except for paths relative to the default subvolume.
This would require just implementing a command to query the default subvolume which is certainly upstreamable. For completeness one could also allow setting location of /boot/grub/grub.cfg explicitly which would go into load.cfg making grub-install "just work". The main problem of this is the fact that /boot as a grub location is really imprinted deep in the subconscious. Ironically, in the past grub-install *did* support --grub-directory which was then deprecated in favour of always using /boot.
Is there a reason the TW grub patches could not just be submitted upstream? Since it is a configurable option, it should not break anything and other distros "could" just leave it disabled.
In summary, what I am trying to understand is how TW is using whatever btrfs snapshot is set to the default during the boot process without hard coding the snapshot into all the grub configuration files.
grub is using default subvolume so changing default subvolume automatically changes grub configuration.
On TW because of the grub patches, but I'm trying to get full snapper functionality working across distros. Unless I've missed something, based on your responses ( which have been MOST HELPEFUL) It seems that the missing piece is the grub patch which I am hoping could be submitted upstream ? -- Regards, Joe