On Sun, Jul 19, 2015 at 11:58 PM, Andrei Borzenkov wrote:

...

I understand the rationalization, it's easier to get to bootable snapshots and rollbacks this way. I just disagree with a.) taking the user's feature and control away from them, b.) making the boot process essentially invisible when it comes to what subvolume is being used.

You also need to a) somehow designate snapshot to be booted by default

That should be something the bootloader configuration understands. Snapshot doesn't change the need to define the bootloader's need to know what root is, to find the specific kernel + initramfs + root fs to pass off to the kernel. Now what's being done is to say, no we're giving up on all such specifics and we're going to hide it in fs metadata that is Btrfs specific. So the entire boot process has now changed, and only on Btrfs.

and b) handle path changes when snapshot is created (snapshot is different subvolume so any path inside grub.cfg needs adjustment).

This is no different than an LVM thinp snapshot where now it's a different device mapper path for boot and root. If you don't like things modifying grub.cfg, then we need to get serious about fixing whatever problems GRUB upstream has with the bootloaderspec. Either the original or mjg59's variant. And then whatever wants to change the boot process needs to add a new drop in script, rather than modifying bootloader files.

Using default subvolume appears to solve everything nicely ... with expense of need to maintain huge pile of distro-specific patches which make grub incompatible to upstream (and probably other distros as well - did not look at what fedora or ubuntu do).

It's totally incompatible with upstream which has definitively said all paths on btrfs are taken as relative to /subvolid 5. And on SUSE, because of this architectural dependency on default subvolume, has reverted those upstream patches, so that GRUB does paths relative to the default subvolume. Otherwise those 200 /boot/grub2/grub.cfgs are found on the same subvolume rather than relative to their separate subvolumes. Absolute paths make a lot more sense in that it's self describing. A human can read the path in both dmesg and journal and follow the boot process. That's not the case with default subvolume ID. A partial work around is to ask Btrfs upstream to print the subvolid, or maybe even the entire subvol ID's path relative to at mount time. That way anytime any btrfs file system is mounted (even subsequent mounts of subvolumes) that we get a kernel message with the absolute path of that subvolume. But this still takes away a feature meant for the user, and usurps it for a utility to manage behind the scenes. If there's concensus on doing it this way, then tools should do it through libbtrfs, and the user space tools should be changed to either disallow set-default or produce a warning and maybe even require a force flag for the user to change it.

I actually have design draft (from grub side) that requires minimal code changes (and some changes are what I believe should be included for other reasons) and solves all these issues. But it also means changes to installer, yast, snapper ... if anyone is willing to cooperate on fixing those, let's setup OBS project and try to implement it. Then it can be present as alternative solution; and I am pretty confident that GRUB changes that are required will be suitable for inclusion upstream.

Do you have a basic outline for this draft? I'm fairly dense when it comes to reading code. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Mon, Jul 20, 2015 at 11:23 AM, Andrei Borzenkov wrote:

В Mon, 20 Jul 2015 10:52:15 -0600 Chris Murphy пишет:

...

On Sun, Jul 19, 2015 at 11:58 PM, Andrei Borzenkov wrote:

...

...

I understand the rationalization, it's easier to get to bootable snapshots and rollbacks this way. I just disagree with a.) taking the user's feature and control away from them, b.) making the boot process essentially invisible when it comes to what subvolume is being used.

You also need to a) somehow designate snapshot to be booted by default

That should be something the bootloader configuration understands. Snapshot doesn't change the need to define the bootloader's need to know what root is, to find the specific kernel + initramfs + root fs to pass off to the kernel.

When you have multiple specifications for kernel+initramfs+root you need to select which one to use. Be these specifications on subvolumes, filesystems or in $BOOT/loader/entries.

...

Now what's being done is to say, no we're giving up on all such specifics and we're going to hide it in fs metadata that is Btrfs specific. So the entire boot process has now changed, and only on Btrfs.

As snapper (rollbacks) are specific to btrfs this is rather logical.

It's logical only insofar as it was easier to achieve in the shortest amount of time. Not that it's either appropriate or good design, especially long term. As proof of concept it's OK but has a lot of consequences as baggage.

...

...

and b) handle path changes when snapshot is created (snapshot is different subvolume so any path inside grub.cfg needs adjustment).

This is no different than an LVM thinp snapshot where now it's a different device mapper path for boot and root. If you don't like things modifying grub.cfg,

I CAN NOT modify grub.cfg because it is on read-only snapshot. At least that was absolute requirement when SUSE created this code.

...

then we need to get serious about fixing whatever problems GRUB upstream has with the bootloaderspec. Either the original or mjg59's variant. And then whatever wants to change the boot process needs to add a new drop in script, rather than modifying bootloader files.

Again - one of the original requirements that lead to current design was booting snapshot without any preparations. I am not sure current code fulfills it, but it was attempted.

I'm not criticizing the attempt. (open)SUSE has progressed Btrfs, and bootable snapshots and rollback more than any other distro. But ostree and rpm-ostree do this much more elegantly and in a discoverable fashion and correctly modifies the bootloader configuration in a way that's at least in a documented specification. Whereas all of this work on Btrfs has a number of negative consequences.

...

...

change it.

...

I actually have design draft (from grub side) that requires minimal code changes (and some changes are what I believe should be included for other reasons) and solves all these issues. But it also means changes to installer, yast, snapper ... if anyone is willing to cooperate on fixing those, let's setup OBS project and try to implement it. Then it can be present as alternative solution; and I am pretty confident that GRUB changes that are required will be suitable for inclusion upstream.

Do you have a basic outline for this draft? I'm fairly dense when it comes to reading code.

Default bootloader in fixed subvolume, e.g. /mboot. "Local bootloader" in root subvolume, e.g in $subvol/boot Master bootloader multiboots local bootloader from $subvol/boot/xxx/core.img, passing it subvolume as parameter. Default subvolume to boot from either as subvolume property (preferable if btrfs developers agree) or as environment block in fixed location, e.g. /mboot/bootconfig.

In this scheme /mboot is installed just once and can be left alone. It has static grub.cfg that looks for available subvolumes and presents them in menu. Switching root environment means just setting variable in /mboot/bootconfig.

Required changes on grub side are

- make grub-mkconfig scripts generate btrfs paths relative to run-time subvolumes (current SUSE patches effectively do this already).

- make grub accept parameters passed via multiboot/EFI (everyone agrees it is needed, the question is mostly semantic). On EFI it may not even be needed because grub already knows where it was booted from, so likely can extract subvolume (with some help from btrfs).

OK I'm not opposed to anything in particular. But here's the problem. It's yet another idea on how to boot a linux OS, and it doesn't appear to include any aspect of the BootLoaderSpec.[1][2] Linux multiboot sucks. There are sufficient departures from upstream GRUB, and lack of installer cooperation by all the distros to ensure this continues to be difficult. While BLS is imperfect, it tries to solve this particular problem. And tries to solve it regardless of bootloader, filesystem, and type of firmware. OSTree leverages much of the BLS, and appears to be the direction RH/Fedora are going to go in. SUSE is doing something completely different than BLS, or GRUB upstream, or what you're proposing. And what you're proposing doesn't marry any of these two paradigms. So at the moment what I'm seeing is a significant divergence in the way linux distros are going to boot in the future, rather than any sense of cooperation, to the degree they will become completely incompatible rather than just difficult.

Downside is, the more run-time magic is added to grub.cfg the more incompatible with os-prober idea it becomes. OTOH nothing prevents os-prober from probing for /mboot ...

I thought upstream wanted to replace os-prober sooner than later? -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

*sigh* Please ignore this thread. It was supposed to be a continuation of one on-going on the Factory list and I made a mistake adding this list. Sorry. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org