[opensuse-factory] Default btrfs subvolume after a rollback

newer
[opensuse-factory] Tumbleweed is...

Chris Murphy

15 Jul 2015 15 Jul '15

21:06

I just found this thread, and read the whole thing. The central problem is snapper changes the default subvolume to achieve (persistent, rw) rollbacks. This is a design flaw because the default subvolume is intended to be user domain. It's inappropriate that a utility has usurped a user domain shortcut feature for its own use (although I understand why that path was taken because the alternative requires a lot more work and coordination among other teams). And as a result the current design makes everything more complicated and fragile for users to understand and troubleshoot. And it's wrong to do that. So I've opened a bug: https://github.com/openSUSE/snapper/issues/178 We'll see what happens. -- Chris Murphy -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Show replies by date

Chris Murphy

17 Jul 17 Jul

15:25

On Wed, Jul 15, 2015 at 3:06 PM, Chris Murphy wrote:

...

https://github.com/openSUSE/snapper/issues/178

We'll see what happens.

That bug has been closed upstream saying that it works because of SUSE's preferred behavior. So I've filed an openFATE bug report in its place. https://features.opensuse.org/319292 -- Chris Murphy -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Andrei Borzenkov

17:23

В Fri, 17 Jul 2015 09:25:45 -0600 Chris Murphy пишет:

...

On Wed, Jul 15, 2015 at 3:06 PM, Chris Murphy wrote:

...
https://github.com/openSUSE/snapper/issues/178

We'll see what happens.

That bug has been closed upstream saying that it works because of SUSE's preferred behavior. So I've filed an openFATE bug report in its place.

https://features.opensuse.org/319292

I think one reason for this was desire to be able to roll back grub configuration. As grub menu is static, if you rollback /boot you also want to rollback /boot/grub/grub.cfg to match it. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Chris Murphy

20 Jul 20 Jul

19:58

I screwed up replying to one of these messages by adding the regular (non-factory) list for subsequent replies. So this is the partial correction to fix this by bringing it back to the proper list. On Mon, Jul 20, 2015 at 11:23 AM, Andrei Borzenkov wr= ote:

...

=D0=92 Mon, 20 Jul 2015 10:52:15 -0600 Chris Murphy =D0=BF=D0=B8=D1=88=D0=B5=D1=82:

...
On Sun, Jul 19, 2015 at 11:58 PM, Andrei Borzenkov = wrote:

...
...
I understand the rationalization, it's easier to get to bootable snapshots and rollbacks this way. I just disagree with a.) taking the user's feature and control away from them, b.) making the boot proces= s essentially invisible when it comes to what subvolume is being used.

You also need to a) somehow designate snapshot to be booted by default

That should be something the bootloader configuration understands. Snapshot doesn't change the need to define the bootloader's need to know what root is, to find the specific kernel + initramfs + root fs to pass off to the kernel.

When you have multiple specifications for kernel+initramfs+root you need to select which one to use. Be these specifications on subvolumes, filesystems or in $BOOT/loader/entries.

...
Now what's being done is to say, no we're giving up on all such specifics and we're going to hide it in fs metadata that is Btrfs specific. So the entire boot process has now changed, and only on Btrfs.

As snapper (rollbacks) are specific to btrfs this is rather logical.

It's logical only insofar as it was easier to achieve in the shortest amount of time. Not that it's either appropriate or good design, especially long term. As proof of concept it's OK but has a lot of consequences as baggage.

...

...
...
and b) handle path changes when snapshot is created (snapshot is different subvolume so any path inside grub.cfg needs adjustment).

This is no different than an LVM thinp snapshot where now it's a different device mapper path for boot and root. If you don't like things modifying grub.cfg,

I CAN NOT modify grub.cfg because it is on read-only snapshot. At least that was absolute requirement when SUSE created this code.

...
then we need to get serious about fixing whatever problems GRUB upstream has with the bootloaderspec. Either the original or mjg59's variant. And then whatever wants to change the boot process needs to add a new drop in script, rather than modifying bootloader files.

Again - one of the original requirements that lead to current design was booting snapshot without any preparations. I am not sure current code fulfills it, but it was attempted.

I'm not criticizing the attempt. (open)SUSE has progressed Btrfs, and bootable snapshots and rollback more than any other distro. But ostree and rpm-ostree do this much more elegantly and in a discoverable fashion and correctly modifies the bootloader configuration in a way that's at least in a documented specification. Whereas all of this work on Btrfs has a number of negative consequences.

...

...

...
change it.

...
I actually have design draft (from grub side) that requires minimal code changes (and some changes are what I believe should be included for other reasons) and solves all these issues. But it also means changes to installer, yast, snapper ... if anyone is willing to cooperate on fixing those, let's setup OBS project and try to implement it. Then it can be present as alternative solution; and I am pretty confident that GRUB changes that are required will be suitable for inclusion upstream.

Do you have a basic outline for this draft? I'm fairly dense when it comes to reading code.

Default bootloader in fixed subvolume, e.g. /mboot. "Local bootloader" in root subvolume, e.g in $subvol/boot Master bootloader multiboots local bootloader from $subvol/boot/xxx/core.img, passing it subvolume as parameter. Default subvolume to boot from either as subvolume property (preferable if btrfs developers agree) or as environment block in fixed location, e.g. /mboot/bootconfig.

In this scheme /mboot is installed just once and can be left alone. It has static grub.cfg that looks for available subvolumes and presents them in menu. Switching root environment means just setting variable in /mboot/bootconfig.

Required changes on grub side are

- make grub-mkconfig scripts generate btrfs paths relative to run-time subvolumes (current SUSE patches effectively do this already).

- make grub accept parameters passed via multiboot/EFI (everyone agrees it is needed, the question is mostly semantic). On EFI it may not even be needed because grub already knows where it was booted from, so likely can extract subvolume (with some help from btrfs).

OK I'm not opposed to anything in particular. But here's the problem. It's yet another idea on how to boot a linux OS, and it doesn't appear to include any aspect of the BootLoaderSpec.[1][2] Linux multiboot sucks. There are sufficient departures from upstream GRUB, and lack of installer cooperation by all the distros to ensure this continues to be difficult. While BLS is imperfect, it tries to solve this particular problem. And tries to solve it regardless of bootloader, filesystem, and type of firmware. OSTree leverages much of the BLS, and appears to be the direction RH/Fedora are going to go in. SUSE is doing something completely different than BLS, or GRUB upstream, or what you're proposing. And what you're proposing doesn't marry any of these two paradigms. So at the moment what I'm seeing is a significant divergence in the way linux distros are going to boot in the future, rather than any sense of cooperation, to the degree they will become completely incompatible rather than just difficult.

...

Downside is, the more run-time magic is added to grub.cfg the more incompatible with os-prober idea it becomes. OTOH nothing prevents os-prober from probing for /mboot ...

I thought upstream wanted to replace os-prober sooner than later? [1] http://www.freedesktop.org/wiki/Specifications/BootLoaderSpec/ [2] http://www.freedesktop.org/wiki/MatthewGarrett/BootLoaderSpec/ --- Chris Murphy -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Michael Chang

21 Jul 21 Jul

07:35

On Mon, Jul 20, 2015 at 01:58:38PM -0600, Chris Murphy wrote:

...

I screwed up replying to one of these messages by adding the regular (non-factory) list for subsequent replies. So this is the partial correction to fix this by bringing it back to the proper list.

On Mon, Jul 20, 2015 at 11:23 AM, Andrei Borzenkov wr= ote:

...
=D0=92 Mon, 20 Jul 2015 10:52:15 -0600 Chris Murphy =D0=BF=D0=B8=D1=88=D0=B5=D1=82:

...
On Sun, Jul 19, 2015 at 11:58 PM, Andrei Borzenkov = wrote:

...
...
I understand the rationalization, it's easier to get to bootable snapshots and rollbacks this way. I just disagree with a.) taking the user's feature and control away from them, b.) making the boot proces= s essentially invisible when it comes to what subvolume is being used.

You also need to a) somehow designate snapshot to be booted by default

That should be something the bootloader configuration understands. Snapshot doesn't change the need to define the bootloader's need to know what root is, to find the specific kernel + initramfs + root fs to pass off to the kernel.

When you have multiple specifications for kernel+initramfs+root you need to select which one to use. Be these specifications on subvolumes, filesystems or in $BOOT/loader/entries.

...
Now what's being done is to say, no we're giving up on all such specifics and we're going to hide it in fs metadata that is Btrfs specific. So the entire boot process has now changed, and only on Btrfs.

As snapper (rollbacks) are specific to btrfs this is rather logical.

It's logical only insofar as it was easier to achieve in the shortest amount of time. Not that it's either appropriate or good design, especially long term. As proof of concept it's OK but has a lot of consequences as baggage.

...
...
...
and b) handle path changes when snapshot is created (snapshot is different subvolume so any path inside grub.cfg needs adjustment).

This is no different than an LVM thinp snapshot where now it's a different device mapper path for boot and root. If you don't like things modifying grub.cfg,

I CAN NOT modify grub.cfg because it is on read-only snapshot. At least that was absolute requirement when SUSE created this code.

...
then we need to get serious about fixing whatever problems GRUB upstream has with the bootloaderspec. Either the original or mjg59's variant. And then whatever wants to change the boot process needs to add a new drop in script, rather than modifying bootloader files.

Again - one of the original requirements that lead to current design was booting snapshot without any preparations. I am not sure current code fulfills it, but it was attempted.

I'm not criticizing the attempt. (open)SUSE has progressed Btrfs, and bootable snapshots and rollback more than any other distro. But ostree and rpm-ostree do this much more elegantly and in a discoverable fashion and correctly modifies the bootloader configuration in a way that's at least in a documented specification. Whereas all of this work on Btrfs has a number of negative consequences.

...
...

...
change it.

...
I actually have design draft (from grub side) that requires minimal code changes (and some changes are what I believe should be included for other reasons) and solves all these issues. But it also means changes to installer, yast, snapper ... if anyone is willing to cooperate on fixing those, let's setup OBS project and try to implement it. Then it can be present as alternative solution; and I am pretty confident that GRUB changes that are required will be suitable for inclusion upstream.

Do you have a basic outline for this draft? I'm fairly dense when it comes to reading code.

Default bootloader in fixed subvolume, e.g. /mboot. "Local bootloader" in root subvolume, e.g in $subvol/boot Master bootloader multiboots local bootloader from $subvol/boot/xxx/core.img, passing it subvolume as parameter. Default subvolume to boot from either as subvolume property (preferable if btrfs developers agree) or as environment block in fixed location, e.g. /mboot/bootconfig.

In this scheme /mboot is installed just once and can be left alone. It has static grub.cfg that looks for available subvolumes and presents them in menu. Switching root environment means just setting variable in /mboot/bootconfig.

I did try the approach similar to yours, but ended up in failure. If my memory serves me well, the problem of multiboots local bootloader", for eg. multiboot $subvol/boot/grub/core.img is that we cannot change it's $prefix and also all varaible gets reset. That means we cannot pass variable to it, while it's possible through source a config file. That means prefix can make it reference to an deleted snapshot from which it got snapshotted. Another shortcoming is, you may not be able to go back to upper menu after booting core.img.

...

...
Required changes on grub side are

- make grub-mkconfig scripts generate btrfs paths relative to run-time subvolumes (current SUSE patches effectively do this already).

- make grub accept parameters passed via multiboot/EFI (everyone agrees it is needed, the question is mostly semantic). On EFI it may not even be needed because grub already knows where it was booted from, so likely can extract subvolume (with some help from btrfs).

That sounds like an attempt to fix the problem I have as described above. Thanks, Michael

...

OK I'm not opposed to anything in particular. But here's the problem. It's yet another idea on how to boot a linux OS, and it doesn't appear to include any aspect of the BootLoaderSpec.[1][2]

Linux multiboot sucks. There are sufficient departures from upstream GRUB, and lack of installer cooperation by all the distros to ensure this continues to be difficult. While BLS is imperfect, it tries to solve this particular problem. And tries to solve it regardless of bootloader, filesystem, and type of firmware.

OSTree leverages much of the BLS, and appears to be the direction RH/Fedora are going to go in.

SUSE is doing something completely different than BLS, or GRUB upstream, or what you're proposing. And what you're proposing doesn't marry any of these two paradigms.

So at the moment what I'm seeing is a significant divergence in the way linux distros are going to boot in the future, rather than any sense of cooperation, to the degree they will become completely incompatible rather than just difficult.

...
Downside is, the more run-time magic is added to grub.cfg the more incompatible with os-prober idea it becomes. OTOH nothing prevents os-prober from probing for /mboot ...

I thought upstream wanted to replace os-prober sooner than later?

[1] http://www.freedesktop.org/wiki/Specifications/BootLoaderSpec/ [2] http://www.freedesktop.org/wiki/MatthewGarrett/BootLoaderSpec/

--- Chris Murphy -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Andrei Borzenkov

08:18

On Tue, Jul 21, 2015 at 10:35 AM, Michael Chang wrote:

...

...
...
Default bootloader in fixed subvolume, e.g. /mboot. "Local bootloader" in root subvolume, e.g in $subvol/boot Master bootloader multiboots local bootloader from $subvol/boot/xxx/core.img, passing it subvolume as parameter. Default subvolume to boot from either as subvolume property (preferable if btrfs developers agree) or as environment block in fixed location, e.g. /mboot/bootconfig.

In this scheme /mboot is installed just once and can be left alone. It has static grub.cfg that looks for available subvolumes and presents them in menu. Switching root environment means just setting variable in /mboot/bootconfig.

I did try the approach similar to yours, but ended up in failure. If my memory serves me well, the problem of multiboots local bootloader",

for eg.

multiboot $subvol/boot/grub/core.img

is that we cannot change it's $prefix and also all varaible gets reset. That means we cannot pass variable to it, while it's possible through source a config file. That means prefix can make it reference to an deleted snapshot from which it got snapshotted.

Another shortcoming is, you may not be able to go back to upper menu after booting core.img.

As master bootloader is at fixed location it is always possible to chain back to it from any other core.img.

...

...
...
Required changes on grub side are

- make grub-mkconfig scripts generate btrfs paths relative to run-time subvolumes (current SUSE patches effectively do this already).

- make grub accept parameters passed via multiboot/EFI (everyone agrees it is needed, the question is mostly semantic). On EFI it may not even be needed because grub already knows where it was booted from, so likely can extract subvolume (with some help from btrfs).

That sounds like an attempt to fix the problem I have as described above.

Yes. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Michael Chang

08:54

On Tue, Jul 21, 2015 at 11:18:24AM +0300, Andrei Borzenkov wrote:

...

On Tue, Jul 21, 2015 at 10:35 AM, Michael Chang wrote:

...
Another shortcoming is, you may not be able to go back to upper menu after booting core.img.

As master bootloader is at fixed location it is always possible to chain back to it from any other core.img.

Yes, that sounds reasonable to solve it by providing a menu entry to chain back to master bootloader. But the behavior I was referring to was using ESC to go upper layer which is more intutive.

...

...
...
...
Required changes on grub side are

- make grub-mkconfig scripts generate btrfs paths relative to run-time subvolumes (current SUSE patches effectively do this already).

- make grub accept parameters passed via multiboot/EFI (everyone agrees it is needed, the question is mostly semantic). On EFI it may not even be needed because grub already knows where it was booted from, so likely can extract subvolume (with some help from btrfs).

That sounds like an attempt to fix the problem I have as described above.

Yes.

It may not work if $prefix is hardcoded in core.img but I think we can force to build with load.cfg and make necessary substitution there. Thanks, Michael

...

-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

3208

Age (days ago)

3214

Last active (days ago)

List overview

Download

6 comments

3 participants

participants (3)

Andrei Borzenkov
Chris Murphy
Michael Chang