http://bugzilla.suse.com/show_bug.cgi?id=978593 Tomáš Chvátal <tchvatal@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|jsrain@suse.com |mchang@suse.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=978593 http://bugzilla.suse.com/show_bug.cgi?id=978593#c4 Michael Chang <mchang@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tchvatal@suse.com Flags| |needinfo?(tchvatal@suse.com | |) --- Comment #4 from Michael Chang <mchang@suse.com> --- Would you please help providing these files 1. yast2 logs 2. /etc/default/grub 3. /boot/grub2/grub.cfg 4. /boot/grub2/x86_64-efi/load.cfg Thanks. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=978593 http://bugzilla.suse.com/show_bug.cgi?id=978593#c6 --- Comment #6 from Tomáš Chvátal <tchvatal@suse.com> --- Created attachment 675742 --> http://bugzilla.suse.com/attachment.cgi?id=675742&action=edit grub.cfg -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=978593 http://bugzilla.suse.com/show_bug.cgi?id=978593#c8 Tomáš Chvátal <tchvatal@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(tchvatal@suse.com | |) | --- Comment #8 from Tomáš Chvátal <tchvatal@suse.com> --- Created attachment 675744 --> http://bugzilla.suse.com/attachment.cgi?id=675744&action=edit yast-logs.tar.bz2 Hopefully thats all you need :) -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=978593 http://bugzilla.suse.com/show_bug.cgi?id=978593#c10 Alexander Graf <agraf@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |agraf@suse.com, | |glin@suse.com, | |ihno@suse.com, | |jeffm@suse.com, | |ohering@suse.com, | |snwint@suse.com --- Comment #10 from Alexander Graf <agraf@suse.com> --- We've stumbled over an issue that looks very similar to this one, so let's hijack this bug in the hope that it really is the same problem. I see 2 issues overlapping themselves here. 1) Btrfs naming --------------- I had a quick chat with Olaf who ran into a similar issue with Xen's pvgrub. The basic problem boils down to difference between upstream grub2 btrfs handling and SUSE grub2 btrfs handling. While upstream uses full path names including the subvolume id: /@/boot/grub2 we omit the subvolume id and directly use a relative path into some default subvolume: /boot/grub2 This leads to a number of problems. In this case, it means that grub2-install puts the upstream style path into the prefix template in grubaa64.efi while our btrfs driver code expects the downstream style path. In the pvgrub case, it means that grub2 compiled from upstream sources can't read our grub.cfg properly, since the paths don't match. I guess this warrants a new bug id though and we should leave this bug to the yast2-bootloader issue that secure boot should be disabled on AArch64. 2) removable media binary To support systems that don't have working NVRAM to store EFI boot entries, I added code that determines we're running on such a system and in that case writes grub2 in the default EFI\BOOT\BOOTAA64.efi path: https://github.com/openSUSE/perl-bootloader/commit/bfd36e38188ccfbb737ffe57d... That code however gets invoked during installation without /sys mounted, so the heuristics fail and we always create a removable fallback binary. The net result of this is that you get a removable media bootaa64.efi (or bootx64.efi) which has a default btrfs path embedded that doesn't work because our btrfs paths look different. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=978593 http://bugzilla.suse.com/show_bug.cgi?id=978593#c12 --- Comment #12 from Michael Chang <mchang@suse.com> ---

That code however gets invoked during installation without /sys mounted, so the heuristics fail and we always create a removable fallback binary.

It's likely efivarfs.ko not there ..

The net result of this is that you get a removable media bootaa64.efi (or bootx64.efi) which has a default btrfs path embedded that doesn't work because our btrfs paths look different.

I think yast2 bootloader should also put that pbl logic to invoke grub2-install to get everything consistent. That pbl script may invoke too early before /etc/default/grub get correctly written by yast. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=978593 http://bugzilla.suse.com/show_bug.cgi?id=978593#c14 --- Comment #14 from Michael Chang <mchang@suse.com> --- (In reply to Alexander Graf from comment #10)

1) Btrfs naming ---------------

(Sigh) This is actually bothering me all the time, and I think even we need a new bug separated from this one.

I had a quick chat with Olaf who ran into a similar issue with Xen's pvgrub.

The basic problem boils down to difference between upstream grub2 btrfs handling and SUSE grub2 btrfs handling. While upstream uses full path names including the subvolume id:

/@/boot/grub2

Yes, upstream use this relatively straight path scheme so that btrfs could be treated like any other filesystems as if no subvolumes there ..

we omit the subvolume id and directly use a relative path into some default subvolume:

/boot/grub2

We need to support booting into snapshots and also rollback via btrfs-set-default so that relative path in a subvol is more straightforward as you could switch different subvols to boot without changing the grub.cfg. But the downside is also it's boot file could be varied by your subvol which is ambiguous to know where it really located from the config at times (unlike the absolute path scheme is always obvious). And also to access files outside the subvol is an issue, currently we use btrfs-mount-subvol to get a subvol in a path. In short it's more headache to get your config correct than absolute path. It's hard really hard, because upstream has settled with absolute path scheme and wouldn't allow it to break. :(

This leads to a number of problems. In this case, it means that grub2-install puts the upstream style path into the prefix template in grubaa64.efi while our btrfs driver code expects the downstream style path.

The path is controlled by SUSE_BTRFS_SNAPSHOT_BOOTING="true|false" (default false) and will affect grub2-install and grub2-mkconfig.

In the pvgrub case, it means that grub2 compiled from upstream sources can't read our grub.cfg properly, since the paths don't match.

My only suggestion is disable btrfs snapshot booting and SUSE_BTRFS_SNAPSHOT_BOOTING=false to use upstream path scheme. Sorry I don't have solution to satisfy both ends. Thanks -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=978593 http://bugzilla.suse.com/show_bug.cgi?id=978593#c16 --- Comment #16 from Alexander Graf <agraf@suse.com> --- (In reply to Michael Chang from comment #15)

Why? Wouldn't the "normal" EFI systems covered by that check as well ? (ie it will not append --no-nvram --removable)

Exactly, so for the "normal" systems nothing changes over today's state. We still have a broken removable file because pbl installed it, so if we end up in the fallback case for whatever reason (nvram invalidated in between executions, wrong boot order configured manually, etc) we still boot the wrong one. In my case, I usually end up with the removable fallback because I use qemu's -bios option which treats all nvram variables as volatile, so after a reset they're gone. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=978593 http://bugzilla.suse.com/show_bug.cgi?id=978593#c18 --- Comment #18 from Michael Chang <mchang@suse.com> --- (In reply to Alexander Graf from comment #16)

(In reply to Michael Chang from comment #15)

...

Why? Wouldn't the "normal" EFI systems covered by that check as well ? (ie it will not append --no-nvram --removable)

Exactly, so for the "normal" systems nothing changes over today's state. We still have a broken removable file because pbl installed it, so if we end up in the fallback case for whatever reason (nvram invalidated in between executions, wrong boot order configured manually, etc) we still boot the wrong one.

Maybe as a temporary fix we should check for ARM patform in that pbl patch (or even yast in the future), we should leave that default efi loader in x86 hanldled by shim's fallback mode (ie shim-install). It's even more hard to say as an analogy to mbr whether we should "update" that file or not from our system. It maybe owned by other distro, much more likely on x86.. It should really be an option for x86 but I think ARM is fine with that.

In my case, I usually end up with the removable fallback because I use qemu's -bios option which treats all nvram variables as volatile, so after a reset they're gone.

The vars are store in ESP, IIRC, as file name /NvVars to be restore during OVMF booting. But that's a hack indeed. (And I don't know whether it's also the case for ARM). -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=978593 http://bugzilla.suse.com/show_bug.cgi?id=978593#c20 --- Comment #20 from Michael Chang <mchang@suse.com> --- (In reply to Alexander Graf from comment #19)

(In reply to Michael Chang from comment #18)

...

(In reply to Alexander Graf from comment #16)

...

(In reply to Michael Chang from comment #15)

I think if what you say is true, the best solution would be to just include efivars.ko in the installer system. Then the rest should resolve itself automatically, because we will detect the correct type of system again.

There may be some misunderstanding here, the "efivars" seen already built-in in kernel while "efivarsfs" is a kernel module and not available in install system. At least from my test ls /sys/firmware/efi/efivars outputs nothing but ls /sys/firmware/efi/vars outputs plenty of files. In your patch it's testing /sys/firmware/efi/efivars ...

Also, whether we use shim or grub2-install is completely orthogonal over whether we want to install the bootloader as --removable or not. Shim vs grub2-install is a software concept, NVRAM available or not is a hardware problem - some systems simply don't have any storage for it.

OK. I think you're right here. I just want to highlight that shim also provide a similar "fallback" feature that will "restore" the boot entries if it was placed under default boot path (eg /boot/efi/BOOTX64.efi) to restore the missing boot entries after a firmware upgrade which may erase all entries. It will only get installed there if that path is not occupied. I am not really into this part of work (Gary did it) so wouldn't be able to say more about it so basically JFYI. :) Thanks. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=978593 http://bugzilla.suse.com/show_bug.cgi?id=978593#c22 --- Comment #22 from Michael Chang <mchang@suse.com> --- Thanks .. so my understanding is To fix this bug, include efivarfs.ko in install system. To fix bsc#978157 (aarch64), fix yast2 bootloader by the same pbl logic. Is it correct ? Thanks. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=978593 http://bugzilla.suse.com/show_bug.cgi?id=978593#c24 --- Comment #24 from Michael Chang <mchang@suse.com> --- (In reply to Alexander Graf from comment #23)

(In reply to Michael Chang from comment #22)

I think we're getting close:

https://github.com/agraf/yast-bootloader/commit/ 3c5219575796c9316ab8d74f06381308ee0feedd

https://github.com/agraf/yast-bootloader/commit/ a133a4c66409a3174dc37f8b18188a0fc38ed00e

What's left is the installer change.

Wow. You are really amazing quick! :) -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=978593 http://bugzilla.suse.com/show_bug.cgi?id=978593#c27 Josef Möllers <josef.moellers@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED CC| |josef.moellers@suse.com Resolution|--- |FIXED --- Comment #27 from Josef Möllers <josef.moellers@suse.com> --- Fixed, see comment#25 and comment#26. -- You are receiving this mail because: You are on the CC list for the bug.