Hello: I would like to set up a Leap 15.1 system that boots from a md raid1 array. I have a sample configuration from a working debian system that boots from raid1 array. However there are 2 things in the sample configuration I don't understand. The working debian grub.cfg has this: ### BEGIN /etc/grub.d/10_linux ### menuentry 'Debian GNU/Linux, with Linux 3.2.0-4-686-pae' --class debian --class gnu-linux --class gnu --class os { load_video insmod gzio insmod raid insmod mdraid1x insmod part_msdos insmod part_msdos insmod ext2 set root='(mduuid/4245........a8)' search --no-floppy --fs-uuid --set=root 68b........26e echo 'Loading Linux 3.2.0-4-686-pae ...' linux /boot/vmlinuz-3.2.0-4-686-pae root=UUID=68b........26e ro nomodeset quiet echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-3.2.0-4-686-pae } My questions: 1. It seems that grub sets the root device using the array's uuid (mduuid). Than it searches for file-system id, and sets the root to that file system. Isn't it enough to set the root using only one of these methods? Either by mduuid or by fs-uuid? Is it necessary to use both? What occurs if the the fs-uuid of the mduuid device's file system is not equal to the one given in the search command? 2. insmod part_msdos command is given 2 times. I guess it would be OK to use only once. Is this correct? Thanks, Istvan
24.12.2020 20:08, Istvan Gabor пишет:
Hello:
I would like to set up a Leap 15.1 system that boots from a md raid1 array.
I have a sample configuration from a working debian system that boots from raid1 array. However there are 2 things in the sample configuration I don't understand.
The working debian grub.cfg has this:
### BEGIN /etc/grub.d/10_linux ### menuentry 'Debian GNU/Linux, with Linux 3.2.0-4-686-pae' --class debian --class gnu-linux --class gnu --class os { load_video insmod gzio insmod raid insmod mdraid1x insmod part_msdos insmod part_msdos insmod ext2 set root='(mduuid/4245........a8)' search --no-floppy --fs-uuid --set=root 68b........26e echo 'Loading Linux 3.2.0-4-686-pae ...' linux /boot/vmlinuz-3.2.0-4-686-pae root=UUID=68b........26e ro nomodeset quiet echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-3.2.0-4-686-pae }
My questions:
1. It seems that grub sets the root device using the array's uuid (mduuid). Than it searches for file-system id, and sets the root to that file system. Isn't it enough to set the root using only one of these methods? Either by mduuid or by fs-uuid? Is it necessary to use both?
This is generated by scripts. Nobody bothered fine tune these scripts to consider such inter-dependencies. In general it cannot be assumed that device enumeration at boot time is the same as device enumeration in booted system, so the first assignment sets default based on guessing and search makes sure correct device is found even if grub guessed wrong.
What occurs if the the fs-uuid of the mduuid device's file system is not equal to the one given in the search command?
The value of "root" variable does not change.
2. insmod part_msdos command is given 2 times. I guess it would be OK to use only once. Is this correct?
Yes.
Thu, 24 Dec 2020 20:53:55 +0300 időpontban Andrei Borzenkov írta:
24.12.2020 20:08, Istvan Gabor пишет:
Hello:
I would like to set up a Leap 15.1 system that boots from a md raid1 array.
[LONG SNIP] Andrei, thanks for the quick answer. On my sytems I always use grub chainloading. I install on the HD (MBR) the chainloader grub menu/loader, and on the operating system's partition the systems grub boot loader. But those systems are not raid systems. Is it doable to chainload a raid1 file system with grub? If yes, how? Grub-install did not let me install grub on the md device. Thanks, Istvan
24.12.2020 20:53, Andrei Borzenkov пишет:
24.12.2020 20:08, Istvan Gabor пишет:
Hello:
I would like to set up a Leap 15.1 system that boots from a md raid1 array.
I have a sample configuration from a working debian system that boots from raid1 array. However there are 2 things in the sample configuration I don't understand.
The working debian grub.cfg has this:
### BEGIN /etc/grub.d/10_linux ### menuentry 'Debian GNU/Linux, with Linux 3.2.0-4-686-pae' --class debian --class gnu-linux --class gnu --class os { load_video insmod gzio insmod raid insmod mdraid1x insmod part_msdos insmod part_msdos insmod ext2 set root='(mduuid/4245........a8)' search --no-floppy --fs-uuid --set=root 68b........26e echo 'Loading Linux 3.2.0-4-686-pae ...' linux /boot/vmlinuz-3.2.0-4-686-pae root=UUID=68b........26e ro nomodeset quiet echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-3.2.0-4-686-pae }
My questions:
...
What occurs if the the fs-uuid of the mduuid device's file system is not equal to the one given in the search command?
The value of "root" variable does not change.
That's not quite correct. If fs-uuid in search command is found on some other device than mduuid device then "root" variable will be set to refer to this other device. If fs-uuid in search command is not found anywhere, the value if "root" variable does not change.
Fri, 25 Dec 2020 09:32:43 +0300 időpontban Andrei Borzenkov írta:
24.12.2020 20:53, Andrei Borzenkov пишет:
24.12.2020 20:08, Istvan Gabor пишет:
Hello:
I would like to set up a Leap 15.1 system that boots from a md raid1 array.
I have a sample configuration from a working debian system that boots from raid1 array. However there are 2 things in the sample configuration I don't understand.
The working debian grub.cfg has this:
### BEGIN /etc/grub.d/10_linux ### menuentry 'Debian GNU/Linux, with Linux 3.2.0-4-686-pae' --class debian --class gnu-linux --class gnu --class os { load_video insmod gzio insmod raid insmod mdraid1x insmod part_msdos insmod part_msdos insmod ext2 set root='(mduuid/4245........a8)' search --no-floppy --fs-uuid --set=root 68b........26e echo 'Loading Linux 3.2.0-4-686-pae ...' linux /boot/vmlinuz-3.2.0-4-686-pae root=UUID=68b........26e ro nomodeset quiet echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-3.2.0-4-686-pae }
My questions:
...
What occurs if the the fs-uuid of the mduuid device's file system is not equal to the one given in the search command?
The value of "root" variable does not change.
That's not quite correct. If fs-uuid in search command is found on some other device than mduuid device then "root" variable will be set to refer to this other device. If fs-uuid in search command is not found anywhere, the value if "root" variable does not change.
Thank you Andrei. I understand this. I still could not make grub2 raid1 chainloading. I could install grub2 on the raid1 (/dev/md113) device: # grub2-install --force /dev/md113 Installing for i386-pc platform. grub2-install: warning: the drive name `mduuid/b...f6' in device.map is incorrect. Using hostdisk//dev/md113 instead. Please use the form [hfc]d[0-9]* (E.g. `hd0' or `cd'). grub2-install: warning: File system `ext2' doesn't support embedding. grub2-install: warning: Embedding is not possible. GRUB can only be installed in this setup by using blocklists. However, blocklists are UNRELIABLE and their use is discouraged.. Installation finished. No error reported. This is grub.cfg: set timeout=3 menuentry "openSUSE Leap 15.1" { insmod part_msdos insmod ext2 insmod mdraid1x set root='mduuid/b...f6' search --no-floppy -l Leap -s linux /boot/vmlinuz root=LABEL=Leap noresume initrd /boot/initrd } In grub2-emu the correct menu appears in the emulator, that is grub is installed on the raid1 (/dev/md113) device. I installed the chainloader on the mbrs of /dev/sda and /dev/sdb. # grub2-install --boot-directory=/boot/grub2.chainload /dev/sda Installing for i386-pc platform. Installation finished. No error reported. # grub2-install --boot-directory=/boot/grub2.chainload /dev/sdb Installing for i386-pc platform. Installation finished. No error reported. The chainloader's config is this: set timeout=3 menuentry "openSUSE Leap 15.1" { insmod part_msdos insmod ext2 insmod mdraid1x insmod chain set root='mduuid/b...f6' search --no-floppy -l Leap -s chainloader +1 } This shows the correct menu when I start the computer. But when it tries to boot/chainload the raid1 device, grub hangs, only showing: GRUB It seems that the chainloader can't find or load grub installed on the raid1 device? How can I solve this? I tried the above configurations with commented search lines and insmod lines but there was no difference. Thanks, Istvan
26.12.2020 15:38, Istvan Gabor пишет:
Fri, 25 Dec 2020 09:32:43 +0300 időpontban Andrei Borzenkov írta:
24.12.2020 20:53, Andrei Borzenkov пишет:
24.12.2020 20:08, Istvan Gabor пишет:
Hello:
I would like to set up a Leap 15.1 system that boots from a md raid1 array.
I have a sample configuration from a working debian system that boots from raid1 array. However there are 2 things in the sample configuration I don't understand.
The working debian grub.cfg has this:
### BEGIN /etc/grub.d/10_linux ### menuentry 'Debian GNU/Linux, with Linux 3.2.0-4-686-pae' --class debian --class gnu-linux --class gnu --class os { load_video insmod gzio insmod raid insmod mdraid1x insmod part_msdos insmod part_msdos insmod ext2 set root='(mduuid/4245........a8)' search --no-floppy --fs-uuid --set=root 68b........26e echo 'Loading Linux 3.2.0-4-686-pae ...' linux /boot/vmlinuz-3.2.0-4-686-pae root=UUID=68b........26e ro nomodeset quiet echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-3.2.0-4-686-pae }
My questions:
...
What occurs if the the fs-uuid of the mduuid device's file system is not equal to the one given in the search command?
The value of "root" variable does not change.
That's not quite correct. If fs-uuid in search command is found on some other device than mduuid device then "root" variable will be set to refer to this other device. If fs-uuid in search command is not found anywhere, the value if "root" variable does not change.
Thank you Andrei. I understand this.
I still could not make grub2 raid1 chainloading.
I could install grub2 on the raid1 (/dev/md113) device:
# grub2-install --force /dev/md113 Installing for i386-pc platform. grub2-install: warning: the drive name `mduuid/b...f6' in device.map is incorrect. Using hostdisk//dev/md113 instead. Please use the form [hfc]d[0-9]* (E.g. `hd0' or `cd'). grub2-install: warning: File system `ext2' doesn't support embedding. grub2-install: warning: Embedding is not possible. GRUB can only be installed in this setup by using blocklists. However, blocklists are UNRELIABLE and their use is discouraged.. Installation finished. No error reported.
The result is grub bootblock in the first sector of /dev/md113 which is configured to load core.img from the absolute disk location on /dev/md113 (which corresponds to the file /boot/grub2/i386-pc/core.img).
This is grub.cfg:
set timeout=3 menuentry "openSUSE Leap 15.1" { insmod part_msdos insmod ext2 insmod mdraid1x set root='mduuid/b...f6' search --no-floppy -l Leap -s linux /boot/vmlinuz root=LABEL=Leap noresume initrd /boot/initrd }
In grub2-emu the correct menu appears in the emulator, that is grub is installed on the raid1 (/dev/md113) device.
You do realize that grub-emu does know how to interpret "hostdisk" and boot time grub code most likely does not?
I installed the chainloader on the mbrs of /dev/sda and /dev/sdb.
# grub2-install --boot-directory=/boot/grub2.chainload /dev/sda Installing for i386-pc platform. Installation finished. No error reported.
# grub2-install --boot-directory=/boot/grub2.chainload /dev/sdb Installing for i386-pc platform. Installation finished. No error reported.
You must have some reasons to use these commands because I do not understand what these commands are supposed to achieve.
The chainloader's config is this:
set timeout=3 menuentry "openSUSE Leap 15.1" { insmod part_msdos insmod ext2 insmod mdraid1x insmod chain set root='mduuid/b...f6'
That's wrong. $root at this point must point to physical BIOS disk that will be considered "boot device" by loaded code. Linux MD RAID device cannot be BIOS disk by definition.
search --no-floppy -l Leap -s chainloader +1 }
Huh? In the previous step you "installed chainloader" (whatever you mean here) on MBR of two disks. Now you load block from entirely different disk location which has nothing to do with these two MBR. How exactly these two steps are related? Unless you have firmware partitioned RAID (like IMSM). But as you do not describe your complete configuration I can only guess.
This shows the correct menu when I start the computer. But when it tries to boot/chainload the raid1 device, grub hangs, only showing: GRUB
This means the initial boot sector was loaded and it tried to load core.img to continue boot and likely succeeded to load something (otherwise we should have seen different error) but code that it loaded hung.
It seems that the chainloader can't find or load grub installed on the raid1 device?
Of course not. chainloader is command that emulates PC BIOS boot protocol^Wconventions. PC BIOS knows nothing about Linux MD RAID, so you cannot tell PC BIOS to boot from Linux MD RAID. What happens here - you load the first block from your MD RAID1 (because *grub* knows how to access it) and jump to it. It gets as parameter some BIOS disk number (it does not matter which one, in this case grub will fall back to disk C - 0x80 - because you did not provide valid disk number). Grub boot block was configured to load core.img from absolute disk locations *relative to /dev/md113*. Now grub interprets these locations *relative to physical disk start*. So it loads some garbage at these locations and jumps to it.
How can I solve this?
If the question is "how you boot with /boot/grub on MD RAID1" - you simply install grub on MBR, that's all. You can also chainload this MBR from another grub (making sure to set correct $root) and it should also work. If the question is "how I chainload grub from /boot/grub on MD RAID1 if I cannot install on MBR" - well, you need separate physical partition(s) with file system that supports embedding to install boot block into. In practice this is only btrfs or you could (try to) convince grub maintainers to support installing boot block directly in raw partition without filesystem. Currently this is explicitly prohibited. You will still need separate physical partitions to install grub into them. Alternatively you could directly load core.img using multiboot /boot/grub2/i386-pc/core.img assuming this core.img was configured to find its /boot/grub on MD RAID1. You will need to skip actual installation of boot block using grub-install --no-bootsector This won't be integrated in default bootloader management framework on openSUSE. Which is also doable ... if someone is willing to implement and maintain it.
27.12.2020 10:52, Andrei Borzenkov пишет:
26.12.2020 15:38, Istvan Gabor пишет:
# grub2-install --force /dev/md113 Installing for i386-pc platform. grub2-install: warning: the drive name `mduuid/b...f6' in device.map is incorrect. Using hostdisk//dev/md113 instead. Please use the form [hfc]d[0-9]* (E.g. `hd0' or `cd').
And that is wrong as well. device.map maps *whole disks* - it allows you to declare that device /dev/XXX will be seen by BIOS during boot as *whole disk* so grub can assume this device will be available. You Linux MD RAID is not whole disk, it is composed of disk partitions. Nor will it be seen by BIOS as the "whole disk" (or as any device that BIOS understands). device.map is really never needed - in all cases I can think of you use "search" command to find filesystem that contains information you need (/boot/grub for grub modules or /boot for Linux kernel).
On 12/26/20 6:38 AM, Istvan Gabor wrote:
anywhere, the value if "root" variable does not change.
Thank you Andrei. I understand this.
I still could not make grub2 raid1 chainloading.
Istvan, I think you are confusing yourself. The only requirement for booting from raid1 is that the mdadm module have been included in the mkinitrd process so that it is available to assemble the arrays as boot time. If I recall correctly, openSUSE includes mdadm in the initrd by default. So the only thing you would need to do is use the UUID for the array holding the boot partition. I run all of my servers on raid1, and I have separate /boot, /, /home (and in some cases /var and /srv) arrays. On openSUSE there wasn't any additional configuration needed. On Arch I have to add the mdadm module to the list of modules compiled into the initrd -- that's it. Once you are to the point of installing the boot loader (mbr type) you can install the boot loader to BOTH physical disks in your array (you only need to install to one, but I would recommend installing to BOTH so in the even of a failed disk containing the mbr, you can install a new disk, boot in degraded mode from the other physical disk in the array and rebuild the array without ever needing the install .iso. So the question you need to answer is whether mdadm is already included in the initrd, and if so, then just install the bootloader into the mbr of one (or both) the physical disk(s) that a part of the array and use the UUID's of the array with grub. If there are other changes needed for UEFI boot, then someone else will need to chime in as I have happily avoided UEFI so far... -- David C. Rankin, J.D.,P.E.
participants (3)
-
Andrei Borzenkov
-
David C. Rankin
-
Istvan Gabor