[opensuse-kubic] Raspberry Pi 4 Feedback and Questions
I just installed MicroOS on my Raspberry Pi 4, and first off: awesome work with getting this working! I was amazed to see GRUB, some kind of EFI loader, and more all working on the Pi 4 -- it's amazing to see the Pi evolve into an actual full-fledged computer. However, I discovered a workflow that seems to be poorly supported as of now. My goal is to get MicroOS booting from a USB SSD (for better performance over the MicroSD card), and I ran into all sorts of troubles along the way. The major issues were: 1. Somehow it seems the dracut-pre-mount hook is deleting the 2nd partition during Ignition firstboot; if I dd the base image onto my SSD and use an ignition file, I can see /dev/sda2 is a proper btrfs filesystem; however, after letting MicroOS start to boot, inspecting the disk after shows the filesystem has been wiped from /dev/sda2 2. I'm not sure if this is just how baremetal images are supposed to work, but I can't seem to disable Ignition/Combustion after the first boot. I tried deleting the ignition cmdline argument from /etc/default/grub and running `transactional-update grub.cfg`, but then it complains about generators/dependencies failing for MicroOS... and then it boots successfully anyways 3. Once I got what I thought was a "proper" install on a MicroSD card, I tried partcloning the MicroOS btrfs partition to my SSD and booting that. Sure enough it worked, but it kept throwing errors about `mmc1: Timeout waiting for hardware cmd interrupt`. It would seem that somewhere something is hardcoded to expect a MicroSD card, and it is VERY mad that I'm trying to only use my SSD :( I got some "screenshots" of the tty with these issues, and I also have the rdsosreport.txt from trying to install directly on the SSD, but I can certainly try the process again and help get any more info to help debug and improve this experience! Screenshots: https://imgur.com/a/IpMF0lw rdsosreport: https://gist.github.com/steelcowboy/5148d854b3d83d84c734e5a0830e927d Please let me know if I can do anything to help resolve some of these issues and make the Raspberry Pi experience better! Thanks! Jim --------------------------------------------------- James Heald Production Engineer Facebook james.r.heald@gmail.com -- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
Am 07.09.20 um 02:18 Uhr schrieb Jim Heald:
I just installed MicroOS on my Raspberry Pi 4, and first off: awesome work with getting this working! I was amazed to see GRUB, some kind of EFI loader, and more all working on the Pi 4 -- it's amazing to see the Pi evolve into an actual full-fledged computer.
However, I discovered a workflow that seems to be poorly supported as of now. My goal is to get MicroOS booting from a USB SSD (for better performance over the MicroSD card), and I ran into all sorts of troubles along the way. The major issues were:
1. Somehow it seems the dracut-pre-mount hook is deleting the 2nd partition during Ignition firstboot; if I dd the base image onto my SSD and use an ignition file, I can see /dev/sda2 is a proper btrfs filesystem; however, after letting MicroOS start to boot, inspecting the disk after shows the filesystem has been wiped from /dev/sda2
I strongly assume that during resizing /dev/sda2 the start sector of the second partition is moved. Some snippets from the log file:
Units: sectors of 1 * 512 = 512 bytes I/O size (minimum/optimal): 512 bytes / 33553920 bytes
Device Boot Start End Sectors Size Id Type /dev/sda1 2048 34815 32768 16M c W95 FAT32 (LBA) /dev/sda2 65535 468862127 468796593 223.6G 83 Linux
33553920 / 512 = 65535, so it seems that the partition is aligned at the optimal position, but that's not where KIWI originally created it. Looking at the log of a RPi booted from a regular USB flash drive I can see an optimal sector size of 512 bytes, and the partition starts at 34816. This is most likely a KIWI bug in one of its dracut scripts - when resizing partitions it must be made sure that the start sector doesn't change.
2. I'm not sure if this is just how baremetal images are supposed to work, but I can't seem to disable Ignition/Combustion after the first boot. I tried deleting the ignition cmdline argument from /etc/default/grub and running `transactional-update grub.cfg`, but then it complains about generators/dependencies failing for MicroOS... and then it boots successfully anyways
Ignition and Combustion need a writable subvolume called /boot/writable to store the information. Is that subvolume properly mounted and / or does it contain a file called firstboot_happened?
3. Once I got what I thought was a "proper" install on a MicroSD card, I tried partcloning the MicroOS btrfs partition to my SSD and booting that. Sure enough it worked, but it kept throwing errors about `mmc1: Timeout waiting for hardware cmd interrupt`. It would seem that somewhere something is hardcoded to expect a MicroSD card, and it is VERY mad that I'm trying to only use my SSD :(
At least on a RPi3 test setup the system is booting perfectly fine without a MicroSD card, so that doesn't seem to be a generic MicroOS problem. As those messages are kernel messages I would assume that hardware support for RPi4 is still a bit lacking...
I got some "screenshots" of the tty with these issues, and I also have the rdsosreport.txt from trying to install directly on the SSD, but I can certainly try the process again and help get any more info to help debug and improve this experience!
Screenshots: https://imgur.com/a/IpMF0lw rdsosreport: https://gist.github.com/steelcowboy/5148d854b3d83d84c734e5a0830e927d
Please let me know if I can do anything to help resolve some of these issues and make the Raspberry Pi experience better! Thanks!
Someone may need to create a KIWI bug report ;-) Thanks for your feedback by the way! Cheers, Ignaz -- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
Hi Jim, Ignaz,
Could you point me to some doc / notes on how you setup rpi to boot on USB SSD ?
I'll align my setup to make tests with that configuration (and not
only with sdcards)
thanks
Paul
On Mon, Sep 7, 2020 at 5:32 PM Ignaz Forster
Am 07.09.20 um 02:18 Uhr schrieb Jim Heald:
I just installed MicroOS on my Raspberry Pi 4, and first off: awesome work with getting this working! I was amazed to see GRUB, some kind of EFI loader, and more all working on the Pi 4 -- it's amazing to see the Pi evolve into an actual full-fledged computer.
However, I discovered a workflow that seems to be poorly supported as of now. My goal is to get MicroOS booting from a USB SSD (for better performance over the MicroSD card), and I ran into all sorts of troubles along the way. The major issues were:
1. Somehow it seems the dracut-pre-mount hook is deleting the 2nd partition during Ignition firstboot; if I dd the base image onto my SSD and use an ignition file, I can see /dev/sda2 is a proper btrfs filesystem; however, after letting MicroOS start to boot, inspecting the disk after shows the filesystem has been wiped from /dev/sda2
I strongly assume that during resizing /dev/sda2 the start sector of the second partition is moved. Some snippets from the log file:
Units: sectors of 1 * 512 = 512 bytes I/O size (minimum/optimal): 512 bytes / 33553920 bytes
Device Boot Start End Sectors Size Id Type /dev/sda1 2048 34815 32768 16M c W95 FAT32 (LBA) /dev/sda2 65535 468862127 468796593 223.6G 83 Linux
33553920 / 512 = 65535, so it seems that the partition is aligned at the optimal position, but that's not where KIWI originally created it. Looking at the log of a RPi booted from a regular USB flash drive I can see an optimal sector size of 512 bytes, and the partition starts at 34816.
This is most likely a KIWI bug in one of its dracut scripts - when resizing partitions it must be made sure that the start sector doesn't change.
2. I'm not sure if this is just how baremetal images are supposed to work, but I can't seem to disable Ignition/Combustion after the first boot. I tried deleting the ignition cmdline argument from /etc/default/grub and running `transactional-update grub.cfg`, but then it complains about generators/dependencies failing for MicroOS... and then it boots successfully anyways
Ignition and Combustion need a writable subvolume called /boot/writable to store the information. Is that subvolume properly mounted and / or does it contain a file called firstboot_happened?
3. Once I got what I thought was a "proper" install on a MicroSD card, I tried partcloning the MicroOS btrfs partition to my SSD and booting that. Sure enough it worked, but it kept throwing errors about `mmc1: Timeout waiting for hardware cmd interrupt`. It would seem that somewhere something is hardcoded to expect a MicroSD card, and it is VERY mad that I'm trying to only use my SSD :(
At least on a RPi3 test setup the system is booting perfectly fine without a MicroSD card, so that doesn't seem to be a generic MicroOS problem. As those messages are kernel messages I would assume that hardware support for RPi4 is still a bit lacking...
I got some "screenshots" of the tty with these issues, and I also have the rdsosreport.txt from trying to install directly on the SSD, but I can certainly try the process again and help get any more info to help debug and improve this experience!
Screenshots: https://imgur.com/a/IpMF0lw rdsosreport: https://gist.github.com/steelcowboy/5148d854b3d83d84c734e5a0830e927d
Please let me know if I can do anything to help resolve some of these issues and make the Raspberry Pi experience better! Thanks!
Someone may need to create a KIWI bug report ;-) Thanks for your feedback by the way!
Cheers, Ignaz -- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
-- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
Am 08.09.20 um 13:12 Uhr schrieb Paul Gonin:
Hi Jim, Ignaz,
Could you point me to some doc / notes on how you setup rpi to boot on USB SSD ? I'll align my setup to make tests with that configuration (and not only with sdcards)
From what I understand booting from USB mass storage may have to be enabled first: https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/msd.... (For the test I was using Fabian's RPi in the office, and that one just booted from the attached USB flash drive.)
The MicroOS image can just be dumped onto the drive with dd. Cheers, Ignaz -- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
Hi all, sorry for the delayed response!
Ignaz, thank you for that analysis! I've gone ahead and filed the
following bug for this issue:
https://bugzilla.suse.com/show_bug.cgi?id=1176472
I had /boot/writeable properly mounted with the firstboot file, but
I'm curious then: should Ignition/Combustion look for that first and
report success if they see it? I'd have to try and repro, but my
experience suggested those were ALWAYS looking for a volume with the
label "ignition" to be attached.
And understood that the mmc issues are probably with the kernel; I'll
see if I can find any kernel bug reports of that nature :)
Appreciate all the help!
Jim
---------------------------------------------------
James Heald
Production Engineer
Facebook
james.r.heald@gmail.com
Jim
---------------------------------------------------
James Heald
Production Engineer
Facebook
james.r.heald@gmail.com
On Thu, Sep 10, 2020 at 12:59 AM Ignaz Forster
Am 08.09.20 um 13:12 Uhr schrieb Paul Gonin:
Hi Jim, Ignaz,
Could you point me to some doc / notes on how you setup rpi to boot on USB SSD ? I'll align my setup to make tests with that configuration (and not only with sdcards)
From what I understand booting from USB mass storage may have to be enabled first: https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/msd.... (For the test I was using Fabian's RPi in the office, and that one just booted from the attached USB flash drive.)
The MicroOS image can just be dumped onto the drive with dd.
Cheers, Ignaz
-- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
Also just following up on some things: 1. I was able to fix the mmc issue by adding `dtparam=sd_poll_once` to /boot/efi/config.txt; see https://bugzilla.suse.com/show_bug.cgi?id=1176576 2. I'm still having trouble with Ignition; even though /boot/writable/firstboot_happened exists, I still see:
[ TIME ] Timed out waiting for /dev/disk/by-label/ignition
on every reboot. In /proc/cmdline I see:
```
BOOT_IMAGE=/boot/Image-5.8.4-1-default
root=UUID=e0428e2f-3a39-4034-a146-62fcfe5183a8 rd.timeout=60
console=ttyS0,115200n8 console=tty0 quiet net.ifnames=0
ignition.platform.id=metal rd.neednet=0
```
What can I look into to debug why it's still looking for an Ignition
volume after firstboot?
Jim
---------------------------------------------------
James Heald
Production Engineer
Facebook
james.r.heald@gmail.com
On Tue, Sep 8, 2020 at 4:12 AM Paul Gonin
Hi Jim, Ignaz,
Could you point me to some doc / notes on how you setup rpi to boot on USB SSD ? I'll align my setup to make tests with that configuration (and not only with sdcards)
thanks Paul
On Mon, Sep 7, 2020 at 5:32 PM Ignaz Forster
wrote: Am 07.09.20 um 02:18 Uhr schrieb Jim Heald:
I just installed MicroOS on my Raspberry Pi 4, and first off: awesome work with getting this working! I was amazed to see GRUB, some kind of EFI loader, and more all working on the Pi 4 -- it's amazing to see the Pi evolve into an actual full-fledged computer.
However, I discovered a workflow that seems to be poorly supported as of now. My goal is to get MicroOS booting from a USB SSD (for better performance over the MicroSD card), and I ran into all sorts of troubles along the way. The major issues were:
1. Somehow it seems the dracut-pre-mount hook is deleting the 2nd partition during Ignition firstboot; if I dd the base image onto my SSD and use an ignition file, I can see /dev/sda2 is a proper btrfs filesystem; however, after letting MicroOS start to boot, inspecting the disk after shows the filesystem has been wiped from /dev/sda2
I strongly assume that during resizing /dev/sda2 the start sector of the second partition is moved. Some snippets from the log file:
Units: sectors of 1 * 512 = 512 bytes I/O size (minimum/optimal): 512 bytes / 33553920 bytes
Device Boot Start End Sectors Size Id Type /dev/sda1 2048 34815 32768 16M c W95 FAT32 (LBA) /dev/sda2 65535 468862127 468796593 223.6G 83 Linux
33553920 / 512 = 65535, so it seems that the partition is aligned at the optimal position, but that's not where KIWI originally created it. Looking at the log of a RPi booted from a regular USB flash drive I can see an optimal sector size of 512 bytes, and the partition starts at 34816.
This is most likely a KIWI bug in one of its dracut scripts - when resizing partitions it must be made sure that the start sector doesn't change.
2. I'm not sure if this is just how baremetal images are supposed to work, but I can't seem to disable Ignition/Combustion after the first boot. I tried deleting the ignition cmdline argument from /etc/default/grub and running `transactional-update grub.cfg`, but then it complains about generators/dependencies failing for MicroOS... and then it boots successfully anyways
Ignition and Combustion need a writable subvolume called /boot/writable to store the information. Is that subvolume properly mounted and / or does it contain a file called firstboot_happened?
3. Once I got what I thought was a "proper" install on a MicroSD card, I tried partcloning the MicroOS btrfs partition to my SSD and booting that. Sure enough it worked, but it kept throwing errors about `mmc1: Timeout waiting for hardware cmd interrupt`. It would seem that somewhere something is hardcoded to expect a MicroSD card, and it is VERY mad that I'm trying to only use my SSD :(
At least on a RPi3 test setup the system is booting perfectly fine without a MicroSD card, so that doesn't seem to be a generic MicroOS problem. As those messages are kernel messages I would assume that hardware support for RPi4 is still a bit lacking...
I got some "screenshots" of the tty with these issues, and I also have the rdsosreport.txt from trying to install directly on the SSD, but I can certainly try the process again and help get any more info to help debug and improve this experience!
Screenshots: https://imgur.com/a/IpMF0lw rdsosreport: https://gist.github.com/steelcowboy/5148d854b3d83d84c734e5a0830e927d
Please let me know if I can do anything to help resolve some of these issues and make the Raspberry Pi experience better! Thanks!
Someone may need to create a KIWI bug report ;-) Thanks for your feedback by the way!
Cheers, Ignaz -- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
-- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
participants (3)
-
Ignaz Forster
-
Jim Heald
-
Paul Gonin