[yast-devel] Re: Configuring the storage proposal for Raspberry Pi
On 09/08/2017 01:38 PM, Andreas Färber wrote:
Hi Ancor,
Sorry for the delay. I've skimmed through the whole document, but let's tap into the collective Arm brains in CC for maximum input. :)
CCing to yast-devel as well. There is some important technical considerations about booting ARM here not to be missed.
What I am not seeing for the <partitions/> section is a selection for GPT vs. MBR partitioning type? For the Raspberry Pi and some other boards (e.g., Marvell) we need to force the old MBR format due to bootloader requirements.
It's indeed not there. We have a separate function in the code to calculate the preferred partition type. So far, it simply returns GPT for most devices (except DASD devices). We can make the function smarter to take the current architecture into account (which is most likely the more straightforward approach and what I suggest) or we can make it configurable in the control.xml.
Where do these XML proposals for SLES/openSUSE/SLES4SAP/CaaSP get maintained? Inside YaST GitHub code or in an external product package?
https://github.com/yast/skelcd-control-SLES https://github.com/yast/skelcd-control-openSUSE etc.
The SLES proposal looks fine for ARM (except Raspberry Pi), but openSUSE may need changes to the <volumes/> proposal - in particular for some platforms we need to force a separate /boot partition in U-Boot-readable format (i.e., ext4). I only spot try_separate_home, but not ..._boot? We would need that a) for .dtb files and b) whenever UEFI support is not available and the kernel+initrd needs to be loaded by U-Boot itself rather than by GRUB.
Something that may not be completely clear in the document. The volumes proposed there are the ones desired for general system operation. In addition, YaST will always create the partitions needed for boot. But that's not configured via control.xml because the partitions needed for boot depend on too many factors and are independent of the product. Is not something for the release manager to decide and write in advance, it's simply dictated by the concrete technologies being used in the system. Which partitions do we propose then? Glad you asked: :-) https://github.com/yast/yast-storage-ng/blob/master/doc/boot-requirements.md
How would we reserve space before the volumes? For some openSUSE/SLES platforms user volumes can only start at e.g. ~10 MB, with various bootloader blobs going before that.
Again, that's not something to specify in control.xml but to take into account while calculating the above-mentioned boot requirements. As you can see, there is no ARM section in that document (which is auto-generated from the automated tests and, thus, from the final implementation). That's something we are trying to fix during the current Scrum sprint and I guess Steffen already contacted you about that.
This will usually depend on the installation target. If installing to a new SATA disk then the default proposal is probably okay, whereas if we're trying to install to an SD card or eMMC device we just booted from, then certain partitions or offsets may need to be preserved. One proposal from Alex had been to have some kind of file preinstalled that would give YaST a hint on how to handle the partitions. Won't work if there's no filesystem partitions, of course.
In the end though we'll need to test this on various platforms to see how this works out. Background of some of the above is that we are looking to reduce the amount of openSUSE JeOS images in favor of the official installer, where the user would then need to create compatible partitions - no need to have all that be auto-magical from day one, but suitable UI/XML options would help.
Cheers, Andreas
Thanks for all the information. As said, most of it is more relevant to calculate the boot requirements than to improve the control.xml format. But is still VERY useful. Cheers.
Am 30.08.2017 um 17:36 schrieb Ancor Gonzalez Sosa:
In SLE15 (and thus openSUSE 15) the code calculating the proposed partitioning (and LVM) layout is not only new but completely different.
We took the opportunity to rethink how each product can configure the storage proposal via control.xml. So please tell us if the new suggested format for the <partitioning> section fits your needs regarding the automatic proposal for weird^H ARM devices. ;-) The present requirements and the foreseeable ones.
If you enjoy horror stories, we have a huge document explaining the differences of the old and the new proposals.
https://github.com/yast/yast-storage-ng/blob/master/doc/old_and_new_proposal...
If you prefer fast-paced stories, you can fast-forward to the section "Proposed settings for yast-storage-ng: by example". If you need background for understanding it, check the previous section "How the new proposal distributes the space". And in you want a more deep explanation of the format, please read "Proposed settings for yast-storage-ng: the details".
Of course, I'm not discouraging you from reading the full document from top to bottom (even the opposite, is probably rather interesting) but I know your time is precious.
And talking about time, we want to start implementing this next week, so having your feedback in advance would be very important.
Thanks.
-- Ancor González Sosa YaST Team at SUSE Linux GmbH -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 11.09.17 10:16, Ancor Gonzalez Sosa wrote:
On 09/08/2017 01:38 PM, Andreas Färber wrote:
Hi Ancor,
Sorry for the delay. I've skimmed through the whole document, but let's tap into the collective Arm brains in CC for maximum input. :)
CCing to yast-devel as well. There is some important technical considerations about booting ARM here not to be missed.
What I am not seeing for the <partitions/> section is a selection for GPT vs. MBR partitioning type? For the Raspberry Pi and some other boards (e.g., Marvell) we need to force the old MBR format due to bootloader requirements.
It's indeed not there. We have a separate function in the code to calculate the preferred partition type. So far, it simply returns GPT for most devices (except DASD devices). We can make the function smarter to take the current architecture into account (which is most likely the more straightforward approach and what I suggest) or we can make it configurable in the control.xml.
We really only have 2 cases on AArch64: 1) GPT 2) Leave label intact Case 2 is relevant if the firmware also sits on the same storage device we're installing onto and the code that loads that firmware does not know how to read a GPT but expects partitions. Or if space needs to get preserved, like a 10MB offset for firmware that Andreas mentioned. Then too we shouldn't touch the existing partitions (and partition table).
Where do these XML proposals for SLES/openSUSE/SLES4SAP/CaaSP get maintained? Inside YaST GitHub code or in an external product package?
https://github.com/yast/skelcd-control-SLES https://github.com/yast/skelcd-control-openSUSE etc.
The SLES proposal looks fine for ARM (except Raspberry Pi), but openSUSE may need changes to the <volumes/> proposal - in particular for some platforms we need to force a separate /boot partition in U-Boot-readable format (i.e., ext4). I only spot try_separate_home, but not ..._boot? We would need that a) for .dtb files and b) whenever UEFI support is not available and the kernel+initrd needs to be loaded by U-Boot itself rather than by GRUB.
Something that may not be completely clear in the document. The volumes proposed there are the ones desired for general system operation. In addition, YaST will always create the partitions needed for boot. But that's not configured via control.xml because the partitions needed for boot depend on too many factors and are independent of the product. Is not something for the release manager to decide and write in advance, it's simply dictated by the concrete technologies being used in the system. Which partitions do we propose then? Glad you asked: :-)
https://github.com/yast/yast-storage-ng/blob/master/doc/boot-requirements.md
How would we reserve space before the volumes? For some openSUSE/SLES platforms user volumes can only start at e.g. ~10 MB, with various bootloader blobs going before that.
Again, that's not something to specify in control.xml but to take into account while calculating the above-mentioned boot requirements. As you can see, there is no ARM section in that document (which is auto-generated from the automated tests and, thus, from the final implementation). That's something we are trying to fix during the current Scrum sprint and I guess Steffen already contacted you about that.
This will usually depend on the installation target. If installing to a new SATA disk then the default proposal is probably okay, whereas if we're trying to install to an SD card or eMMC device we just booted from, then certain partitions or offsets may need to be preserved. One proposal from Alex had been to have some kind of file preinstalled that would give YaST a hint on how to handle the partitions. Won't work if there's no filesystem partitions, of course.
In the end though we'll need to test this on various platforms to see how this works out. Background of some of the above is that we are looking to reduce the amount of openSUSE JeOS images in favor of the official installer, where the user would then need to create compatible partitions - no need to have all that be auto-magical from day one, but suitable UI/XML options would help.
Cheers, Andreas
Thanks for all the information. As said, most of it is more relevant to calculate the boot requirements than to improve the control.xml format. But is still VERY useful.
Outside of the boot setup, I don't think we should be any different from x86_64 :). Alex -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Hi Ancor, Seems I misunderstood your email then? I thought you wanted to know what quirks we need facilities for to support installation on "weirder" Arm boards, that are not yet covered in that link. If you're saying that all those special cases are outside the scope of the document, then what did you want me to review in that document? Regards, Andreas Am 11.09.2017 um 10:16 schrieb Ancor Gonzalez Sosa:
On 09/08/2017 01:38 PM, Andreas Färber wrote:
Hi Ancor,
Sorry for the delay. I've skimmed through the whole document, but let's tap into the collective Arm brains in CC for maximum input. :)
CCing to yast-devel as well. There is some important technical considerations about booting ARM here not to be missed.
What I am not seeing for the <partitions/> section is a selection for GPT vs. MBR partitioning type? For the Raspberry Pi and some other boards (e.g., Marvell) we need to force the old MBR format due to bootloader requirements.
It's indeed not there. We have a separate function in the code to calculate the preferred partition type. So far, it simply returns GPT for most devices (except DASD devices). We can make the function smarter to take the current architecture into account (which is most likely the more straightforward approach and what I suggest) or we can make it configurable in the control.xml.
Where do these XML proposals for SLES/openSUSE/SLES4SAP/CaaSP get maintained? Inside YaST GitHub code or in an external product package?
https://github.com/yast/skelcd-control-SLES https://github.com/yast/skelcd-control-openSUSE etc.
The SLES proposal looks fine for ARM (except Raspberry Pi), but openSUSE may need changes to the <volumes/> proposal - in particular for some platforms we need to force a separate /boot partition in U-Boot-readable format (i.e., ext4). I only spot try_separate_home, but not ..._boot? We would need that a) for .dtb files and b) whenever UEFI support is not available and the kernel+initrd needs to be loaded by U-Boot itself rather than by GRUB.
Something that may not be completely clear in the document. The volumes proposed there are the ones desired for general system operation. In addition, YaST will always create the partitions needed for boot. But that's not configured via control.xml because the partitions needed for boot depend on too many factors and are independent of the product. Is not something for the release manager to decide and write in advance, it's simply dictated by the concrete technologies being used in the system. Which partitions do we propose then? Glad you asked: :-)
https://github.com/yast/yast-storage-ng/blob/master/doc/boot-requirements.md
How would we reserve space before the volumes? For some openSUSE/SLES platforms user volumes can only start at e.g. ~10 MB, with various bootloader blobs going before that.
Again, that's not something to specify in control.xml but to take into account while calculating the above-mentioned boot requirements. As you can see, there is no ARM section in that document (which is auto-generated from the automated tests and, thus, from the final implementation). That's something we are trying to fix during the current Scrum sprint and I guess Steffen already contacted you about that.
This will usually depend on the installation target. If installing to a new SATA disk then the default proposal is probably okay, whereas if we're trying to install to an SD card or eMMC device we just booted from, then certain partitions or offsets may need to be preserved. One proposal from Alex had been to have some kind of file preinstalled that would give YaST a hint on how to handle the partitions. Won't work if there's no filesystem partitions, of course.
In the end though we'll need to test this on various platforms to see how this works out. Background of some of the above is that we are looking to reduce the amount of openSUSE JeOS images in favor of the official installer, where the user would then need to create compatible partitions - no need to have all that be auto-magical from day one, but suitable UI/XML options would help.
Cheers, Andreas
Thanks for all the information. As said, most of it is more relevant to calculate the boot requirements than to improve the control.xml format. But is still VERY useful.
Cheers.
Am 30.08.2017 um 17:36 schrieb Ancor Gonzalez Sosa:
In SLE15 (and thus openSUSE 15) the code calculating the proposed partitioning (and LVM) layout is not only new but completely different.
We took the opportunity to rethink how each product can configure the storage proposal via control.xml. So please tell us if the new suggested format for the <partitioning> section fits your needs regarding the automatic proposal for weird^H ARM devices. ;-) The present requirements and the foreseeable ones.
If you enjoy horror stories, we have a huge document explaining the differences of the old and the new proposals.
https://github.com/yast/yast-storage-ng/blob/master/doc/old_and_new_proposal...
If you prefer fast-paced stories, you can fast-forward to the section "Proposed settings for yast-storage-ng: by example". If you need background for understanding it, check the previous section "How the new proposal distributes the space". And in you want a more deep explanation of the format, please read "Proposed settings for yast-storage-ng: the details".
Of course, I'm not discouraging you from reading the full document from top to bottom (even the opposite, is probably rather interesting) but I know your time is precious.
And talking about time, we want to start implementing this next week, so having your feedback in advance would be very important.
Thanks.
-- SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 09/11/2017 12:14 PM, Andreas Färber wrote:
Hi Ancor,
Seems I misunderstood your email then? I thought you wanted to know what quirks we need facilities for to support installation on "weirder" Arm boards, that are not yet covered in that link. If you're saying that all those special cases are outside the scope of the document, then what did you want me to review in that document?
We actually sent a quite similar mail to all release managers. We wanted to see if the format was enough to express their formatting requirements. For example, CaaSP wants a separate /var/lib/docker. SLE4SAP wants a very particular LVM setup, and so on. They cannot achieve that nowadays without dirty hacks or telling the customers to use the expert partitioner. So we were looking for feedback about that. Some stuff like "we always want to propose a separate /var/whatever for performance reasons if hardware X is used". Nevertheless, the information you provided about booting was just the next question we were about to ask you. :-) So you saved us one mail. I understand that the distinction about partitions required for boot (BootRequirements in our jargon) and partitions desired by the release manager for whatever other reasons (described in control.xml) is confusing. Sorry about that. Cheers.
Regards, Andreas
Am 11.09.2017 um 10:16 schrieb Ancor Gonzalez Sosa:
On 09/08/2017 01:38 PM, Andreas Färber wrote:
Hi Ancor,
Sorry for the delay. I've skimmed through the whole document, but let's tap into the collective Arm brains in CC for maximum input. :)
CCing to yast-devel as well. There is some important technical considerations about booting ARM here not to be missed.
What I am not seeing for the <partitions/> section is a selection for GPT vs. MBR partitioning type? For the Raspberry Pi and some other boards (e.g., Marvell) we need to force the old MBR format due to bootloader requirements.
It's indeed not there. We have a separate function in the code to calculate the preferred partition type. So far, it simply returns GPT for most devices (except DASD devices). We can make the function smarter to take the current architecture into account (which is most likely the more straightforward approach and what I suggest) or we can make it configurable in the control.xml.
Where do these XML proposals for SLES/openSUSE/SLES4SAP/CaaSP get maintained? Inside YaST GitHub code or in an external product package?
https://github.com/yast/skelcd-control-SLES https://github.com/yast/skelcd-control-openSUSE etc.
The SLES proposal looks fine for ARM (except Raspberry Pi), but openSUSE may need changes to the <volumes/> proposal - in particular for some platforms we need to force a separate /boot partition in U-Boot-readable format (i.e., ext4). I only spot try_separate_home, but not ..._boot? We would need that a) for .dtb files and b) whenever UEFI support is not available and the kernel+initrd needs to be loaded by U-Boot itself rather than by GRUB.
Something that may not be completely clear in the document. The volumes proposed there are the ones desired for general system operation. In addition, YaST will always create the partitions needed for boot. But that's not configured via control.xml because the partitions needed for boot depend on too many factors and are independent of the product. Is not something for the release manager to decide and write in advance, it's simply dictated by the concrete technologies being used in the system. Which partitions do we propose then? Glad you asked: :-)
https://github.com/yast/yast-storage-ng/blob/master/doc/boot-requirements.md
How would we reserve space before the volumes? For some openSUSE/SLES platforms user volumes can only start at e.g. ~10 MB, with various bootloader blobs going before that.
Again, that's not something to specify in control.xml but to take into account while calculating the above-mentioned boot requirements. As you can see, there is no ARM section in that document (which is auto-generated from the automated tests and, thus, from the final implementation). That's something we are trying to fix during the current Scrum sprint and I guess Steffen already contacted you about that.
This will usually depend on the installation target. If installing to a new SATA disk then the default proposal is probably okay, whereas if we're trying to install to an SD card or eMMC device we just booted from, then certain partitions or offsets may need to be preserved. One proposal from Alex had been to have some kind of file preinstalled that would give YaST a hint on how to handle the partitions. Won't work if there's no filesystem partitions, of course.
In the end though we'll need to test this on various platforms to see how this works out. Background of some of the above is that we are looking to reduce the amount of openSUSE JeOS images in favor of the official installer, where the user would then need to create compatible partitions - no need to have all that be auto-magical from day one, but suitable UI/XML options would help.
Cheers, Andreas
Thanks for all the information. As said, most of it is more relevant to calculate the boot requirements than to improve the control.xml format. But is still VERY useful.
Cheers.
Am 30.08.2017 um 17:36 schrieb Ancor Gonzalez Sosa:
In SLE15 (and thus openSUSE 15) the code calculating the proposed partitioning (and LVM) layout is not only new but completely different.
We took the opportunity to rethink how each product can configure the storage proposal via control.xml. So please tell us if the new suggested format for the <partitioning> section fits your needs regarding the automatic proposal for weird^H ARM devices. ;-) The present requirements and the foreseeable ones.
If you enjoy horror stories, we have a huge document explaining the differences of the old and the new proposals.
https://github.com/yast/yast-storage-ng/blob/master/doc/old_and_new_proposal...
If you prefer fast-paced stories, you can fast-forward to the section "Proposed settings for yast-storage-ng: by example". If you need background for understanding it, check the previous section "How the new proposal distributes the space". And in you want a more deep explanation of the format, please read "Proposed settings for yast-storage-ng: the details".
Of course, I'm not discouraging you from reading the full document from top to bottom (even the opposite, is probably rather interesting) but I know your time is precious.
And talking about time, we want to start implementing this next week, so having your feedback in advance would be very important.
Thanks.
-- Ancor González Sosa YaST Team at SUSE Linux GmbH -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Am 11.09.2017 um 12:27 schrieb Ancor Gonzalez Sosa:
On 09/11/2017 12:14 PM, Andreas Färber wrote:
Hi Ancor,
Seems I misunderstood your email then? I thought you wanted to know what quirks we need facilities for to support installation on "weirder" Arm boards, that are not yet covered in that link. If you're saying that all those special cases are outside the scope of the document, then what did you want me to review in that document?
We actually sent a quite similar mail to all release managers. We wanted to see if the format was enough to express their formatting requirements. For example, CaaSP wants a separate /var/lib/docker. SLE4SAP wants a very particular LVM setup, and so on. They cannot achieve that nowadays without dirty hacks or telling the customers to use the expert partitioner. So we were looking for feedback about that. Some stuff like "we always want to propose a separate /var/whatever for performance reasons if hardware X is used".
Then long answer short, not being a release manager myself: As far as partitions go, with the exception of Raspberry Pi, from my view SLES for ARM should behave just like SLES on x86_64. The partition requirements for Raspberry Pi have been described in fate#323484. If you have any specific questions, please let me know. Cheers, Andreas -- SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
participants (3)
-
Alexander Graf -
Ancor Gonzalez Sosa -
Andreas Färber