[yast-devel] Alignment of the last partition
This question has been raised in IRC many times during storage-ng development, with no satisfactory answer so far. But now we need to take a decision at last. Storage-ng always tries to align the start and end of all new and resized partitions for optimal performance. In addition to that, there is also a given alignment that has to be respected for the whole thing to work (leaving performance aside). See [1] for more info. For example, in the case of a typical GPT partition table in a typical disk, the REQUIRED grain would be 1B (so we can create a partition of any size), but the OPTIMAL grain is 1MiB (so we try to only distribute the space in virtual "disk slices" of 1MiB). In DASD, the typical REQUIRED grain is 48KiB, so no way we can create a partition whose size is not divisible by that number. In addition, the OPTIMAL grain is 3MiB. The point is that I'm not so sure whether aligning the end makes that much sense from the performance POV if a given partition is at the very end of the partition table. Two cases: ========== 1) GPT partition tables. In GPT the last 16.5KiB are always reserved (a backup of the partition table). Let's say the user decides to create a partition using all the available space at the end (or simply a big partition using the whole disk). A very common scenario. What should we do? Persist in our attempt to have everything aligned and leave a gap of 1007.5Kib at the end or really use the space up until the end? If GPT is designed to always have those 16.5KiB stripped from the useful space, I would assume that having a last "slice" of 1007.5Kib instead of 1MiB shouldn't imply a performance penalty. Otherwise, the GPT design is rather... well... not-smart. 2) DASD partition tables. A similar thing happens with DASD. In many situations, the remaining space at the end of the disk is not divisible by 3MiB. Do we want to leave the remainder space unused? If we decide to use also those, let's say, 2MiB at the end of the disk, would we be introducing a performance problem or that rule does not apply to partitions at the end of the disk? Even further, it is possible than the remainder is not divisible by the REQUIRED grain of 48KiB? I wouldn't expect that to happen, but these eyes have seen so many things... ;-) In short: ========= For a partition trying to use the space at the end of the disk: 1) Should we give up on optimal alignment and use REQUIRED alignment for that case? If we do so, would it have any performance impact? 2) Should we, instead, stick to the OPTIMAL alignment also for that final partition and leave gaps at the end of all GPT and many (most) DASDs? 3) Should we even go further than (1) and completely ignore alignment (even the REQUIRED one) assuming that partition tables can always be used up to the end? (Who would build a disk with unusable space after all ;-) ). Thanks guys for the replies. https://github.com/openSUSE/libstorage-ng/blob/master/doc/alignment.md -- Ancor González Sosa YaST Team at SUSE Linux GmbH -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Ancor Gonzalez Sosa composed on 2018-01-15 17:37 (UTC+0100): ...
For example, in the case of a typical GPT partition table in a typical disk, the REQUIRED grain would be 1B (so we can create a partition of any size), but the OPTIMAL grain is 1MiB (so we try to only distribute the space in virtual "disk slices" of 1MiB)....
1MiB at the front is a convention. How did optimal *grain* get to be so big? 20h/32d 512 byte sectors should be sufficient for optimal if block size is a fairly generous 16k, no, and 8 512 byte sectors for those content with 4096 byte block sizes? Largest block size used here is 4096 bytes, with 1024 common. -- "Wisdom is supreme; therefore get wisdom. Whatever else you get, get wisdom." Proverbs 4:7 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Mon, Jan 15, 2018 at 12:35:17PM -0500, Felix Miata wrote:
Ancor Gonzalez Sosa composed on 2018-01-15 17:37 (UTC+0100): ...
For example, in the case of a typical GPT partition table in a typical disk, the REQUIRED grain would be 1B (so we can create a partition of any size), but the OPTIMAL grain is 1MiB (so we try to only distribute the space in virtual "disk slices" of 1MiB)....
1MiB at the front is a convention. How did optimal *grain* get to be so big?
We do the same as parted does here. The target are SSD where the internals are unfortunately hidden (see https://en.wikipedia.org/wiki/Solid-state_drive#Partition_alignment, also linked from libstorage-ng documentation): "A typical practice for personal computers is to have each partition aligned to start at a 1 MB (= 1,048,576 bytes) mark, which covers all common SSD page and block size scenarios, as it is divisible by all commonly used sizes - 1 MB, 512 KB, 128 KB, 4 KB, and 512 bytes. Modern operating system installation software and disk tools handle this automatically." ciao Arvin -- Arvin Schnell, <aschnell@suse.com> Senior Software Engineer, Research & Development SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Hello, On Jan 15 18:42 Arvin Schnell wrote:
On Mon, Jan 15, 2018 at 12:35:17PM -0500, Felix Miata wrote:
Ancor Gonzalez Sosa composed on 2018-01-15 17:37 (UTC+0100): ...
For example, in the case of a typical GPT partition table in a typical disk, the REQUIRED grain would be 1B (so we can create a partition of any size), but the OPTIMAL grain is 1MiB (so we try to only distribute the space in virtual "disk slices" of 1MiB)....
1MiB at the front is a convention. How did optimal *grain* get to be so big?
We do the same as parted does here. The target are SSD where the internals are unfortunately hidden (see https://en.wikipedia.org/wiki/Solid-state_drive#Partition_alignment, also linked from libstorage-ng documentation):
"A typical practice for personal computers is to have each partition aligned to start at a 1 MB (= 1,048,576 bytes) mark, which covers all common SSD page and block size scenarios, as it is divisible by all commonly used sizes - 1 MB, 512 KB, 128 KB, 4 KB, and 512 bytes. Modern operating system installation software and disk tools handle this automatically."
As far as I know 1MiB would be too small for best performance on modern flash storage, cf. https://github.com/rear/rear/issues/1201#issuecomment-281274259 and subsequent comments and links therein. In particular see the parted manual https://www.gnu.org/software/parted/manual/parted.html that reads (excerpts): ----------------------------------------------------------- how to partition a low-end flash device ("low-end", as of 2011/2012). For such devices, you should use 4MiB-aligned partitions (2) ... (2) Cheap flash drives will be with us for a long time to come, and, for them, 1MiB alignment is not enough. Use at least 4MiB-aligned partitions. For details, see Arnd Bergman's article, http://lwn.net/Articles/428584/ and its many comments. ----------------------------------------------------------- To make DASD same (DASD has a factor 3 in 48 KiB = 3 * 4 * 4 KiB) a least common multiple could be 3 * 4 MiB = 12 MiB but flash storage may have a physical block size of 8 MiB which would make the least common multiple 3 * 8 MiB = 24 MiB that could be used as one same default grain value for best performance on traditional rotating disks, on flash storage (from cheap USB sticks up to high-end SSDs), and even on DASD. Out ouf curiosity: Does someone know the reason behind why IBM introduced the unexpected factor 3 via 48 KiB required grain on DASD? Not that I am actually surprised what IBM introduces, cf. https://en.wikipedia.org/wiki/31-bit Kind Regards Johannes Meixner -- SUSE LINUX GmbH - GF: Felix Imendoerffer, Jane Smithard, Graham Norton - HRB 21284 (AG Nuernberg) -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Johannes Meixner wrote:
Out ouf curiosity:
Does someone know the reason behind why IBM introduced the unexpected factor 3 via 48 KiB required grain on DASD?
Not that I am actually surprised what IBM introduces, cf. https://en.wikipedia.org/wiki/31-bit
It was important to distinguish between 24bit and 31bit :-) I grew up with working in 24 and 31bit mode. 48Kb is the size of an IBM 3390 track. Dunno about the factor 3. -- Per Jessen, Zürich (3.5°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Mon, Jan 15, 2018 at 05:37:44PM +0100, Ancor Gonzalez Sosa wrote:
Storage-ng always tries to align the start and end of all new and resized partitions for optimal performance.
AFAIS the end is only aligned to not produce gaps between partitions.
In DASD, the typical REQUIRED grain is 48KiB, so no way we can create a partition whose size is not divisible by that number. In addition, the OPTIMAL grain is 3MiB.
The optimal grain is 3 MiB since it divides by 1 MiB and 48 KiB.
The point is that I'm not so sure whether aligning the end makes that much sense from the performance POV if a given partition is at the very end of the partition table.
I don't think it is important for performance. In general I would not align the end of the last partition and thus avoid unused space at the end. I personally do not mind that little wasted space but other people might be picky about it (e.g. if other partitioning tools report a tiny piece of unused space). ciao Arvin -- Arvin Schnell, <aschnell@suse.com> Senior Software Engineer, Research & Development SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 01/15/2018 06:56 PM, Arvin Schnell wrote:
On Mon, Jan 15, 2018 at 05:37:44PM +0100, Ancor Gonzalez Sosa wrote:
Storage-ng always tries to align the start and end of all new and resized partitions for optimal performance.
AFAIS the end is only aligned to not produce gaps between partitions.
In DASD, the typical REQUIRED grain is 48KiB, so no way we can create a partition whose size is not divisible by that number. In addition, the OPTIMAL grain is 3MiB.
The optimal grain is 3 MiB since it divides by 1 MiB and 48 KiB.
The point is that I'm not so sure whether aligning the end makes that much sense from the performance POV if a given partition is at the very end of the partition table.
I don't think it is important for performance.
In general I would not align the end of the last partition and thus avoid unused space at the end. I personally do not mind that little wasted space but other people might be picky about it (e.g. if other partitioning tools report a tiny piece of unused space).
Ok. Thanks. So I will describe now what I'm going to implement regarding partition creation (and resizing) in the new expert partitioner. If someone disagrees, speak now or forever hold your peace[1]. The user can select (a) Maximum size (b) Custom size (c) Custom region with start block and end block In (a) and (b) ============== The start will be always aligned to OPTIMAL, no matter what. To align the end, two situations are contemplated. If the whole available space is used (always for option (a) and also option (b) if max size is entered), the end will not be aligned, so it will use everything until the last available byte. If the desired size is smaller than the whole free chunk, the end will also be aligned to OPTIMAL, so the remaining free space starts in an optimal point. In (c) ====== Both the start and the end will be always aligned to REQUIRED. In normal disks that means nothing will change, since the REQUIRED grain is 1Byte. In DASD that can mean moving both the start and the end up to 44KiB each. As a result =========== If I'm not mistaken that ensures that (a) and (b): - always produce optimal alignment - never leave a gap if it doesn't make sense On the other hand, option (c) : - always produces required alignment - provides a way to the user to "escape" from optimal alignment Cheers. [1] With "forever" being "until next release" ;-) -- Ancor González Sosa YaST Team at SUSE Linux GmbH -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
participants (5)
-
Ancor Gonzalez Sosa -
Arvin Schnell -
Felix Miata -
Johannes Meixner -
Per Jessen