On Tue, Mar 17, 2015 at 11:43 AM, Greg Freemyer wrote:

Resend without HTML

On Monday, March 16, 2015, Anton Aylward wrote:

...

On 03/16/2015 09:32 AM, Greg Freemyer wrote:

...

If you are going to use drives with 4KB physical sectors you need to avoid filesystems with 1KB blocks.

I'm sure there are many in the silent majority here who don't have the detailed knowledge, or, like me, have let it pass them by since they deal with other matters (such as users and applications), and wonder about one or another implication in that statement.

We've seen the example with mkfs or extFS for various block sizes, but what about other file systems?

And more to the point for most of us:

How can we tell about these things?

* What block size the disks are are

I suppose all late model disks are 4K :-)

* what size the file system blocks are for the file systems in use - Not just extFS but XFS, reiserFS, BtrFS - if the are not 4K what can we do about it?

* Some might ask about the other file systems in /proc/filesystems. - Does it matter with tmpfs? What about when it 'overflows' to /tmp?

* How can we tell if they are aligned? - if they are not, what can we do about it?

Either it matters, and these are the questions that emerge, or it doesn't.

It matters, but for most people, they do normal things and it works out fine. The developers have addressed their needs.

Right. Where it pops up is when physical sector size isn't passing through various layers. I think the libvirt stuff even has this right, but honestly I haven't checked to see if e.g. a qcow2 file on a 512e drive shows up inside the VM as a 512e drive. If blockdev is thwarted in learning the actual physical sector size, for whatever reason, then the mkfs utilities have no chance with defaults. At least on x86: ext[234] You probably don't have to worry about on anything larger than ~500MiB or whatever it is, you get a 4KiB block size. XFS is always a 4KiB block size, but the journal updates can have a smaller unit matching physical sector size. So if physical sector size is 4096 bytes but wrongly reported by blockdev as 512 bytes, then there could be a performance impact with heavy metadata writes causing a lot of RMW in the drive, just for the journal writes however. I'm not sure how noticeable it'd be. Btrfs is always a 4KiB block size, with a 16KiB node/leaf size by default, which is efficiently used (multiple file extent refs can be stored in one leaf as can inline data for small files.)

The defaults simply break in those conditions, so Felix has to concern himself with issues the rest of us never have come up.

Using VM's backed with qcow2 files would make this a lot easier and more space efficient. Resizes are both possible and easier.

If you look at the above you see he has a lot of partitions that start on a odd sector. That is bad today. Typical modern partitioning tools will align all partitions to 1MB. Felix needs to adopt a 21st century partitioning mechanism.

If the MBR or any EBR sector were to corrupt or read error, all subsequent partition info is lost, unless this information is separately backed up. GPT of course has two copies of everything, and are checksummed. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Greg Freemyer composed on 2015-03-17 13:43 (UTC-0400):

Felix is a different kind of creature :)

He takes a TB or larger drive and puts a bunch of small partitions:

The one you're quoting from is 320G. :-) It came from the just retired HD that lead to my "smartctl: is this HD really OK?" thread.

== From one of Felix's posts == Device Boot Start End Blocks Id System /dev/sda1 63 80324 40131 6 FAT16 /dev/sda2 80325 578339 249007+ 1c Hidden W95 FAT32 (LBA) /dev/sda3 * 578340 1397654 409657+ 83 Linux /dev/sda4 1397655 625137344 311869845 5 Extended /dev/sda5 1397718 4466069 1534176 82 Linux swap / Solaris /dev/sda6 4466133 4482134 8001 1 FAT12 /dev/sda7 4482198 4996214 257008+ 6 FAT16 /dev/sda8 4996278 19743884 7373803+ 83 Linux /dev/sda9 19743948 28756349 4506201 83 Linux /dev/sda10 28756413 40226759 5735173+ 83 Linux /dev/sda11 40226823 78959474 19366326 83 Linux /dev/sda12 78959538 81417419 1228941 83 Linux /dev/sda13 81417483 91249199 4915858+ 83 Linux /dev/sda14 91249263 101080979 4915858+ 83 Linux /dev/sda15 101081043 110912759 4915858+ 83 Linux /dev/sda16 110912823 120744539 4915858+ 83 Linux /dev/sda17 120744603 130576319 4915858+ 83 Linux /dev/sda18 130576383 140408099 4915858+ 83 Linux /dev/sda19 140408163 150239879 4915858+ 83 Linux /dev/sda20 150239943 160071659 4915858+ 83 Linux /dev/sda21 160071723 171542069 5735173+ 83 Linux /dev/sda22 171542133 181373849 4915858+ 83 Linux /dev/sda23 252525798 262357514 4915858+ 83 Linux /dev/sda24 262357578 272189294 4915858+ 83 Linux /dev/sda25 272189358 282021074 4915858+ 83 Linux /dev/sda26 282021138 291852854 4915858+ 83 Linux /dev/sda27 291852918 301684634 4915858+ 83 Linux /dev/sda28 301684698 311516414 4915858+ 83 Linux /dev/sda29 311516478 624093119 156288321 83 Linux /dev/sda30 624093183 625137344 522081 e W95 FAT16 (LBA)

======

The defaults simply break in those conditions, so Felix has to concern himself with issues the rest of us never have come up.

If you look at the above you see he has a lot of partitions that start on a odd sector.

When I'm looking, I'm seeing a considerably less odd view than what that fdisk -l output can show. My partitioner provides a quite different view, readily available via log: OS version : Linux 4.0.0 env TERM=linux root on big41 Disk 1 L-Geo from: LVM info at PSN 0x0000003e ACCEPTED as matching DLAT sector Disk 1 forcing : cylinders from 38913 to 38914 L-Geo Disk 1 Cyl : 38914 H:255 S:63 Bps:512 Size : 0x25431582 = 305250 MiB S-Geo Disk 1 Cyl : 38913 H:255 S:63 Bps:512 Size : 0x2542D6C1 = 305242 MiB BIOS Int13 limit : 1024, I13X support needed beyond : 8032.5 MiB MBR crc 054b4eb9 : 0x0c8ca699 = DFSee generic MBR, English messages, I13X DFSee Linux 12.3 : Executing: fdisk -r- Command timestamp : Monday 2015-03-02 05:00:12 +---+--+-----------------+--+--------+--------+-----------+----------------------------------------+---------+ |ID |Dr|Type, description|ux|Format |Related |VolumeLabel|LVM Vol,Part / GPT name | Size MiB| +---------+--------+------------------+ |01*| |Prim 06 FAT16 | 1|FAT16 |MSWIN4.1|P01WINDOS |wv32 P01 WinDOS wv32 P01 WinDOS | 39.2| |02*| |Hide 16 FAT16 | 2|FAT32 |FRDOS5.1|FREEDOS2012|wv32 P02 FreeDOS wv32 P02 FreeDOS | 243.2| |03>| |Prim 83 LinuxNatv| 3|EXT2 |GRUB |wv32boot03 |wv32 P03 realboot wv32 P03 realboot | 400.1| |04 | |Log 82 SunS/SWAP| 5|SWAP |LinuxV1 |SWAPSPACE2 |wv32 P05 swapper wv32 P05 swapper | 1498.2| |05 | |Log 01 FAT12 | 6|FAT12 |mkdosfs |NO NAME |wv32 P06 dummy wv32 P06 dummy | 7.8| |06 | |Log 06 FAT16 | 7|FAT16 |-DFSee- |P06 SS16A |wv32 P07 FAT E: wv32 P07 FAT E: | 251.0| |07*| |Log 83 LinuxNatv| 8|EXT3 |GRUB |wv32os122 |wv32 P08 suse122 wv32 P08 suse122 | 7201.0| |08 | |Log 83 LinuxNatv| 9|EXT3 |Linux |wv32home09 |wv32 P09 /home wv32 P09 /home | 4400.6| |09*| |Log 83 LinuxNatv|10|EXT3 |GRUB |wv32os121 |wv32 P10 suse121 wv32 P10 suse121 | 5600.8| |10 | |Log 83 LinuxNatv|11|EXT3 |Linux |wv32pub11 |wv32 P11 /pub wv32 P11 /pub | 18912.4| |11 | |Log 83 LinuxNatv|12|EXT3 |Linux |wv32usrlcl |wv32 P12 /usr/local wv32 P12 /usr/local| 1200.1| |12*| |Log 83 LinuxNatv|13|EXT3 |GRUB |wv32f21 |wv32 P13 fedora21 wv32 P13 fedora21 | 4800.6| |13*| |Log 83 LinuxNatv|14|EXT3 |GRUB |wv32f22 |wv32 P14 fedora22 wv32 P14 fedora22 | 4800.6| |14*| |Log 83 LinuxNatv|15|EXT3 |GRUB |wv32mag5 |wv32 P15 mageia5 wv32 P15 mageia5 | 4800.6| |15*| |Log 83 LinuxNatv|16|EXT3 |GRUB |wv32os123 |wv32 P16 suse123 wv32 P16 suse123 | 4800.6| |16*| |Log 83 LinuxNatv|17|EXT3 |GRUB |wv32os131 |wv32 P17 suse131 wv32 P17 suse131 | 4800.6| |17*| |Log 83 LinuxNatv|18|EXT3 |GRUB |wv32mag4 |wv32 P18 mageia4 wv32 P18 mageia4 | 4800.6| |18*| |Log 83 LinuxNatv|19|EXT3 |GRUB |wv32os133 |wv32 P19 suse133 wv32 P19 suse133 | 4800.6| |19*| |Log 83 LinuxNatv|20|EXT3 |GRUB |wv32f23 |wv32 P20 fedora23 wv32 P20 fedora23 | 4800.6| |20*| |Log 83 LinuxNatv|21|EXT3 |GRUB |wv32os114 |wv32 P21 suse114 wv32 P21 suse114 | 5600.8| |21*| |Log 83 LinuxNatv|22|EXT3 |GRUB |wv32os132 |wv32 P22 suse132 wv32 P22 suse132 | 4800.6| |22 | |FreeSpace Logical| |-- -- --|-- -- --|- - - - - -| | 34742.1| |22*| |Log 83 LinuxNatv|23|EXT3 |GRUB |wv32buntu |wv32 buntu wv32 buntu | 4800.6| |23 | |Log 83 LinuxNatv|24|unknown |Linux | |test destination test destinaion | 4800.6| |24 | |Log 83 LinuxNatv|25|unknown |Linux | |SUSE tester SUSE tester | 4800.6| |25 | |Log 83 LinuxNatv|26|unknown |Linux | |Fedo tester Fedo tester | 4800.6| |26 | |Log 83 LinuxNatv|27|unknown |Linux | |test destination test destination | 4800.6| |27 | |Log 83 LinuxNatv|28|unknown |Linux | |was s122 XFCE was s122 XFCE | 4800.6| |28 | |Log 83 LinuxNatv|29|EXT3 |Linux |wv32isos |wv32 isospace wv32 isospace | 152625.3| |29 | |Log 0c FAT32-Ext|30|FAT16 |mkfs.fat|NO NAME |wv32 FAT downloads wv32 FAT downloads | 509.8| |30 | |Partial Cylinder | |-- -- --|-- -- --|- - - - - -| | 2.5| DFSee Linux 12.3 : Executing: part -n Command timestamp : Monday 2015-03-02 05:00:51 +---+--+-----------------+--+--------+--------+-------------+--------+---------+ |ID |Dr|Type, description|ux| Begin |End sect| Cylinders | Sector| Size MiB| +---------+--------+------------<wd3200avvs>--------+ |01 | |MBR +Track-0 Area| | 0| 3e| 0 - 0| 3f| 0.031| |01*| |Prim 06 FAT16 | 1| 3f| 139c4| 0 - 4| 13986| 39.2| |02*| |Hide 16 FAT16 | 2| 139c5| 8d323| 5 - 35| 7995f| 243.2| |03>| |Prim 83 LinuxNatv| 3| 8d324| 155396| 36 - 86| c8073| 400.1| |04 | |Log 82 SunS/SWAP| 5| 1553d6| 442595| 87 - 277| 2ed1c0| 1498.2| |05 | |Log 01 FAT12 | 6| 4425d5| 446456| 278 - 278| 3e82| 7.8| |06 | |Log 06 FAT16 | 7| 446496| 4c3c76| 279 - 310| 7d7e1| 251.0| |07*| |Log 83 LinuxNatv| 8| 4c3cb6| 12d448c| 311 - 1228| e107d7| 7201.0| |08 | |Log 83 LinuxNatv| 9| 12d44cc| 1b6c97d| 1229 - 1789| 8984b2| 4400.6| |09*| |Log 83 LinuxNatv|10| 1b6c9bd| 265cfc7| 1790 - 2503| af060b| 5600.8| |10 | |Log 83 LinuxNatv|11| 265d007| 4b4d372| 2504 - 4914| 24f036c| 18912.4| |11 | |Log 83 LinuxNatv|12| 4b4d3b2| 4da54cb| 4915 - 5067| 25811a| 1200.1| |12*| |Log 83 LinuxNatv|13| 4da550b| 5705a2f| 5068 - 5679| 960525| 4800.6| |13*| |Log 83 LinuxNatv|14| 5705a6f| 6065f93| 5680 - 6291| 960525| 4800.6| |14*| |Log 83 LinuxNatv|15| 6065fd3| 69c64f7| 6292 - 6903| 960525| 4800.6| |15*| |Log 83 LinuxNatv|16| 69c6537| 7326a5b| 6904 - 7515| 960525| 4800.6| |16*| |Log 83 LinuxNatv|17| 7326a9b| 7c86fbf| 7516 - 8127| 960525| 4800.6| |17*| |Log 83 LinuxNatv|18| 7c86fff| 85e7523| 8128 - 8739| 960525| 4800.6| |18*| |Log 83 LinuxNatv|19| 85e7563| 8f47a87| 8740 - 9351| 960525| 4800.6| |19*| |Log 83 LinuxNatv|20| 8f47ac7| 98a7feb| 9352 - 9963| 960525| 4800.6| |20*| |Log 83 LinuxNatv|21| 98a802b| a398635| 9964 - 10677| af060b| 5600.8| |21*| |Log 83 LinuxNatv|22| a398675| acf8b99|10678 - 11289| 960525| 4800.6| |22 | |FreeSpace Logical| | acf8b9a| f0d3ca6|11290 - 15718| 43db10d| 34742.1| |22*| |Log 83 LinuxNatv|23| f0d3ce6| fa3420a|15719 - 16330| 960525| 4800.6| |23 | |Log 83 LinuxNatv|24| fa3424a|1039476e|16331 - 16942| 960525| 4800.6| |24 | |Log 83 LinuxNatv|25|103947ae|10cf4cd2|16943 - 17554| 960525| 4800.6| |25 | |Log 83 LinuxNatv|26|10cf4d12|11655236|17555 - 18166| 960525| 4800.6| |26 | |Log 83 LinuxNatv|27|11655276|11fb579a|18167 - 18778| 960525| 4800.6| |27 | |Log 83 LinuxNatv|28|11fb57da|12915cfe|18779 - 19390| 960525| 4800.6| |28 | |Log 83 LinuxNatv|29|12915d3e|2532e7bf|19391 - 38847|12a18a82| 152625.3| |29 | |Log 0c FAT32-Ext|30|2532e7ff|2542d6c0|38848 - 38912| feec2| 509.8| |30 | |Partial Cylinder | |2542d6c1|2542eaaf|38913 - 38913| 13ef| 2.5|

That is bad today. Typical modern partitioning tools will align all partitions to 1MB. Felix needs to adopt a 21st century partitioning mechanism.

Already done, intentionally to a limited extent, not fixin what ain't broke.

Then you can see that many of his partitions are only 2GB of so.

The preponderance of those are 4800.6MB. So far, only my 512n HDs retain adherence to legacy partitioning. Here's an equivalent look at a 2TB 512e on a machine used more for ordinary things, plus Windows, and less for testing. ;-) : OS version : Linux 3.12.36 env TERM=linux root on vizio Disk 1 Warning : Geo size 0x2c9d74c1 smaller than size in sectors 0xe8e088b0, Cylinder count recalculated and set to 243201. Disk 1 L-Geo from: LVM DLAT at PSN 0x0000001f IGNORED, size < end of last partition Disk 1 L-Geo from: MBR table, likely 1-MiB cylinders (like SSD or 4Kb sectors) Disk 1 forcing : cylinders from 243201 to 1907730 Disk 1 forcing : heads from 255 to 64 Disk 1 forcing : sectors from 63 to 32 L-Geo Disk 1 Cyl :1907730 H: 64 S:32 Bps:512 Size : 0xE8E09000 = 1907730 MiB S-Geo Disk 1 Cyl :243201 H:255 S:63 Bps:512 Size : 0xE8E074C1 = 1907726 MiB BIOS Int13 limit : 1024, I13X support needed beyond : 1024.0 MiB MBR crc 3669dc25 : 0xb16c291f = Windows 7, generic English DFSee Linux 12.3 : Executing: fdisk -r- Command timestamp : Thursday 2015-02-05 07:19:12 +---+--+-----------------+--+--------+--------+-----------+----------------------------------------+---------+ |ID |Dr|Type, description|ux|Format |Related |VolumeLabel|LVM Vol,Part / GPT name | Size MiB| +---------+--------+----------<vizio760st2000dm001 >--------+ |01 | |FreeSpace Wasted | |-- -- --|-- -- --|- - - - - -| | 1.0| |01 | |Prim 06 FAT16 | 1|FAT16 |MSWIN4.1|FATBOOT01P |P01 winboot winboot P01 | 400.0| |02 | |Prim 82 SunS/SWAP| 2|SWAP |LinuxV1 |SWAPSPACE2 |P02 dummy dummy P02 | 2000.0| |03>| |Prim 83 LinuxNatv| 3|EXT2 |GRUB |03linboot |P03 linboot linboot P03 | 400.0| |04 | |Log 82 SunS/SWAP| 5|SWAP |LinuxV1 |SWAPSPACE2 |P05 swapper swapper P05 | 4146.0| |05 | |Log 07 Inst-FSys| 6|NTFS |Win NT |WinSYS |P06 winsys winsys P06 | 48000.0| |06 | |Lhid 16 FAT16 | 7|FAT16 |mkfs.fat|FATDATA |P07 FAT DATA E: FAT DATA E: P07 | 252.0| |07*| |Log 83 LinuxNatv| 8|EXT3 |GRUB |08os13164 |P08 os13164 os13164 P08 | 10000.0| |08*| |Log 83 LinuxNatv| 9|EXT3 |Linux |09root2 |P09 root2 root2 P09 | 10000.0| |09*| |Log 83 LinuxNatv|10|EXT3 |Linux |10root3 |P10 root3 root3 P10 | 10000.0| |10*| |Log 83 LinuxNatv|11|EXT3 |Linux |11root4 |P11 root4 root4 P11 | 10000.0| |11 | |Log 83 LinuxNatv|12|EXT3 |Linux |12tmp |P12 /tmp /tmp P12 | 5000.0| |12 | |Log 83 LinuxNatv|13|EXT3 |Linux |13srv |P13 /srv /srv P13 | 4000.0| |13 | |Log 83 LinuxNatv|14|EXT3 |Linux |14usrlcl |P14 /usr/local /usr/local P14 | 2000.0| |14 | |Log 07 Inst-FSys|15|NTFS |Win NT |WinDATA |P15 windata windata P15 | 24000.0| |15 | |Log 83 LinuxNatv|16|EXT3 |Linux |16home |P16 /home /home P16 | 76000.0| |16 | |Log 83 LinuxNatv|17|EXT3 |Linux |17pub |P17 /pub /pub P17 | 235000.0| |17 | |FreeSpace Logical| |-- -- --|-- -- --|- - - - - -| | 468090.0| |17 | |Log 83 LinuxNatv|18|unknown |Linux | |v760isos v760isos | 950000.0| |18 | |Log 83 LinuxNatv|19|unknown |Linux | |winbootbackup winbootbackup | 440.0| |19 | |Log d7 Unknown |20|unknown |NTFS |. |winsysbackup winsysbackup | 48000.0| |20 | |Partial Cylinder | |-- -- --|-- -- --|- - - - - -| | 0.086| DFSee Linux 12.3 : Executing: part -n Command timestamp : Thursday 2015-02-05 07:19:22 +---+--+-----------------+--+--------+--------+-------------+--------+---------+ |ID |Dr|Type, description|ux| Begin |End sect| Cylinders | Sector| Size MiB| +---------+--------+------------<vizio760st>--------+ |01 | |FreeSpace Wasted | | 0| 7ff| 0 - 0| 800| 1.0| |01 | |Prim 06 FAT16 | 1| 800| c87ff| 1 - 400| c8000| 400.0| |02 | |Prim 82 SunS/SWAP| 2| c8800| 4b07ff| 401 - 2400| 3e8000| 2000.0| |03>| |Prim 83 LinuxNatv| 3| 4b0800| 5787ff| 2401 - 2800| c8000| 400.0| |04 | |Log 82 SunS/SWAP| 5| 578820| d917ff| 2801 - 6946| 818fe0| 4146.0| |05 | |Log 07 Inst-FSys| 6| d91820| 6b517ff| 6947 - 54946| 5dbffe0| 48000.0| |06 | |Lhid 16 FAT16 | 7| 6b51820| 6bcf7ff|54947 - 55198| 7dfe0| 252.0| |07*| |Log 83 LinuxNatv| 8| 6bcf820| 7f577ff|55199 - 65198| 1387fe0| 10000.0| |08*| |Log 83 LinuxNatv| 9| 7f57820| 92df7ff|65199 - 75198| 1387fe0| 10000.0| |09*| |Log 83 LinuxNatv|10| 92df820| a6677ff|75199 - 85198| 1387fe0| 10000.0| |10*| |Log 83 LinuxNatv|11| a667820| b9ef7ff|85199 - 95198| 1387fe0| 10000.0| |11 | |Log 83 LinuxNatv|12| b9ef820| c3b37ff|95199 -100198| 9c3fe0| 5000.0| |12 | |Log 83 LinuxNatv|13| c3b3820| cb837ff|100199-104198| 7cffe0| 4000.0| |13 | |Log 83 LinuxNatv|14| cb83820| cf6b7ff|104199-106198| 3e7fe0| 2000.0| |14 | |Log 07 Inst-FSys|15| cf6b820| fe4b7ff|106199-130198| 2edffe0| 24000.0| |15 | |Log 83 LinuxNatv|16| fe4b820|192bb7ff|130199-206198| 946ffe0| 76000.0| |16 | |Log 83 LinuxNatv|17|192bb820|35db77ff|206199-441198|1cafbfe0| 235000.0| |17 | |FreeSpace Logical| |35db7800|6eff47ff|441199-909288|3923d000| 468090.0| |17 | |Log 83 LinuxNatv|18|6eff4820|e2f6c7ff|909289-1859288|73f77fe0| 950000.0| |18 | |Log 83 LinuxNatv|19|e2f6c820|e30487ff|1859289-1859728| dbfe0| 440.0| |19 | |Log d7 Unknown |20|e3048820|e8e087ff|1859729-1907728| 5dbffe0| 48000.0| |20 | |Partial Cylinder | |e8e08800|e8e088af|1907729-1907729| b0| 0.086| Note that Windows isn't living on any primary, only starting from one, and its partitioning isn't yet fully evolved. :-) -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org