A seemingly basic xfs_grow question
All, I'm creating a 30TB xfs filesystem that I want to ensure can grow to 120TB via xfs_grow. This is a pure SSD environment (Using 30TB Micron 6500 ION drives -- https://www.mouser.com/ProductDetail/Micron/MTFDKCC30T7TGR-1BK1DFCYY?qs=ulEa...) so I don't need to worry about RAID layout etc. I'm am worried about setting the AGcount and AGsize settings in mkfs.xfs By default, I get 29 for AGcount and the corresponding AGsize of ~1TB. Supposedly neither are adjustable, but that can't be true during xfs_grow. My (uninformed) guess is that if I quadruple the size of my xfs filesystem (from 30TB to 120TB), then the number of aggregation groups will also quadruple. Do I have that right? Or do I need to be careful in laying out my original aggregation group layout. Thanks, Greg -- Greg Freemyer Advances are made by answering questions. Discoveries are made by questioning answers. — Bernard Haisch
On Wednesday 2024-02-21 18:22, Greg Freemyer wrote:
[mkfs.xfs] By default, I get 29 for AGcount and the corresponding AGsize of ~1TB. Supposedly neither are adjustable, but that can't be true during xfs_grow.
My (uninformed) guess is that if I quadruple the size of my xfs filesystem (from 30TB to 120TB), then the number of aggregation groups will also quadruple.
It's adjustable upwards, but not quite downwards. (`xfs_growfs -D ...` apparently only works to the point where the highest-numbered AG has been shrunk to 64 blocks.)
On Wed, Feb 21, 2024 at 12:36 PM Jan Engelhardt <jengelh@inai.de> wrote:
On Wednesday 2024-02-21 18:22, Greg Freemyer wrote:
[mkfs.xfs] By default, I get 29 for AGcount and the corresponding AGsize of ~1TB. Supposedly neither are adjustable, but that can't be true during xfs_grow.
My (uninformed) guess is that if I quadruple the size of my xfs filesystem (from 30TB to 120TB), then the number of aggregation groups will also quadruple.
It's adjustable upwards, but not quite downwards. (`xfs_growfs -D ...` apparently only works to the point where the highest-numbered AG has been shrunk to 64 blocks.)
Strange I'm going to setup 500 AGs. Each AG with size 15 million 4k blocks. (~60 GB per AG). When I use xfs_grow later on, that should allow room for the AGcount to quadruple in count, or AGsize to quadruple in size. One of the 2 has to happen! This is too much data to have to rebuild the FS a month from now! Thanks, Greg
On Wednesday 2024-02-21 19:04, Greg Freemyer wrote:
[mkfs.xfs] By default, I get 29 for AGcount and the corresponding AGsize of ~1TB. Supposedly neither are adjustable, but that can't be true during xfs_grow.
My (uninformed) guess is that if I quadruple the size of my xfs filesystem (from 30TB to 120TB), then the number of aggregation groups will also quadruple.
It's adjustable upwards, but not quite downwards. (`xfs_growfs -D ...` apparently only works to the point where the highest-numbered AG has been shrunk to 64 blocks.)
When I use xfs_grow later on, that should allow room for the AGcount to quadruple in count, or AGsize to quadruple in size. One of the 2 has to happen!
Like I said, an increase is not the problem.
On Wed, Feb 21, 2024 at 1:13 PM Jan Engelhardt <jengelh@inai.de> wrote:
On Wednesday 2024-02-21 19:04, Greg Freemyer wrote:
[mkfs.xfs] By default, I get 29 for AGcount and the corresponding AGsize of ~1TB. Supposedly neither are adjustable, but that can't be true during xfs_grow.
My (uninformed) guess is that if I quadruple the size of my xfs filesystem (from 30TB to 120TB), then the number of aggregation groups will also quadruple.
It's adjustable upwards, but not quite downwards. (`xfs_growfs -D ...` apparently only works to the point where the highest-numbered AG has been shrunk to 64 blocks.)
When I use xfs_grow later on, that should allow room for the AGcount to quadruple in count, or AGsize to quadruple in size. One of the 2 has to happen!
Like I said, an increase is not the problem.
As far as I know, xfs_grow doesn't support shrinking the filesystem at all. So increasing the size is the only option.
On Wednesday 2024-02-21 19:15, Greg Freemyer wrote:
It's adjustable upwards, but not quite downwards. (`xfs_growfs -D ...` apparently only works to the point where the highest-numbered AG has been shrunk to 64 blocks.)
When I use xfs_grow later on, that should allow room for the AGcount to quadruple in count, or AGsize to quadruple in size. One of the 2 has to happen!
Like I said, an increase is not the problem.
As far as I know, xfs_grow doesn't support shrinking the filesystem at all. So increasing the size is the only option.
Like I said: only works to the point where the highest-numbered AG has been shrunk to a few blocks. # xfs_growfs /mnt meta-data=/dev/loop7 isize=512 agcount=4, agsize=16777216 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=1 = reflink=1 bigtime=1 inobtcount=1 nrext64=1 data = bsize=4096 blocks=50335744, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=32768, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 data blocks changed from 50335744 to 67108864 Plus it cannot be shrunk all at once, but if you loop it: # xfs_growfs -D 50335744 /mnt [EXPERIMENTAL] try to shrink unused space 50335744, old size is 67108864 xfs_growfs: XFS_IOC_FSGROWFSDATA xfsctl failed: No space left on device # for ((i=67108864;i>=0;i-=4096)); do xfs_growfs -D $i /mnt || break; done it will run for a while until 50331648 or something. Note how 50331648/16777216 = 3.00, so effectively, the last AG (4) has now size 0 or something and you can't get rid of it, else the shrinker would have continued to below 50331648 in the example.
W dniu 21.02.2024 o 19:30, Jan Engelhardt pisze:
On Wednesday 2024-02-21 19:15, Greg Freemyer wrote:
It's adjustable upwards, but not quite downwards. (`xfs_growfs -D ...` apparently only works to the point where the highest-numbered AG has been shrunk to 64 blocks.)
When I use xfs_grow later on, that should allow room for the AGcount to quadruple in count, or AGsize to quadruple in size. One of the 2 has to happen!
Like I said, an increase is not the problem.
As far as I know, xfs_grow doesn't support shrinking the filesystem at all. So increasing the size is the only option.
Like I said: only works to the point where the highest-numbered AG has been shrunk to a few blocks.
# xfs_growfs /mnt meta-data=/dev/loop7 isize=512 agcount=4, agsize=16777216 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=1 = reflink=1 bigtime=1 inobtcount=1 nrext64=1 data = bsize=4096 blocks=50335744, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=32768, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 data blocks changed from 50335744 to 67108864
Plus it cannot be shrunk all at once, but if you loop it:
# xfs_growfs -D 50335744 /mnt [EXPERIMENTAL] try to shrink unused space 50335744, old size is 67108864 xfs_growfs: XFS_IOC_FSGROWFSDATA xfsctl failed: No space left on device
# for ((i=67108864;i>=0;i-=4096)); do xfs_growfs -D $i /mnt || break; done
it will run for a while until 50331648 or something. Note how 50331648/16777216 = 3.00, so effectively, the last AG (4) has now size 0 or something and you can't get rid of it, else the shrinker would have continued to below 50331648 in the example.
Whoa! Shrinking XFS! That's something new :) Just when I needed. Thanks for the info. I wrote a blog post about it with some personal experience gained in the process: https://etam-software.eu/blog/2024-02-22-shrinking-xfs-partition-on-lvm.html
participants (3)
-
Adam Mizerski -
Greg Freemyer -
Jan Engelhardt