[opensuse] {MCP?} openSuSE 12.3 changes default XFS mount options
Hi Folks, We were smitten by a rather subtle change in how 12.3-64 mounts XFS files systems. The first indication was a failure of acroread to start for a particular user. Other users on the same system weren't affected. It turns out it's an old problem talked about here: http://forums.adobe.com/message/4721987 We then encountered another problem with an old cross-compiler that was caused by the same issue. It seems that when an XFS file system is mounted with the inode64 option, and the file system exceeds 1-TB, inode numbers exceeding 32-bits can be created. Without the inode64 mount option, inodes are clustered to be addressable by 32-bits. 12.2 and earlier mounted XFS without the inode64 option, as shown here: (12.2 mount command) /dev/sda7 on /export/home type xfs (rw,relatime,attr2,delaylog,noquota) While on 12.3: /dev/sda7 on /export/home type xfs (rw,relatime,attr2,inode64,noquota) Our workaround is to stop using XFS for home directories. We'll continue to use XFS for huge RAID arrays handling lots of large files. This isn't a bug in openSuSE, IHMO, just a change that illuminates bugs in other 32-bit packages. My thanks to Dan L. who "found" the solution. (Dan isn't on this list) Regards, Lew -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Lew Wolfgang wrote:
Hi Folks,
We were smitten by a rather subtle change in how 12.3-64 mounts XFS files systems. The first indication was a failure of acroread to start for a particular user. Other users on the same system weren't affected. It turns out it's an old problem talked about here:
http://forums.adobe.com/message/4721987
Our workaround is to stop using XFS for home directories. We'll continue to use XFS for huge RAID arrays handling lots of large files.
You could change the mount options as a workaround, now that you know the source of the problem. Admittedly, having the inode numbers bumped when you are not expecting them might not be great. I would bet that if you used the 32-bit package -- to match the 32-bit applications, they probably wouldn't use the inode64 option on the mounts.. Conversely, if you used 64-bit apps on a 64-bit machine, they likely wouldn't have any problems. I'm guessing that it was probably a reasonable direction to take to have 64-bit systems be allowed to use 64-bit inodes when needed. The penalty for using 32-bit inodes on a larger disk -- I *believe* is that the inodes have to be stored only in the area addressable by 32-bits of sectors -- though it looks like it might only be 31... ah, it's probably an 'int'... (signed) instead of unsigned... maybe for returning error codes. Anyway -- it means when accessing files located above a 1TB mark, the disk will first seek down below 1TB to read the inode -- then seek up to where the data is. If it is following a directory chain, it will need to fetch each subdirectory from the low area (it might be in memory after the first read, but .. worst case would be they were spread out). Anyway -- just a heads-up on the future performance issues (likely not noticeable if things are mostly below 1TB...But if you go to a RAID put disks in a large volume with lvm, it's not hard to get way beyond 1TB pretty easily. Another workaround -- Create smaller partitions -- I.e. instead of one 2TB partition, create 2 x 1TB partitions... still, it's a hassle.. hopefully you'll be able to get 64-bit updates to use on your 64-bit OS... Otherwise -- why not stay @ 32bit and you'd miss this issue entirely. Some different ideas for solutions might give you more options... Hope it helps.. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 05/11/2013 07:22 PM, Linda Walsh wrote:
Lew Wolfgang wrote:
Hi Folks,
We were smitten by a rather subtle change in how 12.3-64 mounts XFS files systems. The first indication was a failure of acroread to start for a particular user. Other users on the same system weren't affected. It turns out it's an old problem talked about here:
http://forums.adobe.com/message/4721987
Our workaround is to stop using XFS for home directories. We'll continue to use XFS for huge RAID arrays handling lots of large files.
Thanks for the comments Linda.
You could change the mount options as a workaround, now that you know the source of the problem. Admittedly, having the inode numbers bumped when you are not expecting them might not be great.
True, but that wouldn't help for the high inodes already created. I think the problem would still exist.
I would bet that if you used the 32-bit package -- to match the 32-bit applications, they probably wouldn't use the inode64 option on the mounts..
Conversely, if you used 64-bit apps on a 64-bit machine, they likely wouldn't have any problems. I'm guessing that it was probably a reasonable direction to take to have 64-bit systems be allowed to use 64-bit inodes when needed.
Agreed.
The penalty for using 32-bit inodes on a larger disk -- I *believe* is that the inodes have to be stored only in the area addressable by 32-bits of sectors -- though it looks like it might only be 31... ah, it's probably an 'int'... (signed) instead of unsigned... maybe for returning error codes.
Anyway -- it means when accessing files located above a 1TB mark, the disk will first seek down below 1TB to read the inode -- then seek up to where the data is. If it is following a directory chain, it will need to fetch each subdirectory from the low area (it might be in memory after the first read, but .. worst case would be they were spread out).
That's about what the xfs documents say.
Anyway -- just a heads-up on the future performance issues (likely not noticeable if things are mostly below 1TB...But if you go to a RAID put disks in a large volume with lvm, it's not hard to get way beyond 1TB pretty easily.
Another workaround -- Create smaller partitions -- I.e. instead of one 2TB partition, create 2 x 1TB partitions... still, it's a hassle.. hopefully you'll be able to get 64-bit updates to use on your 64-bit OS... Otherwise -- why not stay @ 32bit and you'd miss this issue entirely.
We use multiple partitions bigger than 20-TB containing thousands of 4-GB files. This is why we selected xfs some years ago and haven't had any "known" issues until now. We'll just be careful with home partitions, which don't really need to be xfs in the first place. Acroread wouldn't know what to do with a 4-GB binary file anyway!
Some different ideas for solutions might give you more options... Hope it helps..
Thanks again. I brought it up here in case other folks might run into the issue. Of course, I wonder what issues reiserfs and/or ext4 have? :-) Regards, Lew -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Lew Wolfgang wrote:
True, but that wouldn't help for the high inodes already created. I think the problem would still exist.
Only safe way I could think of might be something like copying the entire disk off to another disk or directory that only has 32 bit inodes, then copy it back to the original. Not the easiest -- depends on needs...but...fyi
We use multiple partitions bigger than 20-TB containing thousands of 4-GB files. This is why we selected xfs some years ago and haven't had any "known" issues until now. We'll just be careful with home partitions, which don't really need to be xfs in the first place. Acroread wouldn't know what to do with a 4-GB binary file anyway!
---- Maybe acroread 9.x doesn't... but I know photoshop and Adobe utils deal with
4GB files... have had image files that were that large -- that are also that large when you save them in PDF format (which you can usually do in many Adobe utils)...
Thanks again. I brought it up here in case other folks might run into the issue. Of course, I wonder what issues reiserfs and/or ext4 have? :-)
Might depend on how they count files... Windows has all the inode type info in 1-4 (usually 1-2) MftZone Regions... So there, I'd think you'd need to run close to 1-2billion files before needing > 32bits... but Windows ntfs isn't known for being able to handle multiple simultaneous reads and writes in different places like XFS was designed for -- that's why they allow them to be spread out -- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (2)
-
Lew Wolfgang -
Linda Walsh