[opensuse] occupied size, ext4 versus ntfs
Hello, I just hit a curious thing. I use two identical (bough together) external usb 5Tb drives (as archives) Fort whatever reason one was formatted EXR4 and the other kept as NTFS I just made a rsync --delete from one to the other, verified they keep the exact same number of files. The ext4 one have 915 Gb free, the other 687 Gb wtf!! is it possible to move such disk from ntfs to ext4 without reformatting all (and copying 4.5Tb data)? thanks jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
I just made a rsync --delete from one to the other, verified they keep the exact same number of files.
The ext4 one have 915 Gb free, the other 687 Gb Maybe there are files in the trash-bin. Use ncdu (or similar tool which shows disk-utilization) to check what eats up your diskspace. Keep a special eye on "System Volume Information" and "$RECYCLE.BIN" on
Hi On 2016-12-01 16:50, jdd wrote: the nthfs disk and on ".Trash" and "lost+found" on the ext4 disk. Best regards Thomas -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 01/12/2016 à 17:18, Thomas Wagner a écrit :
Hi On 2016-12-01 16:50, jdd wrote:
I just made a rsync --delete from one to the other, verified they keep the exact same number of files.
The ext4 one have 915 Gb free, the other 687 Gb Maybe there are files in the trash-bin.
I removed it and really got some memory back on the ntfs disk
Use ncdu (or similar tool which shows disk-utilization) to check what eats up your diskspace.
I have the exact same number of files on the two disks
Keep a special eye on "System Volume Information" and "$RECYCLE.BIN" on the nthfs disk
this disk never have seen windows so no extra file like this and on ".Trash" and "lost+found" on the ext4 disk.
lost+found 16k on the two disks trash removed ext4: 915Gb free, ntfs 703Gb free may be some difference between hidden files by rsync? thanks jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-12-01 19:03, jdd wrote:
Le 01/12/2016 à 17:18, Thomas Wagner a écrit :
Hi On 2016-12-01 16:50, jdd wrote:
I just made a rsync --delete from one to the other, verified they keep the exact same number of files.
The ext4 one have 915 Gb free, the other 687 Gb Maybe there are files in the trash-bin.
I removed it and really got some memory back on the ntfs disk
Use ncdu (or similar tool which shows disk-utilization) to check what eats up your diskspace.
I have the exact same number of files on the two disks
Keep a special eye on "System Volume Information" and "$RECYCLE.BIN" on the nthfs disk
this disk never have seen windows so no extra file like this You said you "kept" the NTFS. So i assumed you used the NTFS that came from stock (as its often the case for external HDDs). In that case these special folder could be there even I you did never connect it to a windows machine. and on ".Trash" and "lost+found" on the ext4 disk.
lost+found 16k on the two disks
trash removed ext4: 915Gb free, ntfs 703Gb free
may be some difference between hidden files by rsync? Depending on how you call rsync, it can skip hidden files. Thus my suggestion to use another tool, like the above mention ncdu.
Best regards Thomas -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-12-02 12:27, Thomas Wagner wrote:
On 2016-12-01 19:03, jdd wrote:
Keep a special eye on "System Volume Information" and "$RECYCLE.BIN" on the nthfs disk
this disk never have seen windows so no extra file like this You said you "kept" the NTFS. So i assumed you used the NTFS that came from stock (as its often the case for external HDDs). In that case these special folder could be there even I you did never connect it to a windows machine.
Now that you mention. By default, after creating an ntfs disk (in Windows) there is some reserved space that is not accessible and lost when the disk is external. But this can be disabled (in windows). Unfortunately, I can't remember the name of the feature. Not the bin. I should have written a note to myself about this... where is it!? Sigh. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
Le 02/12/2016 à 12:27, Thomas Wagner a écrit :
Depending on how you call rsync, it can skip hidden files. Thus my suggestion to use another tool, like the above mention ncdu.
well, I already used "du -d 1", but there are lot of files/folders so each test last very long I simply use rsync -av --delete /run/media/jdd/intenso5to2/ /run/media/jdd/intenso5to/ AFAIK it shouldn't miss the hidden files. There I don't have any hidden windows file, and partitions are identical. I don't know if windows can hide anything that could be unseen by Linux? it may be some kind of file system store because du is identical /run/media/jdd # du -h -d 2 115G ./intenso5to/autres 16K ./intenso5to/lost+found 3,2T ./intenso5to/archives-avant-2016 297G ./intenso5to/archives-2016 3,7T ./intenso5to 115G ./intenso5to2/autres 16K ./intenso5to2/lost+found 297G ./intenso5to2/archives-2016 3,2T ./intenso5to2/archives-avant-2016 3,7T ./intenso5to2 # df -h /dev/sdc1 4,6T 3,7T 701G 85% /run/media/jdd/intenso5to2 /dev/sdi1 4,6T 3,7T 930G 80% /run/media/jdd/intenso5to so be it... thanks jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 12/02/2016 10:54 AM, jdd wrote:
it may be some kind of file system store because du is identical
/run/media/jdd # du -h -d 2 115G ./intenso5to/autres 16K ./intenso5to/lost+found 3,2T ./intenso5to/archives-avant-2016 297G ./intenso5to/archives-2016 3,7T ./intenso5to
115G ./intenso5to2/autres 16K ./intenso5to2/lost+found 297G ./intenso5to2/archives-2016 3,2T ./intenso5to2/archives-avant-2016 3,7T ./intenso5to2
# df -h /dev/sdc1 4,6T 3,7T 701G 85% /run/media/jdd/intenso5to2 /dev/sdi1 4,6T 3,7T 930G 80% /run/media/jdd/intenso5to
so be it...
Not surprised that Linux file system utilities find no difference in files, or that linux tools for freespace fine differences in disk freespace. (These tools aren't even perfectly consistent using linux file systems, and are mostly useless using btrfs file systems). I am surprised at the extent of the of the difference. It looks like it could be lost clusters of disk that perhaps should have been returned to the free disk space, but never quite made it to the free list. This problem is apparently severe enough that there are tools for it, even from microsoft. http://ntfs.com/disk-scan.htm http://www.ntfs.com/recovery-toolkit.htm https://support.microsoft.com/en-us/kb/814594 -- After all is said and done, more is said than done. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 02/12/2016 à 20:16, John Andersen a écrit :
I am surprised at the extent of the of the difference.
yes, it was what surprised me most (a small difference is probably unavoidable) It looks like it
could be lost clusters of disk that perhaps should have been returned to the free disk space, but never quite made it to the free list.
yes, it was my first guess, may be also journal size?
This problem is apparently severe enough that there are tools for it, even from microsoft. http://ntfs.com/disk-scan.htm http://www.ntfs.com/recovery-toolkit.htm https://support.microsoft.com/en-us/kb/814594
I see no fsck for ntfs (not surprising) I will try ntfsfix, but I don't expect if to be useful thanks jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
01.12.2016 18:50, jdd пишет:
is it possible to move such disk from ntfs to ext4 without reformatting all (and copying 4.5Tb data)?
While in theory this is possible, I am not aware of any tool to do it. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-12-01 16:50, jdd wrote:
Hello, I just hit a curious thing.
I use two identical (bough together) external usb 5Tb drives (as archives)
Fort whatever reason one was formatted EXR4 and the other kept as NTFS
I just made a rsync --delete from one to the other, verified they keep the exact same number of files.
The ext4 one have 915 Gb free, the other 687 Gb
wtf!!
Maybe due to the metadata organization, will be different on each.
is it possible to move such disk from ntfs to ext4 without reformatting all (and copying 4.5Tb data)?
Maybe a proprietary tool. I'm thinking of Partition Magic. Are they big files, like videos? You will save even more space by using XFS. On the contrary, if they are very small (1KB) you gain more with Reiserfs. Maybe with btrfs without snapshots. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
Le 01/12/2016 à 19:58, Carlos E. R. a écrit :
is it possible to move such disk from ntfs to ext4 without reformatting all (and copying 4.5Tb data)?
Maybe a proprietary tool. I'm thinking of Partition Magic.
there is one free http://anyfs-tools.sourceforge.net/man8/anyconvertfs.8.html
Are they big files, like videos? You will save even more space by using XFS.
On the contrary, if they are very small (1KB) you gain more with Reiserfs. Maybe with btrfs without snapshots.
many large files (video) but also many small (photos) I don't try to save so much space, I like better security and I trust ext4 for anyconvert, I fear it wont give good results with nearly full 5Tb disk thanks jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-12-01 20:20, jdd wrote:
Le 01/12/2016 à 19:58, Carlos E. R. a écrit :
Are they big files, like videos? You will save even more space by using XFS.
On the contrary, if they are very small (1KB) you gain more with Reiserfs. Maybe with btrfs without snapshots.
many large files (video) but also many small (photos)
I don't try to save so much space, I like better security and I trust ext4
Once I did a compare, and ext4 format reserves a lot of space from the start for the metadata. XFS doesn't, because the reservation is dynamic, runtime. Thus for multimedia files it makes a difference. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
participants (5)
-
Andrei Borzenkov -
Carlos E. R. -
jdd -
John Andersen -
Thomas Wagner