[opensuse-factory] Re: puzzled by rsync
On 12/9/2013 5:36 PM, Felix Miata wrote:
source & target have same filesystem size, block size, filesystem type OS 11.4 running on both
Couple of scenarios to rule out: 1) going from disk1 a -> 500G file b->empty to disk2 a=>empty b->500G file ---- --delete = --delete-during which means it won't try to delete a later encountered directory until it gets to copying 'b'. If your disk is 750G, your copy will die in the middle of copying "a". Maybe delete-before would be better, though theoretically less safe, in that you might be deleting 500G before the copy on disk1 has safely made it to disk2. But if you are tight on disk space.... 2) If your target disk is more fragmented, then it will need more free-space extents(clusters) -- rather than 1 range with 2G, it might need 20 ranges with 100MB each. That means disk 2 will need more "indirect" space to hold the extra ranges. Those are possibilities that come to mind off the top of my head... -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 2013-12-10 00:11 (GMT-0800) L.A. Walsh composed:
Felix Miata wrote:
source & target have same filesystem size, block size, filesystem type OS 11.4 running on both
Couple of scenarios to rule out:
1) going from
disk1 a -> 500G file b->empty
to
disk2 a=>empty b->500G file
---- --delete = --delete-during which means it won't try to delete a later encountered directory until it gets to copying 'b'. If your disk is 750G, your copy will die in the middle of copying "a".
Maybe delete-before would be better, though theoretically less safe, in that you might be deleting 500G before the copy on disk1 has safely made it to disk2. But if you are tight on disk space....
Given the result observed on running it a 2nd time, I believe the above describes close enough what happened, and I should have used delete-before.
2) If your target disk is more fragmented, then it will need more free-space extents(clusters) -- rather than 1 range with 2G, it might need 20 ranges with 100MB each.
That means disk 2 will need more "indirect" space to hold the extra ranges.
Likely this is happening too, but not the primary problem here. Carlos: not btrfs. MC would explain differences, but not necessarily the rsync behavior absent understanding what Linda explained. Michael: no idea why you brought up sparse. The filesystem's sparse files are mostly small. The largest mass of files by number and size are compressed. AFAICT, sparse is about transfer speed, not ultimate space consumption. Andrey: probably only one 1k hard linked file on that filesystem. Nearly nil soft links too. -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 12/10/2013 09:35 AM, Felix Miata wrote:
AFAICT, sparse is about transfer speed, not ultimate space consumption.
au contraire, my friend! $ truncate -s 10T file $ ls -ldog file -rw-r--r-- 1 10995116277760 Dec 10 09:46 file $ du -h file 0 file $ du -h --apparent-size file 10T file $ df -h --output=source,size,used file Filesystem Size Used /dev/sdb 9.8G 5.7G Have a nice day, Berny -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 2013-12-10 09:49 (GMT+0100) Bernhard Voelker composed:
Felix Miata wrote:
AFAICT, sparse is about transfer speed, not ultimate space consumption.
au contraire, my friend!
$ truncate -s 10T file
$ ls -ldog file -rw-r--r-- 1 10995116277760 Dec 10 09:46 file
$ du -h file 0 file
$ du -h --apparent-size file 10T file
$ df -h --output=source,size,used file Filesystem Size Used /dev/sdb 9.8G 5.7G
Too terse for my comprehension. (I've never had an encounter with truncate before.) -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 10 December 2013 09:11, L.A. Walsh
On 12/9/2013 5:36 PM, Felix Miata wrote:
source & target have same filesystem size, block size, filesystem type OS 11.4 running on both
I'm sorry Linda, I cant seem to find Felix's original post, and even if I could, I don't think this is the appropriate mailinglist for this discussion.. opensuse-factory@o.o should be for discussions about the 'next' release of openSUSE, a topic which sometimes also justifies a little discussion about the 'current' state of openSUSE, but I don't think 11.4 fits into either category :) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 12/10/2013 12:40 AM, Richard Brown wrote:
I'm sorry Linda, I cant seem to find Felix's original post...
It were a booboo on my part. Attempted to retype To field to send only 1 copy, but for some reason my address book prefers factory until I get to the '@'... Will have to beat it into submission, er... or something like that.. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
participants (4)
-
Bernhard Voelker -
Felix Miata -
L.A. Walsh -
Richard Brown