[suse-mirror] wrong rsync behavior with stage::opensuse-full-really-everything/opensuse
When using rsync to get a list of files (which means --list-only is either given or implied) from stage::opensuse-full-really-everything/opensuse we get this: lrwxrwxrwx 8 2009/10/29 06:26:06 distribution/11.2/repo/oss/boot/i386/branding lrwxrwxrwx 8 2009/06/16 22:46:03 factory-snapshot/repo/oss/boot/i386/branding lrwxrwxrwx 8 2009/06/16 22:46:03 factory/repo/oss/boot/i386/branding Note the difference in timestamps between the first and the other two. *However* all 3 files are hardlinks to the same inode, as shown by ls -i: 2147487511 distribution/11.2/repo/oss/boot/i386/branding 2147487511 factory-snapshot/repo/oss/boot/i386/branding 2147487511 factory/repo/oss/boot/i386/branding All file creation and updates are done by rsync, which means it's rsync that is creating the hardlinks. Therefore there is a contradiction between what rsync does with the files and its own listing. The problem has been going on for weeks. It also happens with initrd-xenpae and vmlinuz-xenpae in the same directory and in the x86_64 one. BTW, they're all hardlinks to inodes that are symlinks. In order to see if this bug is due to rsync I'd like to see a listing of these files produced by something like ls -li, find or stat. Please post it here. -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org For additional commands, e-mail: mirror+help@opensuse.org
Am Samstag 13 Februar 2010 schrieb Carlos Carvalho:
In order to see if this bug is due to rsync I'd like to see a listing of these files produced by something like ls -li, find or stat. Please post it here.
I bet rsync is just clever enough to figure the inodes are the same? :) The server has it right at least: 536904664 lrwxrwxrwx 2 mirror stage 8 2009-10-29 09:26 distribution/11.2/repo/oss/boot/i386/branding -> openSUSE 2732788160 lrwxrwxrwx 3 mirror stage 8 2009-06-17 03:46 factory/repo/oss/boot/i386/branding -> openSUSE 2732788160 lrwxrwxrwx 3 mirror stage 8 2009-06-17 03:46 factory- snapshot/repo/oss/boot/i386/branding -> openSUSE Greetings, Stephan -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org For additional commands, e-mail: mirror+help@opensuse.org
In order to see if this bug is due to rsync I'd like to see a listing of these files produced by something like ls -li, find or stat. Please post it here.
I bet rsync is just clever enough to figure the inodes are the same? :)
The server has it right at least:
536904664 lrwxrwxrwx 2 mirror stage 8 2009-10-29 09:26 distribution/11.2/repo/oss/boot/i386/branding -> openSUSE 2732788160 lrwxrwxrwx 3 mirror stage 8 2009-06-17 03:46 factory/repo/oss/boot/i386/branding -> openSUSE 2732788160 lrwxrwxrwx 3 mirror stage 8 2009-06-17 03:46 factory- snapshot/repo/oss/boot/i386/branding -> openSUSE
Multiple links on symlink inodes may not be portable across different filesystem types. Apparently, rsync doesn't count on that either. Coolo, can we please remove all symlinks with a link count larger than 1 and re-create them as single-linked symlinks? That would remove the oddities... Thanks, Roman. -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org For additional commands, e-mail: mirror+help@opensuse.org
Am Dienstag 16 Februar 2010 schrieb Roman Drahtmueller:
In order to see if this bug is due to rsync I'd like to see a listing of these files produced by something like ls -li, find or stat. Please post it here.
I bet rsync is just clever enough to figure the inodes are the same? :)
The server has it right at least:
536904664 lrwxrwxrwx 2 mirror stage 8 2009-10-29 09:26 distribution/11.2/repo/oss/boot/i386/branding -> openSUSE 2732788160 lrwxrwxrwx 3 mirror stage 8 2009-06-17 03:46 factory/repo/oss/boot/i386/branding -> openSUSE 2732788160 lrwxrwxrwx 3 mirror stage 8 2009-06-17 03:46 factory- snapshot/repo/oss/boot/i386/branding -> openSUSE
Multiple links on symlink inodes may not be portable across different filesystem types. Apparently, rsync doesn't count on that either.
Coolo, can we please remove all symlinks with a link count larger than 1 and re-create them as single-linked symlinks? That would remove the oddities...
No time for that. And what's worse: it will reappear once I call factory_snapshot again, so it's better if you update that script to do what you think is right. Greetings, Stephan -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org For additional commands, e-mail: mirror+help@opensuse.org
Roman Drahtmueller (draht@suse.de) wrote on 16 February 2010 14:24:
In order to see if this bug is due to rsync I'd like to see a listing of these files produced by something like ls -li, find or stat. Please post it here.
I bet rsync is just clever enough to figure the inodes are the same? :)
The server has it right at least:
536904664 lrwxrwxrwx 2 mirror stage 8 2009-10-29 09:26 distribution/11.2/repo/oss/boot/i386/branding -> openSUSE 2732788160 lrwxrwxrwx 3 mirror stage 8 2009-06-17 03:46 factory/repo/oss/boot/i386/branding -> openSUSE 2732788160 lrwxrwxrwx 3 mirror stage 8 2009-06-17 03:46 factory- snapshot/repo/oss/boot/i386/branding -> openSUSE
Multiple links on symlink inodes may not be portable across different filesystem types. Apparently, rsync doesn't count on that either.
The problem is not the filesystem; factory and factory-snapshot are still hardlinks. If a bad filesystem doesn't support hardlinks of symlinks rsync will create them as separate objects without problem. What's happening here is caused by rsync. The symlinks above were probably all hardlinks in the past, and were separated later. When this happens rsync doesn't break the hardlink at the destination. Here's an example: % mkdir origin % : > origin/file % ln -s file origin/first-symlink "origin/first-symlink" -> "file" % ln origin/first-symlink origin/second-symlink "origin/second-symlink" => "origin/first-symlink" Now we have symlinks that are hardlinks to the same object: % ls -i origin 2686528 file 2686757 first-symlink 2686757 second-symlink rsync does the hardlinking: % rsync -aHi origin/ dest cd+++++++++ ./
f+++++++++ file cL+++++++++ first-symlink -> file hL+++++++++ second-symlink => first-symlink
% ls -i dest 2686762 file 2686760 first-symlink 2686760 second-symlink Note that the symlinks have the same inode, as they should. Now let's break the bond at the origin: % rm origin/second-symlink % ln -s file origin/second-symlink "origin/second-symlink" -> "file" % ls -i origin 2686528 file 2686757 first-symlink 2686765 second-symlink Now the symlinks point to the same file but are independent objects. They also have different dates. However after an update... % rsync -aHi origin/ dest .d..t...... ./ .L..t...... second-symlink -> file % ls -i dest 2686762 file 2686760 first-symlink 2686760 second-symlink Note that in the destination the symlinks *continue* to be hardlinks... Now there will always be a mismatch in the dates between origin and dest: % rsync -aHi origin/ dest .L..t...... first-symlink -> file .L..t...... second-symlink -> file And it goes on forever: % rsync -aHi origin/ dest .L..t...... first-symlink -> file .L..t...... second-symlink -> file % rsync -aHi origin/ dest .L..t...... first-symlink -> file .L..t...... second-symlink -> file And the hardlinking never breaks: % ls -i dest 2686762 file 2686760 first-symlink 2686760 second-symlink This rsync behavior doesn't affect the availability of information in the destination. Also, if any of the symlinks in the origin is changed to another object rsync will replicate it in the destination. So it can be argued that this isn't a bug, it's just non-intuitive. It has the bennefit of speeding up updates, and since hardlinks of symlinks are so rare, it might be worthwhile. Anyway, I think the rsync maintainers know about the issue. -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org For additional commands, e-mail: mirror+help@opensuse.org
participants (3)
-
carlos＠fisica.ufpr.br -
Roman Drahtmueller -
Stephan Kulow