On 03/08/2019 07.41, Andrei Borzenkov wrote:
02.08.2019 22:24, Carlos E. R. пишет:
On 02/08/2019 19.29, Andrei Borzenkov wrote:
02.08.2019 10:59, Carlos E. R. пишет:
On Friday, 2019-08-02 at 06:48 +0300, Andrei Borzenkov wrote:
...
[... trying ...]
Nope (took 9 minutes to run)
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include=/data/storage_** --include=/data/vmware \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
The idea is to copy only some /data/* directories and avoid the rest. Or to not copy /data/** except /data/storage_**
The above command copies none of the /data tree.
Yes, "--exclude /data" prevents rsync from descending into /data. You need "--include /data/storage_** --exclude /data/**" (not sure whether you really need ** or just * here).
Mmm...? I'll try. Now I have to go out to buy a new tea pot, the current one broke. Tea is a crucial part of life :-)
[...]
I think it is working. 80% sure.
No, it is not!
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include /data/storage_** --include=/data/vmware \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
The problem is that it is creating: /mnt/BookTelcontar/001/storage_b, /mnt/BookTelcontar/001/storage_c, instead of /mnt/BookTelcontar/001/data/storage_b, /mnt/BookTelcontar/001/storage_a.
:-(
I cannot reproduce it. rsync sends paths relative to source directory by default and you have single source directory which is /:
bor@bor-Latitude-E5450:~$ LC_ALL=C find /tmp/root/ /tmp/root/ /tmp/root/foo /tmp/root/data /tmp/root/data/copy /tmp/root/data/copy/foo /tmp/root/data/copy/bar /tmp/root/data/skip /tmp/root/data/skip/baz bor@bor-Latitude-E5450:~$ LC_ALL=C rsync -av --include /data/copy** --exclude /data/** --exclude /foo* /tmp/root/ /tmp/newroot/ sending incremental file list ./ data/ data/copy/ data/copy/bar data/copy/foo
sent 232 bytes received 69 bytes 602.00 bytes/sec total size is 0 speedup is 0.00 bor@bor-Latitude-E5450:~$ LC_ALL=C find /tmp/newroot/ /tmp/newroot/ /tmp/newroot/data /tmp/newroot/data/copy /tmp/newroot/data/copy/foo /tmp/newroot/data/copy/bar bor@bor-Latitude-E5450:~$
You may try --relative which send full paths; as your source is filesystem root anyway, it should do what you want.
Oh, I believe I understand what happens - shell expands your /data/storage_* so rsync (quite correctly) sends paths relative to *new* root which is /data in this case.
Ok, tried with --relative. OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable " DESTINO=002 PREVIO=001 rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets --link-dest=root@Isengard::MyBook_Telcontar/$PREVIO \ --relative --include /data/data/vmware** --include=/data/storage_** --include=/data/cripta --include=/data/xtr --include=/data/raid \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO Telcontar:~ # time bckall2mybook Rsync server found - working on it. ---> main part. --link-dest arg does not exist: root@Isengard::MyBook_Telcontar/001 Number of files: 3,760,365 (reg: 3,383,551, dir: 269,267, link: 106,862, dev: 106, special: 579) Number of created files: 3,760,364 (reg: 3,383,551, dir: 269,266, link: 106,862, dev: 106, special: 579) Number of deleted files: 0 Number of regular files transferred: 2,941,453 Total file size: 3.38T bytes Total transferred file size: 3.29T bytes Literal data: 3.29T bytes Matched data: 0 bytes File list size: 120.51M File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 3.29T Total bytes received: 86.33M sent 3.29T bytes received 86.33M bytes 34.52M bytes/sec total size is 3.38T speedup is 1.03 ---> /data part. real 1588m8.714s user 137m23.366s sys 108m19.771s Telcontar:~ # ie, 26 hours - and it missed some directories. using " --relative --include /data/data/vmware** --include=/data/storage_** --include=/data/cripta --include=/data/xtr --include=/data/raid " /mnt/BookTelcontar/002/data/cripta empty /mnt/BookTelcontar/002/data/raid empty /mnt/BookTelcontar/002/data/storage_a good /mnt/BookTelcontar/002/data/storage_b good /mnt/BookTelcontar/002/data/storage_c good /mnt/BookTelcontar/002/data/xtr empty /mnt/BookTelcontar/002/data/vmware does not exist, my mistake (/data/data/vmware**) Those that it missed I did not write **. Editing and running again. [...] after a 67 minutes run, some 23 G bytes more were sent, but there are still missing directories: /mnt/BookTelcontar/002/data/cripta empty /mnt/BookTelcontar/002/data/raid done /mnt/BookTelcontar/002/data/storage_a good /mnt/BookTelcontar/002/data/storage_b good /mnt/BookTelcontar/002/data/storage_c good /mnt/BookTelcontar/002/data/xtr done /mnt/BookTelcontar/002/data/vmware empty output: Number of files: 3,766,415 (reg: 3,388,039, dir: 270,807, link: 106,885, dev: 106, special: 578) Number of created files: 8,375 (reg: 6,810, dir: 1,542, link: 23) Number of deleted files: 0 Number of regular files transferred: 7,974 Total file size: 3.40T bytes Total transferred file size: 23.07G bytes Literal data: 20.07G bytes Matched data: 3.00G bytes File list size: 22.93M File list generation time: 0.015 seconds File list transfer time: 0.000 seconds Total bytes sent: 20.18G Total bytes received: 2.78M sent 20.18G bytes received 2.78M bytes 4.98M bytes/sec total size is 3.40T speedup is 168.69 real 67m28.490s user 1m27.047s sys 2m17.282s The command was: OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable " DESTINO=002 PREVIO=001 rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets --link-dest=root@Isengard::MyBook_Telcontar/$PREVIO \ --relative --include /data/vmware** --include=/data/storage_** --include=/data/cripta/** --include=/data/xtr** --include=/data/raid** \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO "--include /data/vmware**" does not work. Perhaps /data/vmware/** will? But "--include=/data/cripta/**" did not work. What do I do here? It is not consistent - yet it is a computer, it has to be consistent, but I do not see it. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)