[opensuse] rsync --exclude --include question
I have: OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable " rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ / root@Isengard::MyBook_Telcontar/001 It is failing, /data/storage_** is not included. What is the proper way in this tree: /data/something/* /data/somethingelse/* /data/somethingmore/* /data/somethingevenmore/* /data/storage_a/* /data/storage_b/* to copy /data/storage_a, /data/storage_b, and ignore all the rest, without explicitly listing all the rest? I thought that --exclude=/data --include=/data/storage_** would do it. The manual is very confusing to me. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
* Carlos E. R. <robin.listas@telefonica.net> [08-01-19 18:01]:
I have:
OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable "
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ / root@Isengard::MyBook_Telcontar/001
It is failing, /data/storage_** is not included.
What is the proper way in this tree:
/data/something/* /data/somethingelse/* /data/somethingmore/* /data/somethingevenmore/* /data/storage_a/* /data/storage_b/*
to copy /data/storage_a, /data/storage_b, and ignore all the rest, without explicitly listing all the rest? I thought that
--exclude=/data --include=/data/storage_**
would do it. The manual is very confusing to me.
perhaps, --exclude=/data --include-/data/storage_? -- (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri http://en.opensuse.org openSUSE Community Member facebook/ptilopteri Photos: http://wahoo.no-ip.org/piwigo paka @ IRCnet freenode -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 02/08/2019 00.18, Patrick Shanahan wrote:
* Carlos E. R. <robin.listas@telefonica.net> [08-01-19 18:01]:
I have:
OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable "
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ / root@Isengard::MyBook_Telcontar/001
It is failing, /data/storage_** is not included.
What is the proper way in this tree:
/data/something/* /data/somethingelse/* /data/somethingmore/* /data/somethingevenmore/* /data/storage_a/* /data/storage_b/*
to copy /data/storage_a, /data/storage_b, and ignore all the rest, without explicitly listing all the rest? I thought that
--exclude=/data --include=/data/storage_**
would do it. The manual is very confusing to me.
perhaps, --exclude=/data --include-/data/storage_?
A minus (-) symbol? I'll try. [...] No. Telcontar:~ # time bckall2mybook Rsync server found rsync: --include-/data/storage_**: unknown option rsync error: syntax or usage error (code 1) at main.c(1593) [client=3.1.3] I'll try this: rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=+/data/storage_** \ / root@Isengard::MyBook_Telcontar/$DESTINO I think it doesn't work, but will take a while till it exits and I know. I guess it doesn't because I see the network is not active. ie, not transmitting anything. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
On 02/08/2019 00.29, Carlos E. R. wrote:
On 02/08/2019 00.18, Patrick Shanahan wrote:
* Carlos E. R. <> [08-01-19 18:01]:
I'll try this:
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=+/data/storage_** \ / root@Isengard::MyBook_Telcontar/$DESTINO
I think it doesn't work, but will take a while till it exits and I know. I guess it doesn't because I see the network is not active. ie, not transmitting anything.
Nay, doesn't work. What does is call rsync twice: rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=+/data/storage_** \ / root@Isengard::MyBook_Telcontar/001 rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ /data/vmware /data/storage_** /data/Lareiserfs root@Isengard::MyBook_Telcontar/001/data But I don't like it. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
Op vrijdag 2 augustus 2019 01:22:47 CEST schreef Carlos E. R.:
On 02/08/2019 00.29, Carlos E. R. wrote:
On 02/08/2019 00.18, Patrick Shanahan wrote:
* Carlos E. R. <> [08-01-19 18:01]: I'll try this:
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \
--exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=+/data/storage_** \ / root@Isengard::MyBook_Telcontar/$DESTINO
I think it doesn't work, but will take a while till it exits and I know. I guess it doesn't because I see the network is not active. ie, not transmitting anything. Nay, doesn't work. What does is call rsync twice:
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=+/data/storage_** \ / root@Isengard::MyBook_Telcontar/001
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ /data/vmware /data/storage_** /data/Lareiserfs root@Isengard::MyBook_Telcontar/001/data
But I don't like it. And only the second doesn't?
-- Gertjan Lettink a.k.a. Knurpht openSUSE Board Member openSUSE Forums Team -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Friday, 2019-08-02 at 02:04 +0200, Knurpht-openSUSE wrote:
Op vrijdag 2 augustus 2019 01:22:47 CEST schreef Carlos E. R.:
...
And only the second doesn't?
No, maybe I did not explain myself well. I was trying this way, in several variations: OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable " rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=+/data/storage_** \ / root@Isengard::MyBook_Telcontar/001 This method does not include /data/storage_a, for example. So I'm using this other method, a workaround: rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=+/data/storage_** \ / root@Isengard::MyBook_Telcontar/001 rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ /data/vmware /data/storage_** /data/Lareiserfs root@Isengard::MyBook_Telcontar/001/data ie, *two* rsync calls. - -- Cheers, Carlos E. R. (from openSUSE 15.0 x86_64 at Telcontar) -----BEGIN PGP SIGNATURE----- iHoEARECADoWIQQZEb51mJKK1KpcU/W1MxgcbY1H1QUCXUOEURwccm9iaW4ubGlz dGFzQHRlbGVmb25pY2EubmV0AAoJELUzGBxtjUfVd6gAn2hg8PY3Rfwun3mlw7wZ L/sxnQfPAJ4hBHl++l0pw99ITrJuUMjzqu4qPA== =fjwJ -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
* Carlos E. R. <robin.listas@telefonica.net> [08-01-19 18:32]:
On 02/08/2019 00.18, Patrick Shanahan wrote:
* Carlos E. R. <robin.listas@telefonica.net> [08-01-19 18:01]:
I have:
OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable "
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ / root@Isengard::MyBook_Telcontar/001
It is failing, /data/storage_** is not included.
What is the proper way in this tree:
/data/something/* /data/somethingelse/* /data/somethingmore/* /data/somethingevenmore/* /data/storage_a/* /data/storage_b/*
to copy /data/storage_a, /data/storage_b, and ignore all the rest, without explicitly listing all the rest? I thought that
--exclude=/data --include=/data/storage_**
would do it. The manual is very confusing to me.
perhaps, --exclude=/data --include-/data/storage_?
A minus (-) symbol? I'll try.
no, the (-) is a typo, s/b --include=/data/storage_? -- (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri http://en.opensuse.org openSUSE Community Member facebook/ptilopteri Photos: http://wahoo.no-ip.org/piwigo paka @ IRCnet freenode -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
* Patrick Shanahan <paka@opensuse.org> [08-01-19 23:02]:
* Carlos E. R. <robin.listas@telefonica.net> [08-01-19 18:32]:
On 02/08/2019 00.18, Patrick Shanahan wrote:
* Carlos E. R. <robin.listas@telefonica.net> [08-01-19 18:01]:
I have:
OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable "
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ / root@Isengard::MyBook_Telcontar/001
It is failing, /data/storage_** is not included.
What is the proper way in this tree:
/data/something/* /data/somethingelse/* /data/somethingmore/* /data/somethingevenmore/* /data/storage_a/* /data/storage_b/*
to copy /data/storage_a, /data/storage_b, and ignore all the rest, without explicitly listing all the rest? I thought that
--exclude=/data --include=/data/storage_**
would do it. The manual is very confusing to me.
perhaps, --exclude=/data --include-/data/storage_?
A minus (-) symbol? I'll try.
no, the (-) is a typo, s/b --include=/data/storage_?
the "?" includes a single unknown character you could also write --include=/data/storage_?/* which should include /data/storage_a/ and /data/storage_b/ and ... -- (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri http://en.opensuse.org openSUSE Community Member facebook/ptilopteri Photos: http://wahoo.no-ip.org/piwigo paka @ IRCnet freenode -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Content-ID: <alpine.LSU.2.21.1908020942421.5907@Telcontar.valinor> On Thursday, 2019-08-01 at 23:06 -0400, Patrick Shanahan wrote:
* Patrick Shanahan <> [08-01-19 23:02]:
* Carlos E. R. <> [08-01-19 18:32]:
On 02/08/2019 00.18, Patrick Shanahan wrote:
* Carlos E. R. <> [08-01-19 18:01]:
...
perhaps, --exclude=/data --include-/data/storage_?
A minus (-) symbol? I'll try.
no, the (-) is a typo, s/b --include=/data/storage_?
the "?" includes a single unknown character
you could also write --include=/data/storage_?/* which should include /data/storage_a/ and /data/storage_b/ and ...
I tried those or similar combinations, did not work. I'm running now a test with A.B. reply - -- Cheers, Carlos E. R. (from openSUSE 15.0 x86_64 at Telcontar) -----BEGIN PGP SIGNATURE----- iHoEARECADoWIQQZEb51mJKK1KpcU/W1MxgcbY1H1QUCXUPpvxwccm9iaW4ubGlz dGFzQHRlbGVmb25pY2EubmV0AAoJELUzGBxtjUfVsUgAoItuv1QJCgtYC3giqgQv YULjJRPrAJ0d23lIPfluHsjbIovSh+vohKhm8Q== =QK+m -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
02.08.2019 0:59, Carlos E. R. пишет:
I have:
OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable "
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ / root@Isengard::MyBook_Telcontar/001
It is failing, /data/storage_** is not included.
Filters are in ordered list, the first matching filter applies. --include=/data/subdir needs to be before --exclude=/data
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Content-ID: <alpine.LSU.2.21.1908020957180.5907@Telcontar.valinor> On Friday, 2019-08-02 at 06:48 +0300, Andrei Borzenkov wrote:
02.08.2019 0:59, Carlos E. R. пишет:
I have:
OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable "
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ / root@Isengard::MyBook_Telcontar/001
It is failing, /data/storage_** is not included.
Filters are in ordered list, the first matching filter applies. --include=/data/subdir needs to be before --exclude=/data
Oh! That's reverse order to me :-) [... trying ...] Nope (took 9 minutes to run) rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include=/data/storage_** --include=/data/vmware \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO The idea is to copy only some /data/* directories and avoid the rest. Or to not copy /data/** except /data/storage_** The above command copies none of the /data tree. - -- Cheers, Carlos E. R. (from openSUSE 15.0 x86_64 at Telcontar) -----BEGIN PGP SIGNATURE----- iHoEARECADoWIQQZEb51mJKK1KpcU/W1MxgcbY1H1QUCXUPtVxwccm9iaW4ubGlz dGFzQHRlbGVmb25pY2EubmV0AAoJELUzGBxtjUfVDKMAn3olylWhSNsQlxWTX4+n WWEOibU3AJ9iZxCLucd77iRcoRfkYXntWQ56jA== =zBNZ -----END PGP SIGNATURE-----
Carlos E. R. wrote:
The idea is to copy only some /data/* directories and avoid the rest. Or to not copy /data/** except /data/storage_**
Maybe this will do the job: rsync $OPTIONS /data/storage_* /data/vmware \ root@Isengard::MyBook_Telcontar/$DESTINO or rsync $OPTIONS /data --exclude=<whatever-you-dont-want> \ root@Isengard::MyBook_Telcontar/$DESTINO -- Per Jessen, Zürich (19.1°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Friday, 2019-08-02 at 10:23 +0200, Per Jessen wrote:
Carlos E. R. wrote:
The idea is to copy only some /data/* directories and avoid the rest. Or to not copy /data/** except /data/storage_**
Maybe this will do the job:
rsync $OPTIONS /data/storage_* /data/vmware \ root@Isengard::MyBook_Telcontar/$DESTINO
Certainly, that works. But that implies two separate runs: One to copy the system: rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO another to copy the data directories: rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ /data/vmware /data/storage_** root@Isengard::MyBook_Telcontar/$DESTINO/data That is what I'm doing now. I want to combine the two runs in a single one. The only one I know would be to exclude one by one all the directories under /data: cer@Telcontar:~> ls /data Datum Lareiserfs aux_01 cripta main_root_bck partimage storage_a storage_b02 storage_d storage_new_d storage_old_c vmware Isengard MGR backup homedvl other raid storage_b storage_c storage_e storage_old_b test xtr cer@Telcontar:~> Thus: rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --exclude=/data/Datum --exclude=/data/Lareiserfs --exclude=/data/aux_01 --exclude=/data/cripta \ --exclude=/data/main_root_bck ...... / root@Isengard::MyBook_Telcontar/$DESTINO Listing one by one alll of them, except storage_* There must be a filter to tell rsync to include /data/storage_* and exclude the rest of /data.
or
rsync $OPTIONS /data --exclude=<whatever-you-dont-want> \ root@Isengard::MyBook_Telcontar/$DESTINO
Same thing. - -- Cheers, Carlos E. R. (from openSUSE 15.0 x86_64 at Telcontar) -----BEGIN PGP SIGNATURE----- iHoEARECADoWIQQZEb51mJKK1KpcU/W1MxgcbY1H1QUCXUP22xwccm9iaW4ubGlz dGFzQHRlbGVmb25pY2EubmV0AAoJELUzGBxtjUfVT1cAn3grzTQ4rNKKRx8m0Evb 2EzmxRURAJ490AwcGEIcUh5KHAZdXkLvHvWZdQ== =pyIs -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
On Friday, 2019-08-02 at 10:23 +0200, Per Jessen wrote:
Carlos E. R. wrote:
The idea is to copy only some /data/* directories and avoid the rest. Or to not copy /data/** except /data/storage_**
Maybe this will do the job:
rsync $OPTIONS /data/storage_* /data/vmware \ root@Isengard::MyBook_Telcontar/$DESTINO
Certainly, that works. But that implies two separate runs:
One to copy the system: [snip] That is what I'm doing now. I want to combine the two runs in a single one.
Sorry, wasn't part of the spec, scope-creep :-) I would just do two runs, but I understand the technical challenge.
There must be a filter to tell rsync to include /data/storage_* and exclude the rest of /data.
rsync -ax / /data/storage* --exclude=/data $DEST -- Per Jessen, Zürich (19.4°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 02/08/2019 11.24, Per Jessen wrote:
Carlos E. R. wrote:
On Friday, 2019-08-02 at 10:23 +0200, Per Jessen wrote:
Carlos E. R. wrote:
The idea is to copy only some /data/* directories and avoid the rest. Or to not copy /data/** except /data/storage_**
Maybe this will do the job:
rsync $OPTIONS /data/storage_* /data/vmware \ root@Isengard::MyBook_Telcontar/$DESTINO
Certainly, that works. But that implies two separate runs:
One to copy the system: [snip] That is what I'm doing now. I want to combine the two runs in a single one.
Sorry, wasn't part of the spec, scope-creep :-)
I said or implied that from the start, but perhaps not clear enough.
I would just do two runs, but I understand the technical challenge.
There must be a filter to tell rsync to include /data/storage_* and exclude the rest of /data.
rsync -ax / /data/storage* --exclude=/data $DEST
Ah. Maybe. I'll try. Ah, no. The problem with that is that /data/storage_a ends in DEST/storage_a instead of DEST/data/storage_a Trying anyway. [...] Yes, does that. rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / /data/vmware root@Isengard::MyBook_Telcontar/$DESTINO I saw appear on destination, in seconds, DEST/vmware Then I tried sync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / /data/vmware/ root@Isengard::MyBook_Telcontar/$DESTINO And this is worse, the contents of /data/vmware/ appears directly on $DEST without vmware directory. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
02.08.2019 10:59, Carlos E. R. пишет:
Content-ID: <alpine.LSU.2.21.1908020957180.5907@Telcontar.valinor>
On Friday, 2019-08-02 at 06:48 +0300, Andrei Borzenkov wrote:
02.08.2019 0:59, Carlos E. R. пишет:
I have:
OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable "
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ / root@Isengard::MyBook_Telcontar/001
It is failing, /data/storage_** is not included.
Filters are in ordered list, the first matching filter applies. --include=/data/subdir needs to be before --exclude=/data
Oh! That's reverse order to me :-)
[... trying ...]
Nope (took 9 minutes to run)
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include=/data/storage_** --include=/data/vmware \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
The idea is to copy only some /data/* directories and avoid the rest. Or to not copy /data/** except /data/storage_**
The above command copies none of the /data tree.
Yes, "--exclude /data" prevents rsync from descending into /data. You need "--include /data/storage_** --exclude /data/**" (not sure whether you really need ** or just * here).
On 02/08/2019 19.29, Andrei Borzenkov wrote:
02.08.2019 10:59, Carlos E. R. пишет:
On Friday, 2019-08-02 at 06:48 +0300, Andrei Borzenkov wrote:
...
[... trying ...]
Nope (took 9 minutes to run)
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include=/data/storage_** --include=/data/vmware \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
The idea is to copy only some /data/* directories and avoid the rest. Or to not copy /data/** except /data/storage_**
The above command copies none of the /data tree.
Yes, "--exclude /data" prevents rsync from descending into /data. You need "--include /data/storage_** --exclude /data/**" (not sure whether you really need ** or just * here).
Mmm...? I'll try. Now I have to go out to buy a new tea pot, the current one broke. Tea is a crucial part of life :-) [...] I think it is working. 80% sure. No, it is not! rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include /data/storage_** --include=/data/vmware \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO The problem is that it is creating: /mnt/BookTelcontar/001/storage_b, /mnt/BookTelcontar/001/storage_c, instead of /mnt/BookTelcontar/001/data/storage_b, /mnt/BookTelcontar/001/storage_a. :-( Back to the two process method. Thanks anyway. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
On Fri, 2 Aug 2019 21:24:22 +0200 "Carlos E. R." <robin.listas@telefonica.net> wrote:
On 02/08/2019 19.29, Andrei Borzenkov wrote:
02.08.2019 10:59, Carlos E. R. пишет:
On Friday, 2019-08-02 at 06:48 +0300, Andrei Borzenkov wrote:
...
[... trying ...]
Nope (took 9 minutes to run)
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include=/data/storage_** --include=/data/vmware \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
The idea is to copy only some /data/* directories and avoid the rest. Or to not copy /data/** except /data/storage_**
The above command copies none of the /data tree.
Yes, "--exclude /data" prevents rsync from descending into /data. You need "--include /data/storage_** --exclude /data/**" (not sure whether you really need ** or just * here).
Mmm...? I'll try. Now I have to go out to buy a new tea pot, the current one broke. Tea is a crucial part of life :-)
[...]
I think it is working. 80% sure.
No, it is not!
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include /data/storage_** --include=/data/vmware \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
The problem is that it is creating: /mnt/BookTelcontar/001/storage_b, /mnt/BookTelcontar/001/storage_c, instead of /mnt/BookTelcontar/001/data/storage_b, /mnt/BookTelcontar/001/storage_a.
:-(
Sounds like you're almost there. Even with ** :) But I think this is the first we've heard of storage_c and why would expet a general pattern like * (or **!) to exclude that one particularly? You clearly need an exclude for that particular directory before the include for storage_* before the exclude for storage. But are there any more exceptions? PS I agree that tea is most important,but are you sure you haven't been at the Tinto de Verano?
Back to the two process method.
Thanks anyway.
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 02/08/2019 22.58, Dave Howorth wrote:
On Fri, 2 Aug 2019 21:24:22 +0200 "Carlos E. R." <> wrote:
On 02/08/2019 19.29, Andrei Borzenkov wrote:
02.08.2019 10:59, Carlos E. R. пишет:
On Friday, 2019-08-02 at 06:48 +0300, Andrei Borzenkov wrote:
Yes, "--exclude /data" prevents rsync from descending into /data. You need "--include /data/storage_** --exclude /data/**" (not sure whether you really need ** or just * here).
Mmm...? I'll try. Now I have to go out to buy a new tea pot, the current one broke. Tea is a crucial part of life :-)
[...]
I think it is working. 80% sure.
No, it is not!
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include /data/storage_** --include=/data/vmware \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
The problem is that it is creating: /mnt/BookTelcontar/001/storage_b, /mnt/BookTelcontar/001/storage_c, instead of /mnt/BookTelcontar/001/data/storage_b, /mnt/BookTelcontar/001/storage_a.
:-(
Sounds like you're almost there. Even with ** :)
But I think this is the first we've heard of storage_c and why would expet a general pattern like * (or **!) to exclude that one particularly? You clearly need an exclude for that particular directory before the include for storage_* before the exclude for storage. But are there any more exceptions?
No, all "storage_*" directories are to be copied. The problem with this version is that it creates: /mnt/BookTelcontar/001-+-storage_a |-storage_b |-storage_c \-storage_d when the correct tree is: /mnt/BookTelcontar/001/data/-+-storage_a |-storage_b |-storage_c \-storage_d The reason for the --exclude=/data is that there are 24 subdirectories there, and I do not want to copy all of them, only a few. Currently I'm repeating the test you suggested: rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include=/data/vmware/ --include=/data/storage_*/ \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO I have to wait about 10 minutes to see if it going well, when it finishes seeing that there is nothing in "/" to copy now, and starts thinking about /data if it goes well, or exits if not.
PS I agree that tea is most important,but are you sure you haven't been at the Tinto de Verano?
LOL. Sure. I just made a tea without theine to try the new pot :-D A molten iron tea pot, heated directly on the induction stove. <https://www.alcampo.es/compra-online/hogar/casa-y-decoracion/menaje-de-cocina/cafeteras-y-teteras/teteras/menaje-tetera-hierro-05-litros/p/524592> Previous one was made of borosilicated glass, and it broke this morning after a year of service. Similar to this one: <https://www.alcampo.es/compra-online/hogar/casa-y-decoracion/menaje-de-cocina/cafeteras-y-teteras/teteras/bormioli-tetera-06-litros/p/524584> Later I'll have half a glass of Mosel Riesling, from a bottle I got at a sale (the wine surprised me), with pasta, for a light supper :-D Meanwhile, while I talk of trivialities, I wait for the rsync process to start sending files. It is still thinking, which is probably a good signal. I'll wait more [...] (after 37m total) No, it exited without copying anything, except empty directories. I know this for sure because I deleted /mnt/BookTelcontar/001/data/vmware then started the script, and in seconds it created the empty dir. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
02.08.2019 22:24, Carlos E. R. пишет:
On 02/08/2019 19.29, Andrei Borzenkov wrote:
02.08.2019 10:59, Carlos E. R. пишет:
On Friday, 2019-08-02 at 06:48 +0300, Andrei Borzenkov wrote:
...
[... trying ...]
Nope (took 9 minutes to run)
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include=/data/storage_** --include=/data/vmware \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
The idea is to copy only some /data/* directories and avoid the rest. Or to not copy /data/** except /data/storage_**
The above command copies none of the /data tree.
Yes, "--exclude /data" prevents rsync from descending into /data. You need "--include /data/storage_** --exclude /data/**" (not sure whether you really need ** or just * here).
Mmm...? I'll try. Now I have to go out to buy a new tea pot, the current one broke. Tea is a crucial part of life :-)
[...]
I think it is working. 80% sure.
No, it is not!
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include /data/storage_** --include=/data/vmware \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
The problem is that it is creating: /mnt/BookTelcontar/001/storage_b, /mnt/BookTelcontar/001/storage_c, instead of /mnt/BookTelcontar/001/data/storage_b, /mnt/BookTelcontar/001/storage_a.
:-(
I cannot reproduce it. rsync sends paths relative to source directory by default and you have single source directory which is /: bor@bor-Latitude-E5450:~$ LC_ALL=C find /tmp/root/ /tmp/root/ /tmp/root/foo /tmp/root/data /tmp/root/data/copy /tmp/root/data/copy/foo /tmp/root/data/copy/bar /tmp/root/data/skip /tmp/root/data/skip/baz bor@bor-Latitude-E5450:~$ LC_ALL=C rsync -av --include /data/copy** --exclude /data/** --exclude /foo* /tmp/root/ /tmp/newroot/ sending incremental file list ./ data/ data/copy/ data/copy/bar data/copy/foo sent 232 bytes received 69 bytes 602.00 bytes/sec total size is 0 speedup is 0.00 bor@bor-Latitude-E5450:~$ LC_ALL=C find /tmp/newroot/ /tmp/newroot/ /tmp/newroot/data /tmp/newroot/data/copy /tmp/newroot/data/copy/foo /tmp/newroot/data/copy/bar bor@bor-Latitude-E5450:~$ You may try --relative which send full paths; as your source is filesystem root anyway, it should do what you want. Oh, I believe I understand what happens - shell expands your /data/storage_* so rsync (quite correctly) sends paths relative to *new* root which is /data in this case.
On 03/08/2019 07.41, Andrei Borzenkov wrote:
02.08.2019 22:24, Carlos E. R. пишет:
...
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include /data/storage_** --include=/data/vmware \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
The problem is that it is creating: /mnt/BookTelcontar/001/storage_b, /mnt/BookTelcontar/001/storage_c, instead of /mnt/BookTelcontar/001/data/storage_b, /mnt/BookTelcontar/001/storage_a.
:-(
I cannot reproduce it. rsync sends paths relative to source directory by default and you have single source directory which is /:
bor@bor-Latitude-E5450:~$ LC_ALL=C find /tmp/root/ /tmp/root/ /tmp/root/foo /tmp/root/data /tmp/root/data/copy /tmp/root/data/copy/foo /tmp/root/data/copy/bar /tmp/root/data/skip /tmp/root/data/skip/baz bor@bor-Latitude-E5450:~$ LC_ALL=C rsync -av --include /data/copy** --exclude /data/** --exclude /foo* /tmp/root/ /tmp/newroot/ sending incremental file list ./ data/ data/copy/ data/copy/bar data/copy/foo
sent 232 bytes received 69 bytes 602.00 bytes/sec total size is 0 speedup is 0.00 bor@bor-Latitude-E5450:~$ LC_ALL=C find /tmp/newroot/ /tmp/newroot/ /tmp/newroot/data /tmp/newroot/data/copy /tmp/newroot/data/copy/foo /tmp/newroot/data/copy/bar bor@bor-Latitude-E5450:~$
You may try --relative which send full paths; as your source is filesystem root anyway, it should do what you want.
Oh, I believe I understand what happens - shell expands your /data/storage_* so rsync (quite correctly) sends paths relative to *new* root which is /data in this case.
I used "ps afx | grep rsync" to find out if this was happening, but the shell does not expand "--include /data/storage_*". It is in one of the posts I sent. I may test again - it is becoming difficult (slow) as there are 4TB to copy. I'll create a test tree and try then. But I have a working method, which is call rsync twice: echo "---> main part." rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO echo echo "---> /data part." echo rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ /data/vmware /data/cripta /data/xtr /data/raid /data/storage_** \ root@Isengard::MyBook_Telcontar/$DESTINO/data It is not that neat, but it works. Or I could instead exclude one by one the directories in /data that I do not want, that would work. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
Carlos E. R. wrote:
I may test again - it is becoming difficult (slow) as there are 4TB to copy. I'll create a test tree and try then.
If you run rsync with "-nv", maybe that'll give you information on what is being selected? -- Per Jessen, Zürich (23.2°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 03/08/2019 14.10, Per Jessen wrote:
Carlos E. R. wrote:
I may test again - it is becoming difficult (slow) as there are 4TB to copy. I'll create a test tree and try then.
If you run rsync with "-nv", maybe that'll give you information on what is being selected?
-n, --dry-run perform a trial run with no changes made -v, --verbose increase verbosity Maybe. But it spews thousand of lines :-? I saw an option the other day that replicates the directory tree without files. What was it...? rsync -a --include='*/' --exclude='*' /data/Isengard/ \ /home_aux/BLUERAY_OPS/mnt/ Huh. Those include/exclude could distort the result. Question: Do we have an rsync GUI that would generate the command line, clicking on a tree to select/deselect directories? [...] googling There is "Grsync". <https://www.unixmen.com/grsync-gadmin-rsync-graphical-front-end-applications-rsync-tool/> <https://software.opensuse.org/package/grsync?search_term=grsync> Worth a look. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
Carlos E. R. wrote:
On 03/08/2019 14.10, Per Jessen wrote:
Carlos E. R. wrote:
I may test again - it is becoming difficult (slow) as there are 4TB to copy. I'll create a test tree and try then.
If you run rsync with "-nv", maybe that'll give you information on what is being selected?
-n, --dry-run perform a trial run with no changes made -v, --verbose increase verbosity
Maybe. But it spews thousand of lines :-?
That is what grep is for :-) -- Per Jessen, Zürich (22.8°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 03/08/2019 14.43, Per Jessen wrote:
Carlos E. R. wrote:
On 03/08/2019 14.10, Per Jessen wrote:
Carlos E. R. wrote:
I may test again - it is becoming difficult (slow) as there are 4TB to copy. I'll create a test tree and try then.
If you run rsync with "-nv", maybe that'll give you information on what is being selected?
-n, --dry-run perform a trial run with no changes made -v, --verbose increase verbosity
Maybe. But it spews thousand of lines :-?
That is what grep is for :-)
Check: Telcontar:~ # rsync --password-file /root/cripta/keys/rsync.secrets root@Isengard::MyBook_Telcontar drwxr-xr-x 18 2019/08/04 12:41:20 . drwxr-xr-x 0 2019/08/01 14:55:28 000 drwxr-xr-x 384 2018/11/24 11:04:40 001 <=== existing copy drwxr-xr-x 364 2018/11/24 11:04:40 002 <=== new run today Telcontar:~ # DESTINO=002 PREVIO=001 rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets --link-dest=root@Isengard::MyBook_Telcontar/$PREVIO \ --relative --include /data//data/vmware** --include=/data/storage_** \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO responds with (but continues): --link-dest arg does not exist: root@Isengard::MyBook_Telcontar/001 But it does exist, the check above proves it. The shell expands to: Telcontar:~ # ps afxu | grep rsync root 30750 20.2 0.0 68284 7948 pts/7 S+ 12:52 0:12 | | \_ rsync --archive --acls --xattrs --hard-links --sparse --stats --human-readable --password-file /root/cripta/keys/rsync.secrets --link-dest=root@Isengard::MyBook_Telcontar/001 --relative --include /data//data/vmware** --include=/data/storage_** --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ / root@Isengard::MyBook_Telcontar/002 root 30794 0.0 0.0 7432 888 pts/50 S+ 12:53 0:00 | \_ grep --color=auto rsync Telcontar:~ # You can see the asterisks on "--include=..." are not expanded. I can see that "/mnt/BookTelcontar/002/data/storage_a" already exists and is getting populated, so "--relative" is working. Other data directories are already created, but empty yet. data/vmware not. Also, the cpu load on the server rsyncd gets limited to 85%, as seen by "top", after the systemd trick I mentioned in another post. So, it is working except for the --link-dest thing - which is crucial to me, but is another issue. If that doesn't work, I'll have to use btrfs snapshots somehow. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
Carlos E. R. wrote:
The shell expands to:
Telcontar:~ # ps afxu | grep rsync root 30750 20.2 0.0 68284 7948 pts/7 S+ 12:52 0:12 | | \_ rsync --archive --acls --xattrs --hard-links --sparse --stats --human-readable --password-file /root/cripta/keys/rsync.secrets --link-dest=root@Isengard::MyBook_Telcontar/001 --relative --include /data//data/vmware** --include=/data/storage_** --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ / root@Isengard::MyBook_Telcontar/002 root 30794 0.0 0.0 7432 888 pts/50 S+ 12:53 0:00 | \_ grep --color=auto rsync Telcontar:~ #
You can see the asterisks on "--include=..." are not expanded.
Which they surely should be, unless quoted or escaped.
Also, the cpu load on the server rsyncd gets limited to 85%, as seen by "top", after the systemd trick I mentioned in another post.
It's really just a cgroup, it's a neat and easy way to use it.
So, it is working except for the --link-dest thing - which is crucial to me, but is another issue. If that doesn't work, I'll have to use btrfs snapshots somehow.
Or use two rsyncs ... -- Per Jessen, Zürich (24.0°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 04/08/2019 14.22, Per Jessen wrote:
Carlos E. R. wrote:
The shell expands to:
Telcontar:~ # ps afxu | grep rsync root 30750 20.2 0.0 68284 7948 pts/7 S+ 12:52 0:12 | | \_ rsync --archive --acls --xattrs --hard-links --sparse --stats --human-readable --password-file /root/cripta/keys/rsync.secrets --link-dest=root@Isengard::MyBook_Telcontar/001 --relative --include /data//data/vmware** --include=/data/storage_** --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ / root@Isengard::MyBook_Telcontar/002 root 30794 0.0 0.0 7432 888 pts/50 S+ 12:53 0:00 | \_ grep --color=auto rsync Telcontar:~ #
You can see the asterisks on "--include=..." are not expanded.
Which they surely should be, unless quoted or escaped.
But they are not quoted nor escaped - you can see the exact command line I used in the previous post. The reason is, I understand, that bash knows it is an --option in the command line, not a parameter, which do get expanded. Look, pasted from another post, two days ago: ]> cer@Telcontar:~> ps afx | grep rsync ]> 30526 pts/7 S+ 8:23 | | \_ rsync --archive --acls --xattrs --hard-links --sparse --stats --human-readable --password-file /root/cripta/keys/rsync.secrets --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ --include=/data/storage_** /data/vmware /data/cripta /data/storage_a /data/storage_b /data/storage_b02 /data/storage_c /data/storage_d /data/storage_e /data/storage_new_d /data/storage_old_b /data/storage_old_c root@Isengard::MyBook_Telcontar/001/data ]> 32309 pts/50 S+ 0:00 | \_ grep --color=auto rsync ]> ]> You can see that "--include=/data/storage_**" was not expanded. The command line I used is: ]> ]> rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ ]> --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ ]> --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ ]> --include=/data/storage_** \ ]> /data/vmware /data/cripta /data/storage_** root@Isengard::MyBook_Telcontar/$DESTINO/data Notice how the parameters (last line) did get expanded by bash, but not the --options.
Also, the cpu load on the server rsyncd gets limited to 85%, as seen by "top", after the systemd trick I mentioned in another post.
It's really just a cgroup, it's a neat and easy way to use it.
Indeed it is.
So, it is working except for the --link-dest thing - which is crucial to me, but is another issue. If that doesn't work, I'll have to use btrfs snapshots somehow.
Or use two rsyncs ...
I don't see how that will help to create the hard links to the old backup copy. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
Carlos E. R. wrote:
You can see the asterisks on "--include=..." are not expanded.
Which they surely should be, unless quoted or escaped.
But they are not quoted nor escaped - you can see the exact command line I used in the previous post. The reason is, I understand, that bash knows it is an --option in the command line, not a parameter, which do get expanded.
Aha, that's interesting, I was not aware.
So, it is working except for the --link-dest thing - which is crucial to me, but is another issue. If that doesn't work, I'll have to use btrfs snapshots somehow.
Or use two rsyncs ...
I don't see how that will help to create the hard links to the old backup copy.
Maybe create them manually, afterwards? -- Per Jessen, Zürich (27.2°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 04/08/2019 16.29, Per Jessen wrote:
Carlos E. R. wrote:
You can see the asterisks on "--include=..." are not expanded.
Which they surely should be, unless quoted or escaped.
But they are not quoted nor escaped - you can see the exact command line I used in the previous post. The reason is, I understand, that bash knows it is an --option in the command line, not a parameter, which do get expanded.
Aha, that's interesting, I was not aware.
Me neither :-)
So, it is working except for the --link-dest thing - which is crucial to me, but is another issue. If that doesn't work, I'll have to use btrfs snapshots somehow.
Or use two rsyncs ...
I don't see how that will help to create the hard links to the old backup copy.
Maybe create them manually, afterwards?
You mean another rsync run, this time inside the server? I can only do that, besides the CPU load and time (about 10 hours) if the server disk can hold two different copies of the backup, and I suspect it can't. The backup is about 3.9TB, the capacity is 8TB. No, if rsync done to an rsyncd daemon doesn't support --link-dest (as it seems), the best remaining practical method is making use of btrfs snapshots (untested: does btrfs allways detect an overwrite of an existing file as the same file byte per byte and "link" it?). And of course, in that case I still have to send it all, using those 8 hours or more. The other method is not using the rsyncd daemon on the server, but using ssh transport on the client. All the job done on the client. I'll start a new thread about the --link-dest problem later. It should just work. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
On 04/08/2019 16.46, Carlos E. R. wrote:
On 04/08/2019 16.29, Per Jessen wrote:
Carlos E. R. wrote:
You can see the asterisks on "--include=..." are not expanded.
Which they surely should be, unless quoted or escaped.
But they are not quoted nor escaped - you can see the exact command line I used in the previous post. The reason is, I understand, that bash knows it is an --option in the command line, not a parameter, which do get expanded.
Aha, that's interesting, I was not aware.
Me neither :-)
Correction: --include something** is expanded. --include=something** is not expanded. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
On 03/08/2019 07.41, Andrei Borzenkov wrote:
02.08.2019 22:24, Carlos E. R. пишет:
On 02/08/2019 19.29, Andrei Borzenkov wrote:
02.08.2019 10:59, Carlos E. R. пишет:
On Friday, 2019-08-02 at 06:48 +0300, Andrei Borzenkov wrote:
...
[... trying ...]
Nope (took 9 minutes to run)
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include=/data/storage_** --include=/data/vmware \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
The idea is to copy only some /data/* directories and avoid the rest. Or to not copy /data/** except /data/storage_**
The above command copies none of the /data tree.
Yes, "--exclude /data" prevents rsync from descending into /data. You need "--include /data/storage_** --exclude /data/**" (not sure whether you really need ** or just * here).
Mmm...? I'll try. Now I have to go out to buy a new tea pot, the current one broke. Tea is a crucial part of life :-)
[...]
I think it is working. 80% sure.
No, it is not!
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include /data/storage_** --include=/data/vmware \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
The problem is that it is creating: /mnt/BookTelcontar/001/storage_b, /mnt/BookTelcontar/001/storage_c, instead of /mnt/BookTelcontar/001/data/storage_b, /mnt/BookTelcontar/001/storage_a.
:-(
I cannot reproduce it. rsync sends paths relative to source directory by default and you have single source directory which is /:
bor@bor-Latitude-E5450:~$ LC_ALL=C find /tmp/root/ /tmp/root/ /tmp/root/foo /tmp/root/data /tmp/root/data/copy /tmp/root/data/copy/foo /tmp/root/data/copy/bar /tmp/root/data/skip /tmp/root/data/skip/baz bor@bor-Latitude-E5450:~$ LC_ALL=C rsync -av --include /data/copy** --exclude /data/** --exclude /foo* /tmp/root/ /tmp/newroot/ sending incremental file list ./ data/ data/copy/ data/copy/bar data/copy/foo
sent 232 bytes received 69 bytes 602.00 bytes/sec total size is 0 speedup is 0.00 bor@bor-Latitude-E5450:~$ LC_ALL=C find /tmp/newroot/ /tmp/newroot/ /tmp/newroot/data /tmp/newroot/data/copy /tmp/newroot/data/copy/foo /tmp/newroot/data/copy/bar bor@bor-Latitude-E5450:~$
You may try --relative which send full paths; as your source is filesystem root anyway, it should do what you want.
Oh, I believe I understand what happens - shell expands your /data/storage_* so rsync (quite correctly) sends paths relative to *new* root which is /data in this case.
Ok, tried with --relative. OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable " DESTINO=002 PREVIO=001 rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets --link-dest=root@Isengard::MyBook_Telcontar/$PREVIO \ --relative --include /data/data/vmware** --include=/data/storage_** --include=/data/cripta --include=/data/xtr --include=/data/raid \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO Telcontar:~ # time bckall2mybook Rsync server found - working on it. ---> main part. --link-dest arg does not exist: root@Isengard::MyBook_Telcontar/001 Number of files: 3,760,365 (reg: 3,383,551, dir: 269,267, link: 106,862, dev: 106, special: 579) Number of created files: 3,760,364 (reg: 3,383,551, dir: 269,266, link: 106,862, dev: 106, special: 579) Number of deleted files: 0 Number of regular files transferred: 2,941,453 Total file size: 3.38T bytes Total transferred file size: 3.29T bytes Literal data: 3.29T bytes Matched data: 0 bytes File list size: 120.51M File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 3.29T Total bytes received: 86.33M sent 3.29T bytes received 86.33M bytes 34.52M bytes/sec total size is 3.38T speedup is 1.03 ---> /data part. real 1588m8.714s user 137m23.366s sys 108m19.771s Telcontar:~ # ie, 26 hours - and it missed some directories. using " --relative --include /data/data/vmware** --include=/data/storage_** --include=/data/cripta --include=/data/xtr --include=/data/raid " /mnt/BookTelcontar/002/data/cripta empty /mnt/BookTelcontar/002/data/raid empty /mnt/BookTelcontar/002/data/storage_a good /mnt/BookTelcontar/002/data/storage_b good /mnt/BookTelcontar/002/data/storage_c good /mnt/BookTelcontar/002/data/xtr empty /mnt/BookTelcontar/002/data/vmware does not exist, my mistake (/data/data/vmware**) Those that it missed I did not write **. Editing and running again. [...] after a 67 minutes run, some 23 G bytes more were sent, but there are still missing directories: /mnt/BookTelcontar/002/data/cripta empty /mnt/BookTelcontar/002/data/raid done /mnt/BookTelcontar/002/data/storage_a good /mnt/BookTelcontar/002/data/storage_b good /mnt/BookTelcontar/002/data/storage_c good /mnt/BookTelcontar/002/data/xtr done /mnt/BookTelcontar/002/data/vmware empty output: Number of files: 3,766,415 (reg: 3,388,039, dir: 270,807, link: 106,885, dev: 106, special: 578) Number of created files: 8,375 (reg: 6,810, dir: 1,542, link: 23) Number of deleted files: 0 Number of regular files transferred: 7,974 Total file size: 3.40T bytes Total transferred file size: 23.07G bytes Literal data: 20.07G bytes Matched data: 3.00G bytes File list size: 22.93M File list generation time: 0.015 seconds File list transfer time: 0.000 seconds Total bytes sent: 20.18G Total bytes received: 2.78M sent 20.18G bytes received 2.78M bytes 4.98M bytes/sec total size is 3.40T speedup is 168.69 real 67m28.490s user 1m27.047s sys 2m17.282s The command was: OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable " DESTINO=002 PREVIO=001 rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets --link-dest=root@Isengard::MyBook_Telcontar/$PREVIO \ --relative --include /data/vmware** --include=/data/storage_** --include=/data/cripta/** --include=/data/xtr** --include=/data/raid** \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO "--include /data/vmware**" does not work. Perhaps /data/vmware/** will? But "--include=/data/cripta/**" did not work. What do I do here? It is not consistent - yet it is a computer, it has to be consistent, but I do not see it. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
On Mon, 5 Aug 2019 20:10:50 +0200 "Carlos E. R." <robin.listas@telefonica.net> wrote:
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets --link-dest=root@Isengard::MyBook_Telcontar/$PREVIO \ --relative --include /data/vmware** --include=/data/storage_** --include=/data/cripta/** --include=/data/xtr** --include=/data/raid** \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
"--include /data/vmware**" does not work. Perhaps /data/vmware/** will?
But "--include=/data/cripta/**" did not work. What do I do here? It is not consistent - yet it is a computer, it has to be consistent, but I do not see it.
I'm not sure what is not consistent. Neither worked. That is consistent, no? Why do you use a double star (**) rather than a single star (*) BTW? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 05/08/2019 20.52, Dave Howorth wrote:
On Mon, 5 Aug 2019 20:10:50 +0200 "Carlos E. R." <> wrote:
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets --link-dest=root@Isengard::MyBook_Telcontar/$PREVIO \ --relative --include /data/vmware** --include=/data/storage_** --include=/data/cripta/** --include=/data/xtr** --include=/data/raid** \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
"--include /data/vmware**" does not work. Perhaps /data/vmware/** will?
But "--include=/data/cripta/**" did not work. What do I do here? It is not consistent - yet it is a computer, it has to be consistent, but I do not see it.
I'm not sure what is not consistent. Neither worked. That is consistent, no?
But "--include=/data/storage_**" did work. Why not "--include=/data/cripta/**" or "--include /data/vmware**". And "--include=/data/raid**" did work.
Why do you use a double star (**) rather than a single star (*) BTW?
Because it is in the man page. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
05.08.2019 23:26, Carlos E. R. пишет:
But "--include=/data/storage_**" did work. Why not "--include=/data/cripta/**"
For the same reason --include=/data/storage* --exclude=/data* did not work.
or "--include /data/vmware**". And
Because rsync does not see "**", it is consumed by shell.
"--include=/data/raid**" did work.
You seriously do not see the difference between both lines?
On 06/08/2019 05.57, Andrei Borzenkov wrote:
05.08.2019 23:26, Carlos E. R. пишет:
But "--include=/data/storage_**" did work. Why not "--include=/data/cripta/**"
For the same reason --include=/data/storage* --exclude=/data* did not work.
No, because you told me it would work if the --include goes before the --exclude and it is. I'm using the command line you told me to use.
or "--include /data/vmware**". And
Because rsync does not see "**", it is consumed by shell.
It isn't. I proved that. No, the reason is because I wrote: --include /data/vmware** instead of: --include=/data/vmware** Look, the "**" are NOT expanded - this is what the shell expands and what not: Telcontar:~ # ps afxu | grep rsync root 1257 24.6 0.1 71108 11680 pts/7 S+ 11:35 0:00 | | \_ rsync --archive --acls --xattrs --hard-links --sparse --stats --human-readable --password-file /root/cripta/keys/rsync.secrets --link-dest=root@Isengard::MyBook_Telcontar/001 --relative --include=/data/vmware/** --include=/data/storage_** --include=/data/cripta/** --include=/data/xtr** --include=/data/raid** --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ / root@Isengard::MyBook_Telcontar/002
"--include=/data/raid**" did work.
You seriously do not see the difference between both lines?
It helps when somebody else tells one where to look for the obvious. That's why one posts things in internet, because we do not see the problem and need help. Yes, today I saw a difference. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
On Tue, Aug 6, 2019 at 12:44 PM Carlos E. R. <robin.listas@telefonica.net> wrote:
On 06/08/2019 05.57, Andrei Borzenkov wrote:
05.08.2019 23:26, Carlos E. R. пишет:
But "--include=/data/storage_**" did work. Why not "--include=/data/cripta/**"
For the same reason --include=/data/storage* --exclude=/data* did not work.
No, because you told me it would work if the --include goes before the --exclude and it is. I'm using the command line you told me to use.
it did not work the first time you tried it. And then I told you that with --exclude=/data** rsync does not even attempt to descend into /data so it never has chance to match /data/storage**. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 06/08/2019 12.32, Andrei Borzenkov wrote:
On Tue, Aug 6, 2019 at 12:44 PM Carlos E. R. <robin.listas@telefonica.net> wrote:
On 06/08/2019 05.57, Andrei Borzenkov wrote:
05.08.2019 23:26, Carlos E. R. пишет:
But "--include=/data/storage_**" did work. Why not "--include=/data/cripta/**"
For the same reason --include=/data/storage* --exclude=/data* did not work.
No, because you told me it would work if the --include goes before the --exclude and it is. I'm using the command line you told me to use.
it did not work the first time you tried it. And then I told you that with --exclude=/data** rsync does not even attempt to descend into /data so it never has chance to match /data/storage**.
But we are not talking of the first time I did it, we are talking of the current time. Please just tell me if these are correct or incorrect, and if so, how to change them - because i do not understand what you are saying: --include=/data/vmware** --include=/data/storage_** --include=/data/cripta** --include=/data/xtr** --include=/data/raid** --exclude=/data/** -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
On Tue, Aug 6, 2019 at 1:42 PM Carlos E. R. <robin.listas@telefonica.net> wrote:
On 06/08/2019 12.32, Andrei Borzenkov wrote:
On Tue, Aug 6, 2019 at 12:44 PM Carlos E. R. <robin.listas@telefonica.net> wrote:
On 06/08/2019 05.57, Andrei Borzenkov wrote:
05.08.2019 23:26, Carlos E. R. пишет:
But "--include=/data/storage_**" did work. Why not "--include=/data/cripta/**"
For the same reason --include=/data/storage* --exclude=/data* did not work.
No, because you told me it would work if the --include goes before the --exclude and it is. I'm using the command line you told me to use.
it did not work the first time you tried it. And then I told you that with --exclude=/data** rsync does not even attempt to descend into /data so it never has chance to match /data/storage**.
But we are not talking of the first time I did it, we are talking of the current time.
Please just tell me if these are correct or incorrect, and if so, how to change them - because i do not understand what you are saying:
--include=/data/vmware** --include=/data/storage_** --include=/data/cripta**
That's not what you said before. And that should work like other directories.
--include=/data/xtr** --include=/data/raid** --exclude=/data/**
-- Cheers / Saludos,
Carlos E. R. (from 15.0 x86_64 at Telcontar)
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, 1 Aug 2019 23:59:38 +0200 "Carlos E. R." <robin.listas@telefonica.net> wrote:
I have:
OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable "
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ / root@Isengard::MyBook_Telcontar/001
It is failing, /data/storage_** is not included.
Yes, the man page says that the first matching rule is acted upon, so the /data excludes everything below and your include rule is never considered. If you put the include rule first, I think it may work. I don't remember if you need to quote the patterns, BTW.
What is the proper way in this tree:
/data/something/* /data/somethingelse/* /data/somethingmore/* /data/somethingevenmore/* /data/storage_a/* /data/storage_b/*
to copy /data/storage_a, /data/storage_b, and ignore all the rest, without explicitly listing all the rest? I thought that
--exclude=/data --include=/data/storage_**
would do it. The manual is very confusing to me.
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 02/08/2019 12.43, Dave Howorth wrote:
On Thu, 1 Aug 2019 23:59:38 +0200 "Carlos E. R." <robin.listas@telefonica.net> wrote:
I have:
OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable "
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ / root@Isengard::MyBook_Telcontar/001
It is failing, /data/storage_** is not included.
Yes, the man page says that the first matching rule is acted upon, so the /data excludes everything below and your include rule is never considered.
If you put the include rule first, I think it may work.
Yes, that's what Andrei Borzenkov said, but it does not work, or I did another mistake.
I don't remember if you need to quote the patterns, BTW.
Apparently not on the include/exclude options. On the SOURCE part I tried once and got a syntax error. This is what the process looks like now: cer@Telcontar:~> ps afx | grep rsync 30526 pts/7 S+ 8:23 | | \_ rsync --archive --acls --xattrs --hard-links --sparse --stats --human-readable --password-file /root/cripta/keys/rsync.secrets --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ --include=/data/storage_** /data/vmware /data/cripta /data/storage_a /data/storage_b /data/storage_b02 /data/storage_c /data/storage_d /data/storage_e /data/storage_new_d /data/storage_old_b /data/storage_old_c root@Isengard::MyBook_Telcontar/001/data 32309 pts/50 S+ 0:00 | \_ grep --color=auto rsync You can see that "--include=/data/storage_**" was not expanded. The command line I used is: rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ /data/vmware /data/cripta /data/storage_** root@Isengard::MyBook_Telcontar/$DESTINO/data -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
On 8/2/19 7:23 AM, Carlos E. R. wrote:
On 02/08/2019 12.43, Dave Howorth wrote:
On Thu, 1 Aug 2019 23:59:38 +0200 "Carlos E. R." <robin.listas@telefonica.net> wrote:
I have:
OPTIONS="--archive --acls --xattrs --hard-links --sparse --stats --human-readable "
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \ / root@Isengard::MyBook_Telcontar/001
It is failing, /data/storage_** is not included.
Yes, the man page says that the first matching rule is acted upon, so the /data excludes everything below and your include rule is never considered.
If you put the include rule first, I think it may work.
Yes, that's what Andrei Borzenkov said, but it does not work, or I did another mistake.
I don't remember if you need to quote the patterns, BTW.
Apparently not on the include/exclude options. On the SOURCE part I tried once and got a syntax error.
This is what the process looks like now:
cer@Telcontar:~> ps afx | grep rsync 30526 pts/7 S+ 8:23 | | \_ rsync --archive --acls --xattrs --hard-links --sparse --stats --human-readable --password-file /root/cripta/keys/rsync.secrets --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ --include=/data/storage_** /data/vmware /data/cripta /data/storage_a /data/storage_b /data/storage_b02 /data/storage_c /data/storage_d /data/storage_e /data/storage_new_d /data/storage_old_b /data/storage_old_c root@Isengard::MyBook_Telcontar/001/data 32309 pts/50 S+ 0:00 | \_ grep --color=auto rsync
You can see that "--include=/data/storage_**" was not expanded. The command line I used is:
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \
Change this to : --include=/data/storage_*/ ... I think you need the trailing '/' in order to traverse the subdirs. -- Ken Schneider SuSe since Version 5.2, June 1998 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 02/08/2019 16.04, Ken Schneider - openSUSE wrote:
On 8/2/19 7:23 AM, Carlos E. R. wrote:
...
This is what the process looks like now:
cer@Telcontar:~> ps afx | grep rsync 30526 pts/7 S+ 8:23 | | \_ rsync --archive --acls --xattrs --hard-links --sparse --stats --human-readable --password-file /root/cripta/keys/rsync.secrets --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ --include=/data/storage_** /data/vmware /data/cripta /data/storage_a /data/storage_b /data/storage_b02 /data/storage_c /data/storage_d /data/storage_e /data/storage_new_d /data/storage_old_b /data/storage_old_c root@Isengard::MyBook_Telcontar/001/data 32309 pts/50 S+ 0:00 | \_ grep --color=auto rsync
You can see that "--include=/data/storage_**" was not expanded. The command line I used is:
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \
Change this to : --include=/data/storage_*/ ...
I think you need the trailing '/' in order to traverse the subdirs.
It is not needed on the "--exclude", why would it be on the "--include"? The trailing slash on the source does have a meaning: the last directory is created on destination or not. But I can try. Anyway, I'll try. [...] Did not work. sync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_**/ --include=/data/vmware/ \ / root@Isengard::MyBook_Telcontar/$DESTINO -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
On Fri, 2 Aug 2019 19:11:42 +0200 "Carlos E. R." <robin.listas@telefonica.net> wrote:
On 02/08/2019 16.04, Ken Schneider - openSUSE wrote:
On 8/2/19 7:23 AM, Carlos E. R. wrote:
...
This is what the process looks like now:
cer@Telcontar:~> ps afx | grep rsync 30526 pts/7 S+ 8:23 | | \_ rsync --archive --acls --xattrs --hard-links --sparse --stats --human-readable --password-file /root/cripta/keys/rsync.secrets --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ --include=/data/storage_** /data/vmware /data/cripta /data/storage_a /data/storage_b /data/storage_b02 /data/storage_c /data/storage_d /data/storage_e /data/storage_new_d /data/storage_old_b /data/storage_old_c root@Isengard::MyBook_Telcontar/001/data 32309 pts/50 S+ 0:00 | \_ grep --color=auto rsync
You can see that "--include=/data/storage_**" was not expanded. The command line I used is:
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \
Change this to : --include=/data/storage_*/ ...
I think you need the trailing '/' in order to traverse the subdirs.
It is not needed on the "--exclude", why would it be on the "--include"?
The trailing slash on the source does have a meaning: the last directory is created on destination or not. But I can try.
Anyway, I'll try.
[...]
Did not work.
sync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_**/ --include=/data/vmware/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
We keep telling you that the specific include must precede the more general exclude. Regardless of the trailing slashes or anything else, you MUST do this in all trials! I agree with Andrei, why two *? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 02/08/2019 21.14, Dave Howorth wrote:
On Fri, 2 Aug 2019 19:11:42 +0200 "Carlos E. R." <robin.listas@telefonica.net> wrote:
On 02/08/2019 16.04, Ken Schneider - openSUSE wrote:
On 8/2/19 7:23 AM, Carlos E. R. wrote:
...
This is what the process looks like now:
cer@Telcontar:~> ps afx | grep rsync 30526 pts/7 S+ 8:23 | | \_ rsync --archive --acls --xattrs --hard-links --sparse --stats --human-readable --password-file /root/cripta/keys/rsync.secrets --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ --include=/data/storage_** /data/vmware /data/cripta /data/storage_a /data/storage_b /data/storage_b02 /data/storage_c /data/storage_d /data/storage_e /data/storage_new_d /data/storage_old_b /data/storage_old_c root@Isengard::MyBook_Telcontar/001/data 32309 pts/50 S+ 0:00 | \_ grep --color=auto rsync
You can see that "--include=/data/storage_**" was not expanded. The command line I used is:
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \
Change this to : --include=/data/storage_*/ ...
I think you need the trailing '/' in order to traverse the subdirs.
It is not needed on the "--exclude", why would it be on the "--include"?
The trailing slash on the source does have a meaning: the last directory is created on destination or not. But I can try.
Anyway, I'll try.
[...]
Did not work.
sync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_**/ --include=/data/vmware/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
We keep telling you that the specific include must precede the more general exclude. Regardless of the trailing slashes or anything else, you MUST do this in all trials! I agree with Andrei, why two *?
Must have been a typo or paste error. I'll try again to make sure. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
On 02/08/2019 21.26, Carlos E. R. wrote:
On 02/08/2019 21.14, Dave Howorth wrote:
On Fri, 2 Aug 2019 19:11:42 +0200 "Carlos E. R." <robin.listas@telefonica.net> wrote:
On 02/08/2019 16.04, Ken Schneider - openSUSE wrote:
On 8/2/19 7:23 AM, Carlos E. R. wrote:
...
This is what the process looks like now:
cer@Telcontar:~> ps afx | grep rsync 30526 pts/7 S+ 8:23 | | \_ rsync --archive --acls --xattrs --hard-links --sparse --stats --human-readable --password-file /root/cripta/keys/rsync.secrets --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ --include=/data/storage_** /data/vmware /data/cripta /data/storage_a /data/storage_b /data/storage_b02 /data/storage_c /data/storage_d /data/storage_e /data/storage_new_d /data/storage_old_b /data/storage_old_c root@Isengard::MyBook_Telcontar/001/data 32309 pts/50 S+ 0:00 | \_ grep --color=auto rsync
You can see that "--include=/data/storage_**" was not expanded. The command line I used is:
rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_** \
Change this to : --include=/data/storage_*/ ...
I think you need the trailing '/' in order to traverse the subdirs.
It is not needed on the "--exclude", why would it be on the "--include"?
The trailing slash on the source does have a meaning: the last directory is created on destination or not. But I can try.
Anyway, I'll try.
[...]
Did not work.
sync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ --include=/data/storage_**/ --include=/data/vmware/ \ / root@Isengard::MyBook_Telcontar/$DESTINO
We keep telling you that the specific include must precede the more general exclude. Regardless of the trailing slashes or anything else, you MUST do this in all trials! I agree with Andrei, why two *?
Must have been a typo or paste error. I'll try again to make sure.
Well, I'm sure now, it fails: rsync $OPTIONS --password-file /root/cripta/keys/rsync.secrets \ --include=/data/vmware/ --include=/data/storage_*/ \ --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/proc --exclude=/run \ --exclude=/data/** --exclude=/mnt --exclude=/other --exclude=/media --exclude=/var/lib/samba/lock/ --exclude=/var/run/ \ / root@Isengard::MyBook_Telcontar/$DESTINO I posted more details on another post. It does create the directory "/mnt/BookTelcontar/001/data/vmware", as it should, but it is empty. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
participants (7)
-
Andrei Borzenkov -
Carlos E. R. -
Dave Howorth -
Ken Schneider - openSUSE -
Knurpht-openSUSE -
Patrick Shanahan -
Per Jessen