On 05/03/2013 01:42 PM, Carl Hartung wrote:

exact commands that you issued to create the backups and to attempt/implement the 'restores.' .......................

1) #!/bin/sh # # use rsync to backup /home to /dev/sda2 # mount -t ext4 /dev/sda2 /mnt_cron # df cd /home/pisti rsync -avr --delete --delete-after --exclude=/ /home/ /mnt_cron cd # cp /mnt_cron/etc/fstab.5bak /mnt_cron/etc/fstab df umount /mnt_cron cd ...................... 2) #!/bin/sh # # use rsync to backup WHOLE system / to /dev/sdb4 # mount -t ext4 /dev/sdb4 /mnt_cron # df cd rsync -avr --delete --delete-after --exclude=/dev/shm --exclude=/DVD --exclude=/media --exclude=/mnt --exclude=/mnt_cron --exclude=/proc --exclude=/run --exclude=subdomain --exclude=/sys --exclude=/tmp --exclude=/var/lib/ntp --exclude=/var/run --exclude=/var/tmp / /mnt_cron cd # cp /etc/fstab_4B_BAK /mnt_cron/etc/fstab # DEBUG DEBUG DEBUG - No Mount Point SDA4 df umount /mnt_cron cd .............. thank you so much best regards -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 05/03/2013 05:30 PM, Carl Hartung wrote:

I'm curious to know, however: What steps do you take to arrive at this point? Do you boot to rescue mode using the installation DVD?

- booted with OpenSUSE Live DVD ...................... So, I gather you are taking 'live' snapshots from a running system and storing them in the running system, albeit on devices that aren't normally mounted. YMMV but I don't do it this way. ___________________________________________ - indeed : do this throughout the day .............. - also use External usb Disk : consider these 'terrific' :)) ......................

I keep backups completely isolated from running systems in case of catastrophic hardware failures. Why? Because I once experienced a full system meltdown (literally) and learned a very hard lesson. However, if this method works for you and you're happy with it, don't change a thing. One other item: I use the flags '-avAX' followed by '--delete' without the '--delete-after'. >

........................

hth & regards, Carl

- much appreciated : thank you best regards -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Sun, 5 May 2013 03:21:02 +0200 (CEST) Carlos E. R. wrote:

On Friday, 2013-05-03 at 10:30 -0400, Carl Hartung wrote:

...

One other item: I use the flags '-avAX' followed by '--delete' without the '--delete-after'. More below.

I once destroyed a system when attempting to do backup. I was using - --delete and I wrote the wrong source and target directories - as both did not match, the destination was deleted.

Ouch! You're supposed to "engage your brain" [as in 'engaging' gears in a transmission] *before* tapping the 'Enter' key! ;-) This is precisely why I always, always, always _stop_ beforehand to read back and contemplate, to confirm, my typing _before_ committing to the operation. And, when this strategy fails, there is always the extra copy, right? So, when something horrible _does_ happen one can at least restore to a consistent state? Thanks for bringing up such a valuable point, Carlos. The partitions I omitted earlier, for "clarity," were precisely those containing the redundant copies of my backups. :-) regards, Carl -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Sun, 05 May 2013 06:35:40 -0400 Anton Aylward wrote:

Carl Hartung said the following on 05/05/2013 12:31 AM:

...

On Sun, 5 May 2013 03:21:02 +0200 (CEST) Carlos E. R. wrote:

...

On Friday, 2013-05-03 at 10:30 -0400, Carl Hartung wrote:

...

One other item: I use the flags '-avAX' followed by '--delete' without the '--delete-after'. More below.

I once destroyed a system when attempting to do backup. I was using - --delete and I wrote the wrong source and target directories - as both did not match, the destination was deleted.

Ouch!

You're supposed to "engage your brain" [as in 'engaging' gears in a transmission] *before* tapping the 'Enter' key!

;-)

This is precisely why I always, always, always _stop_ beforehand to read back and contemplate, to confirm, my typing _before_ committing to the operation.

Indeed, but it helps if the documentation/man page is clear about such matters. personally I think the section on "delete" is so badly written that I don't understand it however much I stop and contemplate.

My "commit" is simply not to use it.

Perhaps I am crippling myself by not using the capabilities of rsync, but then again, I avoid a misunderstanding and hence avoid wiping my system.

I highly doubt you're "crippling" yourself, Anton. Obviously, other tools can be used. I'll say this, though: Once you've familiarized yourself with rsync enough to use it every day, it becomes an indispensable part of your toolkit. The biggest problem I had was figuring out that the last slash ('/') in the *source* path acts like a "container." From rsync's perspective, it determines at the outset what is (and is not) written to the target: /source/xyz Here, the last slash "contains" the directory "xyz". Used in this form, rsync will copy the directory "xyz" and it's contents to the target. /source/xyz/ Here, the last slash does _not_ "contain" the directory "xyz". Used in this form, only the contents of "xyz" are copied to the target. /target/xyz rsync treats both of these target paths equally, i.e. /target/xyz/ the trailing slash is implied when omitted. This is comparable to the behavior of 'cp', 'rm', 'rmdir', 'mv', etc. Also, I really only use the following options on a regular basis: -a, --archive archive mode Replicates the source directory, recursively, preserving owners, timestamps and permissions (sort of like 'tar' but the target is another directory instead of a file.) Add 'A' and 'X' flags to preserve, respectively, ACLs and extended attributes. Caveat: It operates exactly as stated above when invoked as root. When invoked by a regular user, it writes to the target as that user (local system.) -r, --recursive recurse into directories Recurse and preserve permissions, but not owners or timestamps (i.e. write as current user with current timestamps.) -v, --verbose increase verbosity -A, --acls preserve ACLs -X, --xattrs preserve extended attributes -n, --dry-run perform a trial run with no changes made I often use this option with redirection or piping to casually examine the results of a contemplated rsync operation. It causes rsync to undertake the operation without actually writing any changes to disk. piping example: rsync -rvn --delete /source/ /target | less redirection example: rsync -rvn --delete /source/ /target >somefilename.txt --delete delete extraneous files from destination --delete-during receiver deletes during the transfer --del an alias for --delete-during Notes regarding the above 'flavors' of '--delete': '--delete' deletes items from the target that are not (i.e. "no longer") present at the source. In my experience, this is functionally equivalent to '--delete-during', where writing of new and changed items and deletions occur sequentially, in the order they are evaluated. Omitting '--delete' actually precludes true synchronization since this leaves items untouched at the target when they're not (i.e. are "no longer") present at the source. This is akin to an incremental backup, where items are added and updated, never deleted. Some notes regarding the other 'flavors' of '--delete' which I actually don't ever use: --delete-before deletions occur before new and changed files are written --delete-delay deletions occur after new and changed files are written --delete-after deletions occur after new and changed files are written --delete-excluded delete exclusions from the destination This last one could be used to remove items from a target that were previously, inadvertently, copied for lack of an earlier '--exclude' directive. From what I've read, the other three are meant to satisfy more 'fringe' or obscure scenarios where remote targets have other than native *nix filesystem types, older rsync versions or rsync implementations running on other operating systems. I could be wrong, but that's my take on them ATM. regards, Carl -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carl Hartung said the following on 05/06/2013 07:27 PM:

On Sun, 05 May 2013 06:35:40 -0400 Anton Aylward wrote:

...

[...]

My "commit" is simply not to use it.

Ah, I meant the "delete" options.

...

Perhaps I am crippling myself by not using the capabilities of rsync, but then again, I avoid a misunderstanding and hence avoid wiping my system.

I highly doubt you're "crippling" yourself, Anton. Obviously, other tools can be used. I'll say this, though: Once you've familiarized yourself with rsync enough to use it every day, it becomes an indispensable part of your toolkit.

Actually I use rsync a lot, for moving stuff, usually in preference to 'cp' for a variety of reasons. Regular readers will recall that I use LVM. I often find it necessary to 'split' a nearly full file system. I can create a logical volume (aka partition) and put a file system there then move part of a tree onto it and after deleting the tree from the original FS mount the new partition and the tree there; adjusting the /etc/fstab to make it permanent. So you'd think I'd use the 'delete' options. Well, no. They aren't clear to me.

The biggest problem I had was figuring out that the last slash ('/') in the *source* path acts like a "container." From rsync's perspective, it determines at the outset what is (and is not) written to the target:

I think that is a learning hurdle for many people :-)

I often use this option with redirection or piping to casually examine the results of a contemplated rsync operation. It causes rsync to undertake the operation without actually writing any changes to disk.

piping example:

rsync -rvn --delete /source/ /target | less

Does that tell you what its going to delete?

redirection example:

rsync -rvn --delete /source/ /target >somefilename.txt

--delete delete extraneous files from destination

Not the source? So what does that mean?

--delete-during receiver deletes during the transfer

Does that only work, therefore, if you have a daemon 'receiver'? What does 'during' amount to>? Does that mean if an file of that name exists it is deleted rather than simply over-written? What about files that exist at the target but not at the source? Are they deleted?

--del an alias for --delete-during

Notes regarding the above 'flavors' of '--delete':

'--delete' deletes items from the target that are not (i.e. "no longer") present at the source. In my experience, this is functionally equivalent to '--delete-during', where writing of new and changed items and deletions occur sequentially, in the order they are evaluated.

My worry about this: it seems to be a sync of source and destination with the source being authoritative. That last sentence poses a problem in my mind about building the set of files ...

Omitting '--delete' actually precludes true synchronization since this leaves items untouched at the target when they're not (i.e. are "no longer") present at the source. This is akin to an incremental backup, where items are added and updated, never deleted.

Or you could call it a 'copy' from source to destination. Everything from the source gets copied to the destination. If there is already stuff at the destination then its there and stays there. Call it "accumulation'.

Some notes regarding the other 'flavors' of '--delete' which I actually don't ever use:

--delete-before deletions occur before new and changed files are written

--delete-delay deletions occur after new and changed files are written

--delete-after deletions occur after new and changed files are written

--delete-excluded delete exclusions from the destination

This last one could be used to remove items from a target that were previously, inadvertently, copied for lack of an earlier '--exclude' directive.

All this just confuses me further.

From what I've read, the other three are meant to satisfy more 'fringe' or obscure scenarios where remote targets have other than native *nix filesystem types, older rsync versions or rsync implementations running on other operating systems. I could be wrong, but that's my take on them ATM.

-- "If you want to know what a man's like, take a good look at how he treats his inferiors, not his equals." -- J.K. Rowling, _Harry Potter and the Goblet of Fire -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. said the following on 05/06/2013 10:38 PM:

On Monday, 2013-05-06 at 21:36 -0400, Anton Aylward wrote:

...

Carl Hartung said the following on 05/06/2013 07:27 PM:

...

...

piping example:

rsync -rvn --delete /source/ /target | less

...

Does that tell you what its going to delete?

Yes, but mixed with the thousands of other things it does. And as there is no "--dry-run" above, by the time you notice it may be too late.

Ouch! As in *O*U*C*H*!*!*!

...

...

redirection example:

rsync -rvn --delete /source/ /target >somefilename.txt

--delete delete extraneous files from destination

...

Not the source? So what does that mean?

Not the source :-)

So this isn't doing a "mv" type of operation where the source is deleted after copy. I use rsync as a form of "mv" where "mv" isn't up to it.

Ok, I don't know the differences between the different variations of "delete" (after, during, etc). The important thing is that it deletes things in the target directory. What? An example is easier.

Suppose you do a backup of a directory.

Then you delete some files in the source directory, create new files, change other files - the typical use of a computer.

A week later you do another backup of the same source and same target. Changed and new files will be copied to the destination, but the deleted files... what? They will simply remain in the backup copy.

Oh YES! Isn't that what a 'backup' means? How else might I be able to recover deleted files?

If you have to do a restore, it will be incorrect, because those files you deleted intentionally will be recovered.

Well, that's one view. Still have a copy of something that was deleted and getting it back is another.

Yes, but there is a better way - for backups.

Right. Install a carousel with tape changer and handler and ...

You can have a backup directory for week 1, another for week 2, etc. All of them contain only what was present at the time they were made, no more, no less. Deleted files during the week are in week_1 directory but not in week_2.

Or RCS or Subversion or Git. Or many more variations.

There are scripts that handle this type of backups automatically, like "rsnapshot". But the functionality is that of "--link-dest" in rsync.

Basically:

rsync $OPTIONS --link-dest=$PREVIOUS $SOURCE/ $DESTINATION

Well that isn't a lot of use for the use-cases I have. My use of rsync is across machines or moving a tree to a new partition to free up space. -- Associate yourself with men of good quality if you esteem your own reputation; for 'tis better to be alone than in bad company. --George Washington -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Mon, 06 May 2013 21:36:57 -0400 Anton Aylward wrote:

Carl Hartung said the following on 05/06/2013 07:27 PM:

...

On Sun, 05 May 2013 06:35:40 -0400 Anton Aylward wrote:

...

[...]

My "commit" is simply not to use it.

Ah, I meant the "delete" options.

That's a /huge/ difference, isn't it! ;-) <snipped>

Actually I use rsync a lot, for moving stuff, usually in preference to 'cp' for a variety of reasons.

Regular readers will recall that I use LVM. I often find it necessary to 'split' a nearly full file system. I can create a logical volume (aka partition) and put a file system there then move part of a tree onto it and after deleting the tree from the original FS mount the new partition and the tree there; adjusting the /etc/fstab to make it permanent.

So you'd think I'd use the 'delete' options. Well, no. They aren't clear to me.

To be fair, the '--delete' option isn't needed for the operation you've just described. There's nothing 'extraneous' to delete in the newly created logical volume and filesystem.

...

The biggest problem I had was figuring out that the last slash ('/') in the *source* path acts like a "container." From rsync's perspective, it determines at the outset what is (and is not) written to the target:

I think that is a learning hurdle for many people :-)

I can't for the life of me figure out /why/ that's true, after the fact, but it is.

...

I often use this option with redirection or piping to casually examine the results of a contemplated rsync operation. It causes rsync to undertake the operation without actually writing any changes to disk.

piping example:

rsync -rvn --delete /source/ /target | less

Does that tell you what its going to delete?

Yes. The verbose output (standard out) from this 'dry run' will be piped to 'less' instead of scrolling up and off the screen. Since 'dry run' effectively sends all writes to /dev/null, you have the benefit of previewing beforehand exactly what will be written (deletes and all) if the 'dry run' option is removed.

...

redirection example:

rsync -rvn --delete /source/ /target >somefilename.txt

'-rvn' is "verbose recursion and do a 'dry run'" As in the previous example of piping standard out to 'less', this method lets you open the text file and examine the consequences of a contemplated rsync operation in advance. I use this 'flavor' when the number of files is huge ('less' has it's limits.) I also often use this method to generate complete lists of the contents of directories and even partitions, especially when I'm studying problems ... when "what 'lives' where" sorts of questions come up.

...

--delete delete extraneous files from destination

Not the source? So what does that mean?

'--delete' options never affect the source, only the target.

...

--delete-during receiver deletes during the transfer

Does that only work, therefore, if you have a daemon 'receiver'?

No, it works the same way locally, too.

What does 'during' amount to?

I think all these 'flavors' of the '--delete' option actually yield the identical results, but they allow tailoring one's operations in relation to bandwidth. "during" implies and acts synchronously, which requires healthy enough bandwidth that the 'sender' can know in a timely fashion what is happening at the 'receiver' end. I think the "before", "delayed" and "after" 'flavors' are more asynchronous and are there to provide for situations where bandwidth is less abundant.

Does that mean if an file of that name exists

exists where?

it is deleted rather than simply over-written?

If a file exists only at the target and the '--delete' option is exercised, that file _will_ be deleted from the target.

What about files that exist at the target but not at the source? Are they deleted?

see above

...

--del an alias for --delete-during

Notes regarding the above 'flavors' of '--delete':

'--delete' deletes items from the target that are not (i.e. "no longer") present at the source. In my experience, this is functionally equivalent to '--delete-during', where writing of new and changed items and deletions occur sequentially, in the order they are evaluated.

My worry about this: it seems to be a sync of source and destination with the source being authoritative.

Absolutely! This is what you do if you want to create an exact copy of the 'master.' Don't exercise '--delete' if that isn't what you want.

That last sentence poses a problem in my mind about building the set of files ...

It's always a one-way street. rsync _never_ writes to the 'source', only to the 'target'. A hypothetical: Let's say you've inadvertently allowed two previously identical branches of a project to diverge from each other (I know this _never_ ever happens) such that there's a different mix of older and newer files in each. The only way out of this dilemma is to run a two pass synchronization, once in each direction, _without_ exercising the '--delete' option. This picks up the newest versions of each object in each directory and copies it to the other -- overwriting older files on the target when they exist.

...

Omitting '--delete' actually precludes true synchronization since this leaves items untouched at the target when they're not (i.e. are "no longer") present at the source. This is akin to an incremental backup, where items are added and updated, never deleted.

Or you could call it a 'copy' from source to destination. Everything from the source gets copied to the destination. If there is already stuff at the destination then its there and stays there. Call it "accumulation'.

'Cumulative' mode ;-) <snipped>

...

--delete-excluded delete exclusions from the destination

This last one could be used to remove items from a target that were previously, inadvertently, copied for lack of an earlier '--exclude' directive.

All this just confuses me further.

Okay, forget the three I've snipped out (at least until you're running over two paper cups and a string) but please focus on this last very useful one, '--delete-excluded': Here's the use case scenario: First pass: rsync -rv /set/of/files/ /backup You then realize all your C sources got copied over, too, and that isn't what you wanted. Here's what you do: rsync -rv --exclude='*.c' --delete-excluded /set/of/files/ /backup The '--exclude=' part defines the set of files that are _not_ to be copied to the target. The '--delete-excluded' bit tells rsync to delete files from the target which match the given '--exclude' criteria. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sunday, 2013-05-05 at 00:31 -0400, Carl Hartung wrote:

On Sun, 5 May 2013 03:21:02 +0200 (CEST) Carlos E. R. wrote:

...

On Friday, 2013-05-03 at 10:30 -0400, Carl Hartung wrote:

...

One other item: I use the flags '-avAX' followed by '--delete' without the '--delete-after'. More below.

I once destroyed a system when attempting to do backup. I was using - --delete and I wrote the wrong source and target directories - as both did not match, the destination was deleted.

Ouch!

You're supposed to "engage your brain" [as in 'engaging' gears in a transmission] *before* tapping the 'Enter' key!

;-)

Oh, absolutely, but mistakes do happen. Actually, my intention as I recall, was to test rsync procedures to find out the correct incantation to use, or to verify the backup, or something of the sort.

This is precisely why I always, always, always _stop_ beforehand to read back and contemplate, to confirm, my typing _before_ committing to the operation.

And I did... O:-)

And, when this strategy fails, there is always the extra copy, right? So, when something horrible _does_ happen one can at least restore to a consistent state?

Yes, but it was much older. You see, I was doing a backup... it was not done yet. The disaster hit at the worst point possible. I don't remember now the exact details, how I recovered. I lost /var and other things randomly. I think I copied things from another system, then used rpm verify to replace things, or restored the old backup and then upgraded. I posted about it somewhere. Ah, here: https://forums.opensuse.org/english/get-technical-help-here/applications/477...

Thanks for bringing up such a valuable point, Carlos. The partitions I omitted earlier, for "clarity," were precisely those containing the redundant copies of my backups. :-)

Not every body learns from other people disasters ;-) Which reminds me that I want to buy a sata hd tray and a pair of hd, to plug in and do backups. - -- Cheers, Carlos E. R. (from 12.1 x86_64 "Asparagus" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) iEYEARECAAYFAlGGXVkACgkQtTMYHG2NR9VSKgCcDitK8V5NhMVPvVzHlQipofNT FNoAnAzUr9KWcRhec3g/7Xf2BH1wgJhs =0AKl -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org