[opensuse] Rsync question
Hello list-members, I have a question about Rsync. Like several people here, I use an external USB disk to make daily backups of my critical data. For this, I have written a shell script which mounts the disk as necessary and then calls rsync to do the actual backup. This works very well, except that if I delete a file in my home directory, have rsync do its thing at night, then the next morning that deleted file is gone from my backup too, so I can only go back one day. This is nice for a disk crash, but not for restoring data that was deleted 4 days ago, for instance. Can someone tell me if it is possible to have rsync retain the data in the backup for say another 14 or 30 days before purging them? I have looked at the man page, but haven't found an answer to my question there. Does someone have an idea, please? I use the following command in my shell script to do the rsync backup: $RSYNC -alvz --delete --exclude=Cache --exclude=.beagle --exclude=trash --exclude=.thumbnails /home/<username> $MPOINT --log-file=/var/log/rsync Thanks in advance, Joop -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
J Beris pecked at the keyboard and wrote:
Hello list-members,
I have a question about Rsync. Like several people here, I use an external USB disk to make daily backups of my critical data. For this, I have written a shell script which mounts the disk as necessary and then calls rsync to do the actual backup.
This works very well, except that if I delete a file in my home directory, have rsync do its thing at night, then the next morning that deleted file is gone from my backup too, so I can only go back one day. This is nice for a disk crash, but not for restoring data that was deleted 4 days ago, for instance. Can someone tell me if it is possible to have rsync retain the data in the backup for say another 14 or 30 days before purging them? I have looked at the man page, but haven't found an answer to my question there. Does someone have an idea, please?
I use the following command in my shell script to do the rsync backup: $RSYNC -alvz --delete --exclude=Cache --exclude=.beagle --exclude=trash --exclude=.thumbnails /home/<username> $MPOINT --log-file=/var/log/rsync
Thanks in advance,
Joop
The --delete means to delete files on the remote that are not on the local machine. That is why they are gone on the backup drive. -- Ken Schneider SuSe since Version 5.2, June 1998 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
I use the following command in my shell script to do the rsync backup: $RSYNC -alvz --delete --exclude=Cache --exclude=.beagle -- exclude=trash --exclude=.thumbnails /home/<username> $MPOINT --log- file=/var/log/rsync
The --delete means to delete files on the remote that are not on the local machine. That is why they are gone on the backup drive.
Hi Ken, Thanks for the fast reply. Yes, I know what --delete does, and I *want* rsync to delete files from the target directory...only not right away, but after 14 days or 30 days or so. If I don't use --delete, then rsync will just keep adding things to the backup, but never removing them, as I understand it. Am I wrong about that? Joop -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Monday 2008-07-28 at 14:23 +0200, J Beris wrote:
Thanks for the fast reply. Yes, I know what --delete does, and I *want* rsync to delete files from the target directory...only not right away, but after 14 days or 30 days or so. If I don't use --delete, then rsync will just keep adding things to the backup, but never removing them, as I understand it.
Am I wrong about that?
It is done differently. The new backup is created in another directory, where the files that have not changed are hardlinks to the previous backup, new files are copied, and the deleted files neither. rsync does that, and many scripts make use of that to keep several copies and purge old copies. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4-svn0 (GNU/Linux) iD8DBQFIjb9StTMYHG2NR9URAsJqAJ970fqYLRtUk39mwPIh7TdWWwj11ACgjt6T /FeNMytluwARGfd4XBnOKoM= =NIka -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
J Beris pecked at the keyboard and wrote:
I use the following command in my shell script to do the rsync backup: $RSYNC -alvz --delete --exclude=Cache --exclude=.beagle -- exclude=trash --exclude=.thumbnails /home/<username> $MPOINT --log- file=/var/log/rsync
The --delete means to delete files on the remote that are not on the local machine. That is why they are gone on the backup drive.
Hi Ken,
Thanks for the fast reply. Yes, I know what --delete does, and I *want* rsync to delete files from the target directory...only not right away, but after 14 days or 30 days or so. If I don't use --delete, then rsync will just keep adding things to the backup, but never removing them, as I understand it.
Am I wrong about that?
Partially. rsync does not currently support this feature. Perhaps if you email the author with a feature request it could be added. Or perhaps an incremental type feature could be implemented. -- Ken Schneider SuSe since Version 5.2, June 1998 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
J Beris wrote:
I have a question about Rsync. Like several people here, I use an external USB disk to make daily backups of my critical data. For this, I have written a shell script which mounts the disk as necessary and then calls rsync to do the actual backup.
This works very well, except that if I delete a file in my home directory, have rsync do its thing at night, then the next morning that deleted file is gone from my backup too, so I can only go back one day. This is nice for a disk crash, but not for restoring data that was deleted 4 days ago, for instance. Can someone tell me if it is possible to have rsync retain the data in the backup for say another 14 or 30 days before purging them? I have looked at the man page, but haven't found an answer to my question there. Does someone have an idea, please?
Don't use rsync directly; use one of the backup systems that already provide the functionality you want (and use rsync internally). I use dirvish, I know of rdiff-backup, I expect there are others. I don't know which are in current opensuse distributions. Cheers, Dave -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Don't use rsync directly; use one of the backup systems that already provide the functionality you want (and use rsync internally).
I use dirvish, I know of rdiff-backup, I expect there are others. I don't know which are in current opensuse distributions.
Hi Dave, Thanks for the fast reply. I'll look into dervish and/or rdiff-backup. Perhaps those will provide me with what I need. Thanks, Joop
On Monday 28 July 2008 06:18:20 am Dave Howorth wrote:
J Beris wrote:
Can someone tell me if it is possible to have rsync retain the data in the backup for say another 14 or 30 days before purging them? I have looked at the man page, but haven't found an answer to my question there. Does someone have an idea, please?
Don't use rsync directly; use one of the backup systems that already provide the functionality you want (and use rsync internally).
I use dirvish, I know of rdiff-backup, I expect there are others. I don't know which are in current opensuse distributions.
Have a look at storebackup as well. -- Don -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Dne Monday 28 of July 2008 14:27:58 Don Raboud napsal(a):
On Monday 28 July 2008 06:18:20 am Dave Howorth wrote:
J Beris wrote:
Can someone tell me if it is possible to have rsync retain the data in the backup for say another 14 or 30 days before purging them? I have looked at the man page, but haven't found an answer to my question there. Does someone have an idea, please?
Don't use rsync directly; use one of the backup systems that already provide the functionality you want (and use rsync internally).
I use dirvish, I know of rdiff-backup, I expect there are others. I don't know which are in current opensuse distributions.
Have a look at storebackup as well.
... and rsnapshot :) -- Best regards / s pozdravem Petr Uzel, Packages maintainer --------------------------------------------------------------------- SUSE LINUX, s.r.o. e-mail: petr.uzel@suse.cz Lihovarská 1060/12 tel: +420 284 028 964 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hi! Am Montag, 28. Juli 2008 14:18 schrieb Dave Howorth:
Don't use rsync directly; use one of the backup systems that already provide the functionality you want (and use rsync internally).
I use dirvish, I know of rdiff-backup, I expect there are others. I don't know which are in current opensuse distributions.
Rsnapshot is another option. Regards, Matthias -- Matthias Bach www.marix.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hi, On 07/28/2008 02:18 PM, Dave Howorth wrote:
Don't use rsync directly; use one of the backup systems that already provide the functionality you want (and use rsync internally).
I use dirvish, I know of rdiff-backup, I expect there are others. I don't know which are in current opensuse distributions.
rdiff-backup does not actually use rsync ;)
My favourite one is rsnapshot - http://www.rsnapshot.org/ , it's part of the
distribution.
If you feel it's too complicated to set up, take a look at rsnap -
http://daniel.lorch.cc/projects/rsnap/ , a package for OpenSUSE is available
from my repository in the OpenSUSE Build Service.
Bye,
LenZ
--
------------------------------------------------------------------
Lenz Grimmer
Hello everyone, Thanks a lot for all the suggestions, I now have several programs that I will examine, but most of them are able to do what I want. I just need to find the one that I find easiest to work with. Thanks for your suggestions, Joop
Lenz Grimmer wrote:
rdiff-backup does not actually use rsync ;)
Yes, I thought that little factoid was more than the OP needed to know :) But it's worth bearing in mind, because librsync produces different results to rsync in some hopefully unusual circumstances. But then so do different versions of rsync :) Cheers, Dave -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----Oorspronkelijk bericht----- Van: Dave Howorth [mailto:dhoworth@mrc-lmb.cam.ac.uk] Verzonden: dinsdag 29 juli 2008 12:42 Aan: opensuse@opensuse.org Onderwerp: Re: [opensuse] Rsync question
Lenz Grimmer wrote:
rdiff-backup does not actually use rsync ;)
Yes, I thought that little factoid was more than the OP needed to know :)
What's this? Witholding vital information from people? Since when do *we* have to hide what a command or program actually does? ;-) Joop
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Monday 2008-07-28 at 14:09 +0200, J Beris wrote:
I have a question about Rsync. Like several people here, I use an external USB disk to make daily backups of my critical data. For this, I have written a shell script which mounts the disk as necessary and then calls rsync to do the actual backup.
This works very well, except that if I delete a file in my home directory, have rsync do its thing at night, then the next morning that deleted file is gone from my backup too, so I can only go back one day. This is nice for a disk crash, but not for restoring data that was deleted 4 days ago, for instance. Can someone tell me if it is possible to have rsync retain the data in the backup for say another 14 or 30 days before purging them? I have looked at the man page, but haven't found an answer to my question there. Does someone have an idea, please?
rsync $OPTIONS --link-dest=$YESTERDAY $WHAT $TODAY But it would probably be easier to use any of the several existing scripts that handle that situation nicely. rdiff-backup the previous backups are stored as rdifs. I think it will go very slow over USB. rsnapshot rsync style. http://www.dirvish.org/ pdumpfs (http://0xcc.net/pdumpfs) - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4-svn0 (GNU/Linux) iD8DBQFIjb23tTMYHG2NR9URAkGiAKCY03C/mSPYMnsosL/MRoT4QZHzFwCcCmOZ VtZkhKjac6JoagFR/bkuzSw= =Ak4S -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Monday 28 July 2008 13:38:12 Carlos E. R. wrote:
The Monday 2008-07-28 at 14:09 +0200, J Beris wrote:
I have a question about Rsync. Like several people here, I use an external USB disk to make daily backups of my critical data. For this, I have written a shell script which mounts the disk as necessary and then calls rsync to do the actual backup.
This works very well, except that if I delete a file in my home directory, have rsync do its thing at night, then the next morning that deleted file is gone from my backup too, so I can only go back one day. This is nice for a disk crash, but not for restoring data that was deleted 4 days ago, for instance. Can someone tell me if it is possible to have rsync retain the data in the backup for say another 14 or 30 days before purging them? I have looked at the man page, but haven't found an answer to my question there. Does someone have an idea, please?
rsync $OPTIONS --link-dest=$YESTERDAY $WHAT $TODAY
But it would probably be easier to use any of the several existing scripts that handle that situation nicely.
rdiff-backup the previous backups are stored as rdifs. I think it will go very slow over USB.
rsnapshot rsync style. http://www.dirvish.org/ pdumpfs (http://0xcc.net/pdumpfs)
-- Cheers, Carlos E. R.
This is the script I have used for several years. It works well, and is based on Mike Rubel's script. I believe things like rsnapshot are gui frontends for something similar. The incremental functionality comes from cp being able to make hardlinks to files, so even though rsync has deleted a file in today's backup, the file still exists because there is a hardlink in yesterday's copy. The file is lost only when the hardlink falls off the end (in my case, after 28 days). The script runs as a nightly cronjob by root. 'lacie' is the name of my external USB backup drive. ---------------- #!/bin/bash # step 1: delete the oldest snapshot, if it exists if [ -d /media/lacie/daily.28 ] ; then rm -rf /media/lacie/daily.28 ; fi ; # step 2: shift the middle snapshot(s) back by one, if they exist if [ -d /media/lacie/daily.27 ] ; then mv /media/lacie/daily.27 /media/lacie/daily.28 ; fi ; if [ -d /media/lacie/daily.26 ] ; then mv /media/lacie/daily.26 /media/lacie/daily.27 ; fi ; if [ -d /media/lacie/daily.25 ] ; then mv /media/lacie/daily.25 /media/lacie/daily.26 ; fi ; if [ -d /media/lacie/daily.24 ] ; then mv /media/lacie/daily.24 /media/lacie/daily.25 ; fi ; if [ -d /media/lacie/daily.23 ] ; then mv /media/lacie/daily.23 /media/lacie/daily.24 ; fi ; if [ -d /media/lacie/daily.22 ] ; then mv /media/lacie/daily.22 /media/lacie/daily.23 ; fi ; if [ -d /media/lacie/daily.21 ] ; then mv /media/lacie/daily.21 /media/lacie/daily.22 ; fi ; if [ -d /media/lacie/daily.20 ] ; then mv /media/lacie/daily.20 /media/lacie/daily.21 ; fi ; if [ -d /media/lacie/daily.19 ] ; then mv /media/lacie/daily.19 /media/lacie/daily.20 ; fi ; if [ -d /media/lacie/daily.18 ] ; then mv /media/lacie/daily.18 /media/lacie/daily.19 ; fi ; if [ -d /media/lacie/daily.17 ] ; then mv /media/lacie/daily.17 /media/lacie/daily.18 ; fi ; if [ -d /media/lacie/daily.16 ] ; then mv /media/lacie/daily.16 /media/lacie/daily.17 ; fi ; if [ -d /media/lacie/daily.15 ] ; then mv /media/lacie/daily.15 /media/lacie/daily.16 ; fi ; if [ -d /media/lacie/daily.14 ] ; then mv /media/lacie/daily.14 /media/lacie/daily.15 ; fi ; if [ -d /media/lacie/daily.13 ] ; then mv /media/lacie/daily.13 /media/lacie/daily.14 ; fi ; if [ -d /media/lacie/daily.12 ] ; then mv /media/lacie/daily.12 /media/lacie/daily.13 ; fi ; if [ -d /media/lacie/daily.11 ] ; then mv /media/lacie/daily.11 /media/lacie/daily.12 ; fi ; if [ -d /media/lacie/daily.10 ] ; then mv /media/lacie/daily.10 /media/lacie/daily.11 ; fi ; if [ -d /media/lacie/daily.09 ] ; then mv /media/lacie/daily.09 /media/lacie/daily.10 ; fi ; if [ -d /media/lacie/daily.08 ] ; then mv /media/lacie/daily.08 /media/lacie/daily.09 ; fi ; if [ -d /media/lacie/daily.07 ] ; then mv /media/lacie/daily.07 /media/lacie/daily.08 ; fi ; if [ -d /media/lacie/daily.06 ] ; then mv /media/lacie/daily.06 /media/lacie/daily.07 ; fi ; if [ -d /media/lacie/daily.05 ] ; then mv /media/lacie/daily.05 /media/lacie/daily.06 ; fi ; if [ -d /media/lacie/daily.04 ] ; then mv /media/lacie/daily.04 /media/lacie/daily.05 ; fi ; if [ -d /media/lacie/daily.03 ] ; then mv /media/lacie/daily.03 /media/lacie/daily.04 ; fi ; if [ -d /media/lacie/daily.02 ] ; then mv /media/lacie/daily.02 /media/lacie/daily.03 ; fi ; if [ -d /media/lacie/daily.01 ] ; then mv /media/lacie/daily.01 /media/lacie/daily.02 ; fi ; # step 3: make a hard-link-only (except for dirs) copy of the latest snapshot. # if that exists if [ -d /media/lacie/daily.00 ] ; then cp -al /media/lacie/daily.00 /media/lacie/daily.01 ; fi ; # step 4: rsync from the system into the latest snapshot # - this is the bit that does the actual backup! /usr/bin/rsync -av --no-l --delete --delete-excluded --exclude-from=/home/bob/bin/excludedfiles /etc /home/bob /home/hils /home/leon /home/play /home/Shared_Documents /media/lacie/daily.00 ---------------- -- Bob Registered Linux User #463880 FSFE Member #1300 GPG-FP: A6C1 457C 6DBA B13E 5524 F703 D12A FB79 926B 994E openSUSE 11.0, Kernel 2.6.25.11-0.1-default, KDE 3.5.9 Intel Celeron 2.53GB, 2GB DDR RAM, nVidia GeForce 7600GS -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hi! Am Dienstag, 29. Juli 2008 20:43 schrieb Bob Williams:
I believe things like rsnapshot are gui frontends for something similar.
Rsnapshot ist a command line application. And AFAIK it is targeted mainly for use in cronjobs or in other event triggered scripts. (Of course a user pushing some button can also be a trigger ;) ) Regards, Matthias -- Matthias Bach www.marix.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (9)
-
Bob Williams -
Carlos E. R. -
Dave Howorth -
Don Raboud -
J Beris -
Ken Schneider -
Lenz Grimmer -
Matthias Bach -
Petr Uzel