Can anyone recommend a backup program/script for home user?
I was just about to setup a raid 1 when I realized I didn't want to really accomplish that. What I want is something to backup my system incase I fubar something. So, rather than dd my drive every night, I'm wondering how I can backup my system where only the changed files get backed up rather than the whole drive. I looked at rsync but I'm not sure if that's what I want. Any advice? Tom
Tom Nielsen wrote:
I was just about to setup a raid 1 when I realized I didn't want to really accomplish that. What I want is something to backup my system incase I fubar something. So, rather than dd my drive every night, I'm wondering how I can backup my system where only the changed files get backed up rather than the whole drive. I looked at rsync but I'm not sure if that's what I want.
Tom, Rsync will do what you want. I prefer to use mirrordir with NFS (because historically I've had to keep differing filesystems in sync - including windoze) and mirrordir did the job (not to mention blisteringly fast). After installing mirrordir (rpm is at http://mirrordir.sourceforge.net) the following script, run from cron (as root), mirrors the primary share to backup share: # Script to mirror the /share area on primary to backup drive # Run this on backup machine NOW=`date +%H%I%d%m%Y` echo "mirroring started at $NOW ..." > /tmp/mirr_report mount primary_IP_address:/share /mnt sleep 3 MOUNTED=`df | grep share`
/tmp/mirr_report if test -n "$MOUNTED" ; then /usr/bin/mirrordir -v /mnt /share >> /tmp/mirr_report fi umount /mnt
Also make sure that on the primary machine the share is exported with full access for the backup machine. Like an entry in etc/exports on the primary machine: /share backup_machine(rw,no_root_squash) For this reason only use this on a protected network (your home network behind a firewall should be OK). Damian
--- Damian O'Hara <damian@cognoscente.uk.com> wrote:
Tom Nielsen wrote:
I was just about to setup a raid 1 when I realized I didn't want to really accomplish that. What I want is something to backup my system incase I fubar something. So, rather than dd my drive every night, I'm wondering how I can backup my system where only the changed files get backed up rather than the whole drive. I looked at rsync but I'm not sure if that's what I want.
You might want to look at the following: http://linuxgazette.net/issue78/adam.html#keyfiles :) -- Thomas Adam __________________________________ Do you Yahoo!? Free Pop-Up Blocker - Get it now http://companion.yahoo.com/
-----Original Message----- From: Tom Nielsen <tom@neuro-logic.com> To: Suse <suse-linux-e@suse.com> Date: Fri, 21 Nov 2003 22:29:55 -0800 Subject: [SLE] Can anyone recommend a backup program/script for home user?
I was just about to setup a raid 1 when I realized I didn't want to really accomplish that. What I want is something to backup my system incase I fubar something. So, rather than dd my drive every night, I'm wondering how I can backup my system where only the changed files get backed up rather than the whole drive. I looked at rsync but I'm not sure if that's what I want.
Any advice?
Tom
That's what rsync is used for. I use it at work to keep two servers in sync and it works great. Ken Schneider
On Saturday 22 November 2003 07:29, Tom Nielsen wrote:
I was just about to setup a raid 1 when I realized I didn't want to really accomplish that. What I want is something to backup my system incase I fubar something. So, rather than dd my drive every night, I'm wondering how I can backup my system where only the changed files get backed up rather than the whole drive. I looked at rsync but I'm not sure if that's what I want.
Any advice?
Well, it all depends on where you intend to do the backup. If you have a scsi tape media, then there are tons of backup programs to use. Everywhere from dump, to ... well, you get the picture. Iff, on the other hand, you have a similar setup to most users. You have a normal CD-RW and want to use it to backup your home drive, and btw. there's no need to backup the system, except for the changes to the original system, which can be done best with SuSE's backup tool. You can make it write the files, already prepared for burning to a CD. If you have lot of diskspace, this can also be your normal backup tool To backup your home directory, I suggest cdbkup. It's a sourceforge project, and does full and incrimental backups to a CD. I used it to backup my home directories, before I made an upgrade to SuSE 9.0.
The Saturday 2003-11-22 at 18:15 +0100, Örn Hansen wrote:
Well, it all depends on where you intend to do the backup. If you have a scsi tape media, then there are tons of backup programs to use. Everywhere from dump, to ... well, you get the picture. Iff, on the other hand, you have a similar setup to most users. You have a normal CD-RW and want to use it to backup your home drive, and btw. there's no need to backup the system, except for the changes to the original system, which can be done best with SuSE's backup tool. You can make it write the files, already prepared for burning to a CD. If you have lot of diskspace, this can also be your normal backup tool
It is an incomplete tool, at least on SuSE 8.2. - It doesn't save all configuration files, only those that existed already on the rpms and were modified: thus, new configuration files are not updated. - It doesn't save the rpm patches. - It can be configured to save the rest, but that means _everything_ not on the rpms (the SuSE dvd). - You can configure paths not to save - but the configuration is not saved for the next run. - It creates a copy of everything that is going to be saved, in /tmp, not checking if there is enough space, and not asking for an alternative destination. If space is insufficient, it will crash after _hours_ of process. - Output is a big tar file, compressed, made of one tar per package. It is known that a backup media defect will render the full tar.gz file useless. There are more robust formats. - The process is horribly slow, even before starting to create the tar itself, even on a fast computer. The process can not be stopped and continued later - for example, if it crashes for lack of space, you can not reuse the list of files. - You can browse the list of files to be saved; but this list can not be processed externally. There is not a gui tool (even a text gui) allowing to browse paths, see sizes and dates, comparing to the "real" tree, to add or remove files from the selection list. It is a good idea, but incomplete. I use it, so I can complain :-) I have to compare these backup tools with the old pctools backups for old dos, from the eighties, and... they doesn't reach the level. I'm still waiting for something as fast and reliable as that old software.
To backup your home directory, I suggest cdbkup. It's a sourceforge project, and does full and incrimental backups to a CD. I used it to backup my home directories, before I made an upgrade to SuSE 9.0.
I'll have a look... -- Cheers, Carlos Robinson
Tom Nielsen wrote:
I was just about to setup a raid 1 when I realized I didn't want to really accomplish that. What I want is something to backup my system incase I fubar something. So, rather than dd my drive every night, I'm wondering how I can backup my system where only the changed files get backed up rather than the whole drive. I looked at rsync but I'm not sure if that's what I want.
Any advice?
Tom
rsync is used to back up specific directories every half an hour. It has been working well for last three or so weeks. LW999
On Sunday 23 November 2003 08:38, LW999@hotpost wrote:
Tom Nielsen wrote:
I was just about to setup a raid 1 when I realized I didn't want to really accomplish that. What I want is something to backup my system incase I fubar something. So, rather than dd my drive every night, I'm wondering how I can backup my system where only the changed files get backed up rather than the whole drive. I looked at rsync but I'm not sure if that's what I want.
Any advice?
Tom
rsync is used to back up specific directories every half an hour. It has been working well for last three or so weeks.
I've been seeing the responses on this thread, yet I wonder if I would also have success with rsync. I'm on a standalone, using 9.0 pro. I would like to backup modified and new files to a seperate partition. Rsync appears to be for 'sync'ing files over a network. Would it work for me, or is there something better? Bernd -- "If you want to build a ship, don't drum up the men to gather wood, divide the work, and give orders. Instead, teach them to yearn for the vast and endless sea." Antoine de St. Exupery
On Sunday 23 November 2003 12:09 pm, Bernd wrote:
I've been seeing the responses on this thread, yet I wonder if I would also have success with rsync.
I'm on a standalone, using 9.0 pro. I would like to backup modified and new files to a seperate partition. Rsync appears to be for 'sync'ing files over a network.
Would it work for me, or is there something better?
I think rsync is just what you want. Let's assume you want to keep a backup of the /home/foo/ directory. So you make a directory (hopefully on another drive but for the sake of this example) at /usr/ local/foo2/ Then run the command: rsync -auvzr --delete /foo/ /usr/local/foo2/ This is exactly what I use between machines or locally. If to another machine I add the parm: -e ssh to use ssh to do the connection. The above, run every <once in awhile> will keep /foo2/ in sync with / foo/, and it would only copy the new and updated files once you made your initial run at it. The --delete will delete from /foo2/ anything you have deleted from / foo/ Note the use of the trailing slash. It's important when using rsync to copy directories... By the way, if you add an 'n' to the mix (-auvzrn) it will show you the files that it WOULD copy but it won't actually copy them. For testing. man rsync is your friend. -- +---------------------------------------------------------------------------- + + Bruce S. Marshall bmarsh@bmarsh.com Bellaire, MI 11/23/03 12:28 + +---------------------------------------------------------------------------- + 'If truth is beauty, how come no one has their hair' 'done in the library?' -- Lily Tomlin
Thanks for the info Bruce. I'll be running some tests to put rsync through the paces. As per the other responses, it looks like I can automate that via cron, and have a small script mount and umount the partition. Thanks everyone, for the thread. This has been most helpful to me. Bernd -- "If you want to build a ship, don't drum up the men to gather wood, divide the work, and give orders. Instead, teach them to yearn for the vast and endless sea." Antoine de St. Exupery
Original poster here. I failed to mention that I want to backup my whole hard drive including my root and home directories to another drive on my computer. this is a stand alone machine. I was going to go with a RAID1 setup but found that's really not what I wanted to accomplish since if I screw up one system, the other will go too. Will the below command still do what I want? It sounds like it. How would I setup my second drive? Any special partitions? Should I dd the disk first then use rsync or just rsync to begin with? Thanks, Tom On Sun, 2003-11-23 at 09:32, Bruce Marshall wrote:
On Sunday 23 November 2003 12:09 pm, Bernd wrote:
I've been seeing the responses on this thread, yet I wonder if I would also have success with rsync.
I'm on a standalone, using 9.0 pro. I would like to backup modified and new files to a seperate partition. Rsync appears to be for 'sync'ing files over a network.
Would it work for me, or is there something better?
I think rsync is just what you want. Let's assume you want to keep a backup of the /home/foo/ directory. So you make a directory (hopefully on another drive but for the sake of this example) at /usr/ local/foo2/
Then run the command:
rsync -auvzr --delete /foo/ /usr/local/foo2/
This is exactly what I use between machines or locally. If to another machine I add the parm: -e ssh to use ssh to do the connection.
The above, run every <once in awhile> will keep /foo2/ in sync with / foo/, and it would only copy the new and updated files once you made your initial run at it.
The --delete will delete from /foo2/ anything you have deleted from / foo/
Note the use of the trailing slash. It's important when using rsync to copy directories...
By the way, if you add an 'n' to the mix (-auvzrn) it will show you the files that it WOULD copy but it won't actually copy them. For testing.
man rsync is your friend.
-- +---------------------------------------------------------------------------- + + Bruce S. Marshall bmarsh@bmarsh.com Bellaire, MI 11/23/03 12:28 + +---------------------------------------------------------------------------- + 'If truth is beauty, how come no one has their hair' 'done in the library?' -- Lily Tomlin
On Sunday 23 November 2003 18:13 pm, Tom Nielsen wrote:
Original poster here. I failed to mention that I want to backup my whole hard drive including my root and home directories to another drive on my computer. this is a stand alone machine. I was going to go with a RAID1 setup but found that's really not what I wanted to accomplish since if I screw up one system, the other will go too.
Will the below command still do what I want? It sounds like it. How would I setup my second drive? Any special partitions? Should I dd the disk first then use rsync or just rsync to begin with?
To essentially 'mirror' your first drive, you should: 1)Set up the 2nd drive with the same partitions (and size of partitions) as the first. 2) Make duplicate (but different) mount point in your root directory. Like /home2 to be a copy of /home, /root2 to be a copy of '/' etc. 3) Format those partitions with the filesystem of your choice. 4) Mount them to their proper mount points. (and add entries to /etc/ fstab so that they get mounted at boot time. 5) Run your first rsync to each partition. 6) Make a cron job to do it automatically. Be ware that yes, if you are sync'ing every 24 hours and you make a screwup that you don't catch within those 24 hours, you are going to sync the screwup to your backup drives... but no scheme can prevent that unless you can have multiple generations of backups.
Thanks, Tom
On Sun, 2003-11-23 at 09:32, Bruce Marshall wrote:
On Sunday 23 November 2003 12:09 pm, Bernd wrote:
I've been seeing the responses on this thread, yet I wonder if I would also have success with rsync.
I'm on a standalone, using 9.0 pro. I would like to backup modified and new files to a seperate partition. Rsync appears to be for 'sync'ing files over a network.
Would it work for me, or is there something better?
I think rsync is just what you want. Let's assume you want to keep a backup of the /home/foo/ directory. So you make a directory (hopefully on another drive but for the sake of this example) at /usr/ local/foo2/
Then run the command:
rsync -auvzr --delete /foo/ /usr/local/foo2/
This is exactly what I use between machines or locally. If to another machine I add the parm: -e ssh to use ssh to do the connection.
The above, run every <once in awhile> will keep /foo2/ in sync with / foo/, and it would only copy the new and updated files once you made your initial run at it.
The --delete will delete from /foo2/ anything you have deleted from / foo/
Note the use of the trailing slash. It's important when using rsync to copy directories...
By the way, if you add an 'n' to the mix (-auvzrn) it will show you the files that it WOULD copy but it won't actually copy them. For testing.
man rsync is your friend.
-- +------------------------------------------------------------------- --------- + + Bruce S. Marshall bmarsh@bmarsh.com Bellaire, MI 11/23/03 12:28 + +------------------------------------------------------------------- --------- + 'If truth is beauty, how come no one has their hair' 'done in the library?' -- Lily Tomlin
-- +---------------------------------------------------------------------------- + + Bruce S. Marshall bmarsh@bmarsh.com Bellaire, MI 11/23/03 19:10 + +---------------------------------------------------------------------------- + "He is winding the watch of his wit; by and by it will strike." - William Shakespeare
Tom Nielsen wrote:
Original poster here. I failed to mention that I want to backup my whole hard drive including my root and home directories to another drive on my computer. this is a stand alone machine. I was going to go with a RAID1 setup but found that's really not what I wanted to accomplish since if I screw up one system, the other will go too.
Will the below command still do what I want? It sounds like it. How would I setup my second drive? Any special partitions? Should I dd the disk first then use rsync or just rsync to begin with?
Thanks, Tom
Tom, Attached is a script I developed base on the info from Bruce Marshall. It backs up to my /dev/hdb1 which is mounted as /.snapshot. I've include the script in cron which should execute at 2 AM every day. I'll find out over the next few days is it works Terry -- SuSE Linux 8.2 (i586) 2.4.20-4GB-athlon || Sun Nov 23 18:25:00 CST 2003 6:25pm up 92 days 23:24, 3 users, load average: 0.25, 0.08, 0.10 #!/bin/sh /usr/bin/rsync -aur --delete /archive /.snapshot/ /usr/bin/rsync -aur --delete /bin /.snapshot/ /usr/bin/rsync -aur --delete /boot /.snapshot/ /usr/bin/rsync -aur --delete /data1 /.snapshot/ /usr/bin/rsync -aur --delete /data2 /.snapshot/ /usr/bin/rsync -aur --delete /data3 /.snapshot/ /usr/bin/rsync -aur --delete /data4 /.snapshot/ /usr/bin/rsync -aur --delete /etc /.snapshot/ /usr/bin/rsync -aur --delete /home /.snapshot/ /usr/bin/rsync -aur --delete /lib /.snapshot/ /usr/bin/rsync -aur --delete /opt /.snapshot/ /usr/bin/rsync -aur --delete /root /.snapshot/ /usr/bin/rsync -aur --delete /sbin /.snapshot/ /usr/bin/rsync -aur --delete /srv /.snapshot/ /usr/bin/rsync -aur --delete /tmp /.snapshot/ /usr/bin/rsync -aur --delete /usr /.snapshot/ /usr/bin/rsync -aur --delete /var /.snapshot/
It seems rsync or at least your script is tied to a common basis. I have a three drive setup one drive on each machine all on caddys and one planned. For me I would have to change this to allow for the differing partitions on each. The third drive (planned) will receive winbloat off drive#1 on computer#2 to clear space. Computer#2 has the CDRW backup drive. CWSIV On Sun, 23 Nov 2003 18:33:44 -0600 Terry Eck <tleck@mindspring.com>
Attached is a script I developed base on the info from Bruce Marshall. It backs up to my /dev/hdb1 which is mounted as /.snapshot. I've include the script in cron which should execute at 2 AM every day. I'll find out over the next few days is it works Terry
#!/bin/sh /usr/bin/rsync -aur --delete /archive /.snapshot/ /usr/bin/rsync -aur --delete /bin /.snapshot/ /usr/bin/rsync -aur --delete /boot /.snapshot/ /usr/bin/rsync -aur --delete /data1 /.snapshot/ /usr/bin/rsync -aur --delete /data2 /.snapshot/ /usr/bin/rsync -aur --delete /data3 /.snapshot/ /usr/bin/rsync -aur --delete /data4 /.snapshot/ /usr/bin/rsync -aur --delete /etc /.snapshot/ /usr/bin/rsync -aur --delete /home /.snapshot/ /usr/bin/rsync -aur --delete /lib /.snapshot/ /usr/bin/rsync -aur --delete /opt /.snapshot/ /usr/bin/rsync -aur --delete /root /.snapshot/ /usr/bin/rsync -aur --delete /sbin /.snapshot/ /usr/bin/rsync -aur --delete /srv /.snapshot/ /usr/bin/rsync -aur --delete /tmp /.snapshot/ /usr/bin/rsync -aur --delete /usr /.snapshot/ /usr/bin/rsync -aur --delete /var /.snapshot/
--------------000606050909080400080804 Content-Type: text/plain; charset=us-ascii
-- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com
--------------000606050909080400080804--
________________________________________________________________ The best thing to hit the internet in years - Juno SpeedBand! Surf the web up to FIVE TIMES FASTER! Only $14.95/ month - visit www.juno.com to sign up today!
Tom Nielsen wrote:
I was just about to setup a raid 1 when I realized I didn't want to really accomplish that. What I want is something to backup my system incase I fubar something. So, rather than dd my drive every night, I'm wondering how I can backup my system where only the changed files get backed up rather than the whole drive. I looked at rsync but I'm not sure if that's what I want.
Any advice?
Tom
find / -mtime -1 -print | cpio -ovcB >/dev/rmt0 Guy.
I you are backuping to disk, storebackup is what you are looking for, and more. With a simple .conf file and a cron job, I get backups every night to the backup disk. It keeps 7 daily backups, 5 end-of-week backups, and 3 end-of-month backups. (end-of-month, end-of-week, and daily can all be the same backup). Additionally, the backups are compressed individually on the disk so you can access them. the directoy /install/Backup/2004_02_17_04_00_01/ contains the status of my disk on 17 Feb at 4:01. To save space, it stored identical files only once, using hardlinks to point to the files in preveous backups. It is everything I ever wanted, and more.... Jerry P.S. (I use the version delivered with SuSE 9.0 Pro) On Thu, 2004-02-19 at 22:20, Guy Zelck wrote:
Tom Nielsen wrote:
I was just about to setup a raid 1 when I realized I didn't want to really accomplish that. What I want is something to backup my system incase I fubar something. So, rather than dd my drive every night, I'm wondering how I can backup my system where only the changed files get backed up rather than the whole drive. I looked at rsync but I'm not sure if that's what I want.
Any advice?
Tom
find / -mtime -1 -print | cpio -ovcB >/dev/rmt0
Guy.
On Thu, 2004-02-19 at 16:20, Guy Zelck wrote:
Tom Nielsen wrote:
I was just about to setup a raid 1 when I realized I didn't want to really accomplish that. What I want is something to backup my system incase I fubar something. So, rather than dd my drive every night, I'm wondering how I can backup my system where only the changed files get backed up rather than the whole drive. I looked at rsync but I'm not sure if that's what I want.
Any advice?
Tom
find / -mtime -1 -print | cpio -ovcB >/dev/rmt0
Guy.
That works, but if you have the extra disk space, I like rdiff-backup. It uses an rsync like algorithm to find changes inside a file, but it keeps old revisions around so you can get back to were a file was a few days ago. The only problem with is if something goes wrong in the code. It is written in python on I think, and if something breaks it outputs a whole stack dump. Sort of hard for a casual user to undrstand. Now for an exotic proposal that was inspired by your dd comment: (ie. Don't do this unless you want to experiment. It will take you a while to setup and get working right and you have to recreate your filesystems.) You could use LVM snapshots. With those you could initiate a snapshot at midnight, then all day long any low level disk blocks get saved before the new data is written to disk. If you decide you need a file the way it is on the snapshot, you simply mount the snapshot read-only and copy it back off. If you want any sort of performance, you will want to put your snapshot volume on another physical disk than the main volume, but in the same server. I don't know if you can setup 2 snapshots or not. ie. alternating snapshots for last night and the night before. Using alternating snapshots is not unusual with high-end storage systems, but I don't know if LVM supports it or not. Greg
participants (15)
-
Bernd
-
Bruce Marshall
-
Carl William Spitzer IV
-
Carlos E. R.
-
Damian O'Hara
-
Greg Freemyer
-
Guy Zelck
-
Jerome R. Westrick
-
Ken Schneider
-
LW999@hotpost
-
Terry Eck
-
Thomas Adam
-
Tom Nielsen
-
Viktor
-
Örn Hansen