Backup script implementration and advice.

Hylton Conacher (ZR1HPC)

14 Jan 2006 14 Jan '06

15:18

Over many weeks/months I have bothered the list with queries about fscks and its fellows. It was all to prevent data loss before I get a backup script up an running. I have decided that the easiest way to backup is with a copy of the affected material onto another harddrive. I have a secondary drive(HDD) installed that I have used for copying odds and sods of unimportant information. I have approx 5Gb free on the drive that I can use for backup space. I would like to invoke the script by clicking on an icon(Link to application ?) in KDE, whilst signed on as a normal user. Assuming a partition fails and I end up installing another drive and reinstalling, I should just be able to restore the relevant files off my backup onto the new partitions ie: # cp -a /data4/home /dev/<new hdd partition> My partitioned SuSE drive is HDB and the secondary copy or backup of the important data and config files will be on HDD. Below I have pasted a copy of what I think I need to do on the script as well as the command I expect to be the one I will need to use. The backup drive has a listing in /etc/fstab of: /dev/hdd1 /data4 auto noauto,user 0 2 I would appreciate any input, especially since this is my FIRST script. ****Paste of future backup script********** #!/bin/bash # Sign on as root Unknown cmd # Mount HDD1 mount /dev/hdd1 /data4 # Remove old backup image rm -dr /data4/ # Copy certain directories to HDD and PRESERVe permissions cp -a /home /data4/home cp -a /etc /data4/etc # unmount HDD1 umount /dev/hdd1 #sign off and shutdown shutdown -h now ****************End**************************** -- ======================================================================== Hylton Conacher - Linux user # 229959 at http://counter.li.org Currently using SuSE 9.0 Professional with KDE 3.1 ========================================================================

Show replies by date

Jim Cunning

15 Jan 15 Jan

17:59

New subject: [SLE] Backup script implementration and advice.

On Sat, 2006-01-14 at 07:18, Hylton Conacher (ZR1HPC) wrote: [...]

...

I have decided that the easiest way to backup is with a copy of the affected material onto another harddrive. I have a secondary drive(HDD) installed that I have used for copying odds and sods of unimportant information. I have approx 5Gb free on the drive that I can use for backup space. Why not try storeBackup? It's a (Perl) script that's already written and checked-out, and does what you want to do:

pin 0.29 - package info for storeBackup ------------------------------------------------------------------ *** rpm info ------------------------------------------------------------------ Name : storeBackup Relocations: (not relocatable) Version : 1.19 Vendor: SuSE Linux AG, Nuernberg, Germany Release : 1.1 Build Date: Mon 12 Sep 2005 10:34:43 AM PDT Install date: Tue 15 Nov 2005 03:02:36 PM PST Build Host: niobe.suse.de Group : Productivity/Archiving/Backup Source RPM: storeBackup-1.19-1.1.src.rpm Size : 515369 License: GPL Signature : DSA/SHA1, Tue 13 Sep 2005 07:02:03 AM PDT, Key ID a84edae89c800aca Packager : http://www.suse.de/feedback URL : http://sourceforge.net/projects/storebackup/ Summary : storeBackup is a Disk-to-Disk Backup Tool for Linux -- Jim Cunning

Hylton Conacher (ZR1HPC)

17 Jan 17 Jan

14:13

New subject: [SLE] Backup script implementration and advice.

Jim Cunning wrote:

...

On Sat, 2006-01-14 at 07:18, Hylton Conacher (ZR1HPC) wrote: [...]

...
I have decided that the easiest way to backup is with a copy of the affected material onto another harddrive. I have a secondary drive(HDD) installed that I have used for copying odds and sods of unimportant information. I have approx 5Gb free on the drive that I can use for backup space.

Why not try storeBackup? It's a (Perl) script that's already written and checked-out, and does what you want to do:

pin 0.29 - package info for storeBackup <snip> Thanks. From what you say it is exactly what I am looking for. I checked in YAST and I have version 1.14-13 installed.

Documentation is a little scarce ie '$ man storebackup' didn't produce any documentation and neither did '$ info storebackup'. After a KDE 'storebackup' file search I found some documentation and hope that that contains most of the setup info I need. From what I have read store backup is pretty thorough and whilst I might need it, it would've been nice to script my own as my first script. I'll concentrate instead on configuring storebackup and leave the scripting to start later. :) I'll continue reading thru the storebackup docs I found, and revert when the inevitable queries arise. Please pass on any further docs or URL's that might help in its configuration/explanation. -- ======================================================================== Hylton Conacher - Linux user # 229959 at http://counter.li.org Currently using SuSE 9.0 Professional with KDE 3.1 ========================================================================

John Pettigrew

15 Jan 15 Jan

18:29

New subject: [SLE] Backup script implementration and advice.

In a previous message, "Hylton Conacher (ZR1HPC)" wrote:

...

# Remove old backup image rm -dr /data4/

# Copy certain directories to HDD and PRESERVe permissions cp -a /home /data4/home cp -a /etc /data4/etc

Better (i.e. faster) would be to use rsync, because this only copies what's changed since the last backup. I use a script I picked up from somewhere called make-backup - it's possibly worth googling for that name together with rsync to get the original. Failing that, I can pass it along. (It's headed "mike's handy rotating-filesystem-snapshot utility" and seems to be created by "mrubel".) Another advantage of this script is that it gives reverse incremental backups for free - the most recent backup is the complete one, with those further back in time having only the things that changed since then. HTH John -- John Pettigrew Headstrong Games john@headstrong-games.co.uk Fun : Strategy : Price http://www.headstrong-games.co.uk/ Board games that won't break the bank Knossos: escape the ever-changing labyrinth before the Minotaur catches you!

Hylton Conacher (ZR1HPC)

17 Jan 17 Jan

14:17

New subject: [SLE] Backup script implementration and advice.

...

In a previous message, "Hylton Conacher (ZR1HPC)" wrote:

...
# Remove old backup image rm -dr /data4/

# Copy certain directories to HDD and PRESERVe permissions cp -a /home /data4/home cp -a /etc /data4/etc

Better (i.e. faster) would be to use rsync, because this only copies what's changed since the last backup. I use a script I picked up from somewhere called make-backup - it's possibly worth googling for that name together with rsync to get the original. Failing that, I can pass it along. (It's headed "mike's handy rotating-filesystem-snapshot utility" and seems to be created by "mrubel".) Tnx for the idea but I think I'll concentrate on Joe's idea of storebackup for now. Whilst rsync may be faster becos it is 'skipping' so many files, I would prefer that ALL the files were copied, even if

John Pettigrew wrote: they hadn't changed. Tnx though -- ======================================================================== Hylton Conacher - Linux user # 229959 at http://counter.li.org Currently using SuSE 9.0 Professional with KDE 3.1 ========================================================================

John Pettigrew

19:53

New subject: [SLE] Backup script implementration and advice.

In a previous message, "Hylton Conacher (ZR1HPC)" wrote:

...

Tnx for the idea but I think I'll concentrate on Joe's idea of storebackup for now.

Fair enough :-)

...

Whilst rsync may be faster becos it is 'skipping' so many files, I would prefer that ALL the files were copied, even if they hadn't changed.

I think you missed the point. rsync will copy everything - but if (on a previous backup) you copied a file across and it hasn't changed, rsync won't copy it again, it'll just leave the existing good copy. Much faster than deleting and recopying everything, when most files won't change from one backup to the next (if lots of files are changing, you need to back up more often!). John -- John Pettigrew Headstrong Games john@headstrong-games.co.uk Fun : Strategy : Price http://www.headstrong-games.co.uk/ Board games that won't break the bank Fields of Valour: 2 Norse clans battle on one of 3 different boards

Carlos E. R.

19:57

New subject: [SLE] Backup script implementration and advice.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Tuesday 2006-01-17 at 16:17 +0200, Hylton Conacher (ZR1HPC) wrote:

...

Tnx for the idea but I think I'll concentrate on Joe's idea of storebackup for now. Whilst rsync may be faster becos it is 'skipping' so many files, I would prefer that ALL the files were copied, even if they hadn't changed.

That's not exact. Sync will mantain a full copy of everything, at the moment it run; it just saves time by not copying things it already has and have not changed, but the copy will of course contain those unchanged files. On the other hand, it is not an historic backup, you can not retrieve older versions. There is just one copy of everything - unless you program it to keep several copies, in diferent places. - -- Cheers, Carlos Robinson -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFDzUwUtTMYHG2NR9URApEfAJ45zq83HOTFi2qypHdcr5nalEUUhwCfZQL4 fp7qwmJ+zoqxpBlclkoAfDk= =113i -----END PGP SIGNATURE-----

John Pettigrew

23:08

New subject: [SLE] Backup script implementration and advice.

In a previous message, "Carlos E. R." wrote:

...

On the other hand, [rsync] is not an historic backup, you can not retrieve older versions. There is just one copy of everything - unless you program it to keep several copies, in diferent places.

The nifty thing about this backup script I found is that it uses rsync initially, and then uses cp -al to copy only links to those files (i.e. it uses no extra space unless files have changed), giving a reverse incremental backup - the latest version is complete (thanks to rsync) and the daily and weekly backups before it contain only the changes to that most-current backup. Anyhow, as long as the OP has found a good solution, that's the main thing! John -- John Pettigrew Headstrong Games john@headstrong-games.co.uk Fun : Strategy : Price http://www.headstrong-games.co.uk/ Board games that won't break the bank Knossos: escape the ever-changing labyrinth before the Minotaur catches you!

Carlos E. R.

18 Jan 18 Jan

01:16

New subject: [SLE] Backup script implementration and advice.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Tuesday 2006-01-17 at 23:08 -0000, John Pettigrew wrote:

...

In a previous message, "Carlos E. R." wrote:

...
On the other hand, [rsync] is not an historic backup, you can not retrieve older versions. There is just one copy of everything - unless you program it to keep several copies, in diferent places.

The nifty thing about this backup script I found is that it uses rsync initially, and then uses cp -al to copy only links to those files (i.e. it uses no extra space unless files have changed), giving a reverse incremental backup - the latest version is complete (thanks to rsync) and the daily and weekly backups before it contain only the changes to that most-current backup.

Ah! I didn't notice that. Interesting :-) I would like a rsync thing that made compressed copies. I just saw tonight something similar in "mkzftree", used to create compressed trees for zisofs/RockRidge cd/dvd backups: -C path, --crib-path path Steal ("crib") files from another directory if it looks (based on name, size, type and modification time) like they match entries in the new filesystem. The "crib tree" is usually the compressed version of an older version of the same workload; this thus allows for "incremental rebuilds" of a compressed filesystem tree. The files are hardlinked from the crib tree to the output tree, so if it is desirable to keep the link count correct the crib path should be deleted before running mkisofs. The crib tree must be on the same filesystem as the output tree. Now, I'll have to translate that to human parlance to understand it... sigh. I'll call it a day. - -- Cheers, Carlos Robinson -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFDzZcBtTMYHG2NR9URAknGAJ9wxhmmVmCZq319VvdU0zOSTYhw/gCglX7M kSXCamsRpClS+HMExE5MCLk= =haz9 -----END PGP SIGNATURE-----

Dave Howorth

10:48

New subject: [SLE] Backup script implementration and advice.

On Tue, 2006-01-17 at 20:57 +0100, Carlos E. R. wrote:

...

Sync will mantain a full copy of everything, at the moment it run; it just saves time by not copying things it already has and have not changed, but the copy will of course contain those unchanged files.

rsync does better than that. It doesn't copy whole files, it copies just the parts that have changed.

...

On the other hand, it is not an historic backup, you can not retrieve older versions. There is just one copy of everything - unless you program it to keep several copies, in diferent places.

There's a program called dirvish (www.dirvish.org) that can keep multiple copies. You can keep them all completely separate but if you keep them all together it uses hard links so that space is minimised. BTW, for compressed backups, you could perhaps use a compressed filesystem to store the backups. Cheers, Dave

Carlos E. R.

19 Jan 19 Jan

00:15

New subject: [SLE] Backup script implementration and advice.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Wednesday 2006-01-18 at 10:48 -0000, Dave Howorth wrote:

...

On Tue, 2006-01-17 at 20:57 +0100, Carlos E. R. wrote:

...
Sync will mantain a full copy of everything, at the moment it run; it just saves time by not copying things it already has and have not changed, but the copy will of course contain those unchanged files.

rsync does better than that. It doesn't copy whole files, it copies just the parts that have changed.

I understood it "transmits" the changed parts, but the copy, the file, would be whole.

...

...
On the other hand, it is not an historic backup, you can not retrieve older versions. There is just one copy of everything - unless you program it to keep several copies, in diferent places.

There's a program called dirvish (www.dirvish.org) that can keep multiple copies. You can keep them all completely separate but if you keep them all together it uses hard links so that space is minimised.

Ah, interesting, John Pettigrew mentioned some thing like that the other day.

...

BTW, for compressed backups, you could perhaps use a compressed filesystem to store the backups.

Such as? I use compressed CDs, but I don't know a compressed R/W file I can use in Linux. - -- Cheers, Carlos Robinson -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFDztoXtTMYHG2NR9URAoOqAJwNaLdx3LTUSRvSffQ0JMZPa0yORgCeJnuh CCRQenWdIVNA5ljoaMUedHU= =xNer -----END PGP SIGNATURE-----

Ken Schneider

17 Jan 17 Jan

21:12

New subject: [SLE] Backup script implementration and advice.

On Tue, 2006-01-17 at 16:17 +0200, Hylton Conacher (ZR1HPC) wrote:

...

...
In a previous message, "Hylton Conacher (ZR1HPC)" wrote:

...
# Remove old backup image rm -dr /data4/

# Copy certain directories to HDD and PRESERVe permissions cp -a /home /data4/home cp -a /etc /data4/etc

Better (i.e. faster) would be to use rsync, because this only copies what's changed since the last backup. I use a script I picked up from somewhere called make-backup - it's possibly worth googling for that name together with rsync to get the original. Failing that, I can pass it along. (It's headed "mike's handy rotating-filesystem-snapshot utility" and seems to be created by "mrubel".) Tnx for the idea but I think I'll concentrate on Joe's idea of storebackup for now. Whilst rsync may be faster becos it is 'skipping' so many files, I would prefer that ALL the files were copied, even if

John Pettigrew wrote: they hadn't changed.

But if you are copying all of the files to the -same- place every time there is -no- need to copy unchanged files. -- Ken Schneider UNIX since 1989, linux since 1994, SuSE since 1998

6678

Age (days ago)

6683

Last active (days ago)

List overview

Download

11 comments

6 participants

participants (6)

Carlos E. R.
Dave Howorth
Hylton Conacher (ZR1HPC)
Jim Cunning
John Pettigrew
Ken Schneider