[opensuse] splitting up an rsync archive
Hi all, This may be a dumb question... but I am still gonna' ask it. Let's say I've got an rsync backup archive and it is getting close to filling up the disk on my backup server. is there an easy way to split up that archive so as to alleviate the load? Thanks. Boris. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On 05/17/2010 01:27 PM, Boris Epstein pecked at the keyboard and wrote:
Hi all,
This may be a dumb question... but I am still gonna' ask it.
Let's say I've got an rsync backup archive and it is getting close to filling up the disk on my backup server. is there an easy way to split up that archive so as to alleviate the load?
Thanks.
Boris.
Unless there is a reason to keep old versions of files I would use the --delete option to rsync. That may free up a lot of space. You can always test by using the -n (dry-run) switch to test first. You might be amazed at the number of files that were deleted locally long ago that are still on an rsync backup. -- Ken Schneider SuSe since Version 5.2, June 1998 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday, 2010-05-17 at 15:13 -0400, Ken Schneider - openSUSE wrote:
On 05/17/2010 01:27 PM, Boris Epstein pecked at the keyboard and wrote:
Hi all,
This may be a dumb question... but I am still gonna' ask it.
Let's say I've got an rsync backup archive and it is getting close to filling up the disk on my backup server. is there an easy way to split up that archive so as to alleviate the load?
Not that I know. But you might use LVM on the target destination to add disks.
Unless there is a reason to keep old versions of files I would use the --delete option to rsync. That may free up a lot of space. You can always test by using the -n (dry-run) switch to test first. You might be amazed at the number of files that were deleted locally long ago that are still on an rsync backup.
Or use more convoluted backup procedures. For example, you can use "--link-dest" to keep "snapshots", with one directory per backup; new files are added/deleted on the new directory, but existing files are hardlinked instead from the old directory. You can restore any version of any file you made a backup some day, but not modified files get only copied once. When full, you can move the older directories elsewhere, or delete them. Still, the target needs have more space than the source. There are scripts that automate this, like rsnapshot. Perhaps dirvish. Another is rdiff-backup, which keeps old versions are rdifss instead, saving space but incrementing cpu usage. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkvxsFcACgkQtTMYHG2NR9WGdQCfZa+WyRBeY6BzZGjgjv3jHRRG eRgAn3Nr0x6ZzpIfPCKB9kKoy2/yIUdm =UHPs -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Mon, May 17, 2010 at 5:08 PM, Carlos E. R. <robin.listas@telefonica.net> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Monday, 2010-05-17 at 15:13 -0400, Ken Schneider - openSUSE wrote:
On 05/17/2010 01:27 PM, Boris Epstein pecked at the keyboard and wrote:
Hi all,
This may be a dumb question... but I am still gonna' ask it.
Let's say I've got an rsync backup archive and it is getting close to filling up the disk on my backup server. is there an easy way to split up that archive so as to alleviate the load?
Not that I know. But you might use LVM on the target destination to add disks.
Unless there is a reason to keep old versions of files I would use the --delete option to rsync. That may free up a lot of space. You can always test by using the -n (dry-run) switch to test first. You might be amazed at the number of files that were deleted locally long ago that are still on an rsync backup.
Or use more convoluted backup procedures. For example, you can use "--link-dest" to keep "snapshots", with one directory per backup; new files are added/deleted on the new directory, but existing files are hardlinked instead from the old directory. You can restore any version of any file you made a backup some day, but not modified files get only copied once.
When full, you can move the older directories elsewhere, or delete them. Still, the target needs have more space than the source.
There are scripts that automate this, like rsnapshot. Perhaps dirvish. Another is rdiff-backup, which keeps old versions are rdifss instead, saving space but incrementing cpu usage.
- -- Cheers, Carlos E. R.
-----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux)
iEYEARECAAYFAkvxsFcACgkQtTMYHG2NR9WGdQCfZa+WyRBeY6BzZGjgjv3jHRRG eRgAn3Nr0x6ZzpIfPCKB9kKoy2/yIUdm =UHPs -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Thanks. I actually am using rsnapshot. So could I perhaps just move some old backups off the server and free some space that way? Boris. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday, 2010-05-17 at 17:22 -0400, Boris Epstein wrote:
I actually am using rsnapshot.
So could I perhaps just move some old backups off the server and free some space that way?
Yes, I think so. Move the older backups to somewhere else (perhaps with compression). The files that have not changed will have hardlinks to newer copies, so they will not be actually deleted, which is correct. Changed or deleted filles will be deleted and save some space, which is what you want. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkvx0V8ACgkQtTMYHG2NR9XToACffLb99vzqqxQb0q2d6lNwPRac IasAn0kt/MICkGuI236IuRwX+DlDZAZZ =gghC -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Mon, May 17, 2010 at 1:27 PM, Boris Epstein <borepstein@gmail.com> wrote:
Hi all,
This may be a dumb question... but I am still gonna' ask it.
Let's say I've got an rsync backup archive and it is getting close to filling up the disk on my backup server. is there an easy way to split up that archive so as to alleviate the load?
Thanks.
Boris.
You can break the archive into segments lots of ways. ie. By directory, filesize, filetype, etc. "man rsync" will give you the details. ie. I've used --min-size and --max-size to push small files to one backup and large files to another. --max-size=SIZE --min-size=SIZE Greg -- Greg Freemyer Head of EDD Tape Extraction and Processing team Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer CNN/TruTV Aired Forensic Imaging Demo - http://insession.blogs.cnn.com/2010/03/23/how-computer-evidence-gets-retriev... The Norcross Group The Intersection of Evidence & Technology http://www.norcrossgroup.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On 05/17/2010 12:27 PM, Boris Epstein wrote:
Hi all,
This may be a dumb question... but I am still gonna' ask it.
Let's say I've got an rsync backup archive and it is getting close to filling up the disk on my backup server. is there an easy way to split up that archive so as to alleviate the load?
Thanks.
Boris.
Yes, It's quite easy as long as you have directories to work with and not one single directory with lots of files. I would just figure out what directories I wanted where and then do something like: rsync -uav ./backup/[A-La-l]** myserver:/my/destination rsync -uav ./backup/[M-Zm-z]** myNEWserver:/my/NEW/destination -- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (5)
-
Boris Epstein -
Carlos E. R. -
David C. Rankin -
Greg Freemyer -
Ken Schneider - openSUSE