Backup to CD/DVD -- recommended software?
I've got a pretty common need: backing up a directory to CD/DVD. The obvious solution is to use K3B -- but alas, it doesn't work right. When K3B creates a project for a directory it saves the directory contents at the time of creating the project. Then, when it attempts to write the directory at future times, it uses that very same file list, even though some members and submembers of the directory may have been deleted and (worse) others may have been added. This particular bit of behavior is to me at least inexplicable, but I see no way around it other than to create a new project every time -- which, for various reasons, I do not want to do. So can someone recommend a CD-writing utility with functionality similar to K3B but that does not have that particular design flaw? I'd much prefer something that's part of the standard SuSE distribution, and I definitely don't want commercial software. Paul
On Wed, 2005-01-12 at 13:26, Paul W. Abrahams wrote:
I've got a pretty common need: backing up a directory to CD/DVD. The obvious solution is to use K3B -- but alas, it doesn't work right. When K3B creates a project for a directory it saves the directory contents at the time of creating the project. Then, when it attempts to write the directory at future times, it uses that very same file list, even though some members and submembers of the directory may have been deleted and (worse) others may have been added. This particular bit of behavior is to me at least inexplicable, but I see no way around it other than to create a new project every time -- which, for various reasons, I do not want to do.
Whenever K3B asks if I want to same the current "project" I just tell it no. Try starting with a "new" project and pick the source dir again. -- Ken Schneider UNIX since 1989 SuSE since 1998 * Only reply to the list please*
On Wednesday 12 January 2005 13:26, Paul W. Abrahams wrote:
I've got a pretty common need: backing up a directory to CD/DVD. The obvious solution is to use K3B -- but alas, it doesn't work right. When K3B creates a project for a directory it saves the directory contents at the time of creating the project. Then, when it attempts to write the directory at future times, it uses that very same file list, even though some members and submembers of the directory may have been deleted and (worse) others may have been added. This particular bit of behavior is to me at least inexplicable, but I see no way around it other than to create a new project every time -- which, for various reasons, I do not want to do.
So can someone recommend a CD-writing utility with functionality similar to K3B but that does not have that particular design flaw? I'd much prefer something that's part of the standard SuSE distribution, and I definitely don't want commercial software.
I use a cron script that runs at night which pipes the results of mkisofs into cdrecord. This works fine as long as the files don't change during the burning process. Something like this: mkisofs -v -R /data | cdrecord -v fs=12 speed=4 dev=ATA:1,1,0 - If your a competent script writer (I'm not) you could improve on this by creating the iso first, burn it, then delete the iso image. That would nearly eliminate any chance of files changing. Jeff
The Wednesday 2005-01-12 at 14:46 -0500, Jeffrey Laramie wrote:
I use a cron script that runs at night which pipes the results of mkisofs into cdrecord. This works fine as long as the files don't change during the burning process. Something like this:
mkisofs -v -R /data | cdrecord -v fs=12 speed=4 dev=ATA:1,1,0 -
If your a competent script writer (I'm not) you could improve on this by creating the iso first, burn it, then delete the iso image. That would nearly eliminate any chance of files changing.
With an intermediate step, you can also compress the image. The linux kernel can later mount the compressed cdrom or dvd transparently. I don't know if any gui can do that. -- Cheers, Carlos Robinson
Carlos E. R. wrote:
The Wednesday 2005-01-12 at 14:46 -0500, Jeffrey Laramie wrote:
I use a cron script that runs at night which pipes the results of mkisofs into cdrecord. This works fine as long as the files don't change during the burning process. Something like this:
mkisofs -v -R /data | cdrecord -v fs=12 speed=4 dev=ATA:1,1,0 -
If your a competent script writer (I'm not) you could improve on this by creating the iso first, burn it, then delete the iso image. That would nearly eliminate any chance of files changing.
With an intermediate step, you can also compress the image. The linux kernel can later mount the compressed cdrom or dvd transparently. I don't know if any gui can do that.
Really? That's interesting. How to do this? I would expect that this would work better than this: I made a compressed tarball of my stuff, about 500MB after compressing, 950MB before. I copied the .tgz to a DVDRAM. Then I clicked the .tgz in Konqueror, which uses whatever to open it and display it just like an ordinary directory, which of course it is not. This is VERY convenient for small archives, but for a 500MB .tgz on a slow medium, it is terrible. It can take literally 15 minutes to access a single file. If I do not compress the tarball, and just copy the .tar to the DVDRAM, then it works much better, and is actually a fair solution, since it also optimizes the write speed to the DVDRAM. Writing a single large file (of whatever size) is fastest, compared to writing many small files, since that requires the drive to seek seek seek all the time (to write the directory entries and data, which are in different physical locations.) Since I have only about 0.9-1.0GB of daily data to backup, I don't mind using an uncompressed .tar on the DVDRAM. The disk is still 4x larger than my backup size, so it will be many years before my home dir expands to be too much for the DVDRAM. So I think what you are talking about is different. The .tgz is not a *filesystem* proper, while a compressed ISO is. So I would presume that this could work much faster than my attempt at accessing a 500MB .tgz. Good day! -- ____________________________________ Christopher R. Carlen Principal Laser/Optical Technologist Sandia National Laboratories CA USA crcarle@sandia.gov
The Wednesday 2005-02-02 at 08:24 -0800, Christopher Carlen wrote:
mkisofs -v -R /data | cdrecord -v fs=12 speed=4 dev=ATA:1,1,0 -
If your a competent script writer (I'm not) you could improve on this by creating the iso first, burn it, then delete the iso image. That would nearly eliminate any chance of files changing.
With an intermediate step, you can also compress the image. The linux kernel can later mount the compressed cdrom or dvd transparently. I don't know if any gui can do that.
Really? That's interesting. How to do this?
...
So I think what you are talking about is different. The .tgz is not a *filesystem* proper, while a compressed ISO is. So I would presume that this could work much faster than my attempt at accessing a 500MB .tgz.
Correct. Read access is immediate: just mount the dvd and read any file directly with any tool: konkeror, cp, mc, whatever. But creating it is cumbersome. I do: 1) Create a compressed copy of the files you want to put on the CD/DVD. For example: mkzftree --parallelism 3 $SOURCE $DESTINATION.CMP The destination directory must _not_ exist, the command creates it. There are not many options, see the man page (mkzftree - Create a zisofs/RockRidge compressed file tree) 2) Create the iso image. For example: mkisofs -z -R -quiet -graft-points\ -P "Not Published, private backup" \ -p "Carlos E. R. .... dot es" \ -V "System Backup, Linux" \ -o $DESTINO_ISO $DESTINO_CMP \ download/=/home/cer/download/ \ compilaciones__tars/=/home/cer/compilaciones/_tars/ \ _usr_src_packages_packages/=/usr/src/packages/ \ patches/=/var/lib/YaST/patches/i386/update/7.3/ The only important switch there is "-z". It is added by a patch to the mkisofs tool, but SuSE has been adding it since just after SuSE 7.3. I know because at the time I added it manually to test it. 2.5) Delete the compressed tree if you need the space. 3) Burn the image. I use xcdroast for that, but any tool will do. 4) Use the CDrom or DVD - I have tested it for dvds with SuSE 9.1, it works. The kernel needs zisofs support, but SuSE kernels have it. There are other compressed filesystems in 9.2, but I haven't tested them because I haven't updated yet. The procedure works fine. The problem is that you need a lot of space: first to create the compressed tree, more than half the original tree. Then space for the iso images: but you can create one at a time, or on the fly and burn. A secondary and not trivial problem is to adjust what to put on each image so that it is "small" enough to fit in the available media - this is complicated because no gui knows about zisofs. My trick is to divide the original compressed tree into several trees, using mc for moving files and calculating sizes. So... the overall process is slow. The result is perfect. A great improvement would be a mkisofs tool that cold compress on the fly. However, I doubt it could calculate exact size in advance. Obviously, if you have very large files, bigger than a CD or a DVD, you can not use this method. -- Cheers, Carlos Robinson
Paul W. Abrahams wrote:
I've got a pretty common need: backing up a directory to CD/DVD. The obvious solution is to use K3B -- but alas, it doesn't work right. When K3B creates a project for a directory it saves the directory contents at the time of creating the project. Then, when it attempts to write the directory at future times, it uses that very same file list, even though some members and submembers of the directory may have been deleted and (worse) others may have been added. This particular bit of behavior is to me at least inexplicable, but I see no way around it other than to create a new project every time -- which, for various reasons, I do not want to do.
So can someone recommend a CD-writing utility with functionality similar to K3B but that does not have that particular design flaw? I'd much prefer something that's part of the standard SuSE distribution, and I definitely don't want commercial software.
Actually, for something repetitive like this, a GUI program is NOT the obvious solution. A script is. Here is the script I have used for CD-R(W) backup of a specific directory until just now beginning to experiment with DVDs: First the backup.settings file: ---------------------------------- # The backup.settings file CDRW_SCSI_DEV=ATA:3,0,0 CDRW_DEV_PATH=/dev/cdrw ---------------------------------- Next is the CD-RW backup script. Just take out the -blank option for CD-R, and a few other minor tweaks to use it on CD-R: ---------------------------------- #!/bin/bash echo We are in the datasheets.CD-RW.backup script! source ~/backup.scripts/backup.settings mkisofs -o ~/cd-rw/datasheets.backup.iso -R -J -v -graft-points ~/datasheets echo \n du -h ~/cd-rw/datasheets.backup.iso echo \n echo Press Enter to continue or Ctrl-Cto exit... read REPLY cdrecord dev=$CDRW_SCSI_DEV -v blank=fast -dao ~/cd-rw/datasheets.backup.iso cdrecord dev=$CDRW_SCSI_DEV -eject cdrecord dev=$CDRW_SCSI_DEV -load echo \n md5sum ~/cd-rw/datasheets.backup.iso md5sum $CDRW_DEV_PATH echo \n echo Press Enter or close when done... read REPLY exit --------------------------------------------------- Obviously it desires a directory in your home called "cd-rw" to create ISOs. It verifies that the burn=the ISO file afterwards. I have had many troubles with this, failing to match md5sums or getting IO errors reading back the disk. After a reboot and manually doing the md5sums would work fine. This is a driver/drive resetting issue. That is the reason for the eject/load actions, which fixes it usually, but I have one machine that still gives problems. xcdroast uses some options to readcd to do this correctly, and someday I may try that more sophisticated approach. But this is the basic idea of how to automate the backup process. Connect this script to a KDE panel button and you have a one-click backup. Far more efficient than using a GUI program. It pays to learn the CLI tools for CD/DVD burning. But don't feel bad if they seem formidable. I have many hours of experimenting and frustration invested in even this simple script. Good day! -- ____________________________________ Christopher R. Carlen Principal Laser/Optical Technologist Sandia National Laboratories CA USA crcarle@sandia.gov
On Tuesday 01 February 2005 5:59 pm, Christopher Carlen wrote:
Actually, for something repetitive like this, a GUI program is NOT the obvious solution. A script is.
I considered using a script -- and I do know how to use the utilities. But as far as I can tell, the process operates "blind" -- there's no visible indication of its progress. K3B has a nice progress bar, and that's why I wanted to use it. Also, if something goes wrong there's a very visible error message. Any ideas on how to construct a script that will do the right thing and also display a progress bar? Or is that not possible? I'm trying to set this up so that someone who doesn't know Linux can just come over to the terminal and click on a couple of buttons, and be able to fix that which is easily fixable. Associating a script with a desktop button would be perfect if not for the issue of the progress bar and error messages. Paul
Paul W. Abrahams wrote:
On Tuesday 01 February 2005 5:59 pm, Christopher Carlen wrote:
Actually, for something repetitive like this, a GUI program is NOT the obvious solution. A script is.
I considered using a script -- and I do know how to use the utilities. But as far as I can tell, the process operates "blind" -- there's no visible indication of its progress. K3B has a nice progress bar, and that's why I wanted to use it. Also, if something goes wrong there's a very visible error message.
Any ideas on how to construct a script that will do the right thing and also display a progress bar? Or is that not possible?
It's actually not so bad. mkisofs outputs a lot of info. My script has no logic to deal with errors, only gives an opportunity to verify that mkisofs succeeded, and check that the .iso will fit on the disk. Then press enter to proceed to writing. This could be automated more by someone with better scripting abilities. Mine are obviously rudimentary. Then cdrecord outputs considerable information. You can actually watch the progress quite well, in terms of both speed and a display of "xxx or yyy MB written" or something like that. Actually, I like using the tools directly better than having the bar graphs. But, each to his own.
I'm trying to set this up so that someone who doesn't know Linux can just come over to the terminal and click on a couple of buttons, and be able to fix that which is easily fixable. Associating a script with a desktop button would be perfect if not for the issue of the progress bar and error messages.
Paul
Good day! -- ____________________________________ Christopher R. Carlen Principal Laser/Optical Technologist Sandia National Laboratories CA USA crcarle@sandia.gov
participants (5)
-
Carlos E. R. -
Christopher Carlen -
Jeffrey Laramie -
Ken Schneider -
Paul W. Abrahams