----- Original Message ----- From: "Erik Jakobsen" To: Sent: Wednesday, January 07, 2009 11:18 AM Subject: [opensuse] rsync ?

Hi.

openSUSE 11.1 and KDE4.1.3

I have a server, that can made backup with rsync to for instance my 11.1 computer. There is a name called "module", that I don't really understand. It shall be written on the server before the backup.

"NAS and a linux rsync server". The module name means the task/mission name you setup in the linux rsync server."

Is what I see. And this is what I don't really understand. What to do ?

Think of a module like a samba/windows share or nfs export. You define some real path on the server machine, and give it any sort of name you want, and the client can access that share by that name, and they have access to only that directory and any under it. So: on the server (lets call it "tank"), assume you want to allow clients to access a directory /path/to/backups/boxa on tank: In /etc/rsyncd.conf you define a module (a share, or export) which is for backups of a client box named boxa. ---snip--- [boxa] path = /path/to/backups/boxa auth users = usera ---snip--- Above, I have said that only user "usera" may access the "boxa" module. If using native rsync (not ssh or rsh), rsync users have nothing to do with system users on either server or client. They are defined solely within /etc/rsyncd.secrets on the server. So, also on tank: in /etc/rsyncd.secrets ---snip--- usera:passa ---snip--- Now, from boxa (the client) you can use an rsync command like this, say to backup the /u drive to the boxa dir on tank on boxa: rsync -avz --del /u usera@tank::boxa Assuming several other config elements and permissions and firewall details have also been tended to, and assuming you entered "passa" when asked, this will copy /u from boxa, to create, then subsequently update, /path/to/backups/boxa/u on tank. -- Brian K. White brian@aljex.com http://www.myspace.com/KEYofR +++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++. filePro BBx Linux SCO FreeBSD #callahans Satriani Filk! -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Greg Freemyer wrote:

...

modules are used when you use an rsync daemon. Check the rsync man page for how to configure rsync daemons.

Cheers, Dave

Be aware that using a rsync daemon and the associated complexity is totally optional.

I always just use plain old rsync on the client. If there is not a rsync server available on the remote machine, it uses ssh to connect to the remote and fire up an instance of rsync on that end and communications goes from there.

Takes zero upfront config assuming you have ssh logins available.

Greg

Many thanks for the replies to my query. Not that I understand much of it, but I will study the man page, and hope I get more insight. -- Med venlig hilsen / Best regards Erik Jakobsen OZ4KK eja@urbakken.dk Registered Linux user #114875 - http://counter.li.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

On Thu, Jan 8, 2009 at 11:11 AM, Erik Jakobsen wrote:

Greg Freemyer wrote:

...

...

modules are used when you use an rsync daemon. Check the rsync man page for how to configure rsync daemons.

Cheers, Dave

Be aware that using a rsync daemon and the associated complexity is totally optional.

I always just use plain old rsync on the client. If there is not a rsync server available on the remote machine, it uses ssh to connect to the remote and fire up an instance of rsync on that end and communications goes from there.

Takes zero upfront config assuming you have ssh logins available.

Greg

Many thanks for the replies to my query. Not that I understand much of it, but I will study the man page, and hope I get more insight.

If you found this confusing, avoid the daemon mode unless you absolutely need it. Are you trying to rsync between 2 directories on one computer, or between 2 computers? If 2: Can you ssh between the 2 machines? Is rsync installed on both machines? Greg -- Greg Freemyer Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf The Norcross Group The Intersection of Evidence & Technology http://www.norcrossgroup.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Greg Freemyer wrote:

...

Many thanks for the replies to my query. Not that I understand much of it, but I will study the man page, and hope I get more insight.

If you found this confusing, avoid the daemon mode unless you absolutely need it.

Are you trying to rsync between 2 directories on one computer, or between 2 computers?

If 2:

Can you ssh between the 2 machines?

Is rsync installed on both machines?

Greg

Thank you very much Greg. I have a HDD with backup from a NAS server done via USB HDD. I want to restore the backup on another NAS server, and it also has USB on it. And yes I can ssh and rsync is installed on the NAS server. So this is in a way a rsync between 2 directories. -- Med venlig hilsen / Best regards Erik Jakobsen OZ4KK eja@urbakken.dk Registered Linux user #114875 - http://counter.li.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Anton Aylward wrote:

...

Erik Jakobsen said the following on 01/08/2009 11:11 AM:

...

...

Many thanks for the replies to my query. Not that I understand much of it, but I will study the man page, and hope I get more insight.

Isn't there an O'reilly book or page on it?

Did you see any ?. I think your reply is as good as "I have a problem,

David C. Rankin wrote: pleas help me". Can that be used to anything ?

No book needed.

In konqueror just enter #rsync or from the cli, just do rsync --help or man rsync. The man page is really quite good.

Ok David and thanks for your reply. I will try to follow your example. That what you wrote here is a real help. -- Med venlig hilsen / Best regards Erik Jakobsen OZ4KK eja@urbakken.dk Registered Linux user #114875 - http://counter.li.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Erik Jakobsen wrote:

...

Anton Aylward wrote:

...

Erik Jakobsen said the following on 01/08/2009 11:11 AM:

...

...

Many thanks for the replies to my query. Not that I understand much of it, but I will study the man page, and hope I get more insight.

Isn't there an O'reilly book or page on it?

Did you see any ?. I think your reply is as good as "I have a problem,

David C. Rankin wrote: pleas help me". Can that be used to anything ?

...

No book needed.

In konqueror just enter #rsync or from the cli, just do rsync --help or man rsync. The man page is really quite good.

Ok David and thanks for your reply. I will try to follow your example. That what you wrote here is a real help.

Erik, You will love rsync. Seriously. If I want to backup a set of files/directories, I do one of two things. If the list is small, I just right a small script with individual rsync calls. For example to backup my ~/.kde, ~/tmp and ~/linux/boxes directories to a usb stick that mounts as /media/KINGSTON, I would simply create a 'david' and 'david/linux' directory under /media/KINGSTON and write a 3 line script: #!/bin/bash rsync -auv ~/.kde /media/KINGSTON/david rsync -auv ~/tmp /media/KINGSTON/david rsync -auv ~/linux/boxes /media/KINGSTON/david/linux I would probably add to that a test to make sure KINGSTON was mounted to keep from filling up /media with my stuff: #!/bin/bash if mount | grep --quiet KINGSTON; then rsync -auv ~/.kde /media/KINGSTON/david rsync -auv ~/tmp /media/KINGSTON/david rsync -auv ~/linux/boxes /media/KINGSTON/david/linux else echo -e "\nKINGSTON isn't mounted -- try again\n" fi There are really only 2 things you need to remember about rsync to not get messed up: (1) a trailing slash on the source files means copy everything in the directory, no trailing slash means copy the directory itself along with the contents; example: rsync ~/.kde/ /media/KINGSTON means dump everything under the .kde directory in /media/KINGSTON, while: rsync ~/.kde /media/KINGSTON means copy the .kde directory itself to /media/KINGSTON/.kde and (2) if you do use the --files-from option, you must remember to explicitly give the -r option to make rsync copy recursively. Other than that it is just picking the options from rsync --help you want to use. Oh, and rule (3), the -n option is your friend! If I have a long list of files I want to copy, then I use --files-from to specify the text file to read the list of files to copy from. Let's say I wanted to backup parts of my linux directory to /media/KINGSTON/linux-save. The parts I want to backup are: gpg GPGQuickStart gpg-Quickstart.html gpl-2.0.txt HOWTO-encryptRootSwapHome.pdf icewm.mail.setting images iqcnumber irc kde kernel ldap linux_lBM-harden-desktop.pdf madwifi mail mount mozilla mysql network_IBM-Build_Linux_test_Network.pdf notes nut nvidia openoffice openSuSE I would create a text file containing the above list of files and directories and then just use 1 rsync command to backup everything at once. Create the /media/KINGSTON/linux-save directory. Then if I saved the above list in the file 'bkupfiles', then I can use rsync as follows: rsync -auvr --files-from=bkupfiles ~/linux /media/KINGSTON/linux-save (remembering rule 2. above) Which basically says rsync all the files-from bkupfiles beginning with the base directory of ~/linux and recursively copy all files and directories listed from ~/linux/<read from file> to the backup location of /media/KINGSTON/linux-save while preserving user, group, file creation time, special files and links. Another great rsync option for backup is the --delete option that will get rid of files on the backup set if they have been deleted from the source files. All in all, rsync is pretty simple. For remote backups, see the -e ssh option. Once you play with rsync for a while and see what it can do, you will wonder how you ever got along without it. Another great benefit of using a script to call rsync is that once you have it working the way you want, just throw it into a cron job to run a 0400 and forget about it. It will do the work for you while you sleep ;-) Seriously, looking at rsync --help, the only options I really ever use are: Options -v, --verbose increase verbosity -q, --quiet suppress non-error messages -a, --archive archive mode; equals -rlptgoD (no -H,-A,-X) -r, --recursive recurse into directories -R, --relative use relative path names -u, --update skip files that are newer on the receiver -l, --links copy symlinks as symlinks -p, --perms preserve permissions -o, --owner preserve owner (super-user only) -g, --group preserve group -t, --times preserve times -n, --dry-run show what would have been transferred ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -e, --rsh=COMMAND specify the remote shell to use --delete delete extraneous files from destination dirs --numeric-ids don't map uid/gid values by user/group name --exclude=PATTERN exclude files matching PATTERN --files-from=FILE read list of source-file names from FILE --port=PORT specify double-colon alternate port number --progress show progress during transfer (-h) --help show this help (-h works with no other options) -- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Greg Freemyer wrote:

On Thu, Jan 8, 2009 at 12:31 PM, Erik Jakobsen wrote:

...

Greg Freemyer wrote:

...

...

Many thanks for the replies to my query. Not that I understand much of it, but I will study the man page, and hope I get more insight.

If you found this confusing, avoid the daemon mode unless you absolutely need it.

Are you trying to rsync between 2 directories on one computer, or between 2 computers?

If 2:

Can you ssh between the 2 machines?

Is rsync installed on both machines?

Greg

Thank you very much Greg.

I have a HDD with backup from a NAS server done via USB HDD.

If the backup was made by rsync, then the backup is just a simple copy of the data being backed up.

The directory structure should be in place.

rsync is basically just a directory tree copy tool that has optimizations to copy only changes if possible.

So if you are copying a 10 MB file between 2 computers and 9.5 MB is the same as it was the last time you ran rsync, then rsync will compare the old copy to the new copy and just copy over the .5 MB of changes.

The key thing to understand is that when rsync is done the source and the destination should be identical.

...

I want to restore the backup on another NAS server, and it also has USB on it.

And yes I can ssh and rsync is installed on the NAS server.

So this is in a way a rsync between 2 directories.

If the data on the USB drive is just a rsync'ed copy of the original you don't even have to use rsync to restore it. (but you can easily enough).

Just plug the USB drive into the computer you want to restore to.

It should automount in the /media directory.

Use a file browser (Konqueror or Dolphin) to go the /media folder and see if you see the USB drive.

Then open that folder and you should see the files that were backed up.

Then just use the file browser to make a copy on the server wherever it is you want them.

Greg

Erik, Also, in my example I left out the machine-to-machine copying that you are talking about. That is what rsync was made for. Take for example 2 machines on my local lan that I want to backup files between. The normal form of rsync for this purpose is: rsync user@machine1:/path/to/src/files user@machine2:/path/to/destination Taking a real world example, my laptop is 'alchemy' and my server is 'nirvana'. I routinely download pictures from my family's digital cameras to my laptop and then rsync them to my server for permanent keeping. On my laptop I download the pictures to /home/david/pictures/family/20090108-deb or /home/david/pictures/family/20090108-sydney for example depending on whose camera they came from. On the server nirvana, all the pictures are stored in their original directories under /home/samba/pictures/family_pictures/year2009/ for example. After downloading new pictures to my laptop, I just call rsync with the same source and destination locations and it takes care of the rest: rsync -rtv david@alchemy:/home/david/pictures/family/ /home/samba/pictures/family_pictures/year2009/ I also backup the data from my law office across the internet each night using rsync and ssh. I use ssh keys for authentication so no password is needed allowing rsync to be run from a cron job. For remote rsync use, all you need to do is specify -e ssh as an option to tell rsync to use ssh as the shell. The remote backups go like this: rsync -arze ssh --delete --files-from=/home/david/linux/scripts/backup/workfiles david@rbpllc.com:/home/samba/ /home/samba/law The workfiles are: # cat workfiles office rankin joint/rankinguillory joint/rankinbertin joint/rankinallen joint/rga forms computer/hardware computer/software closed/rankin closed/rb closed/rg egwfiles/backup egwfiles/mydms scans I have the calls in a script that also give me a daily report of whether the backup succeeded or failed: if /usr/bin/rsync -arze ssh --delete --files-from=/home/david/linux/scripts/backup/workfiles david@rbpllc.com:/home/samba/ /home/samba/law then echo .... Rankin Law Firm backup ------------- [OK] >> $LOGFILE else echo .... Rankin Law Firm backup ------------- [Failed] >> $LOGFILE fi I also backup the mail and websites with 2 more rsync calls. That keeps a duplicate copy of my working files offsite at my home server as well as at work (In case the building burns, etc...) Spend 2 hours learning rsync and it will save you hundreds in the future. The scripting possibilities are endless. I wrote a generic little rsync backup that reads the source, destination and options from a single file and performs the backups accordingly. I'll include it here just for an example: #!/bin/bash --norc # ## This rsyncs the files specified in /home/backup/.data/rsfiles to this server # ## Check to make sure the script is run as root # ROOT_UID=0 E_NOTROOT=67 if [ "$UID" -ne "$ROOT_UID" ]; then echo -e "\nYou must be root to run this script.\nUser: $USER, UID: $UID can't! See below...\n" show_usage exit $E_NOTROOT fi LOGDIR=/home/backup/log LOGFILE=${LOGDIR}/${HOSTNAME}-rsserver.log LOGZIPFILE=${LOGFILE}.bz2 SRVFILES=/home/backup/.data/rsfiles [[ -d $LOGDIR ]] || { sudo mkdir -p /home/backup/log; sudo chown david:dcr $LOGDIR; } [[ -d $LOGDIR ]] || { echo -e "\n\tUnable to Create $LOGDIR\n"; exit 1; } [[ -r $SRVFILES ]] || { echo -e "\n\t$SRVFILES unreadable\n"; exit 1; } [[ -e $LOGZIPFILE ]] && bunzip2 --force $LOGZIPFILE #while [[ $# -gt 0 ]]; do case "$1" in -v|--verbose ) VERBOSE=1; echo -e "\ncase -v | --verbose activated: VERBOSE=${VERBOSE}" ;; -t|--test ) VERBOSE=2; echo -e "\ncase -t | --test activated: VERBOSE=${VERBOSE}" ;; esac [[ ! $1 ]] && VERBOSE=0 # shift #done # ## Begin rsync # echo "$(date '+%b %e %T') ${0##*/} ---------- backup to $HOSTNAME started ----------" >> $LOGFILE { while read OPT SRC DEST; do # check for comment lines if [[ ${OPT} =~ ['#'';'] ]]; then # echo "\${OPT} contains a comment: ${OPT}" continue fi # provide verbose output if cli agrument given case ${VERBOSE} in 0 ) if rsync ${OPT} ${SRC} ${DEST} >> /dev/null 2>&1; then echo "$(date '+%b %e %T') ${0##*/} rsync of ${SRC} to ${DEST} Succeeded" >> $LOGFILE else echo "$(date '+%b %e %T') ${0##*/} rsync of ${SRC} to ${DEST} FAILED" >> $LOGFILE fi;; 1 ) echo -e "\nOPT = ${OPT}; SRC = ${SRC}; DEST = ${DEST}" echo -e "CLI = rsync ${OPT} ${SRC} ${DEST}" if rsync ${OPT} ${SRC} ${DEST} >> $LOGFILE 2>&1; then echo -e "rsync of ${SRC}\t\tSucceeded" echo "$(date '+%b %e %T') ${0##*/} rsync of ${SRC} to ${DEST} Succeeded" >> $LOGFILE else echo -e "rsync of ${SRC}\t\tFailed!" echo "$(date '+%b %e %T') ${0##*/} rsync of ${SRC} to ${DEST} FAILED" >> $LOGFILE fi;; 2 ) echo -e "\nOPT = ${OPT}; SRC = ${SRC}; DEST = ${DEST}" echo -e "CLI = rsync ${OPT} ${SRC} ${DEST}";; * ) echo -e "\n\tUsage: ${0##*/} [ -t print file list and exit, -v verbose ]\n" exit 1;; esac done } < "$SRVFILES" # echo "$(date '+%b %e %T') ${0##*/} ---------- backup to $HOSTNAME complete ----------" >> $LOGFILE bzip2 $LOGFILE exit 0 Where the data file 'rsfiles' is: [15:55 ecstasy/usr/local/bin] # cat /home/backup/.data/rsfiles ## Datafile for the rsync script. Each line below is formatted as: ## ## rsync options source file location destination ## -av\ --delete david@nirvana:/home/samba/skyline /home/samba # ## www files and directories # -av david@nirvana:/srv/www/css /srv/www -av david@nirvana:/srv/www/dcr /srv/www -av david@nirvana:/srv/www/gd /srv/www -av david@nirvana:/srv/www/icons /srv/www -av david@nirvana:/srv/www/images /srv/www -av david@nirvana:/srv/www/js /srv/www -av david@nirvana:/srv/www/skyline-live.session /srv/www/ecstasy-live.session -av david@nirvana:/srv/www/skyline-live.webprj /srv/www/ecstasy-live.webprj -av david@nirvana:/srv/www/templates /srv/www -av david@nirvana:/srv/www/toolbars /srv/www -av david@nirvana:/srv/www/vhosts /srv/www # ## website content files and directories # -av david@nirvana:/srv/www/htdocs/linux /srv/www/htdocs -av david@nirvana:/srv/www/htdocs/law/license /srv/www/htdocs/law -av david@nirvana:/srv/www/htdocs/**.php /srv/www/htdocs -av david@nirvana:/srv/www/htdocs/favicon.ico /srv/www/htdocs -av david@nirvana:/srv/www/tmp/skyline /srv/www/tmp Now you know as much about rsync as I do. Have fun.... -- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org