I have tried tar, rsyn, and rdiff backup all of which have had similar memory leak problems causing kernel panics. Tar caused the problem when writing many files to a single large archive. Rsync and rdiff-backup caused problems with the high volume of files.
I looked over the log files in the last 3 crashes ( 12 oct, 9 sept, 1 sept) and I couldn't asociate it with any cron or script or process.... only if one of my script hangs... that is again strange, because I do only copy, database dumps...
I also do rsync with another share from network.
I have tried tar, rsyn, and rdiff backup all of which have had similar memory leak problems causing kernel panics. Tar caused the problem when writing many files to a single large archive. Rsync and rdiff-backup caused problems with the high volume of files.
I have now resorted to "batching" my backups with date stamps inserted into log files so that I can trace any further errors to a direct script and time. When backing up using rsynch, try inserting date stamps one dir at a time, that way at least you will know if a perticular dir/process is causing
problem.
Bit of a pain in the ass, but less of a pain than a panic.
The dir structure I backup is basically: /top/of/dirstructure/0/0/0/0/0 To /top/of.dir/structure/9/9/9/9/9
With each dir containing 0/0/0/0/0 0/0/0/0/1 0/0/0/0/2 0/0/0/0/...9 0/0/0/1/0 0/0/0/1/1 0/0/0/1/2 Etc
So large number of dirs.
I have scripted
#!/bin/bash
rm -rf /var/log/backup.log echo "Start Dir Struc 0" `date` /var/log/backup.log Rdiff/rsync /top/of/dirstructure/0/ echo "End Dir Struc 0" `date` /var/log/backup.log sleep 2 echo "Start Dir Struc 1" `date` /var/log/backup.log Rdiff/rsync /top/of/dirstructure/1/ echo "End Dir Struc 1" `date` /var/log/backup.log sleep 2
Etc...
This way, I was able to ensure that if anything went wrong I would always know very closely where and could check my dir contents for file allocations that were haywire.
Also, breaking it up into many processes while it makes the backup
High volume of files with rsync? That doesn't make any sense. Rsync is designed to compare the two systems to decide what files have changed, as opposed to everything. Also, does reiserfs not have a filesystem dumping utility like the `dump/restore` utilities for ext2/3 ? That is really what you should be using, if one exists. If not, I would get ahold of the developer folks at Novell and ask them why they haven't developed one yet. :-) Unfortunately, I fear as though we've wandered off the path of on-topic discussions for this list. I certainly don't mind it, but others might. What other suse mailing lists are out there? Tim Rainier Information Services, Kalsec, INC trainier@kalsec.com "b@rry" wrote on 10/13/2005 06:11:32 AM: the process
take much longer, I have not had a single issue since doing this...