Message-Id: <5.0.2.1.0.20001214210037.02806e80@claborn.net> Date: Thu, 14 Dec 2000 21:29:24 -0600 From: wilson@claborn.net (Jonathan Wilson) Subject: Best backup system/plan for maximum safety - what do y'all do? Howdy, Now that we've spent a good deal of time setting up our system of servers, it's occurred to us that we should be dome something more rigorous then once a week manual "cp -a /etc /backup/$DATE" First off, we do NOT want to buy a commercial app like Arieka or BRU. We want to use Free Software, and will write it myself if we can't find something good enough. Also I'm NOT going to use tape. Hard drive space is cheap, and so are CDs. And faster, IMHO. That said, I've had a good look over freshmeat and found more then enough apps/scripts that use tar, cpio, ssh and various compressions... enough to satisfy me. So what I'm really concerned about is not the tools to do this with but the methodology of it. I can think of several considerations: 1. At least some level of backup needs to happen every day, i.e. at least a back up of /etc to a local tarball. 2. Though we have lots of space and can get more, we can't keep archives forever. There probably should be some sort of trailing increments (whatever that means..) like every day for the past 4 weeks, every other day of 2 and 3 months ago, Sunday and Wednesday of 4 and 5 months ago, once a month for a year back after that. Or whatever..... maybe some sort of versioning system like CVS 3. We need to be able to get single files back out of the archive without a big to-do. You know, like if someone edit's Apache's conf file right before they leave and we find out the next morning that it's screwed up and we want to go back a day. This is my primary reason for not wanting to use tapes, and wanting to use hard disks. 4. We have local workstations/servers and remote (co-located) ones. We'll probably swap data - both local back ups and remote backups need to be stored BOTH locally and remotely. Why? Well, if our office were to burn down, for instance, all of our local data would already be stored in town. If our ISP burns down, we'll have all of our server data stored locally. 5. we need to be able to do a very quick reinstall if, say, a hard drive totally fails, or a cracker breaks in (in both cases it would require starting from scratch). 6. We've thought about combining a "honeypot", as they are some times called, with a back up box. (A honeypot, as I understand it, is basically a box that is not running any services at all. Once a night (or whenever) it makes a connection to remote machines to collect log files (and I'm thinking back up files). All ports are closed and no incoming connection is ever accepted (it would be a local box with only direct keyboard control - no telnet or ssh). 7. It HAS to be automated. We've found that anything that waits for human intervention doesn't get done reliably. 8. It needs to have some sort of failover/failproof -ness. Something like it emails 2 other servers whenever the job is complete. Something should run on the other servers that checks for that email every night. If they don't receive the email they can email the admins and/or page their beepers. 9. (kind of a repeat) We MUST be able to do a quick restore of a whole drive in case of emergency. I've never had to restore a whole drive before, but I've been around people that /thought/ they had a back up system going well, and after a disk failure, they found out the hard way that they didn't really have a way to restore a whole drive. That's no good. What's the best solution - some sort of drive image we can dd back onto the drive? That would take a LOT of space. The idea here is we need a total restore - we don't have time to reinstall Linux form our distro CD then manually copy the files we need out of our backups. <p>Any and all advise is very much appreciated. I'd really like to hear from people who have working backup systems. Is there anything I forgot, or does anyone see something wrong with anything I said? Thanks! JW ---------------------------------------- Jonathan Wilson System Administrator Cedar Creek Software http://www.cedarcreeksoftware.com Central Texas IT http://www.centraltexasit.com