data backup; my security risks.
dear group, Can you give me some insight on how to store data (photos) savely and what kind of procedure I can follow best. We produce about 200Gig raw images a year. the workflow now is: work on a XP-box and copy the raw's and the jpeg's to a SuSE-box with winscp as a backup on removable harddisks. (I stopped making cd-s as we produce to much data, storing raw's and jpeg's) Will the removable disks last (ide western-digital)? now and again they're used on the system. We were used to shoot on slide film, now our entire workflow has changed and it troubles me to meet the time when images can't be retreived. regards, piet
HD's, esp Western Digital's, had experience failing more then others so I would invest in a good AIT-3 tape drive. I'm sure there's drives compatible with SuSE esp the newer versions. I'd try reading up on the GFS method (grandfather, father, son) of backup sceduling so you can optomize it based on your setup/use.
I would also look at making dated tar archives of your backups that you would store on the SuSE box (again you need to decide which you want to store locally eg. most recient or oldest or whatever) that way you have them accessable locally and you can store the tapes offsite (highly suggested). Based on your budget restrictions/how much time your willing to spend, see where you want to go with it.
You could also look at getting a dvd burner for the XP box and burning the most critical images/clients straight to DVD. It would probably be the cheapest but not the most logical as DVD's are just as fragile as any other CD media. Again you have tons of options, these are just a few off the top of my head.
Matt
piet
piet schrieb:
dear group, Can you give me some insight on how to store data (photos) savely and what kind of procedure I can follow best. We produce about 200Gig raw images a year. the workflow now is: work on a XP-box and copy the raw's and the jpeg's to a SuSE-box with winscp as a backup on removable harddisks. (I stopped making cd-s as we produce to much data, storing raw's and jpeg's)
Will the removable disks last (ide western-digital)? now and again they're used on the system.
We were used to shoot on slide film, now our entire workflow has changed and it troubles me to meet the time when images can't be retreived. regards, piet
Sounds like a photostudio or medical photo industry. I would do so: - main system raid 5, scsi - temporary backup over nas (network attached storage) for latest data - nas is synced with your data every day over night - daily/weekly backups of all media (with tar) on (with gfs like the solution in the other mail) 200 GB or greater tapes (e.g. http://www.storagebysony.com/products/productmain.asp?id=142) or DVD-RAM Jukebox A DVD jukebox has the advantage that you can later access the data within the dvd-tower, e.g. here: http://www.cddimensions.com/cd_jukebox/dvd-jukebox.asp 45 - 700 discs/max. 6 drives per jukebox There are even solutions from kodak for medical imaging storage. O.K. this may cost a hell, but lost data can cost more money depending on the importance of that data. Philippe
Philippe Vogel wrote:
piet schrieb:
dear group, Can you give me some insight on how to store data (photos) savely and what kind of procedure I can follow best. We produce about 200Gig raw images a year. the workflow now is: work on a XP-box and copy the raw's and the jpeg's to a SuSE-box with winscp as a backup on removable harddisks. (I stopped making cd-s as we produce to much data, storing raw's and jpeg's)
Will the removable disks last (ide western-digital)? now and again they're used on the system.
We were used to shoot on slide film, now our entire workflow has changed and it troubles me to meet the time when images can't be retreived. regards, piet
Sounds like a photostudio or medical photo industry.
I would do so:
- main system raid 5, scsi - temporary backup over nas (network attached storage) for latest data - nas is synced with your data every day over night - daily/weekly backups of all media (with tar) on (with gfs like the solution in the other mail)
200 GB or greater tapes (e.g. http://www.storagebysony.com/products/productmain.asp?id=142) or DVD-RAM Jukebox
A DVD jukebox has the advantage that you can later access the data within the dvd-tower, e.g. here:
http://www.cddimensions.com/cd_jukebox/dvd-jukebox.asp
45 - 700 discs/max. 6 drives per jukebox
There are even solutions from kodak for medical imaging storage. O.K. this may cost a hell, but lost data can cost more money depending on the importance of that data.
Philippe
thanks for replying. well your guess is close, we = two freelance photographers and in the old days storage wasn't so much a worry but finding images was, now shooting digital for almost 2 years, I begin to worry a bit about storage. The financial aspect might be a bottle neck. thanks again, piet
Hi, just my 2 cents: If money is an issue why not backup on a RAID system only? This can be a quite cheap SATA-NAS (just bought one with 1.2 TB netto for about 5,000 euros). With RAID-5 and a couple of spare-disks I guess you are save enough, especially compared to single HDs. ;) Of course it is best to do additional tape backups - but that is quite expensive for these amounts of data and I doubt that you need this extra savety. Regards Peter
Philippe Vogel wrote:
piet schrieb:
dear group, Can you give me some insight on how to store data (photos) savely and what kind of procedure I can follow best. We produce about 200Gig raw images a year. the workflow now is: work on a XP-box and copy the raw's and the jpeg's to a SuSE-box with winscp as a backup on removable harddisks. (I stopped making cd-s as we produce to much data, storing raw's and jpeg's)
Will the removable disks last (ide western-digital)? now and again they're used on the system.
We were used to shoot on slide film, now our entire workflow has changed and it troubles me to meet the time when images can't be retreived. regards, piet
Sounds like a photostudio or medical photo industry.
I would do so:
- main system raid 5, scsi - temporary backup over nas (network attached storage) for latest data - nas is synced with your data every day over night - daily/weekly backups of all media (with tar) on (with gfs like the solution in the other mail)
200 GB or greater tapes (e.g. http://www.storagebysony.com/products/productmain.asp?id=142) or DVD-RAM Jukebox
A DVD jukebox has the advantage that you can later access the data within the dvd-tower, e.g. here:
http://www.cddimensions.com/cd_jukebox/dvd-jukebox.asp
45 - 700 discs/max. 6 drives per jukebox
There are even solutions from kodak for medical imaging storage. O.K. this may cost a hell, but lost data can cost more money depending on the importance of that data.
Philippe
thanks for replying. well your guess is close, we = two freelance photographers and in the old days storage wasn't so much a worry but finding images was, now shooting digital for almost 2 years, I begin to worry a bit about storage.
The financial aspect might be a bottle neck.
thanks again, piet
Hello,
if money is not the Problem, why u dont use True Image
:), u can safe the complete HD while ur system is
workin and can write it back via CD and your System is
fully avail after a short time. Runs perfect with
kernel 2.4.xx and costs only 699 :)
Best Regards
Boris
--- Peter Romianowski
Hi,
just my 2 cents:
If money is an issue why not backup on a RAID system only? This can be a quite cheap SATA-NAS (just bought one with 1.2 TB netto for about 5,000 euros). With RAID-5 and a couple of spare-disks I guess you are save enough, especially compared to single HDs. ;)
Of course it is best to do additional tape backups - but that is quite expensive for these amounts of data and I doubt that you need this extra savety.
Regards Peter
Philippe Vogel wrote:
piet schrieb:
dear group, Can you give me some insight on how to store data (photos) savely and what kind of procedure I can follow best. We produce about 200Gig raw images a year. the workflow now is: work on a XP-box and copy the raw's and the jpeg's to a SuSE-box with winscp as a backup on removable harddisks. (I stopped making cd-s as we produce to much data, storing raw's and jpeg's)
Will the removable disks last (ide western-digital)? now and again they're used on the system.
We were used to shoot on slide film, now our entire workflow has changed and it troubles me to meet the time when images can't be retreived. regards, piet
Sounds like a photostudio or medical photo industry.
I would do so:
- main system raid 5, scsi - temporary backup over nas (network attached storage) for latest data - nas is synced with your data every day over night - daily/weekly backups of all media (with tar) on (with gfs like the solution in the other mail)
200 GB or greater tapes (e.g.
http://www.storagebysony.com/products/productmain.asp?id=142)
or DVD-RAM Jukebox
A DVD jukebox has the advantage that you can later access the data within the dvd-tower, e.g. here:
http://www.cddimensions.com/cd_jukebox/dvd-jukebox.asp
45 - 700 discs/max. 6 drives per jukebox
There are even solutions from kodak for medical
imaging storage.
O.K. this may cost a hell, but lost data can cost more money depending on the importance of that data.
Philippe
thanks for replying. well your guess is close, we = two freelance photographers and in the old days storage wasn't so much a worry but finding images was, now shooting digital for almost 2 years, I begin to worry a bit about storage.
The financial aspect might be a bottle neck.
thanks again, piet
-- Check the headers for your unsubscription address For additional commands, e-mail: suse-security-help@suse.com Security-related bug reports go to security@suse.de, not here
___________________________________________________________ Gesendet von Yahoo! Mail - Jetzt mit 100MB Speicher kostenlos - Hier anmelden: http://mail.yahoo.de
Becuse if the building burns down, or you have a fire and both units take water damage, or someone breaks in and steals them, etc, etc you will cry. It doesn't take a major catastrophic event to wipe out the computer equipment in a building or render the data unretrievable. There's a reason people invest in offsite backups. Personally I have two file servers at home (I work from home) on seperate UPS's. I also have the critical data on a remote server and I burn DVD's/CD's which I place in my safety deposit box. If my house burns down, or the server colo burns down I'm not utterly f**ked. Kurt Seifried, kurt@seifried.org A15B BEE5 B391 B9AD B0EF AEB0 AD63 0B4E AD56 E574 http://seifried.org/security/
On 11/24/04 8:09 PM, "Kurt Seifried"
Personally I have two file servers at home (I work from home) on seperate UPS's. I also have the critical data on a remote server and I burn DVD's/CD's which I place in my safety deposit box. If my house burns down, or the server colo burns down I'm not utterly f**ked.
Kurt, You could also get DSL or Cable on, say your parents, inlaw's, brothers, sister's house and have your servers Rsync to a server at that location. That gives them fast internet access, maybe split the cost(?) and you have another off site system... _______________________________________ D-Link wireless with 4 port wired... Gave them wireless and wired... Just put the server and router somewhere no kids will touch it... I have seen people screw a cheap 1u box to the (finished) basement ceiling. This is what he did: UPS - $50 Router - $10 1u box - $200 Mo-board - $150(?) 250 gig - $120 512 Ram - $100(?) DSL $360 _______ $990/first year - good night sleep Personally, I would have gone with a used box... -- Thanks, George Suppose you were an idiot and suppose you were a member of Congress. But I repeat myself. Mark Twain
On Thursday 25 November 2004 01:09, Kurt Seifried wrote:
Becuse if the building burns down, or you have a fire and both units take water damage, or someone breaks in and steals them, etc, etc you will cry. It doesn't take a major catastrophic event to wipe out the computer equipment in a building or render the data unretrievable. There's a reason people invest in offsite backups. Personally I have two file servers at home (I work from home) on seperate UPS's. I also have the critical data on a remote server and I burn DVD's/CD's which I place in my safety deposit box. If my house burns down, or the server colo burns down I'm not utterly f**ked.
One word.... "verify" Okay, you're possibly fine with your thousand backups, but if someone relies on a single backup, or even two, there's always a chance that the backup files lose the plot. It would be a little embarrassing to find that the CD (for example) you have all your data on is useless just as you're attempting to restore from it! If using removable media as a backup, you may have a utility in whatever backup software you use to verify against a database of what is supposed to be there. If you're just copying from a hard drive to a CD/DVD/tape then at least create a list of md5s which you store in a number of safe places, with possibly a hard copy in a filing cabinet or two ;-) If you're copying to another hard disk somewhere then you can use AIDE/Tripwire or whatever to monitor the system for any changes. While you're at it, do the same for your primary disk storage systems. Remember to update the database when you make an intentional change. If you don't and you get used to cron emailing you a list of changes you'll ignore them and not notice _something's_happened_. Trust me on this! Anyway, there's no point in backing anything up unless you can be sure the backup works. Tom. Tom.
participants (8)
-
Boris Trenn -
george -
Kurt Seifried -
MB -
Peter Romianowski -
Philippe Vogel -
piet -
Thomas Knight