[opensuse] simple data security: RAID1 or rsync?
after seeing a few threads re. RAID arrays, problems booting off them, and extra strain the surviving member of an array is put under when it has to sync a new HDD after it's (only) brother's failure, i doubt the wisdom of my original idea to set up two identical 2TB drives as RAID1 array. wouldn't it be safer (not to mention simpler) to just rsync the simple directory tree i'm concerned about to the other HDD every 15 min or so? if either of the two HDDs fails, the other one still has the important data. it's not my desktop, but a file server i'm putting together for my employer. the only thing he cares about is preserving the data in question. if or when one of the HDDs fails, there's no need to preserve the operating system or anything else. -- phani. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 05/06/2012 03:15 AM, phanisvara das wrote:
after seeing a few threads re. RAID arrays, problems booting off them, and extra strain the surviving member of an array is put under when it has to sync a new HDD after it's (only) brother's failure, i doubt the wisdom of my original idea to set up two identical 2TB drives as RAID1 array.
wouldn't it be safer (not to mention simpler) to just rsync the simple directory tree i'm concerned about to the other HDD every 15 min or so? if either of the two HDDs fails, the other one still has the important data.
it's not my desktop, but a file server i'm putting together for my employer. the only thing he cares about is preserving the data in question. if or when one of the HDDs fails, there's no need to preserve the operating system or anything else.
Hi, I'm just a desktop "business" user - accounting, business files, etc. and have two identical 500GB drives. I started with RAID and came to the same conclusion with the hassles, then I switched to rsync. However, I just do rsync about two or three times a week. With your setup, it will probably be a wash - meaning, every time you run rsync, it has to check the whole tree EVERY TIME to see what has been updated, deleted, added, etc. which takes time for the primary drive (at least from my understanding of what is going on behind the scenes). So to run rsync every 15 minutes or so, is going to bog down the primary drive. Also, with rsync (at least with my limited knowledge of the options), if you rename a file - it deleted the old one and copies the new one - instead of remembering you only did a rename. At least from my humble opinion, you should stick with RAID. Duaine - Hechler Piano & Organ Services -- Duaine Hechler Piano, Player Piano, Pump Organ Tuning, Servicing& Rebuilding Reed Organ Society Member Florissant, MO 63034 (314) 838-5587 dahechler@att.net www.hechlerpianoandorgan.com -- Home& Business user of Linux - 11 years -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sun, 06 May 2012 14:13:59 +0530, Duaine Hechler <dahechler@att.net> wrote:
I started with RAID and came to the same conclusion with the hassles, then I switched to rsync. However, I just do rsync about two or three times a week. With your setup, it will probably be a wash - meaning, every time you run rsync, it has to check the whole tree EVERY TIME to see what has been updated, deleted, added, etc. which takes time for the primary drive (at least from my understanding of what is going on behind the scenes). So to run rsync every 15 minutes or so, is going to bog down the primary drive. Also, with rsync (at least with my limited knowledge of the options), if you rename a file - it deleted the old one and copies the new one - instead of remembering you only did a rename. At least from my humble opinion, you should stick with RAID.
thank you for your toughts. i think you're right that the cost of unconditionally rsync-ing the whole drive every 15 min would be too high. but thinking through the setup i'm planning, that wouldn't be necessary. it's (almost) computer illiterate library staff who are scanning & photographing old books and manuscripts, saving the resulting image files. i won't let them put things into the final archives, but they'll save everything into their HOMEs, from where i'll pick up the images remotely, rename them according to some standard still to be made up (date, time, location in the filename), and move them into the actual archive. means i'd have to rsync the HOMEs frequently during work hours, with not too many changes, and once the archive when i'm done renaming & moving everything at night. to me this doesn't sound like too much strain from rsync, but i'll have to get deeper into rsync and it's various options. i'm afraid i'm overlooking something obvious, that's why i'm asking here, where many have much more experience with this type of thing. -- phani. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
phanisvara das wrote:
i think you're right that the cost of unconditionally rsync-ing the whole drive every 15 min would be too high. but thinking through the setup i'm planning, that wouldn't be necessary. [snip] i'm afraid i'm overlooking something obvious, that's why i'm asking here, where many have much more experience with this type of thing.
RAID1 would be a lot easier and lot less effort (than re-inventing the wheel). -- Per Jessen, Zürich (14.0°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 5/6/2012 4:25 AM, Per Jessen wrote:
phanisvara das wrote:
i think you're right that the cost of unconditionally rsync-ing the whole drive every 15 min would be too high. but thinking through the setup i'm planning, that wouldn't be necessary. [snip] i'm afraid i'm overlooking something obvious, that's why i'm asking here, where many have much more experience with this type of thing.
RAID1 would be a lot easier and lot less effort (than re-inventing the wheel).
Exactly. And as for the "strain" when one member dies, how is this "strain" different than the daily thrashing of rsync? I have software raid 1, and raid 5 arrays where a disk has failed and the rebuild happened without any un-availability, and happened in the background such that the users were never even aware it was happening. I use Rsync between machines (some offsite) and raid 1 within a machine. I'm a belt and suspenders sort of guy. Raid, especially Raid 1 is as easy as falling off a log these days. -- _____________________________________ ---This space for rent--- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-05-06 21:39, John Andersen wrote:
And as for the "strain" when one member dies, how is this "strain" different than the daily thrashing of rsync?
It is the daily load on one disk + the load of replication. - -- Cheers / Saludos, Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk+m1dsACgkQIvFNjefEBxoszQCfS9tKBBA6eS3GplHcg9HSUU4v XxkAoLEP2NoqD6bn7ntkUj9EpBdPE7Mv =Ridf -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, 07 May 2012 01:09:13 +0530, John Andersen <jsamyth@gmail.com> wrote:
And as for the "strain" when one member dies, how is this "strain" different than the daily thrashing of rsync?
not much, only that it happens all at once, perhaps when the disk is already well advanced in it's life.
I have software raid 1, and raid 5 arrays where a disk has failedand the rebuild happened without any un-availability, and happened in the background such that the users were never even aware it was happening.
I use Rsync between machines (some offsite) and raid 1 within a machine. I'm a belt and suspenders sort of guy.
:) i like that; but right now i can't afford both, so have to go for one or the other.
Raid, especially Raid 1 is as easy as falling off a log these days.
that's what i'm suspecting. never used raid; only LVM once, when trying out fedora, and i didn't like much either of them. would it be correct to say security-wise it's a toss, RAID1 or rsync? if the answer to that was "yes," i'd go with RAID. less trouble day-to-day, and i'd learn something new. -- phani. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
phanisvara das said the following on 05/06/2012 03:51 PM:
would it be correct to say security-wise it's a toss, RAID1 or rsync? if the answer to that was "yes," i'd go with RAID. less trouble day-to-day, and i'd learn something new.
Since security encompasses all of * Availability * Integrity * Confidentiality yes, it doesn't matter whether you use RAID for availability or rsync for backups (aka integrity), you can call it Security if you want. -- "Ahhh. A man with a sharp wit. Someone ought to take it away from him before he cuts himself." - Peter da Silva -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, 07 May 2012 03:34:09 +0530, Anton Aylward <opensuse@antonaylward.com> wrote:
* Availability * Integrity * Confidentiality
no, for me in this context "security" means mainly integrity. availability is secondary as long as the data survives. confidentiality isn't a consideration in the local network, and the 'server' won't be available over the internet except via SSH with DSA key.
yes, it doesn't matter whether you use RAID for availability or rsync for backups (aka integrity), you can call it Security if you want.
ok, not calling it security. what i'm interested in is integrity, and that's better served with rsync you say; thanks. -- phani. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
phanisvara das said the following on 05/06/2012 05:25 AM:
i'm afraid i'm overlooking something obvious, that's why i'm asking here, where many have much more experience with this type of thing.
I suspect we all overlook things - its the old "Where you stand depends on where you sit"//"Context is everything". But Per has a few pertinent observations. 1. RAID is about availability, not backup. 2. Keeping the librarian's /home on the file server using NFS, a very well established technique that dates back to SUN and the 1980s, does address part of your problem. 3. We've discussed using LVM instead of RAID in the past. Its easier to set up LVM mirroring of partitions *AND* you can use LVM to do disk-to-disk backup. 4. Ultimately backup means transferring to another media; tape, removable disk, or something, and archiving it securely. This has nothing to do with RAID or RSYNC. I know, first hand, that a) NFS mount of users home directory works well. b) LVM is easy to set up if you start the disk with it, but pernickity if you try to 'convert' and existing disk, partition or drive. You need to have experience (i.e. have got it wrong catastrophically in the past and figured out why). So start with a new set of drives. If you do that its easy. c) LVM snapshot is in real-time until you tell it to stop. d) You can backup snapshots onto DVD. I have a policy of using 4G partitions to facilitate that. Its nice to have them as mountable file systems. I'm of the opinion that for what I'm doing, RAID-per-RAID is too complex, involves too many decisions and too many drives. With LVM I can use any number of any drive of any old size according to my budget, add them or remove them. KISS. But YMMV. -- A program designed for inputs from people is usually stressed beyond breaking point by computer-generated inputs. -- Dennis Ritchie -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-05-06 11:25, phanisvara das wrote:
i'm afraid i'm overlooking something obvious, that's why i'm asking here, where many have much more experience with this type of thing.
You need both raid and a backup, and IMO it is more important the backup. If you make a goof and delete something, the data will be deleted on all the disks of the array. If you have a good backup strategy, you can recover deleted files. It is as simple as that. All that you do is done to the entire raid array. Any bad thing happens to the array - you are protected only from one type of failure: a disk failure. Note that with rsync you can have a history of changes: the new directory has the new files, and hardlinks to the old files that exist in the previous directory. But doing backups every 15 minutes is excessive wear. You would need a third disk with daily backups, and powered off the rest of the day. - -- Cheers / Saludos, Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk+mc8oACgkQIvFNjefEBxpbGgCbBa7EW4y6D3HpGjGubTwdtTfz xF0AoKgrjigzHKVj/wWWmY83YsTg3cMV =B7/a -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sunday, May 06, 2012 08:51 AM Carlos E. R. wrote:
On 2012-05-06 11:25, phanisvara das wrote:
i'm afraid i'm overlooking something obvious, that's why i'm asking here, where many have much more experience with this type of thing.
You need both raid and a backup, and IMO it is more important the backup.
If you make a goof and delete something, the data will be deleted on all the disks of the array. If you have a good backup strategy, you can recover deleted files.
It is as simple as that.
All that you do is done to the entire raid array. Any bad thing happens to the array - you are protected only from one type of failure: a disk failure.
Note that with rsync you can have a history of changes: the new directory has the new files, and hardlinks to the old files that exist in the previous directory.
But doing backups every 15 minutes is excessive wear. You would need a third disk with daily backups, and powered off the rest of the day.
fwiw on my primary machine which requires 100% uptime I use RAID with an additional disk which I mirror with rsync as a failover. In my rsync script I use rsync parms to retain a different fstab and menu.lst which enables it to be bootable at any time. I run the script via cron during lunch and in the evening, so it's not a real time mirror but it's close enough. I also maintain a copy on another machine on my gb lan as protection against a catastrophic failure (i.e., not just disk failure) of the primary machine. Cron runs this rsync script once every evening. Storage is cheap nowadays. Once this is all set up, everything is automatic and I don't have to pay any attention to it, other than the scripts mailing me a message indicating success or failure in the run. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-05-06 16:55, Dennis Gallien wrote:
Storage is cheap nowadays.
No, it is not. All the hard disk manufacturers had their main factories in the same country and they were flooded. Prices doubled and have not come down since. - -- Cheers / Saludos, Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk+mkewACgkQIvFNjefEBxpKUACgnBp1poMo8Xl18XS3lzuxZWFe srcAoIvngWxZnG54yRe8jpbcs0fUdDCk =XYpj -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sunday, May 06, 2012 08:59:56 AM Carlos E. R. wrote:
On 2012-05-06 16:55, Dennis Gallien wrote:
Storage is cheap nowadays.
No, it is not. All the hard disk manufacturers had their main factories in the same country and they were flooded. Prices doubled and have not come down since.
Depends on your point of view. A few years ago when drives weren't so big, I thought $100 for a 20g hard drive was a great price. Now I can get 2Tb for about $120. The prices are coming back down. Just slowly. Mike -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sunday, May 06, 2012 10:59 AM Carlos E. R. wrote:
On 2012-05-06 16:55, Dennis Gallien wrote:
Storage is cheap nowadays.
No, it is not. All the hard disk manufacturers had their main factories in the same country and they were flooded. Prices doubled and have not come down since.
Well, OK, not *right now*. But before that disaster. And I meant relatively speaking. My last WD TB was about the same price as the comparable drive with ~a tenth of the capacity 5 years earlier; that's what I was ref'g to. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2012-05-06 16:55, Dennis Gallien wrote:
Storage is cheap nowadays.
No, it is not. All the hard disk manufacturers had their main factories in the same country and they were flooded. Prices doubled and have not come down since.
The price per Gb is still much less than it was ten years ago. -- Per Jessen, Zürich (16.9°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sun, 06 May 2012 20:25:56 +0530, Dennis Gallien <dwgallien@gmail.com> wrote:
fwiw on my primary machine which requires 100% uptime I use RAID with an additional disk which I mirror with rsync as a failover.
thanks to all of you for your replies. i understand things better now, mainly that RAID isn't so much to keep data safe, but to keep the server available in case of HDD failure. to keep data safe there needs to be another copy, preferably on a different machine. problem is that while storage is much cheaper than a few years ago, that's a relative statement, and for us it's neither cheap nor, and that's even more important, easily available. the only TB size HDDs one gets in kolkata these days are those "green" ones, that park every few seconds, only to be awakened again a couple sec.s later. there's supposedly a DOS utility to adjust that, but google finds mixed reports re. this. i can't easily get another 2TB drive now and will have to make do with two of them for the time being. for us data security is very important, while availability comes second. if that file server isn't available, there's still 'normal' sized HDDs in the workstations, plus a battery of USB drives here & there. (that used to be our "data storage solution" until now.) this way there's no problem saving new images, and access to the old one isn't time critical. (there's a separate web server, which holds the images used for publication, and doesn't depend on the file server in real time.) in this scenario i think it's best to go ahead without RAID, but rsyncing everything important to the second HDD -- until i manage to get another 2TB HDD, when i can implement both, RAID & backup copy. thanks again, -- phani. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Hello, On Sun, 06 May 2012, phanisvara das wrote:
the only TB size HDDs one gets in kolkata these days are those "green" ones, that park every few seconds, only to be awakened again a couple sec.s later. there's supposedly a DOS utility to adjust that, but google finds mixed reports re. this.
I too have only those disks, but they don't park. You can adjust that with 'hdparm', namely the Parameter -S. Also, look at your power-management settings in yast. Or adjust the pm-profile setting SATA_ALPM="max_performance" in the profile. I just copied the 'balanced' to a new name, changed that setting and then activated that profile (/etc/pm-profiler.conf resp. /etc/sysconfig/pm-profiler).
i can't easily get another 2TB drive now and will have to make do with two of them for the time being.
You should. Having identical drives of the same age in a RAID ... HTH, -dnh -- "All mushrooms are edible. However, some of them only once" -- Ino!~ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sun, 06 May 2012 21:58:51 +0530, David Haller <dnh@opensuse.org> wrote:
Hello,
On Sun, 06 May 2012, phanisvara das wrote:
the only TB size HDDs one gets in kolkata these days are those "green" ones, that park every few seconds, only to be awakened again a couple sec.s later. there's supposedly a DOS utility to adjust that, but google finds mixed reports re. this.
I too have only those disks, but they don't park. You can adjust that with 'hdparm', namely the Parameter -S. Also, look at your power-management settings in yast. Or adjust the pm-profile setting
SATA_ALPM="max_performance"
in the profile. I just copied the 'balanced' to a new name, changed that setting and then activated that profile (/etc/pm-profiler.conf resp. /etc/sysconfig/pm-profiler).
ah, that's interesting. which brand / model do you use?
i can't easily get another 2TB drive now and will have to make do with two of them for the time being.
You should. Having identical drives of the same age in a RAID ...
i know... :( -- phani. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 05/06/2012 11:36 AM, phanisvara das wrote:
On Sun, 06 May 2012 21:58:51 +0530, David Haller <dnh@opensuse.org> wrote:
Hello,
On Sun, 06 May 2012, phanisvara das wrote:
the only TB size HDDs one gets in kolkata these days are those "green" ones, that park every few seconds, only to be awakened again a couple sec.s later. there's supposedly a DOS utility to adjust that, but google finds mixed reports re. this.
I too have only those disks, but they don't park. You can adjust that with 'hdparm', namely the Parameter -S. Also, look at your power-management settings in yast. Or adjust the pm-profile setting
SATA_ALPM="max_performance"
in the profile. I just copied the 'balanced' to a new name, changed that setting and then activated that profile (/etc/pm-profiler.conf resp. /etc/sysconfig/pm-profiler).
ah, that's interesting. which brand / model do you use?
Although, I'm running a server, per say, IMHO, I'd use Hitachi brand drives. The reasoning is that, because of money, I have to buy used drives and they have never failed me yet. I outgrow them first. My current set of drives, I've had for three years - again these are - used - drives and my desktop is running 24/7. -- Duaine Hechler Piano, Player Piano, Pump Organ Tuning, Servicing& Rebuilding Reed Organ Society Member Florissant, MO 63034 (314) 838-5587 dahechler@att.net www.hechlerpianoandorgan.com -- Home& Business user of Linux - 11 years -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sun, 06 May 2012 23:31:16 +0530, Duaine Hechler <dahechler@att.net> wrote:
Although, I'm running a server, per say, IMHO, I'd use Hitachi brand drives. The reasoning is that, because of money, I have to buy used drives and they have never failed me yet. I outgrow them first. My current set of drives, I've had for three years - again these are - used - drives and my desktop is running 24/7.
sounds good; i'll have to see how they're available here. in india i don't buy second hand though. if it's any good, people want almost as much as new, and anything cheaper is most likely useless. (hell, even buying new hardware you'll get ripped off if you don't know the dealer, or exactly what to look for. plenty of bootlegging going on.) but i also have to look for 'economic choices', there's not much money available. generally the market is swamped with what everybody uses at a particular time. (and that's what the few reliable dealers i know have available.) dealers don't like to stock HDDs because the prizes are very unstable -- never know what your stock is worth tomorrow. that was the case before the flood in thailand, and now it's the same, only a couple of times more costly. so they don't keep a wide variety of models, but purchase whatever they get good deals for in large quantities. -- phani. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Duaine Hechler said the following on 05/06/2012 02:01 PM:
Although, I'm running a server, per say, IMHO, I'd use Hitachi brand drives. The reasoning is that, because of money, I have to buy used drives and they have never failed me yet. I outgrow them first. My current set of drives, I've had for three years - again these are - used - drives and my desktop is running 24/7.
I buy new [or what the dealer tells me is new], and use LVM, so I can use a patchwork of sizes/specs. Lack of availability of any specific size isn't a problem with LVM. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 5/6/2012 3:00 PM, Anton Aylward wrote:
Duaine Hechler said the following on 05/06/2012 02:01 PM:
Although, I'm running a server, per say, IMHO, I'd use Hitachi brand drives. The reasoning is that, because of money, I have to buy used drives and they have never failed me yet. I outgrow them first. My current set of drives, I've had for three years - again these are - used - drives and my desktop is running 24/7.
I buy new [or what the dealer tells me is new], and use LVM, so I can use a patchwork of sizes/specs. Lack of availability of any specific size isn't a problem with LVM.
You can mix and match drive types with Software raid too. As for mixing and matching size with LVM, its almost always an accident waiting to happen. When your partition is scattered across multiple drives, a failure of any one will likely take down everything. LVM was not made for reliability, it was made to allow building big volumes from several smaller devices. Its really not the right tool for the job, even if it can be forced into that task. http://serverfault.com/questions/279571/lvm-dangers-and-caveats -- _____________________________________ ---This space for rent--- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sun, 06 May 2012 18:31:09 -0700 John Andersen <jsamyth@gmail.com> wrote:
http://serverfault.com/questions/279571/lvm-dangers-and-caveats
Nice one! Thanks, John. Been following this thread with interest ;-) Carl -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
John Andersen said the following on 05/06/2012 09:31 PM:
As for mixing and matching size with LVM, its almost always an accident waiting to happen. When your partition is scattered across multiple drives, a failure of any one will likely take down everything.
Yes, you *CAN* configure LVM that way. it is one of the *MANY* ways you can configure LVM and because of that it is a reason many people find LVM confusing and difficult to use. It is fraught with decision alternatives you have to make. That being said those alternative also allow you to do things like mirroring, yes *REAL* mirroring. And you have the flexibility of doing it on a file system basis, not just a drive basis. Yes, you can also stripe across spindles and scatter-gather across spindles with LVM and yes that will cause a catastrophe if you loose a spindle. But no-one is forcing you to use that rather than mirroring. The same applies with RAID. There are many ways of doing it and not all offer reliability in the event of a loss of a spindle. The OP mentioned the need to protect the data but no the OS. using different strategies for different file systems is a capability of LVM not enjoyed by RAID.
LVM was not made for reliability, it was made to allow building big volumes from several smaller devices.
That too. LVM has many capabilities. Don't focus on just one. Hmm. Some configurations of RAID can be said to allow building big volumes by spreading a file system across several small devices as well. It all depends on how to choose to configure things, doesn't it?
Its really not the right tool for the job, even if it can be forced into that task.
I wouldn't call it 'forcing' any more than choosing a particular RAID is 'forcing'.
http://serverfault.com/questions/279571/lvm-dangers-and-caveats
Some of that is so wrong as misleading that its not worth discussing. That you can screw-up something though ignorance and inexperience applies just as much to RAID as LVM ... or anything else for that matter -- there are many examples of that in the software world! -- Leadership is understanding people and involving them to help you do a job. That takes all of the good characteristics, like integrity, dedication of purpose, selflessness, knowledge, skill, implacability, as well as determination not to accept failure. ~ Admiral Arleigh A. Burke -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sun, 06 May 2012 22:00:59 -0400 Anton Aylward <opensuse@antonaylward.com> wrote:
John Andersen said the following on 05/06/2012 09:31 PM:
As for mixing and matching size with LVM, its almost always an accident waiting to happen. When your partition is scattered across multiple drives, a failure of any one will likely take down everything.
Yes, you *CAN* configure LVM that way. it is one of the *MANY* ways you can configure LVM and because of that it is a reason many people find LVM confusing and difficult to use. It is fraught with decision alternatives you have to make.
That being said those alternative also allow you to do things like mirroring, yes *REAL* mirroring. And you have the flexibility of doing it on a file system basis, not just a drive basis.
Yes, you can also stripe across spindles and scatter-gather across spindles with LVM and yes that will cause a catastrophe if you loose a spindle. But no-one is forcing you to use that rather than mirroring.
The same applies with RAID. There are many ways of doing it and not all offer reliability in the event of a loss of a spindle.
The OP mentioned the need to protect the data but no the OS. using different strategies for different file systems is a capability of LVM not enjoyed by RAID.
LVM was not made for reliability, it was made to allow building big volumes from several smaller devices.
That too. LVM has many capabilities. Don't focus on just one.
Hmm. Some configurations of RAID can be said to allow building big volumes by spreading a file system across several small devices as well.
It all depends on how to choose to configure things, doesn't it?
Its really not the right tool for the job, even if it can be forced into that task.
I wouldn't call it 'forcing' any more than choosing a particular RAID is 'forcing'.
http://serverfault.com/questions/279571/lvm-dangers-and-caveats
Some of that is so wrong as misleading that its not worth discussing.
That you can screw-up something though ignorance and inexperience applies just as much to RAID as LVM ... or anything else for that matter -- there are many examples of that in the software world!
Anton, I'd recommend you actually study the lengthy thread at John's link. Overall, the comments there are quite positive about LVM2 in spite of some specific caveats (see 'pvmove data loss', in particular, plus a decidedly steep(er) learning curve, even for admins.) I really enjoy frank discussions addressing pros and cons from people like you two -- each with lots of contemporary hands-on experience -- and, imho, the thread John referred us to falls into that category. regards, Carl -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carl Hartung said the following on 05/06/2012 10:28 PM:
Some of that is so wrong as misleading that its not worth discussing.
That you can screw-up something though ignorance and inexperience applies just as much to RAID as LVM ... or anything else for that matter -- there are many examples of that in the software world!
Anton, I'd recommend you actually study the lengthy thread at John's link. Overall, the comments there are quite positive about LVM2 in spite of some specific caveats (see 'pvmove data loss', in particular, plus a decidedly steep(er) learning curve, even for admins.)
I did, and as you quote rightly observe, the COMMENTS are positive. But some of the ARTICLE is wrong and misleading. You point - LVM2 - is also pertinent. Many of the criticisms of LVM therein applied to LVM1. Having tried both, I'd say that some forms of RAID have a steep learning curve as well :-) Once you get beyond a certain level of complexity and criticality, there is a lot of software that is not for beginners. I get that feeling with GIMP, for example. I've got the beginners books but nothing in them is what I actually want to do ... Don't get me started on the options available for GPG .... -- sarchasm (n): The gulf between the author of sarcastic wit and the person who doesn't get it. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 05/06/12 22:00, Anton Aylward pecked at the keyboard and wrote:
John Andersen said the following on 05/06/2012 09:31 PM:
As for mixing and matching size with LVM, its almost always an accident waiting to happen. When your partition is scattered across multiple drives, a failure of any one will likely take down everything.
Yes, you *CAN* configure LVM that way. it is one of the *MANY* ways you can configure LVM and because of that it is a reason many people find LVM confusing and difficult to use. It is fraught with decision alternatives you have to make.
That being said those alternative also allow you to do things like mirroring, yes *REAL* mirroring. And you have the flexibility of doing it on a file system basis, not just a drive basis.
Yes, you can also stripe across spindles and scatter-gather across spindles with LVM and yes that will cause a catastrophe if you loose a spindle. But no-one is forcing you to use that rather than mirroring.
The same applies with RAID. There are many ways of doing it and not all offer reliability in the event of a loss of a spindle.
The OP mentioned the need to protect the data but no the OS. using different strategies for different file systems is a capability of LVM not enjoyed by RAID.
LVM was not made for reliability, it was made to allow building big volumes from several smaller devices.
That too. LVM has many capabilities. Don't focus on just one.
Hmm. Some configurations of RAID can be said to allow building big volumes by spreading a file system across several small devices as well.
(R)edundant (A)rray of (I)nexpensive (D)isks -- Ken Schneider SuSe since Version 5.2, June 1998 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Ken Schneider - openSUSE said the following on 05/06/2012 11:23 PM:
(R)edundant (A)rray of (I)nexpensive (D)isks
On my local craigslist and kijiji there are many places that sell "ex-equipment" parts and "off-lease" machines. Newspaper articles occasions do an "Ain't It Awful" expose that the disks in these machines have not been sanitized of PII. But they *are* *(I)nexpensive*! -- When you are in any contest you should work as if there were - to the very last minute - a chance to lose it. Dwight D. Eisenhower -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, 7 May 2012 11:30:59 Anton Aylward wrote: [..]
The OP mentioned the need to protect the data but no the OS. using different strategies for different file systems is a capability of LVM not enjoyed by RAID.
Really? Like, running RAID 1 on two filesystems and linear RAID to stitch together a few left-over chunks of unpartitioned space into a single filesystem (and, yes, I could have done the same with LVM), whilst other filesystems on the same physical disks (which are not as critical) are not RAIDed at all? All easily done with mdraid. Been doing it that way for years. -- ========================================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au ========================================================================== -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Hello, On Sun, 06 May 2012, phanisvara das wrote:
On Sun, 06 May 2012 21:58:51 +0530, David Haller <dnh@opensuse.org> wrote:
On Sun, 06 May 2012, phanisvara das wrote:
the only TB size HDDs one gets in kolkata these days are those "green" ones, that park every few seconds, only to be awakened again a couple sec.s later. [..] I too have only those disks, but they don't park. [..] ah, that's interesting. which brand / model do you use?
Well, the parking has nothing to do with brand or model, not sure if I made that clear. Mostly Samsung (the newest already is a Seagate[1]), but also WD and Seagate. Too bad, the one 2T WD SATA is losing sectors and one of the 1.5T Seagates (ST1500...AS) too. Maybe I'll try Hitachi if I can't find the Samsung-Seagates anymore. Basically, it's pretty much irrelevant, there's only 3.5 Manufacturers left anyway (WD, Hitachi and Seagate/ex-Samsung, Samsung being the .5 until it's fully integrated). Oh, and Toshiba with 2.5" drives. Others (cnMemory etc.) used to repackage Samsungs. In the last years I mostly used Samsungs, currently: $ lsscsi [0:0:0:0] disk ATA SAMSUNG HD204UI 1AQ1 /dev/sda [1:0:0:0] disk ATA SAMSUNG HD204UI 1AQ1 /dev/sdb [2:0:0:0] disk ATA SAMSUNG HD204UI 1AQ1 /dev/sdc [3:0:0:0] disk ATA SAMSUNG HD203WI 1AN1 /dev/sdd [4:0:0:0] disk ATA SAMSUNG HD203WI 1AN1 /dev/sde [5:0:0:0] disk ATA SAMSUNG HD203WI 1AN1 /dev/sdf [8:0:0:0] disk ATA SAMSUNG HD204UI 1AQ1 /dev/sdg [9:0:0:0] disk ATA SAMSUNG HD204UI 1AQ1 /dev/sdh [18:0:1:0] disk ATA WDC WD5000AVJB-6 05.0 /dev/sdi [..] The WD is a 500G PATA model. And the others were bought over ~2 or more years, with a couple months inbetween. Above mentioned Seagate-Samsung is in the other box, replacing a WD 20..EARX? and a Seagate ST1500xxxAS. BTW: the last 2TB HD204UI I bought before the flood in Thailand cost me ~65 EUR (plus packaging and postage), then, prices more than tripled(!), and the Seagate-Samsung (same model) still cost me 100 EUR two weeks ago. HTH & just my 2¢, -dnh [1] 2000GB Seagate Barracuda Green ST2000DL004 (HD204UI), still with a funny mixed Seagate/Samsung Label -- Warning: Some of my best mistakes are yet to be made. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, 07 May 2012 02:44:39 +0530, David Haller <dnh@opensuse.org> wrote:
Well, the parking has nothing to do with brand or model, not sure if I made that clear.
a while ago i looked at reports about those green drives, used with various linux distros, and i remember that some claimed that certain disks / brands were honoring those param.s, others not. but there were discrepancies between different reports, they weren't all saying the same re. those hdparms, and a dos utility to be used with some free dos implementation. in general i got the idea that it's something i would like to avoid. if you have better experience i'm glad for you, but it doesn't alleviate all my worries, or dislike for those green things. there have been quite a few complaints about them, perhaps newer batches have been fixed. that doesn't mean that older, sub-standard drives aren't still being sold to india... -- phani. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, 07 May 2012 02:44:39 +0530, David Haller <dnh@opensuse.org> wrote:
The WD is a 500G PATA model. And the others were bought over ~2 or more years, with a couple months inbetween. Above mentioned Seagate-Samsung is in the other box, replacing a WD 20..EARX? and a Seagate ST1500xxxAS.
so you're using both, seagate & samsung green variety, with good results. perhaps i'll better wait for somebody coming over from US, which will happen sooner or later, and ask them to bring one or two (once i get the money approved, that is). -- phani. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Hello, On Mon, 07 May 2012, phanisvara das wrote:
On Mon, 07 May 2012 02:44:39 +0530, David Haller <dnh@opensuse.org> wrote:
The WD is a 500G PATA model. And the others were bought over ~2 or more years, with a couple months inbetween. Above mentioned Seagate-Samsung is in the other box, replacing a WD 20..EARX? and a Seagate ST1500xxxAS.
so you're using both, seagate & samsung green variety, with good results. perhaps i'll better wait for somebody coming over from US, which will happen sooner or later, and ask them to bring one or two (once i get the money approved, that is).
Not sure if the (real) Seagate ST1500...AS are "green" models. The WD 20...EARX in the other box is a "green" one too. Basically you have 3 kinds of drives today: - server drives (specced for 24/7, often 7200 or even 10k min^-1), expensive, usually twice as much as "run of the mill" of the same size - standard models (7200 or 5400 min^-1, occasionally inbetween) said to be specced for about 8/7, cheap, used to need ~10W, now usually ~6W - "green" models (mostly 5400, some 7200 min^-1) said to be specced for about 8/7, cheap, ~6W Basically, unless you shell out the money for server drives, don't look at names, look at the specs (e.g. cache size, MTBF) and the price ;) For me, "top" disk performance is irrelevant. The HD20[34][UW]I are supposed to do 130-160 MB/s (or more). Two runs of [lsscsi]: [1:0:0:0] disk ATA SAMSUNG HD204UI 1AQ1 /dev/sdb [3:0:0:0] disk ATA SAMSUNG HD203WI 1AN1 /dev/sdd # hdparm -t /dev/sdb /dev/sdd /dev/sdb: Timing buffered disk reads: 410 MB in 3.00 seconds = 136.54 MB/sec /dev/sdd: Timing buffered disk reads: 282 MB in 3.01 seconds = 93.60 MB/sec /dev/sdb: Timing buffered disk reads: 410 MB in 3.00 seconds = 136.66 MB/sec /dev/sdd: Timing buffered disk reads: 282 MB in 3.02 seconds = 93.46 MB/sec But that's misleading too: [lsscsi] [0:0:0:0] disk ATA SAMSUNG HD204UI 1AQ1 /dev/sda [4:0:0:0] disk ATA SAMSUNG HD203WI 1AN1 /dev/sde # hdparm -t /dev/sda /dev/sde /dev/sda: Timing buffered disk reads: 388 MB in 3.00 seconds = 129.30 MB/sec /dev/sde: Timing buffered disk reads: 332 MB in 3.02 seconds = 110.09 MB/sec /dev/sda: Timing buffered disk reads: 406 MB in 3.00 seconds = 135.13 MB/sec /dev/sde: Timing buffered disk reads: 226 MB in 3.01 seconds = 75.21 MB/sec But! In practice, with all the seeks and FS-Overhead and whatnot, I feel lucky if I get over 60MB/s while copying (big) files with mc, typical is something from 40 to 50 MB/s. With small files, it's much much lower anyway (I've got a ext3 only system and the disks tend to be full). So, all those "fancy" numbers you find in benchmarks are a datapoint, but in practice, I see little difference in speed over all my disks in this and the other box, even the 2 PATA 500G disks read/write about 40MB/s and there's little difference between the oldest and newest SATA disks. It is noticeable at times, when copying from a fast one to a fast one vs. slow to slow, but not very much. So, as stated before, it is not really relevant what brand/model you get. I've had bad experiences with IBM (DHEA and DTLA series, bought by Hitachi), Quantum (one 3.2G drive), Maxtor (various models from 9.1G to 80G, bought by Seagate), Seagate, WD, and Samsung, and weirdly enough, the worst were the 2 Samsungs that died a few days apart (both 500G IIRC -> cf. "twins" mentioned), where I lost quite a bit of (replaceable) stuff. One disk was just dead, it spun up IIRC, but not even the BIOS found it anymore, from the other, I was able to scrape most stuff off off it. Since then (and before for models from 2 x 160G to 3 or so 400G), the other Samsung drives have been reliable, and that's been a lot (20 or more I think). OTOH, out of 3 ST1500...AS Seagates, only 1 is still ok, 1 is decomissioned, the other is about to, due to losing sectors. The 1 WD20...EAR. SATA too. See my previous mail. OTOOH, other people have quite different experiences, e.g. Samsungs dying like flies and WD/Seagate rock solid. Whatever. And, with you being in Kolkata (right?), you'd have to check it out yourself anyway, as you got quite a different climate that Germany. Just think about heat and humidity (or are those drives in a air-conditioned environment?) Again: it does not really matter what brand/model of a e.g. 2T drive you get. If in doubt: get the cheapest (non-fake/-refurbished etc.). Think about "green"/5400 min^-1 as being a bit less snappy than 7200 min^-1 models, but needing less power and being cooler. Continuous transfer is IMO irrelevant, because if whether you don't get 130 MB/s or don't get 150 MB/s at the application level copying files with, say, 60 MB/s is irrelevant. For me, those "green" drives are fast enough, and as you can guess from what's in this box alone, moving (i.e. copy & delete) 100 GB about is routine ;) I don't care if moving 50GB (of big files, 200MB-2.xGB) takes 15 or 20 mins. I won't be sitting there watching it anyway. You want snappy? Get a SSD ;) One last datapoint: Samsung drives report running at lower temps than WD or Seagate (no idea about Hitachi). Not sure if they are cooler or just report cooler temps. HTH, -dnh -- Perl is the successful attempt to make a braindump executable. -- Lutz Donnerhacke -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 5/6/2012 2:44 PM, phanisvara das wrote:
On Mon, 07 May 2012 02:44:39 +0530, David Haller <dnh@opensuse.org> wrote:
The WD is a 500G PATA model. And the others were bought over ~2 or more years, with a couple months inbetween. Above mentioned Seagate-Samsung is in the other box, replacing a WD 20..EARX? and a Seagate ST1500xxxAS.
so you're using both, seagate & samsung green variety, with good results. perhaps i'll better wait for somebody coming over from US, which will happen sooner or later, and ask them to bring one or two (once i get the money approved, that is).
Im almost certain these aren't made in the US, so you'd be better waiting for someone coming over from Indonesia or whereever WD is making their drives these days. -- _____________________________________ ---This space for rent--- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, 07 May 2012 06:49:51 +0530, John Andersen <jsamyth@gmail.com> wrote:
Im almost certain these aren't made in the US, so you'd be better waiting for someone coming over from Indonesia or whereever WD is making their drives these days.
well, i don't knowm many people in indonesia, and if things there are anything like india, they'd be prone to sell sub-standard quality to their own people while exporting the good stuff to where most of their profits come from. (it's almost impossible to get good indian tea here locally; it all gets exported and only some boutique-like stores sell it in the big cities, at international rates.) -- phani. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
phanisvara das wrote:
after seeing a few threads re. RAID arrays, problems booting off them, and extra strain the surviving member of an array is put under when it has to sync a new HDD after it's (only) brother's failure, i doubt the wisdom of my original idea to set up two identical 2TB drives as RAID1 array.
wouldn't it be safer (not to mention simpler) to just rsync the simple directory tree i'm concerned about to the other HDD every 15 min or so? if either of the two HDDs fails, the other one still has the important data.
I do both - my workstation is RAID1, and I run a daily rsync backup of essential config and data. My plain back office deskyop systems are not RAID, but /home is kept on a fileserver, so a failing desktop is easily replaced. RAID is not about backup, but about availability.
it's not my desktop, but a file server i'm putting together for my employer.
A file server that doesn't use RAID?? -- Per Jessen, Zürich (13.8°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sun, 6 May 2012 17:45:29 phanisvara das wrote:
after seeing a few threads re. RAID arrays, problems booting off them, and extra strain the surviving member of an array is put under when it has to sync a new HDD after it's (only) brother's failure, i doubt the wisdom of my original idea to set up two identical 2TB drives as RAID1 array.
wouldn't it be safer (not to mention simpler) to just rsync the simple directory tree i'm concerned about to the other HDD every 15 min or so? if either of the two HDDs fails, the other one still has the important data.
it's not my desktop, but a file server i'm putting together for my employer. the only thing he cares about is preserving the data in question. if or when one of the HDDs fails, there's no need to preserve the operating system or anything else.
I agree with what others have said. Backups are for data security. RAID is about fault tolerance and/or performance (depending on what flavour of RAID you choose). Don't get the two mixed up. You need both. -- ========================================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au ========================================================================== -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 06/05/12 04:15, phanisvara das wrote:
after seeing a few threads re. RAID arrays, problems booting off them, and extra strain the surviving member of an array is put under when it has to sync a new HDD after it's (only) brother's failure, i doubt the wisdom of my original idea to set up two identical 2TB drives as RAID1 array.
wouldn't it be safer (not to mention simpler) to just rsync the simple directory tree i'm concerned about to the other HDD every 15 min or so? if either of the two HDDs fails, the other one still has the important data.
it's not my desktop, but a file server i'm putting together for my employer. the only thing he cares about is preserving the data in question. if or when one of the HDDs fails, there's no need to preserve the operating system or anything else.
Neither RAID or rsync provide "data security".. in fact no tool or hardware does, that's a very dangerous mindset to adopt.. deep trouble ahead :-) .. now, that being said.. RAID can provide you much better *availability* and disaster recovery aid when things go wrong, but there is almost always a tradeoff for performance. It is not a replacement for backups, that's when rsync comes into play to help the file transfer/sync process. "data security" involves a "process" not only "a bunch of hw and software" but also policies regarding physical/virtual access to the information, data retention and confidentiality. Yes, cheap-ass hardware fails and usually sucks..but human beings screw things up in a much larger scale..big time! so think before you type ! ;-) Cheers!! -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, 07 May 2012 11:09:28 +0530, Cristian Rodríguez <crrodriguez@opensuse.org> wrote:
Neither RAID or rsync provide "data security".. in fact no tool or hardware does, that's a very dangerous mindset to adopt.. deep trouble ahead .. now, that being said.. RAID can provide you much better *availability* and disaster recovery aid when things go wrong, but there is almost always a tradeoff for performance.
performance doesn't bother me in this case, so i would be willing to accept that trade-off.
It is not a replacement for backups, that's when rsync comes into play to help the file transfer/sync process.
so you also say that, if only one can be adopted at this time, backups (rsync) are more suited to prevent data loss. what i was considering was replacing these regular backups by RAID1, mirroring the whole disk, which obviously contains a copy of the data, too.
"data security" involves a "process" not only "a bunch of hw and software" but also policies regarding physical/virtual access to the information, data retention and confidentiality. Yes, cheap-ass hardware fails and usually sucks..but human beings screw things up in a much larger scale..big time! so think before you type !
i did even better, i asked here, with the usual good results -- even if i'm still not sure what i'm going to do in the end. -- phani. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, 07 May 2012 14:28:37 +0530, you wrote in :
...even if i'm still not sure what i'm going to do in the end.
made up my mind now. for the 'server' i'll stick with rsync for now. it's something i know fairly well, and know where to look for the things i don't know yet. that should limit the human-stupidity factor's potential to create disaster. at the same time i'll start playing around with RAID arrays in virtual machines, damaging disks on purpose per example, to see what happens to the data. i'll be going to germany in about a month, and perhaps i can pick up a couple of large disks there, convincing my boss to buy them off me when i get back, otherwise keep them for myself. -- phani. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Cristian Rodríguez said the following on 05/07/2012 01:39 AM:
Neither RAID or rsync provide "data security".. in fact no tool or hardware does, that's a very dangerous mindset to adopt.. deep trouble ahead :-) .. now, that being said.. RAID can provide you much better *availability* and disaster recovery aid when things go wrong, but there is almost always a tradeoff for performance.
I'm sorry, but the accepted tenets of "Security" include the C-I-A triad: - Confidentiality - Integrity and - *Availability* http://en.wikipedia.org/wiki/CIA_triad#Key_concepts <quote> For over twenty years, information security has held confidentiality, integrity and availability (known as the CIA triad) to be the core principles of information security. </quote>
It is not a replacement for backups, that's when rsync comes into play to help the file transfer/sync process.
Correct: as has been pointed out by a number of people including myself, backups are another aspect of security - Integrity. You can also view backups as Availability post catastrophe :-)
"data security" involves a "process" not only "a bunch of hw and software" but also policies regarding physical/virtual access to the information, data retention and confidentiality.
True, but once we get into that we start dealing with issue like business alignment and policy enforcement and revision and ISO27000 and rapidly go beyond the scope of this list. If you want to go there, I can recommend some very active and very serious-minded lists; but this is about SUSE, not INFOSEC or ISO27K or POLICY.
Yes, cheap-ass hardware fails and usually sucks..but human beings screw things up in a much larger scale..big time! so think before you type ! ;-)
All hardware fails. No-one has an infinite budget. Yes, humans are the #1 security threat. But take away the humans and there's no point in having the computer systems, is there. Ultimately as the OP made clear, its about what management wants. As engineers, we know how to make indefinitely reliable systems out of unreliable components; that's what engineering is about - we're Sons of Martha[1]. Well maybe you're not; maybe you're a Son of Mary or Chicken Little. And as Larry said, "there more than one way to do it"[2] [1] http://tricolour.net/som.html [2] http://c2.com/cgi/wiki?ThereIsMoreThanOneWayToDoIt -- All truths are easy to understand once they are discovered; the point is to discover them. -- Galileo Galilei. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, 7 May 2012 15:09:28 Cristian Rodríguez wrote:
[...] Yes, cheap-ass hardware fails and usually sucks..but human beings screw things up in a much larger scale..big time! so think before you type ! ;-)
Cheers!!
To damn right! It is *way* too easy to be logged on with root priveleges, type rm -rf * and then suddenly get that sinking feeling when it takes much longer than expected...then, slowly (or maybe suddenly) the panic sets in when you realise that you didn't cd to the parent of the directory you wanted to remove but, rather to / and, like a good little indian, your rm command is dutifully doing exactly what you (as root) told it to do - removing everything, recursively, from / without asking for further confirmation... BTDT. Once. Only once! If you're quick you catch it before it gets to /sbin (where the statically linked "restore" should be located, and of course you have a backup handy and you *don't* reboot the machine (which, if you do, won't start because /etc, /bin, most of /lib and a good bit of /home are already gone) and can therefore recover it! I rebooted. Bad mistake. Bad, very expensive mistake. We did get it back. Eventually. From a backup. After much unplanned (and very inconvenient) downtime. I learned my lesson! Keep backups and know how to restore them, even on a broken system. TEST the restore process and make sure a) that your backups work and b) you're confident doing it under pressure if you have to. That is a lesson I won't forget in a hurry! -- ========================================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au ========================================================================== -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, 07 May 2012 21:36:55 +0530, Rodney Baker <rodney.baker@iinet.net.au> wrote:
Bad, very expensive mistake. We did get it back. Eventually. From a backup. After much unplanned (and very inconvenient) downtime. I learned my lesson!
i've had such bad experiences, but only with my own machines. no fun, but a good way to learn. the idea to use something in production which i've never really dealt with (RAID), even though it's "easy as falling off a log," isn't very clever, to say it mildly. i'll stick to what i've done before, rsync, and probably put RAID on my desktop, to get the hang of it. -- phani. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 5/6/2012 4:15 AM, phanisvara das wrote:
after seeing a few threads re. RAID arrays, problems booting off them, and extra strain the surviving member of an array is put under when it has to sync a new HDD after it's (only) brother's failure, i doubt the wisdom of my original idea to set up two identical 2TB drives as RAID1 array.
wouldn't it be safer (not to mention simpler) to just rsync the simple directory tree i'm concerned about to the other HDD every 15 min or so? if either of the two HDDs fails, the other one still has the important data.
it's not my desktop, but a file server i'm putting together for my employer. the only thing he cares about is preserving the data in question. if or when one of the HDDs fails, there's no need to preserve the operating system or anything else.
You're essentially asking which is better, beef or pork. Both those scenarios have pros & cons, but both are so simple that there is hardly any real reason to choose one vs the other. Both methods are fine. Both can be made very reliable. Both can be made automatic. Both can be screwed up and leave you with nothing. Do whichever you like best. We can't tell you whether beef or pork will be more to your liking. -- bkw -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Tue, 08 May 2012 00:09:48 +0530, Brian K. White <brian@aljex.com> wrote:
Both those scenarios have pros & cons, but both are so simple that there is hardly any real reason to choose one vs the other. Both methods are fine. Both can be made very reliable. Both can be made automatic. Both can be screwed up and leave you with nothing. Do whichever you like best.
yes, i will. and since i knew nothing about one of the options (RAID), i've asked here. i found the result very interesting and think i've learned a few things from listening. now i'll use that in the praxis, i.e., use rsync for the server, and start using RAID on my own machine, to get some real experience; thanks for sharing your thoughts and knowledge everyone. -- phani. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 5/7/2012 3:06 PM, phanisvara das wrote:
yes, i will. and since i knew nothing about one of the options (RAID), i've asked here. i found the result very interesting and think i've learned a few things from listening. now i'll use that in the praxis, i.e., use rsync for the server, and start using RAID on my own machine, to get some real experience; thanks for sharing your thoughts and knowledge everyone.
I wasn't even looking for info on this at the moment but I am interested since I have a server at home. At first I was using RAID but found it rather complex for my understanding. A sysadmin at work suggested I try rsync and I have been using that now for several years. Perhaps part of the difficulty that I had with RAID was due to using Solaris at the time. I noticed that one of the subtopics here was availability vs reliability. For a home server I don't care as much about availability as reliability. With rsync I had a drive fail recently and having the server down while I fixed it wasn't a big deal. One big advantage that I found with rsync is recovering files that I deleted by mistake. I just recovered them from the server before the next rsync. Thanks to everyone who joined this thread. I learned some interesting things about this subject. Damon Register -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (15)
-
Anton Aylward -
Brian K. White -
Carl Hartung -
Carlos E. R. -
Cristian Rodríguez -
Damon Register -
David Haller -
Dennis Gallien -
Duaine Hechler -
John Andersen -
Ken Schneider - openSUSE -
Mike -
Per Jessen -
phanisvara das -
Rodney Baker