Hello, My present install uses 3 partitions, / (30Gb, btrfs), /home (30 Gb, xfs)and /data, the last for large file storing (around 400Gb, xfs). no lvm. I want to install an other distro (ubuntu) to test some special config only documented on ubuntu AFAIK it's not possible to reduce the size of an xfs filesystem :-( So Here what I did: * copy (rsync, as root) my /home to /data/home-active folder * coment out the /home line in fstab * adding in fstab /data/home-actif /home none bind" it seems to work, but before trashing the original /home partition I would like to know if all is ok, specially in the fstab line thanks jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
jdd@dodin.org wrote:
So Here what I did:
* copy (rsync, as root) my /home to /data/home-active folder
Not sure if you use hardlinks in your $HOME - those files would probably be doubled. No big thing though.
* coment out the /home line in fstab * adding in fstab /data/home-actif /home none bind"
My enties also have the check options '0 0' at the end, though I'm not really sure it's needed. But at least it doesn't fail :D
it seems to work, but before trashing the original /home partition I would like to know if all is ok, specially in the fstab line
OK only if the 'actif' is just a typo in this mail :D -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 26/10/2017 à 14:19, Peter Suetterlin a écrit :
OK only if the 'actif' is just a typo in this mail :D
oh, yes, the list is in english, so I translated, but the fstab was a raw copy :-) I added "actif" (active in English) to remember not to remove this seemingly duplicate folder someday :-)) jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, 26 Oct 2017 13:19:25 +0100 Peter Suetterlin <P.Suetterlin@royac.iac.es> wrote:
jdd@dodin.org wrote:
So Here what I did:
* copy (rsync, as root) my /home to /data/home-active folder
Not sure if you use hardlinks in your $HOME - those files would probably be doubled. No big thing though.
Not if you use the correct options with rsync. It is designed to preserve the hierarchy exactly. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Dave Howorth wrote:
On Thu, 26 Oct 2017 13:19:25 +0100 Peter Suetterlin <P.Suetterlin@royac.iac.es> wrote:
jdd@dodin.org wrote:
So Here what I did:
* copy (rsync, as root) my /home to /data/home-active folder
Not sure if you use hardlinks in your $HOME - those files would probably be doubled. No big thing though.
Not if you use the correct options with rsync. It is designed to preserve the hierarchy exactly.
I know it *can* do it. But it doesn't by default, and can get very slow if it does. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Friday, 2017-10-27 at 13:42 +0100, Peter Suetterlin wrote:
Dave Howorth wrote:
On Thu, 26 Oct 2017 13:19:25 +0100 Peter Suetterlin <P.Suetterlin@royac.iac.es> wrote:
jdd@dodin.org wrote:
So Here what I did:
* copy (rsync, as root) my /home to /data/home-active folder
Not sure if you use hardlinks in your $HOME - those files would probably be doubled. No big thing though.
Not if you use the correct options with rsync. It is designed to preserve the hierarchy exactly.
I know it *can* do it.
But it doesn't by default, and can get very slow if it does.
No, it doesn't get slow just by telling it to keep hardlinks. It gets slow when you tell it to compare files. - -- Cheers, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlnzKq8ACgkQtTMYHG2NR9WZ8QCdFGRJKbuRO9GqPiBIDcFNDbGW 1uQAnik+VIGo40byO4/6jg6+9dF9PEOL =yaIT -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 27/10/17 13:46, Carlos E. R. wrote:
I know it *can* do it.
But it doesn't by default, and can get very slow if it does.
No, it doesn't get slow just by telling it to keep hardlinks. It gets slow when you tell it to compare files.
Depends on how many hardlinks you've got, and how much ram. Last time I tried it, it brought my system near enough to its knees, though that wasn't the current system that has ram coming out of its ears - the old system had less than a gig iirc. Cheers, Wol -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Wols Lists wrote:
On 27/10/17 13:46, Carlos E. R. wrote:
I know it *can* do it.
But it doesn't by default, and can get very slow if it does.
No, it doesn't get slow just by telling it to keep hardlinks. It gets slow when you tell it to compare files.
Depends on how many hardlinks you've got, and how much ram. Last time I tried it, it brought my system near enough to its knees, though that wasn't the current system that has ram coming out of its ears - the old system had less than a gig iirc.
Might indeed be - it's a warning that I well remembered, but I had not used the option myself in quite some time. Might be it was in (ancient) times of meager hardware, too :D -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Fri, 27 Oct 2017 14:20:31 +0100 Peter Suetterlin <P.Suetterlin@royac.iac.es> wrote:
Wols Lists wrote:
On 27/10/17 13:46, Carlos E. R. wrote:
I know it *can* do it.
But it doesn't by default, and can get very slow if it does.
No, it doesn't get slow just by telling it to keep hardlinks. It gets slow when you tell it to compare files.
Depends on how many hardlinks you've got, and how much ram. Last time I tried it, it brought my system near enough to its knees, though that wasn't the current system that has ram coming out of its ears - the old system had less than a gig iirc.
Might indeed be - it's a warning that I well remembered, but I had not used the option myself in quite some time. Might be it was in (ancient) times of meager hardware, too :D
Well I've used it for many years with many millions of hardlinks and it works fine. And as for not doing it by default; I feel sure that if it did, you would complain about that as well. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Friday, 2017-10-27 at 21:45 +0100, Dave Howorth wrote:
On Fri, 27 Oct 2017 14:20:31 +0100 Peter Suetterlin <> wrote:
Wols Lists wrote:
On 27/10/17 13:46, Carlos E. R. wrote:
I know it *can* do it.
But it doesn't by default, and can get very slow if it does.
No, it doesn't get slow just by telling it to keep hardlinks. It gets slow when you tell it to compare files.
Depends on how many hardlinks you've got, and how much ram. Last time I tried it, it brought my system near enough to its knees, though that wasn't the current system that has ram coming out of its ears - the old system had less than a gig iirc.
Might indeed be - it's a warning that I well remembered, but I had not used the option myself in quite some time. Might be it was in (ancient) times of meager hardware, too :D
Well I've used it for many years with many millions of hardlinks and it works fine.
Same here. Even if it would run slower I would use it. - -- Cheers, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlnznbMACgkQtTMYHG2NR9Uf/QCfdqxReg/R+HNUM8CNL9BJmIzR 30kAoIjXy3D/m6QyWmzo+fFqfDM+0kK+ =bzdN -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-10-26 14:19, Peter Suetterlin wrote:
jdd@dodin.org wrote:
So Here what I did:
* copy (rsync, as root) my /home to /data/home-active folder
Not sure if you use hardlinks in your $HOME - those files would probably be doubled. No big thing though.
rsync copies hardlinks and sparse files as such, if you tell it to: -a, --archive archive mode; equals -rlptgoD (no -H,-A,-X) -r, --recursive recurse into directories -l, --links copy symlinks as symlinks -p, --perms preserve permissions -t, --times preserve times -g, --group preserve group -o, --owner preserve owner (super-user only) -D same as --devices --specials --devices preserve device files (super-user only) --specials preserve special files -A, --acls preserve ACLs (implies -p) [non-standard] -X, --xattrs preserve extended attrs (implies -p) [n.s.] -H, --hard-links preserve hard links -h, --human-readable output numbers in a human-readable format --progress show progress during transfer -P same as --partial --progress --partial keep partially transferred files --partial-dir=DIR put a partially transferred file into DIR -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
On 26/10/17 05:35 AM, jdd@dodin.org wrote:
I want to install an other distro (ubuntu) to test some special config only documented on ubuntu
Why not get a new drive. They are cheap enough. I can get a 3T drive now for what I paid for a 1T a few years ago. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 26/10/2017 à 15:39, Anton Aylward a écrit :
On 26/10/17 05:35 AM, jdd@dodin.org wrote:
I want to install an other distro (ubuntu) to test some special config only documented on ubuntu
Why not get a new drive. They are cheap enough. I can get a 3T drive now for what I paid for a 1T a few years ago.
it's already a 500Gb ssd on a laptop with only one place It was not a problem with ext4. I don't even remember why I formatted data as xfs (may be opensuse default) jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Either log in as root (home for root is /root leaving everything in /home idle) or boot from your favorite recovery live media. use gparted to alter the partition layout then move the contents of /home to the new location. be sure to update /etc/fstab... I *think* systemd still uses that. You're on your own if it doesn't On 10/26/17 6:51 AM, jdd@dodin.org wrote:
Le 26/10/2017 à 15:39, Anton Aylward a écrit :
On 26/10/17 05:35 AM, jdd@dodin.org wrote:
I want to install an other distro (ubuntu) to test some special config only documented on ubuntu
Why not get a new drive. They are cheap enough. I can get a 3T drive now for what I paid for a 1T a few years ago.
it's already a 500Gb ssd on a laptop with only one place
It was not a problem with ext4. I don't even remember why I formatted data as xfs (may be opensuse default)
jdd
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 26/10/2017 à 16:03, Bruce Ferrell a écrit :
Either log in as root (home for root is /root leaving everything in /home idle) or boot from your favorite recovery live media. use gparted to alter the partition layout then move the contents of /home to the new location.
don't works with xfs, on the new location side jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Thats odd... It did for me when I did it a few months back on my MD3000i. You *can* have problems if you don't have all the support stuff installed for a given filesystem type On 10/26/17 7:41 AM, jdd@dodin.org wrote:
Le 26/10/2017 à 16:03, Bruce Ferrell a écrit :
Either log in as root (home for root is /root leaving everything in /home idle) or boot from your favorite recovery live media. use gparted to alter the partition layout then move the contents of /home to the new location.
don't works with xfs, on the new location side
jdd
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, 26 Oct 2017 08:20:10 -0700 Bruce Ferrell <bferrell@baywinds.org> wrote:
Thats odd... It did for me when I did it a few months back on my MD3000i.
You *can* have problems if you don't have all the support stuff installed for a given filesystem type
On 10/26/17 7:41 AM, jdd@dodin.org wrote:
Le 26/10/2017 à 16:03, Bruce Ferrell a écrit :
Either log in as root (home for root is /root leaving everything in >> /home idle) or boot from your favorite recovery live media. use >> gparted to alter the partition layout then move the contents of /home >> to the new location.
don't works with xfs, on the new location side
jdd
As Bruce says, it certainly does work. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 26/10/2017 à 18:38, Dave Howorth a écrit :
On Thu, 26 Oct 2017 08:20:10 -0700
don't works with xfs, on the new location side
jdd
As Bruce says, it certainly does work.
one can't shrink an xfs file system as far as I know. of course moving the whole (data) content is a solution, but not without risks of various kind. jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, 26 Oct 2017 18:43:52 +0200 "jdd@dodin.org" <jdd@dodin.org> wrote:
Le 26/10/2017 à 18:38, Dave Howorth a écrit :
On Thu, 26 Oct 2017 08:20:10 -0700
don't works with xfs, on the new location side
jdd As Bruce says, it certainly does work. one can't shrink an xfs file system as far as I know.
of course moving the whole (data) content is a solution, but not without risks of various kind.
Moving the data was what you suggested (and have cut from your reply, why?). You didn't suggest shrinking a partition. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 26/10/2017 à 19:28, Dave Howorth a écrit :
Moving the data was what you suggested (and have cut from your reply,
not me
why?). You didn't suggest shrinking a partition.
read from the beginning, please jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 26/10/17 09:51 AM, jdd@dodin.org wrote:
it's already a 500Gb ssd on a laptop with only one place
I've upgraded 2.5" laptop drives before. I've a couple of old 2.5" ones that I can't imagine what to do with :-( I'm unclear as to your problem.
It was not a problem with ext4. I don't even remember why I formatted data as xfs (may be opensuse default)
Yes, you can shrink ext4 just fine. I'm not sure what the developers made XFS the default for /home. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 26/10/2017 à 16:05, Anton Aylward a écrit :
I've upgraded 2.5" laptop drives before.
it was me that set this ssd, the computer sold with ssd was incredibly expensive I've a couple of old 2.5" ones that I
can't imagine what to do with :-(
I try to sell them, or use them for extra mass storage eventually (I have 3 5Tb usb disks for archive)
I'm unclear as to your problem.
the disk have only 5 partitions (efi): efi, /, swap, /home and /data I need one more. I could easily make /data a bit smaller if it was not xfs, and / is not large enough to accommodate /home so I moved /home to a folder in /data and expect to use the previous home partition for testing Ubuntu and see if some hardware problem are solved there (and solution be ported to leap) hanks jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
As I said, I don't see your problem. The first time I did this I got a size coupler and copied to old 2.5 contents to my big hard drive. The second time I had a net work and rsync'd it. You don't need to change drives if all you want to do is shrink /data. Copy /home and /data to your hard drive. Use gparted to zap the 2 old partitions. Make everything except the extant root a LVM. Set up minimal space as LVs on the LVM for a /home and /data and copy them back as EXT4. You've still got space on the LVM for other stuff and you can now shrink and grow the /home and /data as well. I've learnt to ALWAYS use a LVM for the simple reason that I don't know what the FS sizes are going to be over the lifetime. XFS doesn't make sense for me. Example: I file Photographs initially by year. if the FS gets crowded I create a new LV and move one year off to that & mount it in the old position. I've just done this with PhotoYear2014. While I was at it, I did the same with PhotoYear2017. So the base FS was now very sparse, so I shrunk it back, which freed up LEs that I can now use for something else. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 26/10/2017 à 17:51, Anton Aylward a écrit :
I've learnt to ALWAYS use a LVM for the simple reason that I don't know what the FS sizes are going to be over the lifetime.
I don't want to use an other layer above what is strictly necessary. lvm is not easy to manage when something goes wrong. I also try to restrict to the minimum the move of large file collection. Every time I did I lose something. Link, some odd named files, some hidden files or files with unexpected permissions do not always rsync well. Of course I could do if it was necessary, but my solution seems to do he job better. I only wanted to know if I didn't miss some important setup, and it seems not thanks jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 26/10/17 16:51, Anton Aylward wrote:
I've learnt to ALWAYS use a LVM for the simple reason that I don't know what the FS sizes are going to be over the lifetime.
While I've got masses of disk space (I use my desktop as main storage), again I'm going to be shifting that over to LVM for that exact reason - I don't know what I want where. Hard drives at the bottom, raid on that, lvm on that, and probably btrfs on that. Because I've got a mix of 3TB and 4TB drives, I'll put swap on a raid-10 on the 4TB's, have 1TB of mirror on the 4TB drives for all my root partitions, and then 6TB raid-6 across all four drives (3TB per drive) as /home. That way I can have gentoo as my main system, SuSE because that's what I support elsewhere, and Mint or Ubuntu or Red Hat or whatever for playing with, all in the 1TB mirror. I can't do what you do with photos though, because this is a multi-user system and most of the photos are hard-linked across several accounts. I could use soft links, but that then makes re-organising a royal pain. Cheers, Wol -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 26/10/2017 à 18:16, Wols Lists a écrit :
Hard drives at the bottom, raid on that, lvm on that, and probably btrfs on that.
I hope you wont ever have a faulty drive... jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, 26 Oct 2017 18:37:59 +0200 "jdd@dodin.org" <jdd@dodin.org> wrote:
Le 26/10/2017 à 18:16, Wols Lists a écrit :
Hard drives at the bottom, raid on that, lvm on that, and probably btrfs on that.
I hope you wont ever have a faulty drive...
That's the main reason for having that setup. When a drive fails the machine carries on whilst you buy a replacement and swap it for the failed drive, which rebuilds. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 26/10/2017 à 19:27, Dave Howorth a écrit :
On Thu, 26 Oct 2017 18:37:59 +0200 "jdd@dodin.org" <jdd@dodin.org> wrote:
Le 26/10/2017 à 18:16, Wols Lists a écrit :
Hard drives at the bottom, raid on that, lvm on that, and probably btrfs on that.
I hope you wont ever have a faulty drive...
That's the main reason for having that setup. When a drive fails the machine carries on whilst you buy a replacement and swap it for the failed drive, which rebuilds.
not always, not always like you want, not always when you want. this setup is extremely expensive and is interesting is you want a system as much as possible always running. some failures on openSUSE servers recently show that even very well made system can fail. there are many situation where rebuilding a raid array can be very time consuming and error prone for example, do you have at hand an exact replacement for any disk in the raid array. What if one fail and you have to wait three days to get an other? well, anyway it's not my actual problem, much simpler jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
jdd@dodin.org wrote:
Le 26/10/2017 à 19:27, Dave Howorth a écrit :
On Thu, 26 Oct 2017 18:37:59 +0200 "jdd@dodin.org" <jdd@dodin.org> wrote:
Le 26/10/2017 à 18:16, Wols Lists a écrit :
Hard drives at the bottom, raid on that, lvm on that, and probably btrfs on that.
I hope you wont ever have a faulty drive...
That's the main reason for having that setup. When a drive fails the machine carries on whilst you buy a replacement and swap it for the failed drive, which rebuilds.
not always, not always like you want, not always when you want.
I have to say - I have been running some 20 systems (hosted externally) with RAID1 for more than 12 years. For the last 5-6 years, 1 or 2 drives fail per annum, sometimes more. They are replaced on demand, they rebuild and all done. Exactly as Dave describes it. Internally we use hot spares on every system, no complaints there either.
this setup is extremely expensive and is interesting is you want a system as much as possible always running.
Not _extremely_ expensive, only the price of a second drive. Apart from high availability requirements, it's very useful if you want to avoid restoring a backup just because of a disk failure. The small price of the extra drive is easily paid back.
there are many situation where rebuilding a raid array can be very time consuming and error prone
Yes, the bigger the array, the longer it takes and that does become a problem. Error prone? Not in my experience.
for example, do you have at hand an exact replacement for any disk in the raid array. What if one fail and you have to wait three days to get an other?
You don't need an exact replacement - only one that is the same size of bigger. -- Per Jessen, Zürich (12.9°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 26/10/17 19:24, Per Jessen wrote:
jdd@dodin.org wrote:
Le 26/10/2017 à 19:27, Dave Howorth a écrit :
On Thu, 26 Oct 2017 18:37:59 +0200 "jdd@dodin.org" <jdd@dodin.org> wrote:
Le 26/10/2017 à 18:16, Wols Lists a écrit :
Hard drives at the bottom, raid on that, lvm on that, and probably btrfs on that.
I hope you wont ever have a faulty drive...
That's the main reason for having that setup. When a drive fails the machine carries on whilst you buy a replacement and swap it for the failed drive, which rebuilds.
Eggsackerly.
not always, not always like you want, not always when you want.
I have to say - I have been running some 20 systems (hosted externally) with RAID1 for more than 12 years. For the last 5-6 years, 1 or 2 drives fail per annum, sometimes more. They are replaced on demand, they rebuild and all done. Exactly as Dave describes it. Internally we use hot spares on every system, no complaints there either.
I've not known that many drives fail, actually. However, when they do fail it has been a bit of a disaster. A friend lost a drive - fortunately mostly backed up on other computers. I made a few quid from a friend of a colleague of my wife - his drive failed, the computer shop replaced it but said "sorry can't recover". I said "no fix no fee I'll take a look £100" so the guy said "done". Fortunately the drive had recovered enough for me to do a "dd", and then I just dumped the lot to CDs. Very easy £100 thanks :-) and the guy got a stack of, I think, nine CDs.
this setup is extremely expensive and is interesting is you want a system as much as possible always running.
Not _extremely_ expensive, only the price of a second drive. Apart from high availability requirements, it's very useful if you want to avoid restoring a backup just because of a disk failure. The small price of the extra drive is easily paid back.
Well, I made the mistake of buying two Seagate Barracudas for my current system. Not just any Barracuda, the 3TB version that got slated as being likely to die far quicker than it should. And they're not really suitable for RAID. I've now got myself two Seagate Ironwolf - consumer grade NAS drives - so they'll be fine in a raid. Not much more expensive.
there are many situation where rebuilding a raid array can be very time consuming and error prone
Yes, the bigger the array, the longer it takes and that does become a problem. Error prone? Not in my experience.
And? Once you've got that much data, doing any sort of move is time consuming. Having done rescue dumps of entire hard-drives, I know how long it takes ... And how long does it take to back up a terabyte? My /home is about 2.5TB in size, and it's about two-thirds full.
for example, do you have at hand an exact replacement for any disk in the raid array. What if one fail and you have to wait three days to get an other?
That's why I'm going raid-6. Two parity drives so if one fails who cares. Well, I do because if a second one goes I'm in trouble, but what are the chances of two going that close to each other? Pretty low, I hope, touch wood!
You don't need an exact replacement - only one that is the same size of bigger.
Which is why I'm quite happy mixing two 3TB and two 4TB drives. With btrfs to protect against things like ransomware attacks ... Cheers, Wol -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 26/10/17 02:55 PM, Wols Lists wrote:
I've not known that many drives fail
https://arstechnica.com/information-technology/2017/01/8tb-hgst-disks-show-t... and more recently https://arstechnica.com/gadgets/2017/10/big-hard-disks-may-be-breaking-the-b... "1,240 10+TB hard disks installed, and not a single one has gone bad." <quote> Traditionally, the expected pattern of hard disk failures has been the so-called "bathtub curve:" a spate of failures of new disks ("infant mortality"), as disks that were defective from the factory are shipped into the wild and rapidly stop working, followed by a long period of low failure rates, and then an uptick in failures as the disks pass their engineered lifetime. The initial data from the 10TB and 12TB disks, however, has not shown that pattern. While the data so far is very limited, with 1,240 disks and 14,220 aggregate drive days accumulated so far, none of these disks (both Seagate models) have failed. </quote> It's early days yet, these haven't been hammered with use. But the lack if 'infant mortality" is interesting. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 26/10/2017 à 21:44, Anton Aylward a écrit :
It's early days yet, these haven't been hammered with use. But the lack if 'infant mortality" is interesting.
if there is no failure, raid is not even necessary :-)) jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On Thu, 2017-10-26 at 23:12 +0200, jdd@dodin.org wrote:
Le 26/10/2017 à 21:44, Anton Aylward a écrit :
It's early days yet, these haven't been hammered with use. But the lack if 'infant mortality" is interesting.
if there is no failure, raid is not even necessary :-))
Or even better...setting up a giant RAID0 array with them! Five disks and you've got the size of our enterprise SAN! :P
jdd
-----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEE7GM/Dul8WSWn72odQ1nEo4DFCIUFAlnyU6EACgkQQ1nEo4DF CIXhmAf/Xoyihf58IMvbXJweHLGoMDMsV7e/QyO4kdQ1A6Wy4bX3noNyBWLulRMy RK9ZAqtpiSC6VTZM3kANRTwUh+J4dAmzuTdJi4nqOdDSyGnuC1iNpB04dyUlsTWC 5CVufoG2GZmxVHMhuwqlT3pplw0W0G0ML4ortKjJ8WIHHWFr5hrK9FHPxpC35uRc LktDronpvT9B9QbUkckpzHSfoMTqfLhwfe6bSfuclq2HnPqOL1DrbbSTwU9rCpPE +MHAqi3qQ/K88hNCiuZSGw0R1/4yvdYHAFlJ5EH68eGWJDhed20cwhdzJOeCm9Ff LbYHu79a+2J69XczQOv4io0UhYeGoQ== =5niZ -----END PGP SIGNATURE-----
On Thu, 26 Oct 2017 15:44:02 -0400 Anton Aylward <opensuse@antonaylward.com> wrote:
On 26/10/17 02:55 PM, Wols Lists wrote:
I've not known that many drives fail
https://arstechnica.com/information-technology/2017/01/8tb-hgst-disks-show-t...
and more recently https://arstechnica.com/gadgets/2017/10/big-hard-disks-may-be-breaking-the-b... "1,240 10+TB hard disks installed, and not a single one has gone bad." <quote> Traditionally, the expected pattern of hard disk failures has been the so-called "bathtub curve:" a spate of failures of new disks ("infant mortality"), as disks that were defective from the factory are shipped into the wild and rapidly stop working, followed by a long period of low failure rates, and then an uptick in failures as the disks pass their engineered lifetime.
The initial data from the 10TB and 12TB disks, however, has not shown that pattern. While the data so far is very limited, with 1,240 disks and 14,220 aggregate drive days accumulated so far, none of these disks (both Seagate models) have failed. </quote>
It's early days yet, these haven't been hammered with use. But the lack if 'infant mortality" is interesting.
I don't think that's a fair summary of the data in the second article. Yes it shows zero failures for the Seagate 10 TB and 12 TB, but it also shows zero failures for the Toshiba and WDC 3 TB and the Toshiba 4 TB and the HGST 8 TB, so there's nothing special about the larger sizes. Indeed, given the extremely low drive-days for the 12 TB drives, it would perhaps be more remarkable if there WERE any failures. Given the absymal figures for other Seagate drives, I suppose it does go some way to rehabilitating them, but personally there's still no way I would touch anything from Seagate or WDC. (Jury is still out on HGST under new ownership, for me) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Wols Lists wrote:
On 26/10/17 19:24, Per Jessen wrote:
jdd@dodin.org wrote:
not always, not always like you want, not always when you want.
I have to say - I have been running some 20 systems (hosted externally) with RAID1 for more than 12 years. For the last 5-6 years, 1 or 2 drives fail per annum, sometimes more. They are replaced on demand, they rebuild and all done. Exactly as Dave describes it. Internally we use hot spares on every system, no complaints there either.
I've not known that many drives fail, actually.
At warranty expiration + 1 the error rate is 100% :-) Much depends on how many drives you have running. The absolute number of failures is directly proportional to the number of drives, but if you have e.g. less than 10, the absolute number of failures is quite small.
computer shop replaced it but said "sorry can't recover". I said "no fix no fee I'll take a look £100" so the guy said "done". Fortunately the drive had recovered enough for me to do a "dd", and then I just dumped the lot to CDs. Very easy £100 thanks :-) and the guy got a stack of, I think, nine CDs.
Good one.
this setup is extremely expensive and is interesting is you want a system as much as possible always running.
Not _extremely_ expensive, only the price of a second drive. Apart from high availability requirements, it's very useful if you want to avoid restoring a backup just because of a disk failure. The small price of the extra drive is easily paid back.
Well, I made the mistake of buying two Seagate Barracudas for my current system. Not just any Barracuda, the 3TB version that got slated as being likely to die far quicker than it should. And they're not really suitable for RAID.
Yes, the type of drive and the expected duty cycle are both important considerations, very true. I do this professionally, and we also stage purchases, to have drives with different production dates. An array with e.g. 10 drives from jan-feb-mar-etc reduces the probability of a dual drive failure, though it's difficult say how much.
there are many situation where rebuilding a raid array can be very time consuming and error prone
Yes, the bigger the array, the longer it takes and that does become a problem. Error prone? Not in my experience.
And? Once you've got that much data, doing any sort of move is time consuming. Having done rescue dumps of entire hard-drives, I know how long it takes ...
The longer it takes, the longer you are vulnerable, that's all. That's why we have hot spares in all local arrays. (the external machines don't have the room). -- Per Jessen, Zürich (12.6°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, 26 Oct 2017 20:24:54 +0200 Per Jessen <per@computer.org> wrote:
jdd@dodin.org wrote:
for example, do you have at hand an exact replacement for any disk in the raid array. What if one fail and you have to wait three days to get an other?
You don't need an exact replacement - only one that is the same size of bigger.
Indeed, and replacing one half by a larger disk, restoring then replacing the other half with an equally larger disk, restoring again and then growing the LVM and file system to fill the disks is one way to expand storage. Of course, installing the two new disks as a new RAID LVM and then copying the content is another way if you have the spare ports. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 26/10/17 02:10 PM, jdd@dodin.org wrote:
this setup is extremely expensive and is interesting is you want a system as much as possible always running.
Once upon a time labour - man power - was cheap and equipment expensive. These days it is the other way round. Because of that we cost things in terms of time and effort. Work out the cost of downtime in term of human inconvenience. Even a day at minimum wage is more than the local street price of a 1T drive here. Sysadmins might be paid more than the cost of a 1T drive per hour. At that price point, if you can save an hour's time by using, as per points out, with a simple RAID1, mirroring, then you're winning. Then again, there are hybrid version of RAID, RAID6, multi-level versions such as RAID60, RAID100, ways of trading space efficiency, fault tolerance, ... There are also collection that aren't termed RAID but offer similar kinds of functionality and don't require all drives to be the same size. You can do things like that with LVM and I understand it is possible with BtrFS. In addition, ZFS, which is sort of like BtrFS in its "one FS to rule them all" -- them being the disk drives - attitude towards drives, can achieve this sort of things as well. How "expensive" or not is up to the decision process, but it is an expense that needs to be compared to other expenses. https://en.wikipedia.org/wiki/ZFS#Software_RAID_using_ZFS You can do RAID-like things with LVM such as mirroring, striping. You are not limited to one drive. You can implement different policies for different file systems. LVM is great when you have a bunch of drives that are different sizes :-) RAID is simply and only a method to store your data safely whereas LVM is a software which can extend the performance of a RAID. RAID *can* be done in hardware whereas LVM is software. https://www.centos.org/docs/5/html/Cluster_Logical_Volume_Manager/LV_create.... Please note: you can create more than one mirror. See also: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/htm... for recovering from a mirror failure As for striping, you can do that with LVM and control the number of devices used in the striping, the interleaving. With LVM you can use stripe mapping across the drives as you would in RAID-0, with the capacity being the same as RAID-0. However, LVM allows you to also use the remaining space for additional volume groups. In the case of mdadm and software RAID-0 on Linux, you cannot grow a RAID-0 group. You can only grow a RAID-1, RAID-5, or RAID-6 array. This means that you can’t add drives to an existing RAID-0 group without having to rebuild the entire RAID group but having to restore all the data from a backup. However, with LVM you can easily grow a logical volume. But, you cannot use stripe mapping to add a drive to an existing striped logical volume because you can’t interleave the existing stripes with the new stripes. However you can extend a a LV with another ... and you can mix and match the degree of striping. And yes, you can create a LV that is BOTH mirrored and striped. It is just a matter of having enough drives :-) And they don't all have to be of the same size or geometry. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, 26 Oct 2017 15:34:29 -0400 Anton Aylward <opensuse@antonaylward.com> wrote:
In addition, ZFS, which is sort of like BtrFS in its "one FS to rule them all" -- them being the disk drives - attitude towards drives, can achieve this sort of things as well.
In my experience ZFS doesn't work well with just spinning rust. You need to give it some SSD as cache as well otherwise it's like swimming in treacle. But if it has loads of RAM, some SSD and loads of spinning rust, and you've updated it, then it does work. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
jdd@dodin.org wrote:
Le 26/10/2017 à 18:16, Wols Lists a écrit :
Hard drives at the bottom, raid on that, lvm on that, and probably btrfs on that.
I hope you wont ever have a faulty drive...
We don't use butterfs, but otherwise same kind of setup. Hardware raid5 or -6 on SATA drives, then LVM, then iSCSI. -- Per Jessen, Zürich (12.4°C) http://www.cloudsuisse.com/ - your owncloud, hosted in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 26/10/2017 à 20:26, Per Jessen a écrit : We don't use butterfs, but otherwise same kind of setup. Hardware raid5
or -6 on SATA drives, then LVM, then iSCSI.
on laptop? we are going too far OT here thanks jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
jdd@dodin.org wrote:
Le 26/10/2017 à 20:26, Per Jessen a écrit :
We don't use butterfs, but otherwise same kind of setup. Hardware raid5
or -6 on SATA drives, then LVM, then iSCSI.
on laptop?
No, not exactly. I was just commenting on the overall setup, which is perfectly sane. -- Per Jessen, Zürich (12.6°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 26/10/17 19:49, Per Jessen wrote:
No, not exactly. I was just commenting on the overall setup, which is perfectly sane.
Not quite :-) Raid-10 on two disks doesn't make particular sense, but if I replace a 3TB with a 4TB and increase it to three disks, then it does :-) Cheers, Wol -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-10-26 11:35, jdd@dodin.org wrote:
Hello,
My present install uses 3 partitions, / (30Gb, btrfs), /home (30 Gb, xfs)and /data, the last for large file storing (around 400Gb, xfs). no lvm.
I want to install an other distro (ubuntu) to test some special config only documented on ubuntu
AFAIK it's not possible to reduce the size of an xfs filesystem :-(
So Here what I did:
* copy (rsync, as root) my /home to /data/home-active folder * coment out the /home line in fstab * adding in fstab /data/home-actif /home none bind"
it seems to work, but before trashing the original /home partition I would like to know if all is ok, specially in the fstab line
It is simple: reboot to your new setup, confirm that /home is not mounted, confirm that you can login and that things work, then go ahead, looks good :-) If in doubt, just rsync home to another disk for backup. See my notes on rsync on another post. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
Le 27/10/2017 à 12:32, Carlos E. R. a écrit :
It is simple: reboot to your new setup, confirm that /home is not mounted, confirm that you can login and that things work, then go ahead, looks good :-)
Already done
If in doubt, just rsync home to another disk for backup.
See my notes on rsync on another post.
I know that, but there are so many options... some of my (old) notes I still find sometime new options or lack in the backups (like zero length files :-(). Last time it was my fault: I stopped the computer forgetting a (very long) rsync copy... I only noticed because I needed some file and couldn't find them... always have more than two copies :-)) jdd jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Friday, 2017-10-27 at 13:23 +0200, jdd@dodin.org wrote:
Le 27/10/2017 à 12:32, Carlos E. R. a écrit :
It is simple: reboot to your new setup, confirm that /home is not mounted, confirm that you can login and that things work, then go ahead, looks good :-)
Already done
If in doubt, just rsync home to another disk for backup.
See my notes on rsync on another post.
I know that, but there are so many options...
some of my (old) notes
I still find sometime new options or lack in the backups (like zero length files :-(). Last time it was my fault: I stopped the computer forgetting a (very long) rsync copy...
I only noticed because I needed some file and couldn't find them... always have more than two copies :-))
Currently, I do my rsync backups explicitly asking for CRC checksum, and do the run twice if I can. - -- Cheers, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlnzGFcACgkQtTMYHG2NR9V46gCeJMIuUrZQrAgG+7HM7nDfCeps /P4An2kSFKusmIC5Py2JJqjVCU5ayNhI =fwYD -----END PGP SIGNATURE-----
participants (9)
-
Anton Aylward -
Bruce Ferrell -
Carlos E. R. -
Christopher Myers -
Dave Howorth -
jdd＠dodin.org -
Per Jessen -
Peter Suetterlin -
Wols Lists