Le 26/10/2017 à 14:19, Peter Suetterlin a écrit :

OK only if the 'actif' is just a typo in this mail :D

oh, yes, the list is in english, so I translated, but the fstab was a raw copy :-) I added "actif" (active in English) to remember not to remove this seemingly duplicate folder someday :-)) jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Dave Howorth wrote:

On Thu, 26 Oct 2017 13:19:25 +0100 Peter Suetterlin wrote:

...

jdd@dodin.org wrote:

...

So Here what I did:

* copy (rsync, as root) my /home to /data/home-active folder

Not sure if you use hardlinks in your $HOME - those files would probably be doubled. No big thing though.

Not if you use the correct options with rsync. It is designed to preserve the hierarchy exactly.

I know it *can* do it. But it doesn't by default, and can get very slow if it does. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 27/10/17 13:46, Carlos E. R. wrote:

...

I know it *can* do it.

But it doesn't by default, and can get very slow if it does.

No, it doesn't get slow just by telling it to keep hardlinks. It gets slow when you tell it to compare files.

Depends on how many hardlinks you've got, and how much ram. Last time I tried it, it brought my system near enough to its knees, though that wasn't the current system that has ram coming out of its ears - the old system had less than a gig iirc. Cheers, Wol -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Fri, 27 Oct 2017 14:20:31 +0100 Peter Suetterlin wrote:

Wols Lists wrote:

...

On 27/10/17 13:46, Carlos E. R. wrote:

...

...

I know it *can* do it.

But it doesn't by default, and can get very slow if it does.

No, it doesn't get slow just by telling it to keep hardlinks. It gets slow when you tell it to compare files.

Depends on how many hardlinks you've got, and how much ram. Last time I tried it, it brought my system near enough to its knees, though that wasn't the current system that has ram coming out of its ears - the old system had less than a gig iirc.

Might indeed be - it's a warning that I well remembered, but I had not used the option myself in quite some time. Might be it was in (ancient) times of meager hardware, too :D

Well I've used it for many years with many millions of hardlinks and it works fine. And as for not doing it by default; I feel sure that if it did, you would complain about that as well. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2017-10-26 14:19, Peter Suetterlin wrote:

jdd@dodin.org wrote:

...

So Here what I did:

* copy (rsync, as root) my /home to /data/home-active folder

Not sure if you use hardlinks in your $HOME - those files would probably be doubled. No big thing though.

rsync copies hardlinks and sparse files as such, if you tell it to: -a, --archive archive mode; equals -rlptgoD (no -H,-A,-X) -r, --recursive recurse into directories -l, --links copy symlinks as symlinks -p, --perms preserve permissions -t, --times preserve times -g, --group preserve group -o, --owner preserve owner (super-user only) -D same as --devices --specials --devices preserve device files (super-user only) --specials preserve special files -A, --acls preserve ACLs (implies -p) [non-standard] -X, --xattrs preserve extended attrs (implies -p) [n.s.] -H, --hard-links preserve hard links -h, --human-readable output numbers in a human-readable format --progress show progress during transfer -P same as --partial --progress --partial keep partially transferred files --partial-dir=DIR put a partially transferred file into DIR -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)

Le 26/10/2017 à 15:39, Anton Aylward a écrit :

On 26/10/17 05:35 AM, jdd@dodin.org wrote:

...

I want to install an other distro (ubuntu) to test some special config only documented on ubuntu

Why not get a new drive. They are cheap enough. I can get a 3T drive now for what I paid for a 1T a few years ago.

it's already a 500Gb ssd on a laptop with only one place It was not a problem with ext4. I don't even remember why I formatted data as xfs (may be opensuse default) jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Le 26/10/2017 à 16:03, Bruce Ferrell a écrit :

Either log in as root (home for root is /root leaving everything in /home idle) or boot from your favorite recovery live media.  use gparted to alter the partition layout then move the contents of /home to the new location.

don't works with xfs, on the new location side jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Thu, 26 Oct 2017 08:20:10 -0700 Bruce Ferrell wrote:

Thats odd... It did for me when I did it a few months back on my MD3000i.

You *can* have problems if you don't have all the support stuff installed for a given filesystem type

On 10/26/17 7:41 AM, jdd@dodin.org wrote:

...

Le 26/10/2017 à 16:03, Bruce Ferrell a écrit :

...

Either log in as root (home for root is /root leaving everything in >> /home idle) or boot from your favorite recovery live media. use >> gparted to alter the partition layout then move the contents of /home >> to the new location.

don't works with xfs, on the new location side

jdd

As Bruce says, it certainly does work. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Thu, 26 Oct 2017 18:43:52 +0200 "jdd@dodin.org" wrote:

Le 26/10/2017 à 18:38, Dave Howorth a écrit :

...

On Thu, 26 Oct 2017 08:20:10 -0700

...

...

...

don't works with xfs, on the new location side

jdd As Bruce says, it certainly does work. one can't shrink an xfs file system as far as I know.

of course moving the whole (data) content is a solution, but not without risks of various kind.

Moving the data was what you suggested (and have cut from your reply, why?). You didn't suggest shrinking a partition. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 26/10/17 09:51 AM, jdd@dodin.org wrote:

...

it's already a 500Gb ssd on a laptop with only one place

I've upgraded 2.5" laptop drives before. I've a couple of old 2.5" ones that I can't imagine what to do with :-( I'm unclear as to your problem.

It was not a problem with ext4. I don't even remember why I formatted data as xfs (may be opensuse default)

Yes, you can shrink ext4 just fine. I'm not sure what the developers made XFS the default for /home. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

As I said, I don't see your problem. The first time I did this I got a size coupler and copied to old 2.5 contents to my big hard drive. The second time I had a net work and rsync'd it. You don't need to change drives if all you want to do is shrink /data. Copy /home and /data to your hard drive. Use gparted to zap the 2 old partitions. Make everything except the extant root a LVM. Set up minimal space as LVs on the LVM for a /home and /data and copy them back as EXT4. You've still got space on the LVM for other stuff and you can now shrink and grow the /home and /data as well. I've learnt to ALWAYS use a LVM for the simple reason that I don't know what the FS sizes are going to be over the lifetime. XFS doesn't make sense for me. Example: I file Photographs initially by year. if the FS gets crowded I create a new LV and move one year off to that & mount it in the old position. I've just done this with PhotoYear2014. While I was at it, I did the same with PhotoYear2017. So the base FS was now very sparse, so I shrunk it back, which freed up LEs that I can now use for something else. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 26/10/17 16:51, Anton Aylward wrote:

I've learnt to ALWAYS use a LVM for the simple reason that I don't know what the FS sizes are going to be over the lifetime.

While I've got masses of disk space (I use my desktop as main storage), again I'm going to be shifting that over to LVM for that exact reason - I don't know what I want where. Hard drives at the bottom, raid on that, lvm on that, and probably btrfs on that. Because I've got a mix of 3TB and 4TB drives, I'll put swap on a raid-10 on the 4TB's, have 1TB of mirror on the 4TB drives for all my root partitions, and then 6TB raid-6 across all four drives (3TB per drive) as /home. That way I can have gentoo as my main system, SuSE because that's what I support elsewhere, and Mint or Ubuntu or Red Hat or whatever for playing with, all in the 1TB mirror. I can't do what you do with photos though, because this is a multi-user system and most of the photos are hard-linked across several accounts. I could use soft links, but that then makes re-organising a royal pain. Cheers, Wol -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Thu, 26 Oct 2017 18:37:59 +0200 "jdd@dodin.org" wrote:

Le 26/10/2017 à 18:16, Wols Lists a écrit :

...

Hard drives at the bottom, raid on that, lvm on that, and probably btrfs on that.

I hope you wont ever have a faulty drive...

That's the main reason for having that setup. When a drive fails the machine carries on whilst you buy a replacement and swap it for the failed drive, which rebuilds. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

jdd@dodin.org wrote:

Le 26/10/2017 à 19:27, Dave Howorth a écrit :

...

On Thu, 26 Oct 2017 18:37:59 +0200 "jdd@dodin.org" wrote:

...

Le 26/10/2017 à 18:16, Wols Lists a écrit :

...

Hard drives at the bottom, raid on that, lvm on that, and probably btrfs on that.

I hope you wont ever have a faulty drive...

That's the main reason for having that setup. When a drive fails the machine carries on whilst you buy a replacement and swap it for the failed drive, which rebuilds.

not always, not always like you want, not always when you want.

I have to say - I have been running some 20 systems (hosted externally) with RAID1 for more than 12 years. For the last 5-6 years, 1 or 2 drives fail per annum, sometimes more. They are replaced on demand, they rebuild and all done. Exactly as Dave describes it. Internally we use hot spares on every system, no complaints there either.

this setup is extremely expensive and is interesting is you want a system as much as possible always running.

Not _extremely_ expensive, only the price of a second drive. Apart from high availability requirements, it's very useful if you want to avoid restoring a backup just because of a disk failure. The small price of the extra drive is easily paid back.

there are many situation where rebuilding a raid array can be very time consuming and error prone

Yes, the bigger the array, the longer it takes and that does become a problem. Error prone? Not in my experience.

for example, do you have at hand an exact replacement for any disk in the raid array. What if one fail and you have to wait three days to get an other?

You don't need an exact replacement - only one that is the same size of bigger. -- Per Jessen, Zürich (12.9°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 26/10/17 02:55 PM, Wols Lists wrote:

I've not known that many drives fail

https://arstechnica.com/information-technology/2017/01/8tb-hgst-disks-show-t... and more recently https://arstechnica.com/gadgets/2017/10/big-hard-disks-may-be-breaking-the-b... "1,240 10+TB hard disks installed, and not a single one has gone bad." <quote> Traditionally, the expected pattern of hard disk failures has been the so-called "bathtub curve:" a spate of failures of new disks ("infant mortality"), as disks that were defective from the factory are shipped into the wild and rapidly stop working, followed by a long period of low failure rates, and then an uptick in failures as the disks pass their engineered lifetime. The initial data from the 10TB and 12TB disks, however, has not shown that pattern. While the data so far is very limited, with 1,240 disks and 14,220 aggregate drive days accumulated so far, none of these disks (both Seagate models) have failed. </quote> It's early days yet, these haven't been hammered with use. But the lack if 'infant mortality" is interesting. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On Thu, 2017-10-26 at 23:12 +0200, jdd@dodin.org wrote:

Le 26/10/2017 à 21:44, Anton Aylward a écrit :

...

It's early days yet, these haven't been hammered with use. But the lack if 'infant mortality" is interesting.

if there is no failure, raid is not even necessary :-))

Or even better...setting up a giant RAID0 array with them! Five disks and you've got the size of our enterprise SAN! :P

jdd

-- http://dodin.org

-----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEE7GM/Dul8WSWn72odQ1nEo4DFCIUFAlnyU6EACgkQQ1nEo4DF CIXhmAf/Xoyihf58IMvbXJweHLGoMDMsV7e/QyO4kdQ1A6Wy4bX3noNyBWLulRMy RK9ZAqtpiSC6VTZM3kANRTwUh+J4dAmzuTdJi4nqOdDSyGnuC1iNpB04dyUlsTWC 5CVufoG2GZmxVHMhuwqlT3pplw0W0G0ML4ortKjJ8WIHHWFr5hrK9FHPxpC35uRc LktDronpvT9B9QbUkckpzHSfoMTqfLhwfe6bSfuclq2HnPqOL1DrbbSTwU9rCpPE +MHAqi3qQ/K88hNCiuZSGw0R1/4yvdYHAFlJ5EH68eGWJDhed20cwhdzJOeCm9Ff LbYHu79a+2J69XczQOv4io0UhYeGoQ== =5niZ -----END PGP SIGNATURE-----

Wols Lists wrote:

On 26/10/17 19:24, Per Jessen wrote:

...

jdd@dodin.org wrote:

...

not always, not always like you want, not always when you want.

I have to say - I have been running some 20 systems (hosted externally) with RAID1 for more than 12 years. For the last 5-6 years, 1 or 2 drives fail per annum, sometimes more. They are replaced on demand, they rebuild and all done. Exactly as Dave describes it. Internally we use hot spares on every system, no complaints there either.

I've not known that many drives fail, actually.

At warranty expiration + 1 the error rate is 100% :-) Much depends on how many drives you have running. The absolute number of failures is directly proportional to the number of drives, but if you have e.g. less than 10, the absolute number of failures is quite small.

computer shop replaced it but said "sorry can't recover". I said "no fix no fee I'll take a look £100" so the guy said "done". Fortunately the drive had recovered enough for me to do a "dd", and then I just dumped the lot to CDs. Very easy £100 thanks :-) and the guy got a stack of, I think, nine CDs.

Good one.

...

...

this setup is extremely expensive and is interesting is you want a system as much as possible always running.

Not _extremely_ expensive, only the price of a second drive. Apart from high availability requirements, it's very useful if you want to avoid restoring a backup just because of a disk failure. The small price of the extra drive is easily paid back.

Well, I made the mistake of buying two Seagate Barracudas for my current system. Not just any Barracuda, the 3TB version that got slated as being likely to die far quicker than it should. And they're not really suitable for RAID.

Yes, the type of drive and the expected duty cycle are both important considerations, very true. I do this professionally, and we also stage purchases, to have drives with different production dates. An array with e.g. 10 drives from jan-feb-mar-etc reduces the probability of a dual drive failure, though it's difficult say how much.

...

...

there are many situation where rebuilding a raid array can be very time consuming and error prone

Yes, the bigger the array, the longer it takes and that does become a problem. Error prone? Not in my experience.

And? Once you've got that much data, doing any sort of move is time consuming. Having done rescue dumps of entire hard-drives, I know how long it takes ...

The longer it takes, the longer you are vulnerable, that's all. That's why we have hot spares in all local arrays. (the external machines don't have the room). -- Per Jessen, Zürich (12.6°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 26/10/17 02:10 PM, jdd@dodin.org wrote:

this setup is extremely expensive and is interesting is you want a system as much as possible always running.

Once upon a time labour - man power - was cheap and equipment expensive. These days it is the other way round. Because of that we cost things in terms of time and effort. Work out the cost of downtime in term of human inconvenience. Even a day at minimum wage is more than the local street price of a 1T drive here. Sysadmins might be paid more than the cost of a 1T drive per hour. At that price point, if you can save an hour's time by using, as per points out, with a simple RAID1, mirroring, then you're winning. Then again, there are hybrid version of RAID, RAID6, multi-level versions such as RAID60, RAID100, ways of trading space efficiency, fault tolerance, ... There are also collection that aren't termed RAID but offer similar kinds of functionality and don't require all drives to be the same size. You can do things like that with LVM and I understand it is possible with BtrFS. In addition, ZFS, which is sort of like BtrFS in its "one FS to rule them all" -- them being the disk drives - attitude towards drives, can achieve this sort of things as well. How "expensive" or not is up to the decision process, but it is an expense that needs to be compared to other expenses. https://en.wikipedia.org/wiki/ZFS#Software_RAID_using_ZFS You can do RAID-like things with LVM such as mirroring, striping. You are not limited to one drive. You can implement different policies for different file systems. LVM is great when you have a bunch of drives that are different sizes :-) RAID is simply and only a method to store your data safely whereas LVM is a software which can extend the performance of a RAID. RAID *can* be done in hardware whereas LVM is software. https://www.centos.org/docs/5/html/Cluster_Logical_Volume_Manager/LV_create.... Please note: you can create more than one mirror. See also: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/htm... for recovering from a mirror failure As for striping, you can do that with LVM and control the number of devices used in the striping, the interleaving. With LVM you can use stripe mapping across the drives as you would in RAID-0, with the capacity being the same as RAID-0. However, LVM allows you to also use the remaining space for additional volume groups. In the case of mdadm and software RAID-0 on Linux, you cannot grow a RAID-0 group. You can only grow a RAID-1, RAID-5, or RAID-6 array. This means that you can’t add drives to an existing RAID-0 group without having to rebuild the entire RAID group but having to restore all the data from a backup. However, with LVM you can easily grow a logical volume. But, you cannot use stripe mapping to add a drive to an existing striped logical volume because you can’t interleave the existing stripes with the new stripes. However you can extend a a LV with another ... and you can mix and match the degree of striping. And yes, you can create a LV that is BOTH mirrored and striped. It is just a matter of having enough drives :-) And they don't all have to be of the same size or geometry. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

jdd@dodin.org wrote:

Le 26/10/2017 à 18:16, Wols Lists a écrit :

...

Hard drives at the bottom, raid on that, lvm on that, and probably btrfs on that.

I hope you wont ever have a faulty drive...

We don't use butterfs, but otherwise same kind of setup. Hardware raid5 or -6 on SATA drives, then LVM, then iSCSI. -- Per Jessen, Zürich (12.4°C) http://www.cloudsuisse.com/ - your owncloud, hosted in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

jdd@dodin.org wrote:

Le 26/10/2017 à 20:26, Per Jessen a écrit :

We don't use butterfs, but otherwise same kind of setup. Hardware raid5

...

or -6 on SATA drives, then LVM, then iSCSI.

on laptop?

No, not exactly. I was just commenting on the overall setup, which is perfectly sane. -- Per Jessen, Zürich (12.6°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Friday, 2017-10-27 at 13:23 +0200, jdd@dodin.org wrote:

Le 27/10/2017 à 12:32, Carlos E. R. a écrit :

...

It is simple: reboot to your new setup, confirm that /home is not mounted, confirm that you can login and that things work, then go ahead, looks good :-)

Already done

...

If in doubt, just rsync home to another disk for backup.

See my notes on rsync on another post.

I know that, but there are so many options...

some of my (old) notes

I still find sometime new options or lack in the backups (like zero length files :-(). Last time it was my fault: I stopped the computer forgetting a (very long) rsync copy...

I only noticed because I needed some file and couldn't find them... always have more than two copies :-))

Currently, I do my rsync backups explicitly asking for CRC checksum, and do the run twice if I can. - -- Cheers, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlnzGFcACgkQtTMYHG2NR9V46gCeJMIuUrZQrAgG+7HM7nDfCeps /P4An2kSFKusmIC5Py2JJqjVCU5ayNhI =fwYD -----END PGP SIGNATURE-----