В Thu, 28 May 2015 18:42:18 +0200 jdd пишет:

Le 28/05/2015 17:54, Andrei Borzenkov a écrit :

...

Is there existing data in this partition?

yes, but the same data (approx, it was the same yesterday)

In this case you likely won't be able to use YaST. But creating manually is easy. bor@opensuse:~/src/lvm2> sudo mdadm --create /dev/md0 --level 1 --raid-devices 1 /dev/loop0 mdadm: '1' is an unusual number of drives for an array, so it is probably a mistake. If you really mean it you will need to specify --force before setting the number of drives. bor@opensuse:~/src/lvm2> sudo mdadm --create /dev/md0 --level 1 --raid-devices 1 --force /dev/loop0 mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90 mdadm: largest drive (/dev/loop0) exceeds size (960K) by more than 1% Continue creating array? y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md0 started. bor@opensuse:~/src/lvm2> sudo mkfs -t ext2 /dev/md0 mke2fs 1.42.12 (29-Aug-2014) Creating filesystem with 960 1k blocks and 120 inodes Allocating group tables: done Writing inode tables: done Writing superblocks and filesystem accounting information: done bor@opensuse:~/src/lvm2> sudo mount -t ext2 /dev/md0 /tmp/x Now copy data from old file system to new on /dev/md0; make sure data is copied. When you are satisfied, unmount old partition and add it to array. bor@opensuse:~/src/lvm2> sudo mdadm --grow /dev/md0 --add --raid-devices 2 /dev/loop1 mdadm: added /dev/loop1 raid_disks for /dev/md0 set to 2 unfreeze bor@opensuse:~/src/lvm2> sudo mdadm --detail --scan ARRAY /dev/md0 metadata=1.2 name=opensuse.site:0 UUID=686f4db4:e2c57d52:fbc1bd79:5ad5e1e3 Now just dump configuration mdadm --detail --scan >> /etc/mdadm.conf Adjust numbers as needed (if you have some arrays already use /dev/md1 etc). Change fstab to use /dev/md0 or new UUID or old LABEL - up to you.

but as it is the same, I can wipe one without problem (not the two of them)

thanks jdd

-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

В Thu, 28 May 2015 19:57:20 +0200 "Carlos E. R." пишет:

Members of a software raid 1 (mirror) can be mounted separately, without raid.

It depends on MD metadata format. Today default is 1.2 which places metadata at the start of device, so actual data starts at some offset and device won't be recognized as filesystem. Previous format was 0.90 that placed metadata at the end of device, thus you could indeed access data on individual mirror pieces normally. There are several variants of 1.x format, one of them places metadata at the end as well.

On 2015-05-28 20:28, Andrei Borzenkov wrote:

В Thu, 28 May 2015 19:57:20 +0200 "Carlos E. R." <> пишет:

...

Members of a software raid 1 (mirror) can be mounted separately, without raid.

It depends on MD metadata format. Today default is 1.2 which places metadata at the start of device, so actual data starts at some offset and device won't be recognized as filesystem.

Ah...

Previous format was 0.90 that placed metadata at the end of device, thus you could indeed access data on individual mirror pieces normally.

I see.

There are several variants of 1.x format, one of them places metadata at the end as well.

Good to know... Thanks. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

Le 28/05/2015 20:45, Per Jessen a écrit :

Carlos E. R. wrote:

...

Actually, IMO, an rsync job is safer than raid, it covers more failure modes, but not as fast.

It depends on how you define "safe". In my book, the longer a system runs without two identical copies, the less safe it is. There is no way an rsync copy will ever catch up with a RAID1 mirror.

problem with raid is that nothing verify disk failures on unused data. the raid purpose is uninterrupted running, not data saving. on the same time rsync does some control, but the daily rsync I do on my server (to an other server) propagates also the errors done and not noticed immediately... no system is perfect jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On May 29, 2015 1:58:23 AM EDT, Per Jessen wrote:

jdd wrote:

...

Le 28/05/2015 20:45, Per Jessen a écrit :

...

Carlos E. R. wrote:

...

Actually, IMO, an rsync job is safer than raid, it covers more failure modes, but not as fast.

It depends on how you define "safe". In my book, the longer a system runs without two identical copies, the less safe it is. There is no way an rsync copy will ever catch up with a RAID1 mirror.

problem with raid is that nothing verify disk failures on unused data. the raid purpose is uninterrupted running, not data saving.

Absolutely.

What is being said? With raid you can setup a background scrub which scans all the member disks and ensures there are no bad blocks and data is consistent. http://en.m.wikipedia.org/wiki/Data_scrubbing#RAID Notice mdraid has support for data scrubbing. Greg -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Le 29/05/2015 15:12, greg.freemyer@gmail.com a écrit :

With raid you can setup a background scrub which scans all the member disks and ensures there are no bad blocks and data is consistent.

http://en.m.wikipedia.org/wiki/Data_scrubbing#RAID

Notice mdraid has support for data scrubbing.

very interesting, first time I hear about it thanks! jdd (there is always something to learn :-) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2015-05-28 20:45, Per Jessen wrote:

Carlos E. R. wrote:

...

Actually, IMO, an rsync job is safer than raid, it covers more failure modes, but not as fast.

It depends on how you define "safe". In my book, the longer a system runs without two identical copies, the less safe it is. There is no way an rsync copy will ever catch up with a RAID1 mirror.

A RAID system does not protect you against data corruption or filesystem corruption or human error. It only protects against HD hardware failure. If you delete a file by mistake, you may still have a copy in the rsync copy - which may have a photo taken each hour, so you can have several versions of the same file. Depends on how much space you dedicate to it. On a RAID, both copies are lost instantly. If my data is so important to need a raid, I absolutely must also have an rsync backup. If I can't have both, then it will only be the rsync copy. It could be designed a system doing the best of both worlds, working like a raid, but one of the sides being always incremental backup type, not a real mirror. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlVnjDwACgkQja8UbcUWM1yK9AD+OHlp7Myic5IBFwaar3+htIQn 2yB3qiFzKcg0wJHuQogBAI7brFGZeEPS2raU/sLOzBcWC3bGQJAc51kuvT8j5Gtp =KLOt -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

jdd wrote:

Le 28/05/2015 23:44, Carlos E. R. a écrit :

...

It could be designed a system doing the best of both worlds, working like a raid, but one of the sides being always incremental backup type, not a real mirror.

incremental backup can be used for light data, not big one.

I make frequent video shots, with sources as big as 100 Gb for an evening, and subsequent use as Blu-Ray video, more than 20Gb, I often have more than 500Gb transient data during editing,impossible to backup incrementally (I only keep the last work, and the sources for 2/3 years...)

as I said, no solution is perfect.

For example, I'm retired and have no more parents, so I have no other place to hold hard disks than my house. I keep a copy in an anti-fire safe at the other side of the house, but all ca be stolen or destroyed by wild fire.

so I keep a light copy on my web site, but it's full copy of photos, but only web size copy of videos...

to be back to the subject, I'm not sure the raid is worth it's cost if the permanent use if not mandatory.

Cost is minimal - a 2Tb disk is less than EUR100, a Seagate 8Tb drive is only EUR250. It is purely a question of your needs. Do you need an up-to-the-second copy at all times, so no data is lost when a disk breaks or is it acceptable to revert to the 24hour old backup? For the former, RAID (maybe DRBD) is the solution, for the latter rsync copying to a NAS box will suffice. As Carlos says, if you also want protection against accidental file deletion (and other fat finger incidents), you need both. -- Per Jessen, Zürich (15.6°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256

On 2015-05-28 20:45, Per Jessen wrote:

...

Carlos E. R. wrote:

...

Actually, IMO, an rsync job is safer than raid, it covers more failure modes, but not as fast.

It depends on how you define "safe". In my book, the longer a system runs without two identical copies, the less safe it is. There is no way an rsync copy will ever catch up with a RAID1 mirror.

A RAID system does not protect you against data corruption or filesystem corruption or human error. It only protects against HD hardware failure. If you delete a file by mistake, you may still have a copy in the rsync copy - which may have a photo taken each hour, so you can have several versions of the same file. Depends on how much space you dedicate to it. On a RAID, both copies are lost instantly.

We were talking about what "safe" is. If you want to safeguard your data against your own mistakes, RAID is certainly not the right tool. A pair of big gardening gloves might be better :-)

If my data is so important to need a raid, I absolutely must also have an rsync backup. If I can't have both, then it will only be the rsync copy.

Your rsync copy still isn't as good as your RAID "copy". The rsync copy is always older, so you will always lose data when you have to use it. -- Per Jessen, Zürich (15.7°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. wrote:

On 2015-05-29 08:05, Per Jessen wrote:

...

Carlos E. R. wrote:

...

We were talking about what "safe" is. If you want to safeguard your data against your own mistakes, RAID is certainly not the right tool. A pair of big gardening gloves might be better :-)

And filesystem crash, and file corruption. Don't forget both.

I have not had either for years and years. I use a mature filesystem. Of course, I also run rsync backups.

But the problem is that many home users set up raid, because their main boards have it, in the belief that it makes their data safe. It doesn't.

Well, let's be honest - it does make their data a lot _safer_. Of all the things that can happen to the data on a harddisk, the major risk is harddisk failure (usually exactly 3 years after purchase), and RAID takes care of that (as long as you replace the failed disk as soon as you can). For the rest of it - fat fingers, flooding, fire, earthquakes - even filesystem crash and file corruption - you need a backup copy. -- Per Jessen, Zürich (23.0°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R. wrote:

On 2015-05-29 08:05, Per Jessen wrote:

...

...

If my data is so important to need a raid, I absolutely must also have an rsync backup.

I use a RAID0 on my workstation -- they aren't just for backups

...

Your rsync copy still isn't as good as your RAID "copy". The rsync copy is always older, so you will always lose data when you have to use it.

Which is why it should be both.

rsync doesn't protect you against data corruption that you don't find out about until next week (or month). With daily incrementals, you can go back to any specific day in the period you keep. With rsync, if you've been infected w/malware both rsync and RAID will just help duplicated. And by 'dailies', I don't mean you have to restore each day to get to your current day, Usually 5 restores is a max to get to any given day.. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

A RAID system does not protect you against data corruption or filesystem corruption or human error. It only protects against HD hardware failure.

True.

If you delete a file by mistake, you may still have a copy in the rsync copy -

As you would have a copy in a snapshot [ZFS, LVM, BTRFS].

which may have a photo taken each hour, so you can have several versions of the same file.

This depends, even with rsync it is possible to create a single instance store [with the often overlooked --link-dest=DIR option]. We backup piles of data every day using --link-dest=DIR [referencing the backup from the previous day] and consume stunningly little disk capacity [and the I/O load the backup is massively reduced]. - read snap-dir from previous day - create new-dir - rsync source to new dir with --link-desk=snap-dir - write snap-der as snap-dir - rinse-repeat.

If my data is so important to need a raid, I absolutely must also have an rsync backup. If I can't have both, then it will only be the rsync copy.

Or data on a RAID in an LVM, snap, rsync the snap to alternate storage, drop snap. It is important to distinguish between redundancy [increases availability, aka uptime], backup [time machine'd copies], and archive [safe copies].

It could be designed a system doing the best of both worlds, working like a raid, but one of the sides being always incremental backup type, not a real mirror.

Yep. -- Adam Tauno Williams mailto:awilliam@whitemice.org GPG D95ED383 Systems Administrator, Python Developer, LPI / NCLA -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Thu, 2015-05-28 at 19:57 +0200, Carlos E. R. wrote:

On 2015-05-28 17:41, jdd wrote:

...

(I can also make an hourly rsync crom job if raid is overkill) Actually, IMO, an rsync job is safer than raid, it covers more failure modes, but not as fast.

In a sense not as fast - or faster. A RAID on one disk means one disk takes double [or worse] the same write load. If the user gets deep into I/O performance is going to suffer. But for ease of use, etc... if some redundancy and some time-machine is desired I'd put LVM on the mirrored partition and create a snapshot every hour, dropping snapshots after a few snapshots deep. Or use Butter FS. -- Adam Tauno Williams mailto:awilliam@whitemice.org GPG D95ED383 Systems Administrator, Python Developer, LPI / NCLA -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2015-05-29 12:36, Adam Tauno Williams wrote:

Or use Butter FS.

Btrfs? Or you mean something else :-? -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

On 05/28/2015 12:57 PM, Carlos E. R. wrote:

...

(I can also make an hourly rsync crom job if raid is overkill) Actually, IMO, an rsync job is safer than raid, it covers more failure modes, but not as fast. Like deleting a file by mistake, or a filesystem corruption.

I disagree -- only for this reason. The rsync job is not 'atomic'. (in a bastardized sense of the word, -- meaning writes will have occurred between the last rsync and the disk failure you are protecting against). mdraid is by far the most flexible and robust and reliable and portable raid solution for your typical desktop or server. (it does not have a chip dependency of dmraid/fake raid [it's a good solution too, just know the limits]). But the purpose of raid is to provide real-time fault tolerance. Protection against a disk failing 'now' while preserving all incremental writes since the last rsync shot. That is why rsync is no substitute for raid. (it is also equally true) That raid is no substitute for rsync. (to separate media) As a general point of interest, both mdraid/dmraid can exist side-by-side in the same install (e.g.): $ cat /proc/partitions major minor #blocks name <snip> 9 0 204608 md0 9 1 52396032 md1 9 2 922944192 md2 9 3 1047552 md3 253 0 244198583 dm-0 253 1 128457 dm-1 253 2 24410736 dm-2 253 3 1951866 dm-3 253 4 217704816 dm-4 So you are not precluded from adding arrays of either type to an existing install. Having run both for the past 15 years, I would lean toward mdraid for one primary reason -- rebuilding after a disk failure. Simply install a new drive, boot, and the rebuild is automatic. Boot as normal, an check progress with 'cat /proc/mdstat'. dmraid rebuilds (if available at all), usually involve booting to the dark side. (which is the real-world means shutting down the system, booting gparted (or using cfdisk or dd or rsync) to recreate partitions and then duplicating the remaining good drive to a new drive) -- David C. Rankin, J.D.,P.E. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org