On Wed, Dec 7, 2016 at 6:31 PM, Greg Freemyer wrote:

On Wed, Dec 7, 2016 at 2:56 AM, Per Jessen wrote:

...

Greg Freemyer wrote:

...

On Tue, Dec 6, 2016 at 2:46 AM, Per Jessen wrote:

...

Greg Freemyer wrote:

...

Got my Drobo today. So far I'm not impressed.

Pros:

I plugged in 8 1TB drives and it saw them and let me thin provision a 16TB volume.

That sounds like a major pro (or con) - 16Tb with only 8Tb space :-) I guess it automagically dealt with RAID levels and such?

Thin provisioning works like that, and yes as I think about it, it is a major pro.

Okay, I thought it was a typo - with LVM, thin provisioning just means the space isn't all allocated, but I doubt if you can allocate a volume that is bigger than what is available in a PV. Well, it has never occurred to me to try it :-) I usually allocate what I need, then extend later on when needed.

A cool thin provisioning feature I just noticed.

This is with a Windows 10 client over iSCSI.

I populated about 3TB of data onto the volume yesterday, composed of about 1 million files. Today I deleted them and my overall array usage dropped.

That means the SCSI commands to unallocate (or erase) unused sectors was issued to the Drobo and the Drobo actually removed the sectors from the allocated data blocks.

I'm extremely surprised that happened. I wasn't even trying to test it. I just noticed that my overall data usage dropped on the Drobo when I did a large folder delete.

I need to setup a iSCSI volume for openSUSE to access, but at least for now as soon as I create a new iSCSI volume via the user interface, it gets accessed by that Windows PC. I'll try to do some more experimenting.

I tried to create a iSCSI volume, then mount if from openSUSE 42.2. Creating it was easy. No success on the openSUSE end, but that might be operator error since I don't know what I'm doing. I ordered a second 10TB drive last week. I also pulled out one of the 1TB drives to leave a empty drive bay for the 10TB. It took about 12 hours to rebuild then. A few minutes ago I popped the new (second) 10 TB drive in. It took a couple minutes, then the overall status of the Drobo showed the extra capacity and the all drives were green. I went ahead and pulled out another 1TB drive to make room for the next 10TB drive install. The Drobo is reporting 24 hours to do the rebuild this time. I'm very surprised. I still only have about 3TB of data on the array. With such long rebuild times, it would be nice if the unit had a "migrate data" feature that allowed the data to be migrated off of a drive without creating a day long window of vulnerability. If it has that, I missed it. This is definitely not the fastest unit in the world, but for the $160 (or so) I paid for it, I'm very happy. I've got it set to 1-disk redundancy. I gather that means a collection of Raid-1 and Raid-5 sub-volumes are being created and aggregated together. The data layout is hidden from me, but it is fully leveraging both of the 10TB drives now that I have 2 installed. With only one, most of the first 10TB went unusable. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Tue, Dec 13, 2016 at 2:35 PM, jdd wrote:

Le 13/12/2016 à 20:29, Greg Freemyer a écrit :

...

This is definitely not the fastest unit in the world, but for the $160 (or so) I paid for it, I'm very happy.

I guess the 10Tb drives are much more expensive :-)

jdd

Well, the drives are re-usable if I decide to dump the Drobo in the future. 10TB drives (~$400) are not the cheapest per TB option, but I decided that I would get the largest I could buy in order not to have to replace them all a couple years from now. In theory, the Drobo I have will hold 8 x 10TB, so that should last me for a while in this use case. FYI: I bought a "surveillance" HDD this last time (Seagate Skyhawk). The first was a Seagate Barracuda. The surveillance drive was almost 20% cheaper. http://www.seagate.com/www-content/product-content/skyhawk/files/skyhawk-ds-... The surveillance drive claims to be pretty robust and that is what I care about more than performance for this situation. I can't help but wonder if they aren't the same drive with different labels and warranties attached. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Tue, Dec 13, 2016 at 6:11 PM, John Andersen wrote:

On 12/13/2016 03:02 PM, Greg Freemyer wrote:

...

I would get the largest I could buy in order not to have to replace them all a couple years from now.

Buys Security Drives.... Goes through multiple 10Tb Drives in a couple years...

You are starting to worry me Greg...

This is work related activity. I work in litigation support. My worry is if I bought 8x4TB as an example, I might fill it up in the next year or two. With 10TB drives, I should be able just keep adding drives and not fill up all 8 slots. Anyway, I rarely wear out drives. I mostly fill them up with potentially important data and put them on the shelf not even powered on. I use the data heavily for a few weeks, then ignore it for a couple years. Most of those drives are way below 50% utilized, but I keep different matters on different physical drives. Once the lawsuit concludes, I destroy the data (wipe the drive). I'm using this Drobo just to create a consolidated onsite backup of those preservation drives. The surveillance drives are actually designed for 7x24 operation and supposedly work well in a RAID array. That is almost a perfect match for my use case, and $70/drive cheaper than a Barracuda. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Le 14/12/2016 à 08:22, Per Jessen a écrit :

I replace SATA drives regularly - we have a lot - the WD RE4 and the older IBM/Hitachi Ultrastar are the best, anything consumer-oriented (3 year warranty) will die almost exactly when the warranty expires.

not that sure, I have some much older than that (an as a simple consumer I buy the cheaper 3 years waranty I can find) My main desktop is 6 years old and still have his original disk (and it's the one I use the most) what I fear the most is inactive drives that don't awake jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Wed, Dec 14, 2016 at 3:02 AM, jdd wrote:

Le 14/12/2016 à 08:22, Per Jessen a écrit :

...

I replace SATA drives regularly - we have a lot - the WD RE4 and the older IBM/Hitachi Ultrastar are the best, anything consumer-oriented (3 year warranty) will die almost exactly when the warranty expires.

not that sure, I have some much older than that (an as a simple consumer I buy the cheaper 3 years waranty I can find)

My main desktop is 6 years old and still have his original disk (and it's the one I use the most)

what I fear the most is inactive drives that don't awake

jdd

There was a run of drives 15 years or so ago. They put on too much grease. It could accumulate on the outer edge of the platter. When powered on the motor wasn't always strong enough to break the platter free of the grease. The fix was to drop the drive from a couple inches (10 cm) above a desk. :) Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

...

The fix was to drop the drive from a couple inches (10 > cm) above a desk. :)

- 10.16 cms = 4 inches ? ....... regards -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Wed, Dec 14, 2016 at 10:08 AM, Per Jessen wrote:

...

I ordered a second 10TB drive last week. I also pulled out one of the 1TB drives to leave a empty drive bay for the 10TB. It took about 12 hours to rebuild then.

Interesting that it rebuilds when a disk has been removed.

Huh? That's what I expect from any decent RAID implementation. Unless this disk was not in use, but that's not clear from the above. ...

...

With such long rebuild times, it would be nice if the unit had a "migrate data" feature that allowed the data to be migrated off of a drive without creating a day long window of vulnerability. If it has that, I missed it.

If it has it, it ought to be invoked automatically I would say. It's first and foremost priority should be to reduce the time in degraded mode.

How do you expect it to be invoked for a disk that is not present anymore? You can only rebuild using redundant information on remaining drives. Such migration must happen while disk is still present. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Andrei Borzenkov wrote:

On Wed, Dec 14, 2016 at 10:08 AM, Per Jessen wrote:

...

...

I ordered a second 10TB drive last week. I also pulled out one of the 1TB drives to leave a empty drive bay for the 10TB. It took about 12 hours to rebuild then.

Interesting that it rebuilds when a disk has been removed.

Huh? That's what I expect from any decent RAID implementation. Unless this disk was not in use, but that's not clear from the above.

When I remove a disk from a raid5 or raid6 array, it just goes into degraded mode. Rebuilding doesn't happen until a new drive is added. -- Posted with knode 4.14 from openSUSE Leap42.1 office34: Cyrix 486DX2/66MHz, 4096Mb RAM, #9 GXE -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Andrei Borzenkov wrote:

On Wed, Dec 14, 2016 at 11:08 AM, Per Jessen wrote:

...

Andrei Borzenkov wrote:

...

On Wed, Dec 14, 2016 at 10:08 AM, Per Jessen wrote:

...

...

I ordered a second 10TB drive last week. I also pulled out one of the 1TB drives to leave a empty drive bay for the 10TB. It took about 12 hours to rebuild then.

Interesting that it rebuilds when a disk has been removed.

Huh? That's what I expect from any decent RAID implementation. Unless this disk was not in use, but that's not clear from the above.

When I remove a disk from a raid5 or raid6 array, it just goes into degraded mode. Rebuilding doesn't happen until a new drive is added.

Which is why hot spare disks exist

Of course, but Greg didn't mention any.

- to have something to rebuild on as soon as failure is detected. I honestly expect that anyone who cares about data availability has one ... :)

Requirements do differ, but I would tend to agree. Running degraded is an open invitation for disaster, the longer the worse.

There are also custom implementations that reserve space on disks and rebuild data from failed disk onto this reserve space. Or simply on free space, as long as there is enough. This company appears to offer such custom implementation:

Greg will be able to tell us. -- Per Jessen, Zürich (0.8°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

jdd wrote:

Le 14/12/2016 à 09:52, Per Jessen a écrit :

...

Requirements do differ, but I would tend to agree. Running degraded is an open invitation for disaster, the longer the worse.

no worst than having no raid...

Well no, but that's hardly the point.

raid is only an availability question - and a price one. When price do not matter (or is low according to the general budget), it's obviously better

Pricey? That's matter of opinion of course, but when a 500Gb is only CHF50, I don't think it's expensive. With enterprise level hardware, it's slightly more expensive (e.g. CHF160 for a 2TB WD RE4), but that's irrelevant.

my main concern on raid is unused data control, that is verification of the integrity of unused data on disk (to prevent unexpected multi disk failure).

Huh? You mean check unused data or unused space? I don't see what you would gain and I am not aware of any enterprise class hardware that will do it. Usually SMART (and other vendor-specific diagnostics) tests are run regularly, that's about it. If you cannot rely on your data being written accurately to your disk, all bets are off. -- Per Jessen, Zürich (1.8°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Andrei Borzenkov wrote:

On Wed, Dec 14, 2016 at 1:40 PM, Per Jessen wrote:

...

...

my main concern on raid is unused data control, that is verification of the integrity of unused data on disk (to prevent unexpected multi disk failure).

Huh? You mean check unused data or unused space? I don't see what you would gain

Consider RAID5 stripe D1, D2, P (two data blocks, one parity). D1 belongs some file and is accessed frequently. D2 happens to be outside of any file or metadata so never explicitly accessed by host.

If D1 is unreadable, it can be reconstructed from D2 and P. Because D1 (and P) are accessed often, there is good probability that problems will be noticed and corrected early. But you are not aware of state of D2 until the very moment you need it to reconstruct your data. If at this moment D2 is unreadable, data is lost. D2 may have failed years ago without you being aware of it.

In principle it hasn't failed until you notice it :-)

...

and I am not aware of any enterprise class hardware that will do it.

*Every* enterprise class hardware performs periodical scrub across the full range of disk blocks to detect and fix latent errors. Most enterprise class hardware I am aware of performs permanent scrub and you cannot turn it off at all.

...

Usually SMART (and other vendor-specific diagnostics) tests are run regularly, that's about it.

We probably have different notion of "enterprise class hardware" ... :)

We use 3ware, HP and IBM. Yes, I know the latter two will scan for bad sectors during period of inactivity, but that's all. They don't go scrubbing down terabytes of data very often if at all. I don't know if the drive electronics does something like that?

...

If you cannot rely on your data being written accurately to your disk, all bets are off.

The fact that data had been accurately written to your disk in the past by no means imply that data can be accurately read today.

Very true, yet we all rely on exactly that. -- Per Jessen, Zürich (2.2°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Wed, Dec 14, 2016 at 6:39 AM, Per Jessen wrote:

Andrei Borzenkov wrote:

...

On Wed, Dec 14, 2016 at 1:40 PM, Per Jessen wrote:

...

...

my main concern on raid is unused data control, that is verification of the integrity of unused data on disk (to prevent unexpected multi disk failure).

Huh? You mean check unused data or unused space? I don't see what you would gain

Consider RAID5 stripe D1, D2, P (two data blocks, one parity). D1 belongs some file and is accessed frequently. D2 happens to be outside of any file or metadata so never explicitly accessed by host.

If D1 is unreadable, it can be reconstructed from D2 and P. Because D1 (and P) are accessed often, there is good probability that problems will be noticed and corrected early. But you are not aware of state of D2 until the very moment you need it to reconstruct your data. If at this moment D2 is unreadable, data is lost. D2 may have failed years ago without you being aware of it.

In principle it hasn't failed until you notice it :-)

...

...

and I am not aware of any enterprise class hardware that will do it.

*Every* enterprise class hardware performs periodical scrub across the full range of disk blocks to detect and fix latent errors. Most enterprise class hardware I am aware of performs permanent scrub and you cannot turn it off at all.

...

Usually SMART (and other vendor-specific diagnostics) tests are run regularly, that's about it.

We probably have different notion of "enterprise class hardware" ... :)

We use 3ware, HP and IBM. Yes, I know the latter two will scan for bad sectors during period of inactivity, but that's all. They don't go scrubbing down terabytes of data very often if at all. I don't know if the drive electronics does something like that?

The term of art is "disk scrubbing" and is exactly the same as "the latter two will scan for bad sectors during period of inactivity". I suspect the HP and IBM do it over the full drive platter surfaces over the course of time. With today's very large drives disk scrubbing is important Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Le 15/12/2016 à 00:43, Greg Freemyer a écrit :

The discussion got me curious, and yes the Drobo does do background data scrubbing.

"Drobo systems continuously scrub data in the background to fix disk errors before they happen. If there are too many errors, we proactively purge the drive and rebuilt using the available space on other drives. "

Also, I can hear mine going to town write now, but there is no client activity at the moment so it should be idle.

good to know :-) thanks jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

jdd wrote:

Le 14/12/2016 à 11:40, Per Jessen a écrit :

...

jdd wrote:

...

...

my main concern on raid is unused data control, that is verification of the integrity of unused data on disk (to prevent unexpected multi disk failure).

Huh? You mean check unused data

data

or unused space? I don't see what you

...

would gain and I am not aware of any enterprise class hardware that will do it. Usually SMART (and other vendor-specific diagnostics) tests are run regularly, that's about it. If you cannot rely on your data being written accurately to your disk, all bets are off.

it's certainly a problem. I read recently a discussion about it (but don't remember the exect name):

loook at this

http://sysadmin1138.net/mt/blog/2012/12/how-multi-disk-failures-happen.shtml

Yes, multi disk failures definitely do happen, the risk increases with the drive size. That's why we use DRBD mirror copies, hot spares and RAID6, staged drive purchases, and make sure each array has a good mix of different drive ages. -- Per Jessen, Zürich (2.6°C) http://www.cloudsuisse.com/ - your owncloud, hosted in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Greg Freemyer wrote:

On Wed, Dec 14, 2016 at 8:07 AM, Per Jessen wrote:

...

jdd wrote:

...

Le 14/12/2016 à 11:40, Per Jessen a écrit :

...

jdd wrote:

...

...

my main concern on raid is unused data control, that is verification of the integrity of unused data on disk (to prevent unexpected multi disk failure).

Huh? You mean check unused data

data

or unused space? I don't see what you

...

would gain and I am not aware of any enterprise class hardware that will do it. Usually SMART (and other vendor-specific diagnostics) tests are run regularly, that's about it. If you cannot rely on your data being written accurately to your disk, all bets are off.

it's certainly a problem. I read recently a discussion about it (but don't remember the exect name):

loook at this

http://sysadmin1138.net/mt/blog/2012/12/how-multi-disk-failures-happen.shtml

Yes, multi disk failures definitely do happen, the risk increases with the drive size. That's why we use DRBD mirror copies, hot spares and RAID6, staged drive purchases, and make sure each array has a good mix of different drive ages.

Now you just need to mix in some surveillance disks :)

Haha - given that they intended for 100% duty cycle, I do see some interesting use cases. Problems I see - apparently they don't support hot-swap, and the 3-year warranty. Pro: they're about 2/3 the price of the WD RE and they have more cache. -- Per Jessen, Zürich (3.2°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Wed, Dec 14, 2016 at 4:08 AM, jdd wrote:

Le 14/12/2016 à 09:52, Per Jessen a écrit :

...

Requirements do differ, but I would tend to agree. Running degraded is an open invitation for disaster, the longer the worse.

no worst than having no raid...

Much worse. A raid 0 with 6 drives is roughly 6 times as likely to fail than a single drive is because the whole thing fails if a single drive fails. A degraded raid 5 is the same. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Wed, Dec 14, 2016 at 3:39 AM, Andrei Borzenkov wrote:

On Wed, Dec 14, 2016 at 11:08 AM, Per Jessen wrote:

...

Andrei Borzenkov wrote:

...

On Wed, Dec 14, 2016 at 10:08 AM, Per Jessen wrote:

...

...

I ordered a second 10TB drive last week. I also pulled out one of the 1TB drives to leave a empty drive bay for the 10TB. It took about 12 hours to rebuild then.

Interesting that it rebuilds when a disk has been removed.

Huh? That's what I expect from any decent RAID implementation. Unless this disk was not in use, but that's not clear from the above.

When I remove a disk from a raid5 or raid6 array, it just goes into degraded mode. Rebuilding doesn't happen until a new drive is added.

Which is why hot spare disks exist - to have something to rebuild on as soon as failure is detected. I honestly expect that anyone who cares about data availability has one ... :)

There are also custom implementations that reserve space on disks and rebuild data from failed disk onto this reserve space. Or simply on free space, as long as there is enough. This company appears to offer such custom implementation: http://www.techrepublic.com/blog/the-enterprise-cloud/how-drobos-beyondraid-..., see also short summary of BeyondRaid implementation in wikipedia article.

So far I like the functionality, but dislike the lack of visibility into what it is really doing. Definitely still experimenting. I'll read those links. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Wed, Dec 14, 2016 at 11:56 AM, Carlos E. R. wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256

On 2016-12-14 08:54, Andrei Borzenkov wrote:

...

On Wed, Dec 14, 2016 at 10:08 AM, Per Jessen <> wrote:

...

...

If it has it, it ought to be invoked automatically I would say. It's first and foremost priority should be to reduce the time in degraded mode.

How do you expect it to be invoked for a disk that is not present anymore? You can only rebuild using redundant information on remaining drives. Such migration must happen while disk is still present.

There might be a feature to start migrating to a new disk while the old disk is still connected and in use. At some point, when migration is completed, it switches over and tells the admin so that he can remove the old disk.

Of course, but it cannot be invoked when disk is pulled out already, which is the context of this sub-discussion.

There might be another feature to sync a hot spare disk periodically so that when it has to enter service the distance is not so great, only recent changes.

It is impossible unless you use RAID1 and with RAID1 it degrades to "triple mirror" that requires you to dedicate "spare" disk for each mirror, which is usually unacceptable as you lose too much space. You cannot "sync spare" because you do not know which disk is going to fail as so do not know what content spare will have. Even with single RAID group, not to mention global hot spare for multiple different RAIDs.

Another feature still would be to create a snapshot of the current status on another (bigger) disk, which can then be removed for archival or backup.

Archival/backup is orthogonal to data availability offered by RAID. But yes, you can replicate data to another device (or RAID) and sometimes transparently switch to another device (which presumes synchronous replication, otherwise any automatic switch over is disastrous). Any such solution obviously doubles amount of needed space. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org