[opensuse] Re: SSD in openSUSE.
Greg Freemyer wrote:
On Wed, Nov 23, 2011 at 7:19 PM, Joachim Schrod <jschrod@acm.org> wrote:
Greg Freemyer wrote:
On Tue, Nov 22, 2011 at 7:44 PM, Joachim Schrod <jschrod@acm.org> wrote:
If you can afford it, don't use 15% of the disk. To get this free space, use secure erase to reset the disk's firmware notion of what you're using, and then don't allocate 15% of the disk during partitioning.
Undocumented feature time. First secure erase is not always implemented on SSDs, so don't trust it.
That's new for me, I haven't found this information via Google.
Can you give more information, maybe perhaps some links, about that issue? I've always thought I can rely on secure erase via hdparm, while respecting the frozen issue/state.
My knowledge is under NDA. I will just say that for a while a major manufacturer returned success for secure erase commands, but if you disassembled (desoldered) the chips and took a look, data was still present.
Yes, that I've read. But that was not my use case. My use case is "SSD firmware uses all sectors for wear levelling after secure erase". I don't care if the data is still on it. Does your NDA-knowledge include information about that use case? Please note that my posted steps include one step to unlock an SSD, so locked devices are covered. Joachim -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Joachim Schrod Email: jschrod@acm.org Roedermark, Germany -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, Nov 24, 2011 at 4:47 AM, Joachim Schrod <jschrod@acm.org> wrote:
Greg Freemyer wrote:
On Wed, Nov 23, 2011 at 7:19 PM, Joachim Schrod <jschrod@acm.org> wrote:
Greg Freemyer wrote:
On Tue, Nov 22, 2011 at 7:44 PM, Joachim Schrod <jschrod@acm.org> wrote:
If you can afford it, don't use 15% of the disk. To get this free space, use secure erase to reset the disk's firmware notion of what you're using, and then don't allocate 15% of the disk during partitioning.
Undocumented feature time. First secure erase is not always implemented on SSDs, so don't trust it.
That's new for me, I haven't found this information via Google.
Can you give more information, maybe perhaps some links, about that issue? I've always thought I can rely on secure erase via hdparm, while respecting the frozen issue/state.
My knowledge is under NDA. I will just say that for a while a major manufacturer returned success for secure erase commands, but if you disassembled (desoldered) the chips and took a look, data was still present.
Yes, that I've read. But that was not my use case. My use case is "SSD firmware uses all sectors for wear levelling after secure erase". I don't care if the data is still on it.
Does your NDA-knowledge include information about that use case?
Please note that my posted steps include one step to unlock an SSD, so locked devices are covered.
Joachim
My NDA knowledge doesn't cover that use case. We were worried about drive sanitation (getting rid of data.) But there are rumors of SSDs that immediately return a success upon receiving a Secure Erase command (ie. less than a second). That seems pretty hard to believe they were even doing anything with the command. ie. It takes several seconds to process a full set of trim commands even from hdparm (which is extremely efficient. See the wiper.sh script that comes with hdparm. It will invoke hdparm on a few different filesystems to trim them fully. fstrim is the new "supported" way, but it is no where near as efficient as how hdparm does it.) Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Greg Freemyer writes:
On Thu, Nov 24, 2011 at 4:47 AM, Joachim Schrod <jschrod@acm.org> wrote:
Yes, that I've read. But that was not my use case. My use case is "SSD firmware uses all sectors for wear levelling after secure erase". I don't care if the data is still on it.
Does your NDA-knowledge include information about that use case?
Please note that my posted steps include one step to unlock an SSD, so locked devices are covered.
Joachim
My NDA knowledge doesn't cover that use case. We were worried about drive sanitation (getting rid of data.)
But there are rumors of SSDs that immediately return a success upon receiving a Secure Erase command (ie. less than a second). That seems pretty hard to believe they were even doing anything with the command. ie. It takes several seconds to process a full set of trim commands even from hdparm (which is extremely efficient. See the wiper.sh script that comes with hdparm. It will invoke hdparm on a few different filesystems to trim them fully. fstrim is the new "supported" way, but it is no where near as efficient as how hdparm does it.)
Thanks a lot for your insightful information. I'll consider that. FWIW, the secure erase command (via hdparm) on my Crucial SSD needed some time to return. Btw, I have forwarded your comments concerning "leave free unpartitioned space" to some friends of mine who do storage designs for large providers for a living. AFAIK, they have access to internal information from disk manufacturers resp. storage subsystem vendors. If information is returned that I'm allowed to share, I'll forward it. Best, Joachim -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Fri, Nov 25, 2011 at 02:54, Joachim Schrod <jschrod@acm.org> wrote:
Btw, I have forwarded your comments concerning "leave free unpartitioned space" to some friends of mine who do storage designs for large providers for a living. AFAIK, they have access to internal information from disk manufacturers resp. storage subsystem vendors. If information is returned that I'm allowed to share, I'll forward it.
These are the guys I'd like to hear from :-) Of anyone, they are the ones with real world experience and hopefully access to more reliable info - vs... mmmm.. Wikipedia. If they can't share vendor specific info, maybe a sanitized or generalized comment? C. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Friday, November 25, 2011 07:14:50 AM C wrote:
On Fri, Nov 25, 2011 at 02:54, Joachim Schrod <jschrod@acm.org> wrote:
Btw, I have forwarded your comments concerning "leave free unpartitioned space" to some friends of mine who do storage designs for large providers for a living. AFAIK, they have access to internal information from disk manufacturers resp. storage subsystem vendors. If information is returned that I'm allowed to share, I'll forward it.
These are the guys I'd like to hear from :-) Of anyone, they are the ones with real world experience and hopefully access to more reliable info - vs... mmmm.. Wikipedia. If they can't share vendor specific info, maybe a sanitized or generalized comment?
C. PLus maybe we could poke them to update Wikipedia too. -- Roger Luedecke openSUSE Ambassador Ind. Repairs and Consulting **Looking for a C++ etc. mentor*** -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, 2011-11-24 at 23:04 -0800, Roger Luedecke wrote:
On Friday, November 25, 2011 07:14:50 AM C wrote:
On Fri, Nov 25, 2011 at 02:54, Joachim Schrod <jschrod@acm.org> wrote:
Btw, I have forwarded your comments concerning "leave free unpartitioned space" to some friends of mine who do storage designs for large providers for a living. AFAIK, they have access to internal information from disk manufacturers resp. storage subsystem vendors. If information is returned that I'm allowed to share, I'll forward it.
These are the guys I'd like to hear from :-) Of anyone, they are the ones with real world experience and hopefully access to more reliable info - vs... mmmm.. Wikipedia. If they can't share vendor specific info, maybe a sanitized or generalized comment?
C. PLus maybe we could poke them to update Wikipedia too.
I feel I should summarize the info in the responses to this question. Unfortunately, there were more questions asked than answers found. Perhaps we can wait and see what Joachim's buddies have to say, if it can be repeated? And then try to summarize from that? Yours sincerely, Roger Oberholtzer OPQ Systems / Ramböll RST Office: Int +46 10-615 60 20 Mobile: Int +46 70-815 1696 roger.oberholtzer@ramboll.se ________________________________________ Ramböll Sverige AB Krukmakargatan 21 P.O. Box 17009 SE-104 62 Stockholm, Sweden www.rambollrst.se -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Fri, Nov 25, 2011 at 7:39 AM, Roger Oberholtzer <roger@opq.se> wrote: <snip>
I feel I should summarize the info in the responses to this question. Unfortunately, there were more questions asked than answers found.
Please feel free to update either of the WIKI pages I posted at the beginning of this, or create a 3rd one. If you do create a 3rd, I'd appreciate it if you update the first 2 to have a reference to it. fyi: I've been surprised how many non-opensuse users have read those first 2 pages. There is not much good SSD sumarization info out there, so google has ranked those 2 pages pretty high for people googling for SSD / linux info. As a matter of fact a simple google search for "SSD trim" has one of the pages as the 7th hit.
Perhaps we can wait and see what Joachim's buddies have to say, if it can be repeated? And then try to summarize from that?
If you do the wiki approach, you can always update it as find out more.
Yours sincerely,
Roger Oberholtzer
Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Fri, Nov 25, 2011 at 13:39, Roger Oberholtzer <roger@opq.se> wrote:
I feel I should summarize the info in the responses to this question. Unfortunately, there were more questions asked than answers found.
Perhaps we can wait and see what Joachim's buddies have to say, if it can be repeated? And then try to summarize from that?
I've been trying to do more digging on this.. came across this on SlashDot today: http://hardware.slashdot.org/story/11/11/30/0344252/is-the-time-finally-righ... Lots of discussion about the lifetime of SSDs etc. Maybe something can be extracted/digested. Most people link back to one or two "authoritative" articles on SSDs and lifetime... but in the end it really seems to all be anecdotal, and no one really knows 100% for sure other than that so far lifetime seems to be a non-issue outside of the thrashing SSDs can get in a datacentre (ie it still seems that for home use, it's a case of don't worry about it, a current/new SSD has a longer life expectancy than your reg hard drive without any taking precautions for reducing write cycles). C. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, 2011-11-30 at 14:00 +0100, C wrote:
On Fri, Nov 25, 2011 at 13:39, Roger Oberholtzer <roger@opq.se> wrote: I've been trying to do more digging on this.. came across this on SlashDot today: http://hardware.slashdot.org/story/11/11/30/0344252/is-the-time-finally-righ... Lots of discussion about the lifetime of SSDs etc. Maybe something can be extracted/digested.
Extracting useful information from a Slashdot discussion? That's funny. Seriously.... I wouldn't bother trying.
Most people link back to one or two "authoritative" articles on SSDs and lifetime... but in the end it really seems to all be anecdotal, and no one really knows 100% for sure other than that so far lifetime seems to be a non-issue outside of the thrashing SSDs can get in a datacentre (ie it still seems that for home use, it's a case of don't worry about it
+1. Just use it and don't worry about it.
, a current/new SSD has a longer life expectancy than your reg hard drive without any taking precautions for reducing write cycles).
I'm not sure about "longer life expectancy than your reg hard drive"; but in any case it doesn't matter. The best thing to do is always: have a spare. It doesn't matter what it is; it your concerned enough about it [because, I presume, you depend on it] to crawl around the Internet to verify your configuration .... buy two [and at only twice the price! ]. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, Nov 30, 2011 at 9:43 AM, Adam Tauno Williams <awilliam@whitemice.org> wrote:
On Wed, 2011-11-30 at 14:00 +0100, C wrote:
On Fri, Nov 25, 2011 at 13:39, Roger Oberholtzer <roger@opq.se> wrote: I've been trying to do more digging on this.. came across this on SlashDot today: http://hardware.slashdot.org/story/11/11/30/0344252/is-the-time-finally-righ... Lots of discussion about the lifetime of SSDs etc. Maybe something can be extracted/digested.
Extracting useful information from a Slashdot discussion? That's funny. Seriously.... I wouldn't bother trying.
Most people link back to one or two "authoritative" articles on SSDs and lifetime... but in the end it really seems to all be anecdotal, and no one really knows 100% for sure other than that so far lifetime seems to be a non-issue outside of the thrashing SSDs can get in a datacentre (ie it still seems that for home use, it's a case of don't worry about it
+1. Just use it and don't worry about it.
, a current/new SSD has a longer life expectancy than your reg hard drive without any taking precautions for reducing write cycles).
I'm not sure about "longer life expectancy than your reg hard drive"; but in any case it doesn't matter. The best thing to do is always: have a spare. It doesn't matter what it is; it your concerned enough about it [because, I presume, you depend on it] to crawl around the Internet to verify your configuration .... buy two [and at only twice the price! ].
SSDs do NOT have a reputation for long life. Per word of mouth, they fail early and have EB (erase block) failures that causes partial data loss. (I have NOT seen studies to back that up, it's just anecdotal.) I suggest for the typical desktop/laptop user, the least of your concerns should be exceeding max. rated lifetime of writes. General lifetime is more important. So while they are extremely fast, I don't consider them as reliable as rotating disk at this point. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, Nov 30, 2011 at 15:51, Greg Freemyer <greg.freemyer@gmail.com> wrote:
SSDs do NOT have a reputation for long life. Per word of mouth, they fail early and have EB (erase block) failures that causes partial data loss.
(I have NOT seen studies to back that up, it's just anecdotal.)
Has anyone asked the question... what is failing when they die? Is it that they exhaust their write cycles early? or they die from other causes? I don't question that there are reports of the early demise of SSDs (there definitely are) , but I do question if they die from write exhaustion. Take this blog entry for example: http://www.codinghorror.com/blog/2011/05/the-hot-crazy-solid-state-drive-sca... where it recounts the horror of one user with multiple SSDs dying in well under one year of use. I really doubt he managed to hit the write cycle ceiling - which is what this discussion here has been primarily focused on - when any one of those drives died.
I suggest for the typical desktop/laptop user, the least of your concerns should be exceeding max. rated lifetime of writes. General lifetime is more important.
So while they are extremely fast, I don't consider them as reliable as rotating disk at this point.
Very valid point :-) C. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Adam Tauno Williams wrote:
I'm not sure about "longer life expectancy than your reg hard drive"; but in any case it doesn't matter. The best thing to do is always: have a spare. It doesn't matter what it is; it your concerned enough about it [because, I presume, you depend on it] to crawl around the Internet to verify your configuration
Absolutely agree with all of this.
.... buy two [and at only twice the price!
But not with this little bit. At least with regular disks - I can't speak for SSDs. But buying two together increases your chances of them sharing a common batch fault and both failing at very near the same time. I generally try to buy from different manufacturers, or at different times. Cheers, Dave -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, 2011-11-30 at 15:01 +0000, Dave Howorth wrote:
Adam Tauno Williams wrote:
I'm not sure about "longer life expectancy than your reg hard drive"; but in any case it doesn't matter. The best thing to do is always: have a spare. It doesn't matter what it is; it your concerned enough about it [because, I presume, you depend on it] to crawl around the Internet to verify your configuration Absolutely agree with all of this. .... buy two [and at only twice the price! But not with this little bit. At least with regular disks - I can't speak for SSDs. But buying two together increases your chances of them sharing a common batch fault and both failing at very near the same time. I generally try to buy from different manufacturers, or at different times.
False. They cannot fail at the "same time'; that isn't possible. You purchased the second as a *SPARE*. You don't use a *SPARE* until you need it [the first one fails]. Once you are using it then it isn't a spare. Maybe they will both fail after six months of use - but that is a year, not six months. And you have six months to buy your new spare after your first one failed. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, Nov 30, 2011 at 10:44 AM, Adam Tauno Williams <awilliam@whitemice.org> wrote:
On Wed, 2011-11-30 at 15:01 +0000, Dave Howorth wrote:
Adam Tauno Williams wrote:
I'm not sure about "longer life expectancy than your reg hard drive"; but in any case it doesn't matter. The best thing to do is always: have a spare. It doesn't matter what it is; it your concerned enough about it [because, I presume, you depend on it] to crawl around the Internet to verify your configuration Absolutely agree with all of this. .... buy two [and at only twice the price! But not with this little bit. At least with regular disks - I can't speak for SSDs. But buying two together increases your chances of them sharing a common batch fault and both failing at very near the same time. I generally try to buy from different manufacturers, or at different times.
False. They cannot fail at the "same time'; that isn't possible. You purchased the second as a *SPARE*. You don't use a *SPARE* until you need it [the first one fails]. Once you are using it then it isn't a spare. Maybe they will both fail after six months of use - but that is a year, not six months. And you have six months to buy your new spare after your first one failed.
Adam, A couple years ago I bought a bunch of drives at once (maybe 20?). As we were managing a large set of data we made 2 working copies and a backup copy on 3 different drives from that set. All 3 failed within 48 hours of each other. (don't ask me why). Fortunately it was a circuit board failure, so we swapped the electronics package from one of the drivew we hadn't used yet and were able to recover the data. The same also happened to people with raid arrays at that time. Suddenly the advice on the mdraid list was to mix and match manufacturers in an array to reduce the likelihood of simultaneous failure. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 11/30/2011 10:59 AM, Greg Freemyer wrote:
On Wed, Nov 30, 2011 at 10:44 AM, Adam Tauno Williams <awilliam@whitemice.org> wrote:
On Wed, 2011-11-30 at 15:01 +0000, Dave Howorth wrote:
Adam Tauno Williams wrote:
I'm not sure about "longer life expectancy than your reg hard drive"; but in any case it doesn't matter. The best thing to do is always: have a spare. It doesn't matter what it is; it your concerned enough about it [because, I presume, you depend on it] to crawl around the Internet to verify your configuration Absolutely agree with all of this. .... buy two [and at only twice the price! But not with this little bit. At least with regular disks - I can't speak for SSDs. But buying two together increases your chances of them sharing a common batch fault and both failing at very near the same time. I generally try to buy from different manufacturers, or at different times.
False. They cannot fail at the "same time'; that isn't possible. You purchased the second as a *SPARE*. You don't use a *SPARE* until you need it [the first one fails]. Once you are using it then it isn't a spare. Maybe they will both fail after six months of use - but that is a year, not six months. And you have six months to buy your new spare after your first one failed.
Adam,
A couple years ago I bought a bunch of drives at once (maybe 20?).
As we were managing a large set of data we made 2 working copies and a backup copy on 3 different drives from that set.
All 3 failed within 48 hours of each other. (don't ask me why).
Fortunately it was a circuit board failure, so we swapped the electronics package from one of the drivew we hadn't used yet and were able to recover the data.
The same also happened to people with raid arrays at that time. Suddenly the advice on the mdraid list was to mix and match manufacturers in an array to reduce the likelihood of simultaneous failure.
Greg
A little miscommunication here I think. You actually just made his point not countered it. You had unused spare drives on the shelf that had their full (however short) life ahead of them and were usable starting at the time the active drives all died. He didn't dispute that all drives that start being used at the same time may die at the same time. He didn't specify "one is a cold spare" when he said "buy two" but that was the implication from context at least by the time of the above post if not the first post. -- bkw -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, 2011-11-30 at 10:59 -0500, Greg Freemyer wrote:
On Wed, Nov 30, 2011 at 10:44 AM, Adam Tauno Williams <awilliam@whitemice.org> wrote:
On Wed, 2011-11-30 at 15:01 +0000, Dave Howorth wrote:
Adam Tauno Williams wrote: As we were managing a large set of data we made 2 working copies and a backup copy on 3 different drives from that set. All 3 failed within 48 hours of each other. (don't ask me why).
I've had the same experience - but my point is: those aren't "spares". If you use them they aren't "spares".
The same also happened to people with raid arrays at that time. Suddenly the advice on the mdraid list was to mix and match manufacturers in an array to reduce the likelihood of simultaneous failure.
That has long been the advice for building RAID arrays. At least purchase drives from different suppliers [and hope they don't drop ship them from the same place]. But manufacturer defect isn't really the same as failure-due-to-usage as is being discussed here in relation to SSDs. A reasonable burn-in of all parts, including spares, is a good idea if you really don't want to end up in a corner. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, 2011-11-30 at 09:43 -0500, Adam Tauno Williams wrote:
+1. Just use it and don't worry about it.
, a current/new SSD has a longer life expectancy than your reg hard drive without any taking precautions for reducing write cycles).
I'm not sure about "longer life expectancy than your reg hard drive"; but in any case it doesn't matter. The best thing to do is always: have a spare. It doesn't matter what it is; it your concerned enough about it [because, I presume, you depend on it] to crawl around the Internet to verify your configuration .... buy two [and at only twice the price! ].
Just use it wisely, _then_ don't worry about it. As tou can read up the thread, i re-installed a 11.3 machine on a 60GB ssd. Booted fast as lightning and worked like a dream. however i forgot to disable swap. After about a year, i tried to upgrade to 11.4 and it started to behave strange. lots and lots and lots of errors on SDA. So because of the countless writings on the SDD, it is a brick right now. Re-formatting still results in errors. So no fairytale or so, but first hand experience. I do still recommend an sdd though ( i replaced mine) But only for the area's that should be considered (mostly) readonly, like /, /boot, /usr /lib /sbin ...., while write intensive area's are on normal disk (swap, /tmp, /data /var/tmp/ /var/spool /var/log etc etc) Oh, and btw, what i heard yesterday at a suse product meeting, it _seems_ that btrfs handels SDD's better as it seems to be capable of distinguishing between traditional hdd en Solid State hdd. hw -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, 2011-11-30 at 17:16 +0100, Hans Witvliet wrote:
Oh, and btw, what i heard yesterday at a suse product meeting, it _seems_ that btrfs handels SDD's better as it seems to be capable of distinguishing between traditional hdd en Solid State hdd.
Or btrfs can be explicitly mounted with the "ssd" option to enable SSD-friendly behavior; if you are worried the autodetection won't kick and and you know you are using an SSD. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, Nov 24, 2011 at 8:54 PM, Joachim Schrod <jschrod@acm.org> wrote:
Btw, I have forwarded your comments concerning "leave free unpartitioned space" to some friends of mine who do storage designs for large providers for a living. AFAIK, they have access to internal information from disk manufacturers resp. storage subsystem vendors. If information is returned that I'm allowed to share, I'll forward it.
FYI: that wasn't my suggestion. I have no knowledge of if that is a good or bad idea. My post was based solely on "If you are going to do that ..." Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (9)
-
Adam Tauno Williams -
Brian K. White -
C -
Dave Howorth -
Greg Freemyer -
Hans Witvliet -
Joachim Schrod -
Roger Luedecke -
Roger Oberholtzer