[opensuse-kernel] SSD / filesystem / TRIM

Thomas Hertweck

25 Aug 2010 25 Aug '10

22:24

Hi! I asked a question on the opensuse-de mailing list and Philipp Thomas gave me a tip to better try the opensuse-kernel mailing list, so here we go: What filesystem should be used for partitions on an SSD? Or does the filesystem not really matter? Are there any special mount options that should be used? Does Linux nowadays fully support TRIM? There is of course some information about these topics on the internet or in the opensuse mailing list archives, but quite a lot of information seems to be outdated or even inconsistent: Some people say the Linux kernel supports TRIM but it's not fully implemented according to the ATA-8 standard. Some people say /tmp or /var should not be put on an SSD because of frequent writes. Some people even say they had problems with SSDs under Linux in general and faced permanent timeout problems. Some people recommend the use of special scripts (TRIM related) that seem to be part of the hdparm package, others say they are no longer required since kernel 2.6.33. I am not an expert on these issues, therefore I (and I am sure others as well) would appreciate some input from the experts regarding these questions. Thanks in advance for shedding some light on the issues. Regards, Thomas -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org

Show replies by date

Greg Freemyer

25 Aug 25 Aug

23:29

On Wed, Aug 25, 2010 at 6:24 PM, Thomas Hertweck wrote:

...

Hi!

I asked a question on the opensuse-de mailing list and Philipp Thomas gave me a tip to better try the opensuse-kernel mailing list, so here we go:

I have started a reference page on the wiki. It needs further elaboration, but it's worth a read: http://en.opensuse.org/SDB:SSD_discard_%28trim%29_support Feel free to add some of the below to the page.

...

What filesystem should be used for partitions on an SSD? Or does the filesystem not really matter? Are there any special mount options that should be used? Does Linux nowadays fully support TRIM?

ext4 and xfs have internal discard support, BUT it is highly non-optimized and as such is not a significant performance benefit. The major issue is the trim commands require the command queue on the SSD be flushed. So interspersing trims with normal i/o is not a great solution. fyi: Windows also intersperses the commands, so it to does not greatly benefit. In my opinion, a better solution for now is to use the wiper.sh script described in the wiki page. wiper.sh depends on fallocate() being supported by the filesystem. Many do including ext3, ext4, xfs. Not sure which others. The big problem with wiper.sh is it has no support for lvm / dm / md setups. And worse I don't think it detects the problem, so if used with partitions on those setups it could cause data loss!

...

There is of course some information about these topics on the internet or in the opensuse mailing list archives, but quite a lot of information seems to be outdated or even inconsistent: Some people say the Linux kernel supports TRIM but it's not fully implemented according to the ATA-8 standard.

True. The kernel at present only provides one contiguous trim range per trim command. And it maxes out at 4GB per range iirc. The spec allows a vectorized list of non-contguous ranges to be passed in. The wiper.sh script does so, but it uses hdparm to bypass most of the block stack.

...

Some people say /tmp or /var should not be put on an SSD because of frequent writes.

That's a historical comment aiui.

...

Some people even say they had problems with SSDs under Linux in general and faced permanent timeout problems.

Don't know about that.

...

Some people recommend the use of special scripts (TRIM related) that seem to be part of the hdparm package, others say they are no longer required since kernel 2.6.33.

With the exception of Intel SSDs I would use the script unless you are using lvm / dm / md. The script is in oS 11.2 and 11.3 It is known to have an issue with Intel SSDs, but it does not cause data loss. The issue is that the script creates payloads of range vectors up to 1MB. The Intel chip only support 512B or range payload per trim. I am not aware of any SDD that is able to handle the kernel implementation more effectively than the script. fyi: oS 11.3 has hdparm v9.28 I just checked and v9.30 was released a couple weeks ago. Possibly it has better payload management. If it was my SSD, I would get the latest hdparm and wiper.sh for sure. I don't know if they're in OBS or not. The source is at: http://sourceforge.net/projects/hdparm/files/

...

I am not an expert on these issues, therefore I (and I am sure others as well) would appreciate some input from the experts regarding these questions. Thanks in advance for shedding some light on the issues.

Regards, Thomas

Greg -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org

Thomas Hertweck

29 Aug 29 Aug

09:32

Hi Greg, Thanks for the information. On 26/08/10 00:29, Greg Freemyer wrote:

...

[...] I have started a reference page on the wiki. It needs further elaboration, but it's worth a read:

http://en.opensuse.org/SDB:SSD_discard_%28trim%29_support

I've read it, thanks for starting this page. I get the impression though, that using an SSD for a non-experienced Linux user is far too complicated at the moment. Given that the use of SSDs increases rapidly, I think something needs to be done to make such hardware easier to use - I think at the end of the day users don't want to think about alignments of filesystems, special mount options, setting up cron jobs etc to use an SSD.

...

[...] ext4 and xfs have internal discard support, BUT it is highly non-optimized and as such is not a significant performance benefit. The major issue is the trim commands require the command queue on the SSD be flushed. So interspersing trims with normal i/o is not a great solution.

fyi: Windows also intersperses the commands, so it to does not greatly benefit.

In my opinion, a better solution for now is to use the wiper.sh script described in the wiki page. wiper.sh depends on fallocate() being supported by the filesystem. Many do including ext3, ext4, xfs. Not sure which others.

The big problem with wiper.sh is it has no support for lvm / dm / md setups. And worse I don't think it detects the problem, so if used with partitions on those setups it could cause data loss!

I have now installed openSUSE 11.3 on a OCZ Vertex 2E SSD. I used a standard ext4 filesystem, the mount options at the moment are "rw,noatime,acl,user_xattr". Unfortunately, wiper.sh doesn't work at all. $> wiper.sh --verbose --commit /dev/sdb1 wiper.sh: Linux SATA SSD TRIM utility, version 2.6, by Mark Lord. wiper.sh: This tool is DANGEROUS! Please read and understand wiper.sh: /usr/share/doc/packages/hdparm/README.wiper wiper.sh: before going any further. rootdev=/dev/sdb1 fsmode2: fsmode=read-write /: fstype=ext4 freesize = 45338732 KB, reserved = 453387 KB Preparing for online TRIM of free space on /dev/sdb1 (ext4 mounted read-write at /). This operation could silently destroy your data. Are you sure (y/N)? y Creating temporary file (44885345 KB).. Syncing disks.. Beginning TRIM operations.. get_trimlist=/sbin/hdparm --fibmap WIPER_TMPFILE.13467 /dev/sdb: trimming 89770696 sectors from 1493 ranges FAILED: Input/output error Removing temporary file.. Syncing disks.. Aborted. No joy, obviously. So what can I do? Should I try the "discard" mount option? Or would that really hurt performance? The motherboard, by the way, is an Asus P6X58D-E, the SSD is attached to one of the 3GB/s SATA ports (Intel chip). There's also a 6GB/s SATA port (Marvell chip) on the board, but I couldn't find Linux support for that (that's why I haven't used this port).

...

[...]

With the exception of Intel SSDs I would use the script unless you are using lvm / dm / md. The script is in oS 11.2 and 11.3

Well,... See above. Regards, Thomas -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org

Greg KH

18:41

On Sun, Aug 29, 2010 at 10:32:57AM +0100, Thomas Hertweck wrote:

...

Hi Greg,

Thanks for the information.

On 26/08/10 00:29, Greg Freemyer wrote:

...
[...] I have started a reference page on the wiki. It needs further elaboration, but it's worth a read:

http://en.opensuse.org/SDB:SSD_discard_%28trim%29_support

I've read it, thanks for starting this page. I get the impression though, that using an SSD for a non-experienced Linux user is far too complicated at the moment.

Not at all. Just plug it in and use it like a "normal" disk and all will work just fine, that's what I do. Nothing special at all and it works _way_ faster than a "normal" disk. thanks, greg k-h -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org

Takashi Iwai

30 Aug 30 Aug

06:29

At Sun, 29 Aug 2010 11:41:17 -0700, Greg KH wrote:

...

On Sun, Aug 29, 2010 at 10:32:57AM +0100, Thomas Hertweck wrote:

...
Hi Greg,

Thanks for the information.

On 26/08/10 00:29, Greg Freemyer wrote:

...
[...] I have started a reference page on the wiki. It needs further elaboration, but it's worth a read:

http://en.opensuse.org/SDB:SSD_discard_%28trim%29_support

I've read it, thanks for starting this page. I get the impression though, that using an SSD for a non-experienced Linux user is far too complicated at the moment.

Not at all. Just plug it in and use it like a "normal" disk and all will work just fine, that's what I do. Nothing special at all and it works _way_ faster than a "normal" disk.

Yes, there seem some misunderstanding. Normal users don't need anything. SSD works just as is, and it's fast enough. If you are adventurous, you can go quest for the performance optimization with TRIM support. But normal people don't need it at this moment. So, arguing about the usability for normal users doesn't make sense for now. Takashi -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org

Thomas Hertweck

09:26

On 30/08/10 07:29, Takashi Iwai wrote:

...

At Sun, 29 Aug 2010 11:41:17 -0700, Greg KH wrote:

...
[...] Not at all. Just plug it in and use it like a "normal" disk and all will work just fine, that's what I do. Nothing special at all and it works _way_ faster than a "normal" disk.

Yes, there seem some misunderstanding. Normal users don't need anything. SSD works just as is, and it's fast enough.

If you are adventurous, you can go quest for the performance optimization with TRIM support. But normal people don't need it at this moment. So, arguing about the usability for normal users doesn't make sense for now.

Thanks for the reassurance. Everybody on the internet seems to be talking about TRIM etc when it comes to SSD, that's why it looks to be so important (even the openSUSE wiki focuses on TRIM). Without having the proper technical background in ATA, filesystem, and SSD technology, it's difficult to know who's right and who's wrong. Anyway, the SSD works fine for me at the moment and indeed a lot faster than a normal HDD. We are currently running some tests to see whether we could use SSD technology for our large Linux clusters where we need fast scratch disks on each cluster node. Regards, Thomas -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org

C

09:39

On Mon, Aug 30, 2010 at 11:26, Thomas Hertweck wrote:

...

On 30/08/10 07:29, Takashi Iwai wrote:

...
At Sun, 29 Aug 2010 11:41:17 -0700, Greg KH wrote:

...
[...] Not at all. Just plug it in and use it like a "normal" disk and all will work just fine, that's what I do. Nothing special at all and it works _way_ faster than a "normal" disk.

Yes, there seem some misunderstanding. Normal users don't need anything. SSD works just as is, and it's fast enough.

If you are adventurous, you can go quest for the performance optimization with TRIM support. But normal people don't need it at this moment. So, arguing about the usability for normal users doesn't make sense for now.

Thanks for the reassurance. Everybody on the internet seems to be talking about TRIM etc when it comes to SSD, that's why it looks to be so important (even the openSUSE wiki focuses on TRIM). Without having the proper technical background in ATA, filesystem, and SSD technology, it's difficult to know who's right and who's wrong. Anyway, the SSD works fine

You're not the only one struggling with this... there's two camps... those that say don't worry, it just works and you see major speed increase, and those who say no no no, trim is crucial, and there are no significant performance differences anyway. Two polar opposite camps with convincing arguments on both sides. I'm interested in your test results. I really would like to try it out too... I'm considering installing root and /home on an SSD and use regular drives for long term storage for things like static media files and documents that are not accessed every day. C. -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org

Greg Freemyer

19:04

On Mon, Aug 30, 2010 at 5:39 AM, C wrote:

...

On Mon, Aug 30, 2010 at 11:26, Thomas Hertweck wrote:

...
On 30/08/10 07:29, Takashi Iwai wrote:

...
At Sun, 29 Aug 2010 11:41:17 -0700, Greg KH wrote:

...
[...] Not at all. Just plug it in and use it like a "normal" disk and all will work just fine, that's what I do. Nothing special at all and it works _way_ faster than a "normal" disk.

Yes, there seem some misunderstanding. Normal users don't need anything. SSD works just as is, and it's fast enough.

If you are adventurous, you can go quest for the performance optimization with TRIM support. But normal people don't need it at this moment. So, arguing about the usability for normal users doesn't make sense for now.

Thanks for the reassurance. Everybody on the internet seems to be talking about TRIM etc when it comes to SSD, that's why it looks to be so important (even the openSUSE wiki focuses on TRIM). Without having the proper technical background in ATA, filesystem, and SSD technology, it's difficult to know who's right and who's wrong. Anyway, the SSD works fine

You're not the only one struggling with this... there's two camps... those that say don't worry, it just works and you see major speed increase, and those who say no no no, trim is crucial, and there are no significant performance differences anyway. Two polar opposite camps with convincing arguments on both sides.

I'm interested in your test results. I really would like to try it out too... I'm considering installing root and /home on an SSD and use regular drives for long term storage for things like static media files and documents that are not accessed every day.

C.

Getting trim support working with SSDs is like giving a race car a final pre-race tune-up. Its a smart thing to do and worth the effort, but a race car is fast even if not perfectly in tune. In my opinion the Windows kernel and Linux kernel trim implementations both offer only marginal performance improvement over standard ATA-7 like SSD performance because they don't leverage the fact a TRIM command can accept multiple trim ranges in the payload. (Red Hat has done benchmarks in which they don't show much advantage from the ext4 --discard option at all. They even submitted a patch to remove the option. The patch was rejected because hardware may eventually exist that works well with the current implementation. It caused a big thread on the ext4 mailing list.) Rad Hat now has proposed a new fitrim() ioctl, but its not performance is not too much better than whats in the 2.6.35 (and older) kernels. The block layer needs to grow a new API in my opinion to handle a discard range vector, not just discrete discard ranges. Who knows maybe 2.6.37 will get that.) For now in Linux, only the hdparm / wiper.sh script even attempts to do that. I saw Thomas failed attempt, so apparently its not fully working yet either but at least it tries. Greg -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org

Greg Freemyer

18:41

On Mon, Aug 30, 2010 at 5:26 AM, Thomas Hertweck wrote:

...

On 30/08/10 07:29, Takashi Iwai wrote:

...
At Sun, 29 Aug 2010 11:41:17 -0700, Greg KH wrote:

...
[...] Not at all. Just plug it in and use it like a "normal" disk and all will work just fine, that's what I do. Nothing special at all and it works _way_ faster than a "normal" disk.

Yes, there seem some misunderstanding. Normal users don't need anything. SSD works just as is, and it's fast enough.

If you are adventurous, you can go quest for the performance optimization with TRIM support. But normal people don't need it at this moment. So, arguing about the usability for normal users doesn't make sense for now.

Thanks for the reassurance. Everybody on the internet seems to be talking about TRIM etc when it comes to SSD, that's why it looks to be so important (even the openSUSE wiki focuses on TRIM). Without having the proper technical background in ATA, filesystem, and SSD technology, it's difficult to know who's right and who's wrong. Anyway, the SSD works fine for me at the moment and indeed a lot faster than a normal HDD. We are currently running some tests to see whether we could use SSD technology for our large Linux clusters where we need fast scratch disks on each cluster node.

Regards, Thomas

Thomas, RE: http://en.opensuse.org/SDB:SSD_discard_%28trim%29_support I've been working on the SSD discard page. In particular, I added an intro to the wiki ssd article to this effect: === This article discusses a specific SSD feature and what some may consider a shortcoming. This "shortcoming" in no way means that SSDs are not a good solution with openSUSE today. Basic SSD operation is supported in all released versions of openSUSE and provides full ATA-7 support and significant performance improvements over rotating disks are seen for most SSDs with most workloads. It is only the new ATA-8 TRIM feature that is not yet fully implemented nor optimized. Not having this single feature fully integrated into openSUSE (or the linux kernel in general) is no reason not to benefit from the significant performance gain available in modern SSDs. === Hopefully that makes the situation more clear. Greg -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org

Bruno Friedmann

18:55

On 08/30/2010 08:41 PM, Greg Freemyer wrote:

...

On Mon, Aug 30, 2010 at 5:26 AM, Thomas Hertweck wrote:

...
On 30/08/10 07:29, Takashi Iwai wrote:

...
At Sun, 29 Aug 2010 11:41:17 -0700, Greg KH wrote:

...
[...] Not at all. Just plug it in and use it like a "normal" disk and all will work just fine, that's what I do. Nothing special at all and it works _way_ faster than a "normal" disk.

Yes, there seem some misunderstanding. Normal users don't need anything. SSD works just as is, and it's fast enough.

If you are adventurous, you can go quest for the performance optimization with TRIM support. But normal people don't need it at this moment. So, arguing about the usability for normal users doesn't make sense for now.

Thanks for the reassurance. Everybody on the internet seems to be talking about TRIM etc when it comes to SSD, that's why it looks to be so important (even the openSUSE wiki focuses on TRIM). Without having the proper technical background in ATA, filesystem, and SSD technology, it's difficult to know who's right and who's wrong. Anyway, the SSD works fine for me at the moment and indeed a lot faster than a normal HDD. We are currently running some tests to see whether we could use SSD technology for our large Linux clusters where we need fast scratch disks on each cluster node.

Regards, Thomas

Thomas,

RE: http://en.opensuse.org/SDB:SSD_discard_%28trim%29_support

I've been working on the SSD discard page. In particular, I added an intro to the wiki ssd article to this effect:

=== This article discusses a specific SSD feature and what some may consider a shortcoming. This "shortcoming" in no way means that SSDs are not a good solution with openSUSE today. Basic SSD operation is supported in all released versions of openSUSE and provides full ATA-7 support and significant performance improvements over rotating disks are seen for most SSDs with most workloads. It is only the new ATA-8 TRIM feature that is not yet fully implemented nor optimized. Not having this single feature fully integrated into openSUSE (or the linux kernel in general) is no reason not to benefit from the significant performance gain available in modern SSDs. ===

Hopefully that makes the situation more clear.

Greg

Very good Greg, with this introduction, I can better understand, and now I'm back in trouble in which ssd to choose. Last week, the choice for a simple new mechanical harddrive was simple :-) -- Bruno Friedmann bruno@ioda-net.ch Ioda-Net Sàrl www.ioda-net.ch openSUSE Member User www.ioda.net/r/osu Blog www.ioda.net/r/blog fsfe fellowship www.fsfe.org (bruno.friedmann (at) fsfe.org ) tigerfoot on irc GPG KEY : D5C9B751C4653227 -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org

Greg Freemyer

19:14

(Putting Mark Lord and linux-ide in cc:) Mark, Thomas has tried to use wiper.sh / hdparm v9.28 to trim a SSD and it failed. See below: This is with the openSUSE 2.6.34 kernel. On Sun, Aug 29, 2010 at 5:32 AM, Thomas Hertweck wrote: <snip>

...

I have now installed openSUSE 11.3 on a OCZ Vertex 2E SSD. I used a standard ext4 filesystem, the mount options at the moment are "rw,noatime,acl,user_xattr". Unfortunately, wiper.sh doesn't work at all.

$> wiper.sh --verbose --commit /dev/sdb1 wiper.sh: Linux SATA SSD TRIM utility, version 2.6, by Mark Lord. wiper.sh: This tool is DANGEROUS! Please read and understand wiper.sh: /usr/share/doc/packages/hdparm/README.wiper wiper.sh: before going any further. rootdev=/dev/sdb1 fsmode2: fsmode=read-write /: fstype=ext4 freesize = 45338732 KB, reserved = 453387 KB Preparing for online TRIM of free space on /dev/sdb1 (ext4 mounted read-write at /). This operation could silently destroy your data. Are you sure (y/N)? y Creating temporary file (44885345 KB).. Syncing disks.. Beginning TRIM operations.. get_trimlist=/sbin/hdparm --fibmap WIPER_TMPFILE.13467 /dev/sdb: trimming 89770696 sectors from 1493 ranges FAILED: Input/output error Removing temporary file.. Syncing disks.. Aborted.

No joy, obviously. So what can I do? <snip>

...

The motherboard, by the way, is an Asus P6X58D-E, the SSD is attached to one of the 3GB/s SATA ports (Intel chip). There's also a 6GB/s SATA port (Marvell chip) on the board, but I couldn't find Linux support for that (that's why I haven't used this port).

Well,... See above.

Regards, Thomas

Mark, any thoughts why it didn't work. Have you added anything relevant to hdparm since v.9.28 that would help? Greg -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org

4993

Age (days ago)

4998

Last active (days ago)

List overview

Download

10 comments

6 participants

participants (6)

Bruno Friedmann
C
Greg Freemyer
Greg KH
Takashi Iwai
Thomas Hertweck

[opensuse-kernel] SSD / filesystem / TRIM

C

tags

participants (6)