[opensuse] tuning tweak from factory
All, There is a discussion on factory currently that the mount option "relatime" can be used to accelerate application launch, etc. (note that relatime is NOT a typo, just a fairly new mount option.) And that the 2.6.30 kernel now uses it by default, so it should be pretty safe. I don't know if it is available in the standard 11.1 kernel, but looks like it is worth investigating. Someone feel like doing some tests? Thanks Greg -- Greg Freemyer Head of EDD Tape Extraction and Processing team Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf The Norcross Group The Intersection of Evidence & Technology http://www.norcrossgroup.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Mon, 2009-06-08 at 11:30 -0400, Greg Freemyer wrote:
All, There is a discussion on factory currently that the mount option "relatime" can be used to accelerate application launch, etc. (note that relatime is NOT a typo, just a fairly new mount option.) And that the 2.6.30 kernel now uses it by default, so it should be pretty safe. I don't know if it is available in the standard 11.1 kernel, but looks like it is worth investigating. Someone feel like doing some tests?
Yes, I believe it is. And it is documented in the mount man page (on 11.1): <quote> relatime Update inode access times relative to modify or change time. Access time is only updated if the previous access time was earlier than the current modify or change time. (Similar to noatime, but doesn't break mutt or other applications that need to know if a file has been read since the last time it was modified.) </quote> "relatime" is a compromise between rigid standards compliance and "noatime" (which, btw, implies the "nodiratime" option) which gives the best performance. *Theoretically* some application somewhere might actually use the atime and thus break with "noatime" set - I've never found one and none of the admin's I know have ever found one. I always run with "noatime" - but it would make a bad default for a distro (hence "relatime"). -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Adam Tauno Williams wrote:
On Mon, 2009-06-08 at 11:30 -0400, Greg Freemyer wrote:
All, There is a discussion on factory currently that the mount option "relatime" can be used to accelerate application launch, etc. (note that relatime is NOT a typo, just a fairly new mount option.) And that the 2.6.30 kernel now uses it by default, so it should be pretty safe. I don't know if it is available in the standard 11.1 kernel, but looks like it is worth investigating. Someone feel like doing some tests?
Yes, I believe it is. And it is documented in the mount man page (on 11.1):
<quote> relatime Update inode access times relative to modify or change time. Access time is only updated if the previous access time was earlier than the current modify or change time. (Similar to noatime, but doesn't break mutt or other applications that need to know if a file has been read since the last time it was modified.) </quote>
"relatime" is a compromise between rigid standards compliance and "noatime" (which, btw, implies the "nodiratime" option) which gives the best performance. *Theoretically* some application somewhere might actually use the atime and thus break with "noatime" set - I've never found one and none of the admin's I know have ever found one. I always run with "noatime" - but it would make a bad default for a distro (hence "relatime").
I have a heavily used cron job that runs every minute on all my production boxes that scans a special temp directory and deletes any file that has not been accessed, as in, read, in 2 minutes. It is used primarily by application software to "fire & forget" to produce a temp file in htdocs and then launch a browser on the users pc to view the file (or in cgi & php code to do similar), without having to either wait around for some sort of acknowledgement from the browser (which would have to be implimented by using a cgi or php to cat all such files instead of simply reading them directly.) or worse, waiting some arbitrary amount of time and then deleting the file. Both of those are unacceptable because it causes the application to hang while waiting to delete the file (Well, I could use a system() command and the "at" service after creating every temp file to cause the file to be deleted N minutes from now without waiting to do it myself. Um, no!), and because there is no such thing as any single amount of time that is a good amount. It will always be making users wait for no reason, and yet will also still always breaking some users download/views/passthru-print jobs. Instead the slick answer is a nice lightweight cron job with a single find -amin command. (I also have a way to do it on systems without gnu find or the -amin option, by touching a timestamp file every minute as part of the same cron job, and having find compare -anewer against the stamp file from 2 minutes ago) The application code just generates it's temp file and immediately fires a command to the users terminal emulator to launch a http url to the file, and proceeds without any pause and without ever thinking, knowing, or caring about that temp file again. The browser starts loading the file within a few seconds if not within one second, and may take as long as it needs to finish reading the file. Once it's been read once, it's in the browsers cache and doesn't matter if it gets deleted immediately after. The cron job runs every minute, and a variable in the script is set for 2 minutes, so the file isn't deleted until 2 minutes after the last _read_ access, which is the only sort of access there ever is other than creation and deletion. The cron job runs as root so it always has permission to delete regardless who or what created a given temp file. And, the cron job is not lazy or sloppy like some of my programmers and it never forgets or fails to clean up a temp file. Really I don't even have any especially good reason for letting these files live even 2 minutes except that it's handy for debugging sometimes, so I have debug options in a lot of scripts and apps that will temporarily dump a lot of stuff into the same directory. 2 minutes is enough time make use of the extra debug logs, and yet even on a very busy server it's short enough that the temp dir never has very many files in it even if I leave debug options on all day waiting for a rare or intermittent problem to happen. Depending on how apache works, possibly I could switch this to -mmin and count 2 or some minutes from creation time and still never have a problem, if apache always loads the full file into ram at the start, or if apache locks the file continuously until the client has it all, even on slow and/or intermittent connections. But, it's not merely apache that reads files from here. Sometimes it's cgi scripts or various processes that lead up to an eventual apache download and the intervening temp files are just as ephemeral as the final output to the browser. Sometimes apache isn't involved at all so it doesn't matter even if I can use -mmin without breaking slow http transactions. Or I could make this special temp directory a mount point on it's own filesystem so that it can have different mount options. Except I can't do that simply because it's impractical. The nature of this dir is very dynamic and I don't want to have to guess any particular size for it ahead of time. Because the files are short-lived, it's usually empty or only a few to a few hundred megs in it. But it's also completely possible to have a spike that goes into the gigs. So I want it to be a dir in the main filesystem or in the main data filesystem. This magic self-cleaning temp dir scheme is very heavily exploited by all our application, cgi, & php code by now, since I came up with it a decade ago. It's exceedingly convenient for programmers and more reliable than programmers at cleaning up. It's used by almost all forms of printing/faxing/emailing/viewing of documents generated by the application, and by scanning/viewing scanned documents in the app and in web services that view the scans and allow users to upload images manually in place of scanning. So I definitely need atime or at _least_ relatime on my main filesystem and for a pretty useful reason. -- bkw -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Mon, Jun 8, 2009 at 6:37 PM, Brian K. White
<snip>
So I definitely need atime or at _least_ relatime on my main filesystem and for a pretty useful reason.
-- bkw
Brian, Sounds like you need to be aware of the details on this one, and need to do some real testing prior to implementing 11.2 I'm sure there will be a mount option to preserve the current atime behaviour, but it appears at present that you will no longer get that behavior by default. I assume it will be as simple as editing fstab. Greg -- Greg Freemyer Head of EDD Tape Extraction and Processing team Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf The Norcross Group The Intersection of Evidence & Technology http://www.norcrossgroup.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
For some background on this change, read http://lwn.net/Articles/326471/ Andreas -- Andreas Jaeger, Director Platform / openSUSE, aj@suse.de SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) Maxfeldstr. 5, 90409 Nürnberg, Germany GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
On 08/06/09 11:30, Greg Freemyer wrote:
I don't know if it is available in the standard 11.1 kernel, but looks like it is worth investigating.
Of course you can use relatime in 11.1 kernels.. however, what you really want is to use "noatime", the only known application that breaks is "mutt" and that can be solved using the $check_mbox_size configuration option or simply using the Maildir format instead. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
2009/6/8 Greg Freemyer
All,
There is a discussion on factory currently that the mount option "relatime" can be used to accelerate application launch, etc. (note that relatime is NOT a typo, just a fairly new mount option.)
And that the 2.6.30 kernel now uses it by default, so it should be pretty safe.
I don't know if it is available in the standard 11.1 kernel, but looks like it is worth investigating.
Someone feel like doing some tests?
relatime is also in the man page for mount on 11.0. I have just made this change and rebooted, and my first impression is good. Firefox was taking forever to start up with lots of disk thrashing and now it starts up quietly in one or two seconds. I will leave it like this and post again if I notice any problems. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Mon, 2009-06-08 at 20:18 +0200, Michael Roberts wrote:
All, There is a discussion on factory currently that the mount option "relatime" can be used to accelerate application launch, etc. (note that relatime is NOT a typo, just a fairly new mount option.) And that the 2.6.30 kernel now uses it by default, so it should be pretty safe. I don't know if it is available in the standard 11.1 kernel, but looks like it is worth investigating. Someone feel like doing some tests? relatime is also in the man page for mount on 11.0. I have just made
2009/6/8 Greg Freemyer
: this change and rebooted, and my first impression is good. Firefox was taking forever to start up with lots of disk thrashing and now it starts up quietly in one or two seconds. I will leave it like this and post again if I notice any problems.
The original Linux commentary is here http://lwn.net/Articles/244830/, it is pretty emphatic. For something like a Cyrus mail store the difference between atime and noatime can be awesome. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (6)
-
Adam Tauno Williams
-
Andreas Jaeger
-
Brian K. White
-
Cristian Rodríguez
-
Greg Freemyer
-
Michael Roberts