Mailinglist Archive: opensuse (1473 mails)

< Previous Next >
Re: [opensuse] tuning tweak from factory
  • From: "Brian K. White" <brian@xxxxxxxxx>
  • Date: Mon, 08 Jun 2009 18:37:47 -0400
  • Message-id: <4A2D92BB.8020908@xxxxxxxxx>


Adam Tauno Williams wrote:
On Mon, 2009-06-08 at 11:30 -0400, Greg Freemyer wrote:
All,
There is a discussion on factory currently that the mount option
"relatime" can be used to accelerate application launch, etc. (note
that relatime is NOT a typo, just a fairly new mount option.)
And that the 2.6.30 kernel now uses it by default, so it should be pretty safe.
I don't know if it is available in the standard 11.1 kernel, but looks
like it is worth investigating.
Someone feel like doing some tests?

Yes, I believe it is. And it is documented in the mount man page (on
11.1):

<quote>
relatime
Update inode access times relative to modify or change time. Access
time is only updated if the previous access time was earlier than the
current modify or change time. (Similar to noatime, but doesn't
break mutt or other applications that need to know if a file has been
read since the last time it was modified.)
</quote>

"relatime" is a compromise between rigid standards compliance and
"noatime" (which, btw, implies the "nodiratime" option) which gives the
best performance. *Theoretically* some application somewhere might
actually use the atime and thus break with "noatime" set - I've never
found one and none of the admin's I know have ever found one. I always
run with "noatime" - but it would make a bad default for a distro (hence
"relatime").

I have a heavily used cron job that runs every minute on all my production boxes that scans a special temp directory and deletes any file that has not been accessed, as in, read, in 2 minutes.

It is used primarily by application software to "fire & forget" to produce a temp file in htdocs and then launch a browser on the users pc to view the file (or in cgi & php code to do similar), without having to either wait around for some sort of acknowledgement from the browser (which would have to be implimented by using a cgi or php to cat all such files instead of simply reading them directly.) or worse, waiting some arbitrary amount of time and then deleting the file. Both of those are unacceptable because it causes the application to hang while waiting to delete the file (Well, I could use a system() command and the "at" service after creating every temp file to cause the file to be deleted N minutes from now without waiting to do it myself. Um, no!), and because there is no such thing as any single amount of time that is a good amount. It will always be making users wait for no reason, and yet will also still always breaking some users download/views/passthru-print jobs.

Instead the slick answer is a nice lightweight cron job with a single find -amin command. (I also have a way to do it on systems without gnu find or the -amin option, by touching a timestamp file every minute as part of the same cron job, and having find compare -anewer against the stamp file from 2 minutes ago)
The application code just generates it's temp file and immediately fires a command to the users terminal emulator to launch a http url to the file, and proceeds without any pause and without ever thinking, knowing, or caring about that temp file again.

The browser starts loading the file within a few seconds if not within one second, and may take as long as it needs to finish reading the file. Once it's been read once, it's in the browsers cache and doesn't matter if it gets deleted immediately after.
The cron job runs every minute, and a variable in the script is set for 2 minutes, so the file isn't deleted until 2 minutes after the last _read_ access, which is the only sort of access there ever is other than creation and deletion. The cron job runs as root so it always has permission to delete regardless who or what created a given temp file. And, the cron job is not lazy or sloppy like some of my programmers and it never forgets or fails to clean up a temp file.
Really I don't even have any especially good reason for letting these files live even 2 minutes except that it's handy for debugging sometimes, so I have debug options in a lot of scripts and apps that will temporarily dump a lot of stuff into the same directory. 2 minutes is enough time make use of the extra debug logs, and yet even on a very busy server it's short enough that the temp dir never has very many files in it even if I leave debug options on all day waiting for a rare or intermittent problem to happen.

Depending on how apache works, possibly I could switch this to -mmin and count 2 or some minutes from creation time and still never have a problem, if apache always loads the full file into ram at the start, or if apache locks the file continuously until the client has it all, even on slow and/or intermittent connections. But, it's not merely apache that reads files from here. Sometimes it's cgi scripts or various processes that lead up to an eventual apache download and the intervening temp files are just as ephemeral as the final output to the browser. Sometimes apache isn't involved at all so it doesn't matter even if I can use -mmin without breaking slow http transactions.

Or I could make this special temp directory a mount point on it's own filesystem so that it can have different mount options. Except I can't do that simply because it's impractical. The nature of this dir is very dynamic and I don't want to have to guess any particular size for it ahead of time. Because the files are short-lived, it's usually empty or only a few to a few hundred megs in it. But it's also completely possible to have a spike that goes into the gigs. So I want it to be a dir in the main filesystem or in the main data filesystem.

This magic self-cleaning temp dir scheme is very heavily exploited by all our application, cgi, & php code by now, since I came up with it a decade ago.
It's exceedingly convenient for programmers and more reliable than programmers at cleaning up.
It's used by almost all forms of printing/faxing/emailing/viewing of documents generated by the application, and by scanning/viewing scanned documents in the app and in web services that view the scans and allow users to upload images manually in place of scanning.

So I definitely need atime or at _least_ relatime on my main filesystem and for a pretty useful reason.

--
bkw
--
To unsubscribe, e-mail: opensuse+unsubscribe@xxxxxxxxxxxx
For additional commands, e-mail: opensuse+help@xxxxxxxxxxxx

< Previous Next >
Follow Ups