On Mon, Jun 3, 2013 at 3:07 PM, Roger Oberholtzer <roger@opq.se> wrote:
On Mon, 2013-06-03 at 12:52 -0400, Greg Freemyer wrote:
On Mon, Jun 3, 2013 at 1:39 PM, Roger Oberholtzer <roger@opq.se> wrote:
On Mon, 2013-06-03 at 09:11 -0400, Greg Freemyer wrote:
O_DIRECT
From the open(2) man page:
"The thing that has always disturbed me about O_DIRECT is that the whole interface is just stupid, and was probably designed by a deranged monkey on some serious mind-controlling substances."--Linus
So of course I will have to try it!
I said it will have a major impact. I didn't say if it would be a good or bad impact!
In general for streaming i/o loads I think it is a good thing. dd as an example has an option to use O_DIRECT for the i/o. For a normal random i/o workload it is probably horrible.
OTOH, database tools will sometimes want to totally control how caching works, so they use O_DIRECT to get as close to the hard drive as they can.
I have three JPEG compression methods that I play with: (1) the standard turbo that comes with openSUSE (used in the systems with the issues described here), (2) a re-implementation of that based on the Intel Performance Primitives (IPP) (same API/ABI), (3) another one based on IPP that deals with the image as an object, via a different API.
#2 is 3-4 times faster than #1, and #3 is 6-8 times faster than #1 on our gray scale images of road surfaces.
All of these will of course provide data of different sizes as the compressed images vary in size. I do not relish the thought of maintaining fixed sized buffers from any of these to manage O_DIRECT disk I/O.
I think I will try a different file system, and then a different disk device in the computer.
I am rather surprised by all of this as each disk is written to by only one thread, and only one file at a time is written. There is not a great demand on writing. The disk access light turns on periodically, once every second or so. So it would seem that the data is being written rather often.
I wonder if there is some other system dynamic I am missing. There is ample memory, and all buffers are pre-allocated. There are 16 cores, and when I see the system load, 10 are 99% idle.
Seriously, post to the XFS list. They get end-user questions fairly often and are pretty friendly about helping out. I've been really surprised how much help the give people working with raid systems. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org