Mailinglist Archive: opensuse (1239 mails)

< Previous Next >
Re: [opensuse] Re: XFS and openSUSE 12.1
  • From: Roger Oberholtzer <roger@xxxxxx>
  • Date: Tue, 04 Jun 2013 23:21:04 +0200
  • Message-id: <1370380864.19897.6.camel@localhost>
On Tue, 2013-06-04 at 11:26 -0700, Linda Walsh wrote:
It sounds more like things build up in memory for 30 minutes,
then something is flushing and/or having to work hard for 5 seconds --
like you've written to memory buffers for 30 minutes then some routine calls
to empty buffers because it is out of space.

Any buffers are filled up very quickly when you write 25 MB/Sec. So
something happening after 30 minutes is probably not related to general
buffering.


This is where I think you'll benet most by using O_DIRECT in your
open call and either a 2nd thread to handle I/O, or async I/O.

You shouldn't have a 5 second pause like that unless your app is doing
lots of little writes and it takes 30 minutes to fill up memory to the
point that it has to empty the buffer, but that sounds like a long time.

The standard jpeg compression library does indeed do little writes. But
these are via fwrite, so they are buffered via the FILE mechanism. I go
not change the buffer sizes.


Do you know if your disc is writing continuously while you execute,
or do you get disk-i/o only ever 30 minutes? ;-)

The disks are accessed every few seconds.

The extra cpu's. If you run iostat, does your output disk see continuous
I/O, or only periodically?
(i.e. "iostat 5 /dev/sdX", where X is the disk you are writing to).

By writing to /dev/null instead of a disk you are avoiding the kernel's
disk-io routines as well as the xfs-io routines. I.e. it could be
either one. That's why trying O_DIRECT can help eliminate the kernel's
I/O buffering routines.

I do not think the kernel is the culprit. I think it is the hard disk
itself. We will be trying some higher performance discs.


--

Yours sincerely,

Roger Oberholtzer

OPQ Systems / Ramböll RST

Office: Int +46 10-615 60 20
Mobile: Int +46 70-815 1696
roger.oberholtzer@xxxxxxxxxx
________________________________________

Ramböll Sverige AB
Krukmakargatan 21
P.O. Box 17009
SE-104 62 Stockholm, Sweden
www.rambollrst.se

--
To unsubscribe, e-mail: opensuse+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse+owner@xxxxxxxxxxxx

< Previous Next >
This Thread