Mailinglist Archive: opensuse (1239 mails)

< Previous Next >
Re: [opensuse] Re: XFS and openSUSE 12.1
  • From: Per Jessen <per@xxxxxxxxxxxx>
  • Date: Wed, 12 Jun 2013 08:24:08 +0200
  • Message-id: <kp9468$aka$1@saturn.local.net>
Roger Oberholtzer wrote:

On Tue, 2013-06-11 at 12:14 -0400, Greg Freemyer wrote:

I think everyone is misunderstanding the situation. I am not doing
anything with or expecting or manipulating a cache. The cache I see is
a totally private thing being done by the OS. The existence of the
cache is not the problem. In fact, if there was no cache I would think
something was wrong.

The problem I am seeing is that the cache is growing and growing to
eat all my memory.

It only grows as long as nothing else needs the memory. Using up
otherwise unused memory as file systems cache seems quite prudent.

In addition, as the cache grows, the periodic writes to disk take
longer and longer. 100% reproducible.

This is more likely the problem.

To be clear: I do not ask for, manipulate or in any other way
influence the cache through any direct action in my application. I am
only writing a single file by a single process. This file is growing
at 25 MB a second (more or less). The file is opened with fopen,
written to, and then closed with fclose(). Files can be big, but never
more than 2GB each.

If this is a systemic problem, I ought to be able to reproduce it, so
yesterday I wrote a little test program to do exactly that. I never saw
your 4-5 second delay, but I did see the IO-rate dropping to about half
a couple of times. Not regularly though. Much smaller system, single
core, 1.5GB RAM, one single harddrive.

I understand that the cache is there so I can possibly read data that
has been recently written. However, I do not see how the kernel can
just grow this cache until my memory is gone.

If something else needs the memory, the kernel will invalidate the cache
and give the memory away.

Especially if the bigger cache also results in significant and
increasingly longer delays in write completions.

Yes, that is the odd thing. Especially your 4-5 second delays.

The workaround that seems to correct the situation is to run this:

while [ 1 ]
do
echo 1 > /proc/sys/vm/drop_caches
sleep 60
done &

It would seem to substantiate your point about the cache being the
issue. How about if you only tried limiting the cache size? (just as
another option). If the file systems cache was kept to e.g. 1Gb.



--
Per Jessen, Zürich (14.6°C)
http://www.dns24.ch/ - free DNS hosting, made in Switzerland.

--
To unsubscribe, e-mail: opensuse+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse+owner@xxxxxxxxxxxx

< Previous Next >
This Thread
Follow Ups