On Wed, 2013-06-12 at 08:24 +0200, Per Jessen wrote:
It only grows as long as nothing else needs the memory. Using up otherwise unused memory as file systems cache seems quite prudent.
Would that this were the case. The memory use increases until process start to be killed. Which is the standard way Linux deals with memory shortages. And it begs the question, why should any file system think it is ok to cache, say 16GB? At least I would not expect this as the default. I have checked and the behavior is the same on openSUSE 12.3 as I see on 12.1
In addition, as the cache grows, the periodic writes to disk take longer and longer. 100% reproducible.
This is more likely the problem.
It is what first caught my attention!
If this is a systemic problem, I ought to be able to reproduce it, so yesterday I wrote a little test program to do exactly that. I never saw your 4-5 second delay, but I did see the IO-rate dropping to about half a couple of times. Not regularly though. Much smaller system, single core, 1.5GB RAM, one single harddrive.
In fact my single process has two files open, each on a separate disk. Maybe that is part of the dynamic. If you let your test app run, and the file it creates grow and grow, how does the cache usage progress? The test app needs to write to open a new file when the previous one is
than the file system file size limit. And just keep doing this.
And the cache usage will grow and grow...
I understand that the cache is there so I can possibly read data that has been recently written. However, I do not see how the kernel can just grow this cache until my memory is gone.
If something else needs the memory, the kernel will invalidate the cache and give the memory away.
I suspect the memory has been invalidated. But it has not been given away. I base that assumption on the fact that my workaround frees the cache immediately. Whether the cache size stops at some reasonable point or not is perhaps a side issue. The question remains: why, as the cache grows, do write calls periodically have longer and longer delays (in the magnitude of seconds)? If the cache is not causing this, then why does freeing it with the workaround result in these delays not happening? -- Roger Onerholtzer -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org