Roger Oberholtzer said the following on 06/12/2013 10:09 AM:
I understand that the kernel is doing this as the result of me making a big file. So of course, ultimately, I am making it happen. What I am trying to say is that I am not telling the kernel to act stupid when making this big file so that all my memory goes to cache and that huge delays are incurred when dealing with the cache.
In effect you are; you are telling it, by not telling it otherwise, that the ageing and cache handling settings that are set at installation 'out of the box' are Ok, BECAUSE YOU HAVE NOT SET THEM UP TO DEAL WITH YOUR CONTEXT!
"maximum capacity of the cache" - there is the core of the problem. It seems that the kernel thinks the max capacity is all my RAM!
Well it is - as others have pointed out. Unless something else - some other process - makes a demand for memory, then using all the available memory for a cache is a sensible DEFAULT strategy. Its the default because you haven't told the system otherwise.
When I limit the cache to at most 4 or so GB (the result of my fix), the system acts as I expect. When I let the system alone to deal with the cache, it grows until the system has a problem.
Right, because the default is well suited to my context, a desktop running in 4G, email, a browser, an editor, a xterm. Real "multi-tasking", varied demands, lots of networking. If I were to try and tune my page ageing I'd be wasting my time. I keep saying Context is Everything and my context is different from your context. The out-of-the-box defaults are "just fine, thank you", for my context.
I am happy to let the system cache things as it usually does. But I am unable to let it cache things as it seems to want to when a big file is written at a steady rate and there is lots of available RAM for the taking. Something in the logic seems to break. I just do not know what.
I am guessing some of the tunable parameters for the kernel VM may help.
I wish someone would tell us if its possible to set cache activity by file system. Really when it comes down to it, your application is write-only. The cache is of no benefit since you aren't reading anything back. Well, OK, it need to cache the inode segments since its creating new file nodes, oh and the directory data. But so long as all you are ding is writing out the files you don't need to cache data. But you do need the cache for the "other" things about Linux. If and only if both a) you have this data on a separate file system from the 'system', and b) each file system can have its won cache then setting the cache(data) size to zero for the file system where you are storing the images will achieve what you want. But if you can't use separate caches then shrinking the cache to zero will hurt other parts of the performance. I think being aggressive about page ageing and flushing aged pages is the best approach in that case. -- How long did the whining go on when KDE2 went on KDE3? The only universal constant is change. If a species can not adapt it goes extinct. That's the law of the universe, adapt or die. -- Billie Walsh, May 18 2013 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org