On Wed, 2013-06-12 at 06:50 -0400, Anton Aylward wrote:
Roger Oberholtzer said the following on 06/11/2013 08:15 AM:
That you don't, that you are treating the cacche as a buffer and its behaving as a cache, is the root of that you are observing and complaining about. I am not doing this. I am only reporting that it seems that as the file system gets written to, the OS cache grows and grows. As the cache grows, the occasional flushes that are done to the disk seem to take longer and longer. Coincidence? Perhaps. Repeatable? 100%.
What you are describing is the behaviour of a cache.
I don't know why you say "I am not doing this". Perhaps you are not actively saying "I have programmed a cache", but the file system is running a cache anyway because that is what the file systems do. It not you, its the system you are running. It applies just as much to XFS as ext4 as any other FS.
I understand that the kernel is doing this as the result of me making a big file. So of course, ultimately, I am making it happen. What I am trying to say is that I am not telling the kernel to act stupid when making this big file so that all my memory goes to cache and that huge delays are incurred when dealing with the cache.
But what you *are* doing when you flush with sync etc etc that you described in previous email is _expecting_ the system cache to behave like a buffer. When you flush a buffer it empties. When you 'flush' a cache all you are doing is making sure that the disk is in sync with the cache. The cache does not empty.
When you mount a FS the cache is empty, but every time you write or read from the FS it sticks in the cache up to the maximum capacity of the cache. The cache algorithm (probably) supposes that the most recently accessed stuff is going to be re-used so only drops "old" stuff.
"maximum capacity of the cache" - there is the core of the problem. It seems that the kernel thinks the max capacity is all my RAM! When I limit the cache to at most 4 or so GB (the result of my fix), the system acts as I expect. When I let the system alone to deal with the cache, it grows until the system has a problem. I am happy to let the system cache things as it usually does. But I am unable to let it cache things as it seems to want to when a big file is written at a steady rate and there is lots of available RAM for the taking. Something in the logic seems to break. I just do not know what. I am guessing some of the tunable parameters for the kernel VM may help. Yours sincerely, Roger Oberholtzer Ramböll RST / Systems Office: Int +46 10-615 60 20 Mobile: Int +46 70-815 1696 roger.oberholtzer@ramboll.se ________________________________________ Ramböll Sverige AB Krukmakargatan 21 P.O. Box 17009 SE-104 62 Stockholm, Sweden www.rambollrst.se -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org