On 2014-06-25 17:43, Anton Aylward wrote:
On 06/25/2014 11:19 AM, Carlos E. R. wrote:
Years ago, in MsDOS, I had a third party disk cache that could be applied to a single partition, IIRC.
That's basically it, dedicate a cache with fixed ram space to a single directory, in this case, "/tmp". Most accesses would work only in RAM, it would be persistent, and size would not be a problem.
Yes but is it worth it? We have inode caching. We have file pathname lookup caching. We have page caching that amounts to file content caching.
The issue is this:
* Is it smart enough? * Would giving preference to /tmp for any of the above degrade performance in other areas?
Sometimes the admin may know things that the kernel does not :-) For instance. If I'm playing a DVD, or a movie from an iso file, I'm not interested in the system caching the entire file. In fact, I'm not interested in caching it at all. Or, if I'm coping a bunch of files, I do want the inodes and such cached, but not the contents, because as in the previous case, I only need the contents once. Read once, write once on another place, be done. Having the previous used file contents in cache when I read the next is a waste. In fact, I learned bout this reading xine docs, where they mentioned using some type of raw read library in order to impede the kernel caching movies. The cache is much more useful on files that often read and written, continuously. Thus if you can tell the kernel the areas not to cache, it can dedicate more effort on others. And the other way round. People like tmp on a tmpfs because it is very fast. Data and metadata are in memory. The problem, for other people, is that the size is limited, or it can impact other programs needing ram. If it grows a lot, it will cause swapping. Will it cause application memory to be swapped out in order to write a temporary file? I do not want that! (On the other hand, what is the purpose of having tmp in ram? The application could directly store that data in ram and skip the effort. If it uses a tmp file is to conserve ram, no? On the other hand, there are intermediary results and pipes and such. Just thinking aloud) If we had an hybrid tmpfs, limited to a fixed ram value, overflowing to disk when needed (not swap, perhaps), or a dedicated and limited cache on a certain directory, it would be ram-speed fast on small usages, and disk based for large files, and everybody could be happy. Worth it? Dunno. For me, probably. :-)
Years ago I worked on a project where the PM didn't understand 'systems'. He didn't understand that C deals with dynamic variables on the stack, he still thought of C as 'Fortran with semicolons' and required that all variables be global. That buggered up many nested loops! I had to metricate the compiler to show that the global segment was via a pointer. This was on the VAX. Global variables were no faster than locals. Another of his idiocies was a lack of understanding of how virtual memory and page caching works. He required each program to be locked in memory. Think about that. I made sure that my code was small (and hence fast) and so small that the memory allocators pretty much ignored it. Because I made heavy use of shared libraries my read code only amounted to a couple of the 4K pages.
:-)
So my question becomes this: unless you have lots of memory (so that you never even swap) then what is the advantage in this? The system is already caching the inodes and paths you use a lot. Your machine might want to swap out unused apps to speed up disk access of running ones.
The idea is to cache file content as well, in order to get the performance of a tmpfs, at least till the usage passes a given amount. That it behaves like a tmpfs till, say, 500 MB are used, and beyond that, like a hard disk, so that you can write terabytes if you need to.
I think this is another case where the machine knows more about what's going on than you do in the 95% case. Yes, we have the fringe use-cases like the phone/image rendering that uses big, persistent temp files, and the development situation where the compiler is using small transient ones. The sysadmin should pay attention. And yes there are ways to tune this. In the limiting case we should be able to creat a tmfs and set
TMP=/var/specialtmp
in the application environment. But will it be honoured?
See 15.1.7 http://doc.opensuse.org/products/draft/SLES/SLES-tuning_sd_draft/cha.tuning....
That's quite complicated to adjust, and get good results... -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)