On 2018-05-21 15:55, Roger Oberholtzer wrote:
I see the following write times when making a file:
# time dd if=/dev/zero of=file.txt count=2096576 bs=4096 2096576+0 records in 2096576+0 records out 8587575296 bytes (8.6 GB, 8.0 GiB) copied, 42.8592 s, 200 MB/s
real 0m45.338s user 0m0.526s sys 0m5.100s
Try playing with options such as "oflag=direct", or "nocache" or "dsync". Dsync just makes sure the data is written to disk asap, but it leaves a copy in cache. Nocache first uses the kernel disk cache mechanism, then flushes it. Direct eliminates the use of the kernel cache altogether.
But something just ain't right. Time to review our application to see if something else has happened. We used to use an Intel library to compress JPEG buffers (memory -> memory). We now use Turbo JPEG to do the same. That seems to be as fast. But it is the only thing we have changed in the code. (Famous last words - but I did verify this in our revision control system.)
Maybe your code can call to flush each file when it writes it, similarly to how 'dd' does it. -- Cheers / Saludos, Carlos E. R. (from 42.3 x86_64 "Malachite" at Telcontar)