On Fri, Jul 25, 2014 at 12:29 PM, Bernhard Voelker <mail@bernhard-voelker.de> wrote:
On 07/25/2014 01:52 PM, Greg Freemyer wrote:
mbuffer!
Very cool. I love that app, but rarely find good use cases for it.
For those who don't know it, mbuffer is often used between apps like tar or cpio and high-speed tape drives. Tape drives in sequential access mode are often far faster than a disk with data scattered all around the place so a large buffer highly optimizes the overall process.
Nice, specialized tool - even network support and other useful features. ;-)
Out of curiosity: for disks and tapes, is there much different from using dd iflag=fullblock bs=..M or dd iflag=fullblock ibs=..M obs=..M in between the pipes?
Thanks & have a nice day, Berny
For tapes yes, but it has been a decade since I did comparative testing: Let's say you have a tape that works really well if you send it 1 MB at a time, but you want to send it a continuous stream for at least 30 seconds of continuous tape operation at a time. A fast tape drive may write to media at 60 MB/sec, so that 1 MB block will barely feed the tape for anytime at all. You want to send it chucks of data 1.8GB or bigger. You can use mbuffer to output the 1MB blocks, but not to start until it has 1800 blocks in the buffer (1.8 GB of data). Now you get to feed the tape drive the 1MB blocks it likes to get and you know that you can send it a continuous stream of those blocks for at least 30 seconds. Also, any data that tar or cpio feed into mbuffer during that 30 seconds time will also get sent to the tape drive as part of the continuous stream of 1MB blocks. When the buffer gets to the low water mark, mbuffer will quit sending 1MB blocks until the buffer fills back up. I guess you could attempt the same with dd bs=1.8GB, but it really would not be the same. As an example assume your data source (tar) can produce data at 1.5 GB per 30 seconds. With mbuffer you would have: 0 - 35 seconds (or so) - mbuffer collecting data, no output 35-65 seconds - mbuffer spitting out the first 1.8 GB of data at full tape speed 65-120 seconds (or so) - mbuffer continuing to run at full tape speed pushing out the data that came from tar during the 35-65 second timeframe. 120 - 175 seconds (or so) - mbuffer still continuing to send out data, but this is data it received from 65-120 seconds That process will continue, so as you can see mbuffer will keep the drive busy for minutes at a time in that scenario, then have a 35 second pause every 5 or 6 minutes. If you did the same with dd and 1.8 GB blocks, you would have a pause at the end of every 1.8GB block because the next block would only be only 90% full and not ready to send to the tape. The end result with dd would be the tape running for 30 seconds at a time with a pause after each 30 seconds. That means more wear and tear on the tape drive and the media at a minimum and I think overall lower tape throughput. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org