[opensuse-project] [ANNOUNCE] bfq disk I/O scheduler
Hi, I have been working for a few years (with Fabio Checconi) on a disk scheduler providing definitely lower latencies than cfq, as well as a higher throughput with most of the test workloads we used (or the same throughput as cfq with the other workloads). We named this scheduler bfq (budget fair queueing). I hope this is the right list for announcing this work. One of the things we measured in our tests is the cold-cache execution time of a command as, e.g., "bash -c exit", "xterm /bin/true" or "konsole -e /bin/true", while the disk was also accessed by different combinations of sequential, or random, readers and/or writers. Depending on which of these background workloads was used, these execution times were five to nine times lower with bfq under 2.6.32. Under 2.6.35 they were instead from six to fourteen times lower. The highest price paid for these lower latencies was a 20% loss of aggregated disk throughput for konsole in case of background workloads made only of sequential requests (due to the fact that bfq of course privileges, more than cfq, the seeky IO needed to load konsole and its dependencies). In contrast, with shorter commands, as bash or xterm, bfq also provided up to 30% higher aggregated throughput. We saw from 15% to 30% higher aggregated throughput also in our only-aggregated-throughput tests. You can find in [1] all the details on our tests and on other nice features of bfq, such as the fact that it perfectly distributes the disk throughput as desired, independently of disk physical parameters like, e.g., ZBR. in [1] you can also find a detailed description of bfq and a short report on the maturity level of the code (TODO list), plus all the scripts used for the tests. The results I mentioned so far have been achieved with the last version of bfq, released about two months ago as patchsets for 2.6.33 or 2.6.34. From a few days a patchset for 2.6.35 is available too, as well as a backport to 2.6.32. The latter has been prepared by Mauro Andreolini, who also helped me a lot with debugging. All these patches can be found here [2]. A few days after being released, this version of bfq has been introduced as the default disk scheduler in the Zen Kernel. It has been adopted as the default disk scheduler in Gentoo Linux too. I also recorded downloads from users with other distributions, as, e.g., Ubuntu and ArchLinux. As of now we received only positive feedbacks from the users. Paolo [1] http://algo.ing.unimo.it/people/paolo/disk_sched/ [2] http://algo.ing.unimo.it/people/paolo/disk_sched/sources.php -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
El 04/08/10 14:26, Paolo Valente escribió:
Interesting, but out of topic on this list, if you push this new scheduler to kernel upstream it will appear in openSUSE as consecuence. -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Cristian Rodríguez ha scritto:
Thanks for the feedback. We already proposed bfq to lkml about two years ago, and it received a warm welcome. Unfortunately the ultimate response of the maintainer was: "ok, it outperforms cfq as you say, but the core algorithm is too deep and hence hard to understand". I'm going to re-propose this new version, which achieves even lower latencies, to lkml, but while waiting for its fate, my idea was to let people/developers of openSUSE know that this scheduler is available. Probably this is not the right channel, but I was not able to find a better one ...
-- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Wed, Aug 4, 2010 at 5:56 PM, Paolo Valente <posta_paolo@yahoo.it> wrote:
Paolo, I assume its compilable as a module? If so, you can build it in your home directory on OBS (openSUSE build service) for whichever distro versions it would support. (ie. openSUSE 11.3 uses the 2.6.34 kernel, older distros use older kernels). You can publish it as a KMP for those distros straight from your home directory and as long as it doesn't violate the terms of use (GPL, etc.) no one should complain. But, assuming it works to you your satisfaction, you could then find a devel repo on OBS that would be appropriate and try submitting your full KMP package to it. It is up to the maintainers of that repo whether or not they accept your submission. Volunteering to maintain it will greatly increase the odds of it being accpeted. If they do, openSUSE users can add that repo instead of your home rpop and get your KMP for openSUSE 11.3, etc.. And then from that repo you can try to submit it to contrib or factory. Submitting with a designated maintainer is mandatory. Factory will become the next full release in March 2011. I don't know who the decision maker(s) would be for accepting it into factory, but the above is the process. If your hoping one of the existing openSUSE maintainers will attempt the above process in your place, you definitely need to ask for a volunteer. As to which mailinglist to ask for that volunteer if no one on your team is willing to do it, my guess would be the opensuse-kernel list. Greg -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Wednesday 2010-08-04 20:26, Paolo Valente wrote:
I can guess your intentions, but if I were to be scientifically precise, I'd say, what happens now? Did you intend to suggest that openSUSE should include it too? Say so - further discussion on the opensuse-kernel mailing list seems inorder. Did you want to showcase it on openSUSE? That too should be no problem. Branch the kernel-source package in our Build Service, add the patch, let it compile. Willing testers can thus try it in an instant (and without spending time on the compile themselves). Good Luck! -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Jan Engelhardt ha scritto:
Did you intend to suggest that openSUSE should include it too? Say so - further discussion on the opensuse-kernel mailing list seems inorder.
As I wrote in my reply to crrodriguez@opensuse.org, my purpose was just to let you know that bfq exists and hopefully does what I wrote it does. Apart from this, and from the desire to let people benefit from what I did :), I do not have a clear idea of how exactly bfq might make its way to openSUSE
Ok, I will have a look at the process
Good Luck! Thanks :)
-- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
El 04/08/10 14:26, Paolo Valente escribió:
Interesting, but out of topic on this list, if you push this new scheduler to kernel upstream it will appear in openSUSE as consecuence. -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Cristian Rodríguez ha scritto:
Thanks for the feedback. We already proposed bfq to lkml about two years ago, and it received a warm welcome. Unfortunately the ultimate response of the maintainer was: "ok, it outperforms cfq as you say, but the core algorithm is too deep and hence hard to understand". I'm going to re-propose this new version, which achieves even lower latencies, to lkml, but while waiting for its fate, my idea was to let people/developers of openSUSE know that this scheduler is available. Probably this is not the right channel, but I was not able to find a better one ...
-- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Wed, Aug 4, 2010 at 5:56 PM, Paolo Valente <posta_paolo@yahoo.it> wrote:
Paolo, I assume its compilable as a module? If so, you can build it in your home directory on OBS (openSUSE build service) for whichever distro versions it would support. (ie. openSUSE 11.3 uses the 2.6.34 kernel, older distros use older kernels). You can publish it as a KMP for those distros straight from your home directory and as long as it doesn't violate the terms of use (GPL, etc.) no one should complain. But, assuming it works to you your satisfaction, you could then find a devel repo on OBS that would be appropriate and try submitting your full KMP package to it. It is up to the maintainers of that repo whether or not they accept your submission. Volunteering to maintain it will greatly increase the odds of it being accpeted. If they do, openSUSE users can add that repo instead of your home rpop and get your KMP for openSUSE 11.3, etc.. And then from that repo you can try to submit it to contrib or factory. Submitting with a designated maintainer is mandatory. Factory will become the next full release in March 2011. I don't know who the decision maker(s) would be for accepting it into factory, but the above is the process. If your hoping one of the existing openSUSE maintainers will attempt the above process in your place, you definitely need to ask for a volunteer. As to which mailinglist to ask for that volunteer if no one on your team is willing to do it, my guess would be the opensuse-kernel list. Greg -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Wednesday 2010-08-04 20:26, Paolo Valente wrote:
I can guess your intentions, but if I were to be scientifically precise, I'd say, what happens now? Did you intend to suggest that openSUSE should include it too? Say so - further discussion on the opensuse-kernel mailing list seems inorder. Did you want to showcase it on openSUSE? That too should be no problem. Branch the kernel-source package in our Build Service, add the patch, let it compile. Willing testers can thus try it in an instant (and without spending time on the compile themselves). Good Luck! -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Jan Engelhardt ha scritto:
Did you intend to suggest that openSUSE should include it too? Say so - further discussion on the opensuse-kernel mailing list seems inorder.
As I wrote in my reply to crrodriguez@opensuse.org, my purpose was just to let you know that bfq exists and hopefully does what I wrote it does. Apart from this, and from the desire to let people benefit from what I did :), I do not have a clear idea of how exactly bfq might make its way to openSUSE
Ok, I will have a look at the process
Good Luck! Thanks :)
-- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
participants (4)
-
Cristian Rodríguez -
Greg Freemyer -
Jan Engelhardt -
Paolo Valente