[opensuse-buildservice] Optimization of OBS scheduler
Hello, I've got a question about OBS scheduling algorithm. My test project has the following package dependencies structure: “A”->”AB”->”B”->”A” “A”->”AA” “B”->”BB” Where relation “A”->”AB” means that package “AB” BuildRequires “A”. So, here we have one cycle: “A”, “AB”, “B” and two packages outside - “AA” and “BB”. For the experiments I use OBS-2.7 with one worker. Every time I get the following building sequence: “A”, “B”, “AB”, “A”, “AA”, “B”, “BB”, “A”, “AB”, “AA” Here everything works right - according to the current scheduler algorithm. But, it seems strange for me that package “AA” is built twice whereas it’s not part of a cycle. It looks like a miss-optimization. In my view, it could be reasonable to build non-cyclic packages only after cyclic ones. If so, “AA” and “BB” should be built only one time. In sum, the question is what if we change the scheduler to postpone building of non-cyclic packages(that doesn’t belong to any cycle) until cyclical builds are finished? Can we get here any build speed-up or there are some corner cases where this approach is not acceptable? Thanks. -- Best regards, Pavel Kopyl -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Hello, Does anybody have an idea about this optimization? I'm going to try it on my private OBS instance, but before doing this I'd like to hear someone's opinion. Thanks. -- Best regards, Pavel Kopyl On 10/27/2016 3:09 PM, Kopyl Pavel wrote:
Hello,
I've got a question about OBS scheduling algorithm.
My test project has the following package dependencies structure:
“A”->”AB”->”B”->”A”
“A”->”AA”
“B”->”BB”
Where relation “A”->”AB” means that package “AB” BuildRequires “A”.
So, here we have one cycle: “A”, “AB”, “B” and two packages outside - “AA” and “BB”.
For the experiments I use OBS-2.7 with one worker.
Every time I get the following building sequence:
“A”, “B”, “AB”, “A”, “AA”, “B”, “BB”, “A”, “AB”, “AA”
Here everything works right - according to the current scheduler algorithm.
But, it seems strange for me that package “AA” is built twice whereas it’s not part of a cycle.
It looks like a miss-optimization. In my view, it could be reasonable to build non-cyclic packages only
after cyclic ones. If so, “AA” and “BB” should be built only one time.
In sum, the question is what if we change the scheduler to postpone building of
non-cyclic packages(that doesn’t belong to any cycle) until cyclical builds are finished?
Can we get here any build speed-up or there are some corner cases where this approach is not acceptable?
Thanks.
--
Best regards,
Pavel Kopyl
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Wed, Nov 02, 2016 at 05:11:21PM +0300, Pavel Kopyl wrote:
Does anybody have an idea about this optimization? I'm going to try it on my private OBS instance, but before doing this I'd like to hear someone's opinion.
It sounds like a good idea to me. You'll need to tweak the handling of cyclic dependencies a bit, so that all packages of a cycle are processed as a block (i.e. with no other package in between). That's currently true for the first pass, but not for the second pass. Would you need to do is set "$notready" of all cycle packages if one of them is building/blocked. Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Thanks for answering. I'll try it. -- Best regards, Pavel Kopyl On 11/02/2016 06:27 PM, Michael Schroeder wrote:
On Wed, Nov 02, 2016 at 05:11:21PM +0300, Pavel Kopyl wrote:
Does anybody have an idea about this optimization? I'm going to try it on my private OBS instance, but before doing this I'd like to hear someone's opinion. It sounds like a good idea to me. You'll need to tweak the handling of cyclic dependencies a bit, so that all packages of a cycle are processed as a block (i.e. with no other package in between). That's currently true for the first pass, but not for the second pass.
Would you need to do is set "$notready" of all cycle packages if one of them is building/blocked.
Cheers, Michael.
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
participants (3)
-
Kopyl Pavel -
Michael Schroeder -
Pavel Kopyl