[opensuse-buildservice] RFC: new repository scheduling options
Hello, micha and me thought about how to configure the upcoming new scheduler modes per repository. We will have new attributes called "rebuild", "block" and "linkedbuild" optionally per repo. And we would like to hear some feedback about them. In addition to that it will be possible to reuse all package sources from other projects. Such a project meta xml might look like this: <project> ... <link project="openSUSE:Tools" /> ... <repository name="openSUSE_11.2" rebuild="local" block="all" linkedbuild="localdep" > <path repository="standard" project="openSUSE.org:openSUSE:11.2"/> <arch>x86_64</arch> </repository> </project> The values of the attributes can be these: <define ns="" name="rebuild-modes"> <choice> <value>transitive</value> <!-- DEFAULT: build on source change and all depending packages including indirect dependencies --> <value>direct</value> <!-- build on source change and direct depending packages --> <value>local</value> <!-- build on source change only --> </choice> </define> <define ns="" name="block-modes"> <choice> <value>all</value> <!-- DEFAULT: block until all packages we depend on are built --> <value>local</value> <!-- like all, but ignore packages from other repositories --> <value>never</value> <!-- immediately start building the packages --> </choice> </define> <define ns="" name="linkedbuild-modes"> <choice> <value>off</value> <!-- DEFAULT: do not build packages from project links --> <value>localdep</value> <!-- only build project linked packages if they depend on a local package --> <value>all</value> <!-- treat packages from project links like local packages --> </choice> </define> does this make sense ? What better names are thinkable ? bye adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Wednesday 2010-04-28 15:10, Adrian Schröter wrote:
And we would like to hear some feedback about them.
I'm gonna love this..
The values of the attributes can be these:
<define ns="" name="rebuild-modes"> <choice> <value>transitive</value> <!-- DEFAULT: build on source change and all depending packages including indirect dependencies --> <value>direct</value> <!-- build on source change and direct depending packages --> <value>local</value> <!-- build on source change only --> </choice> </define>
It should be possible to specify exceptions on a per-package basis. I would set a project to "direct", but to cater for packages using static inclusion of a library (prog -> foo.a), one may need "transitive" in case the library changes as a result of a rebuild (like when it uses __DATE__, though that's a light example; there is probably someone doing worse things that change just because of a rebuild).
<define ns="" name="block-modes"> <choice> <value>all</value> <!-- DEFAULT: block until all packages we depend on are built --> <value>local</value> <!-- like all, but ignore packages from other repositories --> <value>never</value> <!-- immediately start building the packages --> </choice> </define>
What happens if projects have a cyclic linking against another? -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Mittwoch, 28. April 2010 15:20:36 schrieb Jan Engelhardt:
On Wednesday 2010-04-28 15:10, Adrian Schröter wrote:
And we would like to hear some feedback about them.
I'm gonna love this..
The values of the attributes can be these:
<define ns="" name="rebuild-modes"> <choice> <value>transitive</value> <!-- DEFAULT: build on source change and all depending packages including indirect dependencies --> <value>direct</value> <!-- build on source change and direct depending packages --> <value>local</value> <!-- build on source change only --> </choice> </define>
It should be possible to specify exceptions on a per-package basis. I
maybe later, for now this is out of focus.
would set a project to "direct", but to cater for packages using static inclusion of a library (prog -> foo.a), one may need "transitive" in case the library changes as a result of a rebuild (like when it uses __DATE__, though that's a light example; there is probably someone doing worse things that change just because of a rebuild).
there is also other modes thinkable like "build on provides changes", but this is also future. However, it should be no problem to add new modes by this design.
<define ns="" name="block-modes"> <choice> <value>all</value> <!-- DEFAULT: block until all packages we depend on are built --> <value>local</value> <!-- like all, but ignore packages from other repositories --> <value>never</value> <!-- immediately start building the packages --> </choice> </define>
What happens if projects have a cyclic linking against another?
I hope to catch via a cycle detection. Howver only implemented for 50% yet. -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Wed, Apr 28, 2010 at 03:26:46PM +0200, Adrian Schröter wrote:
Am Mittwoch, 28. April 2010 15:20:36 schrieb Jan Engelhardt:
What happens if projects have a cyclic linking against another?
I hope to catch via a cycle detection. Howver only implemented for 50% yet.
A cycle is not an error for the backend, it just doesn't follow the link if the target is already visited. Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Markus Rex, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Mittwoch, 28. April 2010 15:31:30 schrieb Michael Schroeder:
On Wed, Apr 28, 2010 at 03:26:46PM +0200, Adrian Schröter wrote:
Am Mittwoch, 28. April 2010 15:20:36 schrieb Jan Engelhardt:
What happens if projects have a cyclic linking against another?
I hope to catch via a cycle detection. Howver only implemented for 50% yet.
A cycle is not an error for the backend, it just doesn't follow the link if the target is already visited.
okay, but I need nevertheless to detect them to avoid infinite looks when doing a checkout or branch of a not existing package for example. bye adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Wed, Apr 28, 2010 at 08:10:54AM -0500, Adrian Schröter wrote:
Hello,
micha and me thought about how to configure the upcoming new scheduler modes per repository. We will have new attributes called "rebuild", "block" and "linkedbuild" optionally per repo. And we would like to hear some feedback about them.
In addition to that it will be possible to reuse all package sources from other projects.
Such a project meta xml might look like this:
<project> ... <link project="openSUSE:Tools" /> ... <repository name="openSUSE_11.2" rebuild="local" block="all" linkedbuild="localdep" > <path repository="standard" project="openSUSE.org:openSUSE:11.2"/> <arch>x86_64</arch> </repository> </project>
The values of the attributes can be these:
<define ns="" name="rebuild-modes"> <choice> <value>transitive</value> <!-- DEFAULT: build on source change and all depending packages including indirect dependencies --> <value>direct</value> <!-- build on source change and direct depending packages --> <value>local</value> <!-- build on source change only --> </choice> </define>
This looks like an excellent addition.
<define ns="" name="block-modes"> <choice> <value>all</value> <!-- DEFAULT: block until all packages we depend on are built --> <value>local</value> <!-- like all, but ignore packages from other repositories --> <value>never</value> <!-- immediately start building the packages --> </choice> </define>
I'm concerned about the interaction between this option and the previous option. Looks like we have a matrix of 9 combinations, and I'm not sure they all interact so well.
<define ns="" name="linkedbuild-modes"> <choice> <value>off</value> <!-- DEFAULT: do not build packages from project links --> <value>localdep</value> <!-- only build project linked packages if they depend on a local package --> <value>all</value> <!-- treat packages from project links like local packages --> </choice> </define>
At first I did not understand this, but on IRC you pointed me to https://features.opensuse.org/307915 and I have a better grasp now. This also, I believe, adds complexity when combined with the above. A better name would eliminate confusion. My proposal is "projectlink-build-modes", but you indicated on IRC that wasnt good. In any case, this name needs adjustment to eliminate confusion between package _links and project links. -- Michael
On 28.4.2010 15:10, Adrian Schröter wrote:
<define ns="" name="rebuild-modes"> <choice> <value>transitive</value> <!-- DEFAULT: build on source change and all depending packages including indirect dependencies --> <value>direct</value> <!-- build on source change and direct depending packages --> <value>local</value> <!-- build on source change only --> </choice> </define>
Does 'direct' mean "rebuild, if the other package is mentioned in BuildRequires:", "rebuild, if the other package is needed for building", or "rebuild, if the other package is from the same repository"? My use case would be Kernel:HEAD with kernel-$flavor-devel, kernel-devel, kernel-syms and some KMPs for testing. The dependencies look like: foo-kmp BuildRequires: kernel-syms module-init-tools kernel-syms Requires: kernel-devel, kernel-{default,pae,xen}-devel I would like foo-kmp to block until all the kernel-* dependencies are built (block="local"), and I would like to rebuild it iff these kernel-* packages changes. This means not to rebuild if module-init-tools in openSUSE:Factory changes, but rebuild if kernel-default-devel changes, even though it is not directly mentioned in BuildRequires:. Is rebuild="direct" going to provide this? If not, is it possible to have such mode (analogous to rebuild="local" below)? I expect that similar project, e.g. zypp:Head, Yast:Head, would be also interested in such mode (do "correct" rebuilds within the project, but do not trigger on rebuilds in Factory).
<define ns="" name="block-modes"> <choice> <value>all</value> <!-- DEFAULT: block until all packages we depend on are built --> <value>local</value> <!-- like all, but ignore packages from other repositories --> <value>never</value> <!-- immediately start building the packages --> </choice> </define>
<define ns="" name="linkedbuild-modes"> <choice> <value>off</value> <!-- DEFAULT: do not build packages from project links --> <value>localdep</value> <!-- only build project linked packages if they depend on a local package --> <value>all</value> <!-- treat packages from project links like local packages --> </choice> </define>
does this make sense ? What better names are thinkable ?
I think it's going to be huge step forward! Michal -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On 29.4.2010 10:58, Michal Marek wrote:
On 28.4.2010 15:10, Adrian Schröter wrote:
<define ns="" name="rebuild-modes"> <choice> <value>transitive</value> <!-- DEFAULT: build on source change and all depending packages including indirect dependencies --> <value>direct</value> <!-- build on source change and direct depending packages --> <value>local</value> <!-- build on source change only --> </choice> </define>
Does 'direct' mean "rebuild, if the other package is mentioned in BuildRequires:", "rebuild, if the other package is needed for building", or "rebuild, if the other package is from the same repository"? My use case would be Kernel:HEAD with kernel-$flavor-devel, kernel-devel, kernel-syms and some KMPs for testing. The dependencies look like:
foo-kmp BuildRequires: kernel-syms module-init-tools kernel-syms Requires: kernel-devel, kernel-{default,pae,xen}-devel
I would like foo-kmp to block until all the kernel-* dependencies are built (block="local"), and I would like to rebuild it iff these kernel-* packages changes. This means not to rebuild if module-init-tools in openSUSE:Factory changes, but rebuild if kernel-default-devel changes, even though it is not directly mentioned in BuildRequires:. Is rebuild="direct" going to provide this? If not, is it possible to have such mode (analogous to rebuild="local" below)? block="local" of course
I expect that similar project, e.g. zypp:Head, Yast:Head, would be also interested in such mode (do "correct" rebuilds within the project, but do not trigger on rebuilds in Factory).
<define ns="" name="block-modes"> <choice> <value>all</value> <!-- DEFAULT: block until all packages we depend on are built --> <value>local</value> <!-- like all, but ignore packages from other repositories --> <value>never</value> <!-- immediately start building the packages --> </choice> </define>
<define ns="" name="linkedbuild-modes"> <choice> <value>off</value> <!-- DEFAULT: do not build packages from project links --> <value>localdep</value> <!-- only build project linked packages if they depend on a local package --> <value>all</value> <!-- treat packages from project links like local packages --> </choice> </define>
does this make sense ? What better names are thinkable ?
I think it's going to be huge step forward!
Michal
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
participants (5)
-
Adrian Schröter -
Jan Engelhardt -
Michael E Brown -
Michael Schroeder -
Michal Marek