Mailinglist Archive: opensuse-features (47 mails)

< Previous Next >
[openFATE 305015] support noarch via own scheduler in Build Service
Feature changed by: Jan Engelhardt (jengelh)
Feature #305015, revision 12
Title: support noarch via own scheduler in Build Service

Buildservice: Evaluation by engineering manager
Priority
Requester: Desirable
Projectmanager: Desirable

Requested by: Adrian Schröter (adriansuse)
Requested by: Lars Vogdt (lrupp)
Partner organization: openSUSE.org

Description:
We would like to build noarch packages only once to avoid multiple
builds and wasted space on the server. This can be reached by building
noarch packages only via an dedicated scheduler.
However, we need to avoid that have a horrible project configuration,
where all BuildRequired noarch packages needs to get submitted into all
other architecture repository :full trees.

Discussion:
#1: Adrian Schröter (adriansuse) (2010-02-19 10:48:58)
Theoretically already possible, still questionable if we want this. ->
keep it in evaluation

#2: Greg Freemyer (gregfreemyer) (2012-03-30 20:14:05)
I can think of of at least 3 ways to accomplish the goal: 1) Allow
noarch builds of just one arch to be accepted to factory. Then
encourage people to just enable one arch. Hopefully randomly to spread
the load around. 2) Create a new "noarch" architecture and scheduler.
Have that feed whatever actual scheduler has the lightest load at
schedule time. That way even non-Intel machines could build the noarch
packages. 3) Enforce in all schedulers but one, that they ignore noarch
packages. Then document that so everyone knows they can only build
noarch on i586 (as an example). idea 1) seems relatively doable for a
modest effort. idea 2) would require changes lots of places I assume,
so it is not trivial.
idea 3) seems like the least work by far, and it sheds work load from
one of the schedulers. I don't know if that would benefit the overall
OBS load or not.
And I'm sure there are other ways. ===
The question is if the wasted build resources is worth the R&D to
bother with fixing this.

#3: Claudio Freire (klaussfreire) (2012-03-30 20:30:05) (reply to #2)
3) Enforce in all schedulers but one, that they ignore noarch
packages. Then document that so everyone knows they can only build
noarch on i586 (as an example).
I think this is the best option.
For one, making sure noarch packages are built in the same architecture
every time will help avoid spurious deltas: if a package builds
differently on different architectures, but the packager decided that
the difference is inconsequential. For instance, arm zlib needs not
produce the same output as x86 zlib.
For two, i586 and x86_64, AFAIK, use the same hosts. Right? So
spreading the load among those two is useless.
But there are quite a few caveats. One, people should not care about
which arch is enabled, if there is any enabled archs for a noarch
package, it should be built by "the noarch" scheduler (say, the x86_64
scheduler). If there are many archs enabled, only one build should be
scheduled, and build results should be replicated. This might need some
doing.
2) Create a new "noarch" architecture and scheduler. Have that feed
whatever actual scheduler has the lightest load at schedule time. That
way even non-Intel machines could build the noarch packages.
So, more to-the-point would be option 2, but it still doesn't
completely convince me, because it would be quite confusing in the UI.
There also option 1, which I guess would work for factory (but not
other projects). It's already common practice in OBS to only enable one
arch for big noarch packages, so it should be rather easy to do - just
fix factory-auto's validation script to account for noarch packages.
About the benefits, I have a few projects that build hundred-MB noarch
rpms with game data. Those take a lot of I/O resources to build, so the
inefficiency (in that case) is significant. I'm not sure about how many
projects are found in that situation, though.
I would vote to get option 1 implemented ASAP - it should be easy and
immediate. If I wanted to patch it... where should I go to find the
factory-auto script?

+ #5: Jan Engelhardt (jengelh) (2015-08-14 00:06:13) (reply to #2)
+ >[3.] Then document that so everyone knows they can only build noarch
+ on i586 And if you have no x86? Like, a PPC-only pool.
+ Suggestion [2] doing a round-robin seems preferable.

#4: Stefan Brüns (stefanbruens) (2015-07-20 17:10:51)
I think this should be reevaluated - ARM builds are really slow, and e.
g kernel-docs takes a full day to rebuild.




--
openSUSE Feature:
https://features.opensuse.org/305015

< Previous Next >
This Thread