Feature changed by: Jan Engelhardt (jengelh) Feature #305015, revision 12 Title: support noarch via own scheduler in Build Service Buildservice: Evaluation by engineering manager Priority Requester: Desirable Projectmanager: Desirable Requested by: Adrian Schröter (adriansuse) Requested by: Lars Vogdt (lrupp) Partner organization: openSUSE.org Description: We would like to build noarch packages only once to avoid multiple builds and wasted space on the server. This can be reached by building noarch packages only via an dedicated scheduler. However, we need to avoid that have a horrible project configuration, where all BuildRequired noarch packages needs to get submitted into all other architecture repository :full trees. Discussion: #1: Adrian Schröter (adriansuse) (2010-02-19 10:48:58) Theoretically already possible, still questionable if we want this. -> keep it in evaluation #2: Greg Freemyer (gregfreemyer) (2012-03-30 20:14:05) I can think of of at least 3 ways to accomplish the goal: 1) Allow noarch builds of just one arch to be accepted to factory. Then encourage people to just enable one arch. Hopefully randomly to spread the load around. 2) Create a new "noarch" architecture and scheduler. Have that feed whatever actual scheduler has the lightest load at schedule time. That way even non-Intel machines could build the noarch packages. 3) Enforce in all schedulers but one, that they ignore noarch packages. Then document that so everyone knows they can only build noarch on i586 (as an example). idea 1) seems relatively doable for a modest effort. idea 2) would require changes lots of places I assume, so it is not trivial. idea 3) seems like the least work by far, and it sheds work load from one of the schedulers. I don't know if that would benefit the overall OBS load or not. And I'm sure there are other ways. === The question is if the wasted build resources is worth the R&D to bother with fixing this. #3: Claudio Freire (klaussfreire) (2012-03-30 20:30:05) (reply to #2) 3) Enforce in all schedulers but one, that they ignore noarch packages. Then document that so everyone knows they can only build noarch on i586 (as an example). I think this is the best option. For one, making sure noarch packages are built in the same architecture every time will help avoid spurious deltas: if a package builds differently on different architectures, but the packager decided that the difference is inconsequential. For instance, arm zlib needs not produce the same output as x86 zlib. For two, i586 and x86_64, AFAIK, use the same hosts. Right? So spreading the load among those two is useless. But there are quite a few caveats. One, people should not care about which arch is enabled, if there is any enabled archs for a noarch package, it should be built by "the noarch" scheduler (say, the x86_64 scheduler). If there are many archs enabled, only one build should be scheduled, and build results should be replicated. This might need some doing. 2) Create a new "noarch" architecture and scheduler. Have that feed whatever actual scheduler has the lightest load at schedule time. That way even non-Intel machines could build the noarch packages. So, more to-the-point would be option 2, but it still doesn't completely convince me, because it would be quite confusing in the UI. There also option 1, which I guess would work for factory (but not other projects). It's already common practice in OBS to only enable one arch for big noarch packages, so it should be rather easy to do - just fix factory-auto's validation script to account for noarch packages. About the benefits, I have a few projects that build hundred-MB noarch rpms with game data. Those take a lot of I/O resources to build, so the inefficiency (in that case) is significant. I'm not sure about how many projects are found in that situation, though. I would vote to get option 1 implemented ASAP - it should be easy and immediate. If I wanted to patch it... where should I go to find the factory-auto script? + #5: Jan Engelhardt (jengelh) (2015-08-14 00:06:13) (reply to #2) + >[3.] Then document that so everyone knows they can only build noarch + on i586 And if you have no x86? Like, a PPC-only pool. + Suggestion [2] doing a round-robin seems preferable. #4: Stefan Brüns (stefanbruens) (2015-07-20 17:10:51) I think this should be reevaluated - ARM builds are really slow, and e. g kernel-docs takes a full day to rebuild. -- openSUSE Feature: https://features.opensuse.org/305015