openSUSE Build Service October 2020

buildservice@lists.opensuse.org

28 participants
23 discussions

[opensuse-buildservice] Compiler Cache (Ccache) support in Open Build Service

by Sumit Jamgade

Hello, Long running compilation can now leverage ccache[1] to speed up the build process. This feature can be enabled per package in the project config: BuildFlags: useccahe:<packagename> example: https://build.opensuse.org/projects/home:sjamgade:branches:science:machinel… Once enabled per package, it will be enabled for all architectures and repositories. https://openbuildservice.org/help/manuals/obs-user-guide/cha.obs.prjconfig.… (grep ccache) --- The ccache tar file is collected only on *successful builds with changed result*. It is extracted for every build at /.ccache/ if enabled. This build artifact is not versioned and only the latest tar is stored. It can be downloaded from the webui or osc. https://build.opensuse.org/package/binaries/home:sjamgade:branches:science:… osc getbinaries --ccache -M standard home:sjamgade:branches:science:machinelearning tensorflow openSUSE_Tumbleweed x86_64 _ccache.tar osc getbinaries --ccache $prj $pkg $repo $arch _ccache.tar This archive can be further used for local builds too. osc build --pkg-ccache /full/path/to/_ccache. (please use the latest osc) --- Ccache config file is stored in /.ccache/ccache.conf and is writable by the build user. So, it could also be disabled (or configured as per package requirements) echo "disable = true" >> /.ccache/ccache.conf For possible build time comparisons: https://build.opensuse.org/packages/tensorflow:standard/job_history/home:sj… https://w3.nue.suse.com/~sjamgade/ccachereport.html Thank you, Sumit Jamgade and OBS Team [1] - https://ccache.dev/ PS: Some packages (eg: tensorflow) need extra plumbing in spec file to enable access to /.ccache and use of ccache cmd: export CC_PREFIX=ccache; ln -s /.ccache/ ~/.ccache -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner(a)opensuse.org

4 months, 4 weeks

[opensuse-buildservice] Short OBS downtime this Saturday

by Michael Schroeder

Hi OBS users, we need to do a little database maintenance on Saturday at 21:00 UTC. The buildservice will not be available for about 30 minutes. The build service will still schedule and finish builds in that time, but the API and the Web UI will not be available. Thanks for your understanding and have a nice weekend, Michael. -- Michael Schroeder SUSE Software Solutions Germany GmbH mls(a)suse.de GF: Felix Imendoerffer HRB 36809, AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner(a)opensuse.org

4 months, 4 weeks

[opensuse-buildservice] handling one thousand source tarballs in one package

by Ludwig Nussel

Hi, I'm trying to build a package from source that uses npm modules. Doing so involves more than one thousand(!) upstream tarballs. In the hope to make that more manageable I wrote a PoC script¹ to produuce a file includable from the spec and to also download the tarballs. Uploading them to OBS as part of the source is rather annoying though as you can't see the forest for all the trees anymore then. So would be convenient if a service like download_files or download_url could do that on server side so the files would be hidden as _service:* AFAICS services do not have access to previously generated results² though so would have to download those files on every source change which seems rather wasteful and also takes time. OTOH there's a hack in OBS to have a cache directory for tar_scm³ only. So would it be possible to either let services access the previous results or have a cache dir for all? Any other ideas how to handle one thousand source tarballs? cu Ludwig [1] https://github.com/lnussel/nodejs-tarballs [2] https://github.com/openSUSE/osc/blob/7b5d10552352295ebdd21193b6dfbf830e7d23… [3] https://github.com/openSUSE/open-build-service/blob/master/src/backend/run-… -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner(a)opensuse.org

7 months, 3 weeks

[opensuse-buildservice] ARM Architectures on OBS Instance

by Kyle Edwards

Hello all, I recently added aarch64 and armv7el architectures to my OBS instance. I'm building for Ubuntu 16.04 and Ubuntu 20.04. However, both aarch64 and armv7el seem to be unavailable for Ubuntu 16.04 - they both say "unresolvable." When I look for more details, I see a long list of "nothing provides <package>." Is there not an aarch64 and armv7el build available for Ubuntu 16.04 on OBS? In addition, for Ubuntu 20.04, armv7el says "The repository setup is broken, build or publish is not possible." What does this mean? So far the only one of these four builds that I've managed to get working is aarch64 on Ubuntu 20.04. Side note: is armv7el the right architecture to use? There's a bunch of different arm-32 architectures, and it's not clear how the OBS arches correspond to the arches for other distros. Kyle -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner(a)opensuse.org

10 months, 2 weeks

[opensuse-buildservice] Need help with debugging an endless loop of events - bs_sched never takes a break!

by Srinidhi B

Hello List, I need some help in debugging a situation in my private OBS (2.9.x) instance that has been going on for a couple of months!! The schedulers keep scanning projects in a loop - never taking a break. On a Sunday, after 12 hours and 20 minutes of runtime, here are some numbers: # grep -ci "calculating project dependencies" /bs/log/scheduler_*.log /bs/log/scheduler_i586.log:19032 /bs/log/scheduler_local.log:10256 /bs/log/scheduler_x86_64.log:18915 # grep -ci "waiting for an event" /bs/log/scheduler_*.log /bs/log/scheduler_i586.log:0 /bs/log/scheduler_local.log:19150 /bs/log/scheduler_x86_64.log:0 I have not managed to find any hints as to what is causing this loop. Any help or pointers would be greatly appreciated! Let me share what I've found / done so far: 0. This problem started after we applied all available updates for SLES12 SP3 on July 31st. We had to, of course, reboot the system after that. I want to plan an upgrade to SLES12 SP5 but the current situation is making it very difficult to plan the upgrade. I just don't know if I'll making things even worse with an upgrade! 1. Dumped scheduler state by running `obs_admin --dump-state x86_64` and looked inside the state file. A few observations: * 'lookat' element has 1,230 items. * 'lookat_oob' element has 11 items. * 'retryevents' element has 66 items. * Number of items in 'prpfinished' and 'prps' do not match. Difference is 106. * 'changed_med' has 2 entries. 2. /bs/events/lastevents is always filled. After stopping all backend services, there are almost 5,000 entries in the file. 3. /srv/www/obs/api/log/backend_access.log has backend calls which do not show up in the production.log file. I thought, all backend calls must go through the API. Since today is Sunday, I suspected that the project consistency check could have been triggered. While it was indeed triggered, that job was finished 4 hours 23 minutes: Performed ConsistencyCheckJob (Job ID: c492970a-bfd7-4683-abf1-816f5eb5a6b6) from DelayedJob(consistency_check) in 259020.22ms 4. I have checked clockworkd.ruby2.5.clock.output to see if there are any strange jobs that are getting triggered. But the problem that I've always faced with this log is that it only logs when a job is triggered while it never contains when a job finished. When I try to run the "jobs:check" command, I get a MySQL error: $ run_in_api rails.ruby2.5 jobs:check rails aborted! ActiveRecord::StatementInvalid: Mysql2::Error: Unknown column 'created_at' in 'where clause': SELECT COUNT(*) FROM `delayed_jobs` WHERE (attempts = 0 AND created_at < '2020-10-25 10:26:42.114022') bin/rails:4:in `require' bin/rails:4:in `<main>' Caused by: Mysql2::Error: Unknown column 'created_at' in 'where clause' bin/rails:4:in `require' bin/rails:4:in `<main>' Tasks: TOP => jobs:check (See full trace by running task with --trace) Clearly, I'm going nowhere with this and I'm desperately trying to improve the situation. Any hints or pointers would be greatly appreciated. Regards, Srinidhi. N��r��y隊Z)z{.��Wlz��qﮞ˛��m�)z{.��+�:�{Zr�az�'z��j)h��Ǜ�)]��Ǿ� ޮ�^�ˬz��

10 months, 2 weeks

[opensuse-buildservice] Strange differences in 15.1 vs. 15.2 OBS builds

by Manfred Hollstein

Hi there, while this appears to be Rust specific, it is not as I've seen similar strange differences in other complex build situations (e.g. MozillaFirefox), too. The build of rust for Leap 15.1 failed due to the following results of RPMLINT <https://build.opensuse.org/package/live_build_log/devel:languages:rust/rust…>: [15022s] rust-src.noarch: E: env-script-interpreter (Badness: 9) /usr/lib/rustlib/src/rust/library/backtrace/ci/android-sdk.sh /usr/bin/env sh [15022s] rust-src.noarch: E: env-script-interpreter (Badness: 9) /usr/lib/rustlib/src/rust/library/stdarch/ci/dox.sh /usr/bin/env bash [15022s] rust-src.noarch: E: env-script-interpreter (Badness: 9) /usr/lib/rustlib/src/rust/library/stdarch/ci/run-docker.sh /usr/bin/env sh [15022s] rust-src.noarch: E: env-script-interpreter (Badness: 9) /usr/lib/rustlib/src/rust/library/stdarch/ci/run.sh /usr/bin/env sh [15022s] rust-src.noarch: E: env-script-interpreter (Badness: 9) /usr/lib/rustlib/src/rust/library/stdarch/ci/style.sh /usr/bin/env sh [15022s] This script uses 'env' as an interpreter. For the rpm runtime dependency [15022s] detection to work, the shebang #!/usr/bin/env python needs to be patched into [15022s] #!/usr/bin/python otherwise the package dependency generator merely adds a [15022s] dependency on /usr/bin/env rather than the actual interpreter /usr/bin/python. [15022s] Alternatively, if the file should not be executed, then ensure that it is not [15022s] marked as executable or don't install it in a path that is reserved for [15022s] executables. [15022s] ... [15022s] 12 packages and 0 specfiles checked; 6 errors, 27 warnings. Interestingly the same messages are shown in the build log for Leap 15.2, but this build succeeded <https://build.opensuse.org/package/live_build_log/devel:languages:rust/rust…> Here there is one error less (due to not having to build the included LLVM, but rather use the system provided one), but still 5 errors. I worked around it (see <https://build.opensuse.org/request/show/845063>), but I don't understand why OBS behaves differently for OSs older than 15.2 and 15.2 onwards (incl. Tumbleweed). Does anyone have an idea? Fix? TIA, cheers. l8er manfred

10 months, 3 weeks

[opensuse-buildservice] How to specify DEB_BUILD_OPTIONS / environment variables

by Harald Welte

Dear OBS list, I've tried to find out how I can set environment variables while building dpkg packages via OBS. I would have assumed that maybe the project config would provide something like that, but I failed to find any. The concrete usage example is to pass something like DEB_BUILD_OPTIONS=parallel=N which then most sane debian packages turn into "-jN" during make. I also grep'ed for DEB_BUILD_OPTIONS in the OBS source code, but there were no hits. Right now I'm watching a 16-core machine building a Linux kernel with "-j1" since I cannot figure out what is the designated way in OBS to specify such environment variables. Thanks in advance for any assistance. Regards, Harald p.s.: I previously had to switch from public build.opensuse.org to a local installation of OBS due to the public build.opensuse.org workers consistently failing builds with "no space left on device" while trying to build a linux kernel package. And I already had stripped it down to only one variant + flavor + architecture. Not a complaint, just noticing it. I would have expected kernel builds not to be that uncommon. -- - Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner(a)opensuse.org

10 months, 3 weeks

[opensuse-buildservice] Trying to add ARM architectures to my OBS installation

by Kyle Edwards

Hello, I am trying to add aarch64 and armv7hl to my OBS server. In the web interface, I check the boxes for aarch64 and armv7hl (and also uncheck the one for armv7l, since it was checked for some reason.) However, the changes don't seem to take - when I leave the page and come back, aarch64 and armv7hl are unchecked, and armv7l is still checked. Am I doing this right? I don't see an "update" button anywhere on the page (and the "update" button on the main configuration tab doesn't seem to help.) Kyle -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner(a)opensuse.org

10 months, 3 weeks

[opensuse-buildservice] Installing RPM fails

by Roger Oberholtzer

I have just built a couple RPMs as a user on OBS. The repository is not a new one. But the package is new. The package built okay. But when I try to install the RPM, I get this: Download (curl) error for 'http://download.opensuse.org/repositories/home:/rogeroberholtzer/openSUSE_L…': Error code: Curl error 60 Error message: SSL certificate problem: self signed certificate in certificate chain I do not know what this means. It happens in more than one environment. So I suspect that something has happened on OBS related to certificates. But I really do not know what. I googled and found things like this: https://support.microfocus.com/kb/doc.php?id=7024620 I tried what it said, but it did not help. Other solutions were less clear. And most were many years old. This problem has just happened. I do not know if it is an OBS issue or a local issue. -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner(a)opensuse.org

10 months, 3 weeks

[opensuse-buildservice] Add Fedora 33 to openSUSE Build Service?

by Neal Gompa

Hey, Fedora 33 is going to be released on Tuesday. Can we get it added to OBS now? -- 真実はいつも一つ！/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner(a)opensuse.org

10 months, 3 weeks

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openSUSE Build Service October 2020