Hi,
I'm trying to build a package from source that uses npm modules. Doing
so involves more than one thousand(!) upstream tarballs. In the hope to
make that more manageable I wrote a PoC script¹ to produuce a file
includable from the spec and to also download the tarballs. Uploading
them to OBS as part of the source is rather annoying though as you can't
see the forest for all the trees anymore then.
So would be convenient if a service like download_files or download_url
could do that on server side so the files would be hidden as
_service:*
AFAICS services do not have access to previously generated results²
though so would have to download those files on every source change
which seems rather wasteful and also takes time.
OTOH there's a hack in OBS to have a cache directory for tar_scm³ only.
So would it be possible to either let services access the previous
results or have a cache dir for all? Any other ideas how to handle one
thousand source tarballs?
cu
Ludwig
[1] https://github.com/lnussel/nodejs-tarballs
[2]
https://github.com/openSUSE/osc/blob/7b5d10552352295ebdd21193b6dfbf830e7d23…
[3]
https://github.com/openSUSE/open-build-service/blob/master/src/backend/run-…
--
(o_ Ludwig Nussel
//\
V_/_ http://www.suse.com/
SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer
HRB 36809 (AG Nürnberg)
--
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org
To contact the owner, e-mail: opensuse-buildservice+owner(a)opensuse.org
Hello,
Long running compilation can now leverage ccache[1] to speed up the
build process. This feature can be enabled per package in the project
config:
BuildFlags: useccahe:<packagename>
example:
https://build.opensuse.org/projects/home:sjamgade:branches:science:machinel…
Once enabled per package, it will be enabled for all architectures and
repositories.
https://openbuildservice.org/help/manuals/obs-user-guide/cha.obs.prjconfig.…
(grep ccache)
---
The ccache tar file is collected only on *successful builds with changed
result*. It is extracted for every build at /.ccache/ if enabled.
This build artifact is not versioned and only the latest tar is stored.
It can be downloaded from the webui or osc.
https://build.opensuse.org/package/binaries/home:sjamgade:branches:science:…
osc getbinaries --ccache -M standard
home:sjamgade:branches:science:machinelearning tensorflow
openSUSE_Tumbleweed x86_64 _ccache.tar osc getbinaries --ccache $prj
$pkg $repo $arch _ccache.tar
This archive can be further used for local builds too.
osc build --pkg-ccache /full/path/to/_ccache.
(please use the latest osc)
---
Ccache config file is stored in /.ccache/ccache.conf and is writable by
the build user. So, it could also be disabled (or configured as per
package requirements)
echo "disable = true" >> /.ccache/ccache.conf
For possible build time comparisons:
https://build.opensuse.org/packages/tensorflow:standard/job_history/home:sj…https://w3.nue.suse.com/~sjamgade/ccachereport.html
Thank you,
Sumit Jamgade and OBS Team
[1] - https://ccache.dev/
PS:
Some packages (eg: tensorflow) need extra plumbing in spec file to
enable access to /.ccache and use of ccache cmd:
export CC_PREFIX=ccache;
ln -s /.ccache/ ~/.ccache
--
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org
To contact the owner, e-mail: opensuse-buildservice+owner(a)opensuse.org
Hi!
All packages in d:l:R:autoCran turned to state unresolvable
because there is choice :-) :-(
unresolvable: have choice for xorg-x11-fonts-100dpi needed by R-base:
xorg-x11-fonts xorg-x11-fonts-legacy have choice for
xorg-x11-fonts-75dpi needed by R-base: xorg-x11-fonts
xorg-x11-fonts-legacy have choice for python3 needed by
texlive-filesystem: python3 python38
Is this temporarily or will this persist? Do I need to recreate 13556
spec files or not?
In case this change is here to stay: How could I have known? How could
I have found out beforehand, that such a change is imminent?
Where can I read what I have to do now to make the packages build again?
All the best
Detlef
Hello,
The SSL certificate I was using on my OBS server recently expired, and
had to be replaced with a renewed certificate yesterday. However, even
with the renewed certificate, I am having trouble getting packages to
build. In the scheduler log, I see the following:
2020-11-19 15:13:43: [1620] looking at high prio cmake/xUbuntu_16.04
(1/0/2/0/2)
- cmake/xUbuntu_16.04
scanning repo cmake/xUbuntu_16.04...
sorting 2 packages
checking packages
- cmake-xenial (500 remote error: Internal Server Error)
broken: 1
disabled: 1
building: 0, notready: 0, unfinished: 0
took 0 seconds to check the packages
leaf prp, freeing data
2020-11-19 15:13:44: [1620] looking at high prio cmake/xUbuntu_20.04
(0/0/2/0/2)
- cmake/xUbuntu_20.04
scanning repo cmake/xUbuntu_20.04...
sorting 2 packages
checking packages
- cmake-focal (interconnect error: rpc timeout)
broken: 1
disabled: 1
building: 0, notready: 0, unfinished: 0
took 0 seconds to check the packages
leaf prp, freeing data
2020-11-19 15:13:44: [1620] looking at low prio cmake/xUbuntu_16.04
(0/0/1/0/2)
- cmake/xUbuntu_16.04
scanning repo cmake/xUbuntu_16.04...
sorting 2 packages
checking packages
- cmake-xenial (500 remote error: Internal Server Error)
broken: 1
disabled: 1
building: 0, notready: 0, unfinished: 0
took 0 seconds to check the packages
leaf prp, freeing data
2020-11-19 15:13:44: [1620] looking at low prio cmake/xUbuntu_20.04
(0/0/0/0/2)
- cmake/xUbuntu_20.04
scanning repo cmake/xUbuntu_20.04...
sorting 2 packages
checking packages
- cmake-focal (interconnect error: rpc timeout)
broken: 1
disabled: 1
building: 0, notready: 0, unfinished: 0
took 0 seconds to check the packages
leaf prp, freeing data
Kyle
Hi!
In recent days we've seen a many hours of downtime of not only
download.opensuse.org but also mirrors.opensuse.org and
status.opensuse.org I'm not complaining, just stating the facts.
This caused severe downstream problems for e.g. any CI/CD jobs that try
to use the related packages, or builds of Dockerfiles failed, etc.
So I thought, in order to be better prepared for possible future
incidents, I'd want to create a mirror for the network:osmocom project
and it's sub-projects.
According to https://en.opensuse.org/openSUSE:Mirror_infrastructure
the 'buildservice-repos' module on rsync.opensuse.org contains "The
complete content".
However, much to my surprise, when I call
"rsync rsync.opensuse.org::buildservice-repos/network:/osmocom:/nightly/"
I only see some of the distributions (not even all OpenSuSE and
Raspobian_10), but not the majority of them (like all our Debian,
xUbuntu CentOS builds).
This is quite strange, and I'm somewhat surprised given that it's
supposedly "The complete content".
Is this intentional? Then the wiki page with the description should
probably be updated. And in this case: is there some other rsync
module that one can use to mirror all of the content of one's project on
build.opensuse.org?
Or is it a bug? In that case, it would be great if somebody could look
into it.
Thanks to roox I found a wokr-around: rsync from one of the [few]
mirrors that have all of the OBS builds ("repositories") and themselves
offer rsync access. Interestingly, that works. But then why do those
mirrors get "all" of the content, while the official rsync archive
doesn't have it?
Here the respective output:
-----
$ rsync rsync.opensuse.org::buildservice-repos/network:/osmocom:/nightly/
drwxr-xr-x 4,096 2020/10/29 14:50:01 .
drwxr-xr-x 8,192 2020/11/18 11:02:51 Raspbian_10
drwxr-xr-x 121 2020/11/21 02:40:54 openSUSE_Leap_15.1
drwxr-xr-x 141 2020/11/21 04:51:29 openSUSE_Leap_15.1_ARM
drwxr-xr-x 121 2020/11/21 02:40:58 openSUSE_Leap_15.2
drwxr-xr-x 137 2020/11/21 02:41:04 openSUSE_Tumbleweed
-----
vs.
-----
rsync rsync://ftp.gwdg.de/pub/opensuse/repositories/network:/osmocom:/nightly/
drwxr-xr-x 4,096 2020/10/29 14:50:01 .
drwxr-xr-x 134 2020/11/21 04:58:31 CentOS_8_Stream
drwxr-xr-x 8,192 2020/11/21 06:26:01 Debian_10
drwxr-xr-x 8,192 2020/11/21 02:52:28 Debian_8.0
drwxr-xr-x 8,192 2020/11/21 06:27:13 Debian_9.0
drwxr-xr-x 8,192 2020/11/21 05:03:27 Debian_Testing
drwxr-xr-x 8,192 2020/11/21 04:38:53 Debian_Unstable
drwxr-xr-x 12,288 2020/11/18 11:02:51 Raspbian_10
drwxr-xr-x 135 2020/11/20 03:01:23 openSUSE_Factory_ARM
drwxr-xr-x 116 2020/11/21 02:40:54 openSUSE_Leap_15.1
drwxr-xr-x 135 2020/11/21 04:51:29 openSUSE_Leap_15.1_ARM
drwxr-xr-x 116 2020/11/21 02:40:58 openSUSE_Leap_15.2
drwxr-xr-x 131 2020/11/21 02:41:04 openSUSE_Tumbleweed
drwxr-xr-x 8,192 2020/10/29 04:09:53 xUbuntu_16.04
drwxr-xr-x 8,192 2020/11/21 02:57:04 xUbuntu_18.04
drwxr-xr-x 8,192 2020/10/29 04:04:55 xUbuntu_18.10
drwxr-xr-x 8,192 2020/11/21 02:54:52 xUbuntu_19.04
drwxr-xr-x 8,192 2020/11/21 02:59:10 xUbuntu_19.10
drwxr-xr-x 8,192 2020/11/21 03:08:48 xUbuntu_20.04
drwxr-xr-x 8,192 2020/11/21 03:09:06 xUbuntu_20.10
-----
Compare the list of distros at
rsync -av --delete rsync://ftp.gwdg.de/pub/opensuse/repositories/network:/osmocom: /external/packages/repositories/network\:/
Regards,
Harald
--
- Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
Hi all,
I've checked out the user interface overhaul behind the beta switch,
and I have to comment on a specific point: the placement of the
"actions on this page" in the sidebar.
It is quite understandable to have the "global" actions, e.g. "Places",
in the sidebar, which constitutes the top level of the page's
"navigational hierarchy".
However, it is very puzzling to have to find the actions that pertain
to *usually just a tab* of the current page (but not all of them!) there.
I understand the desire to fill the sidebar, but it is really breaking
the hierarchy.
To make it clearer what I mean, let's take the overview page for a
package, where the sidebar has actions such as "delete package".
The tab bar at the top changes what actions are available, but is
below the sidebar in the hierarchy. On the other hand, there are
also a lot of actions which are *not* in the sidebar, such as
"Download package" and "Add file".
On other tabs such as "Users", most of the actions (editing, deleting)
are inline, while the "add" action is in the sidebar, quite far away.
It is, all in all, quite hard to see why some actions have been singled
out and moved to a completely different location, while others have
stayed. (From a user point of view; I can guess the technical side.)
Thanks for your consideration,
Georg
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------