[opensuse-buildservice] Re: New repo for mass building of packages from CRAN

Hi Detlef, I would suggest to throttle the initial upload a little bit (like you said maybe to every second minute if that is suitable) in order to minimize the chance of potential issues. In case we detect any considerable performance losses related to it, you will hear back. I wish you success! Regards, Lukas On 10/26/20 10:37 AM, Detlef Steuer wrote:

*Ping*

If I do not hear back anything, I'll start producing R-Rpms tomorrow.

All the best Detlef

Am Thu, 22 Oct 2020 11:37:50 +0200 schrieb Detlef Steuer :

...

Hi Adrian,

I have created d:l:R:autoCRAN to hold the generated packages and bring most of cran.r-project.org to all users of SuSE Linux.

Before I start populating that directory I want to state my plans and ask if such a usage is ok with OBS.

At the moment cran holds 16.5k libraries for the R language, around 11k libraries can be created already automatically, but I'm actively improving the script and hope to reach 90% of CRAN soon(tm).

That means there will be >= 15k packages in the repo with some heavy dependencies and huge recompiles when a core package changes. My experience in the home:dsteuer repo showed that the packages compile relatively fast on OBS' big machines. Therefore I don't expect trouble for OBS.

All packages will be compiled successfully outside of OBS before uploading (modulo bugs in the scripts, of course). So I don't expect too much bitrot there.

All packages will be kept uptodate through cron jobs outside of OBS and will only be uploaded after the updates compile successfully, too. I expect 20 to 50 updates a day after the initial upload.

Following our former discussions: is it ok for me to push CRAN to OBS, even with this number of packages?

Should I throttle the initial upload to a package a minute or every second minute or is it ok to push after each build? My guess is around two packages a minute for a few days, if I push after each successful build. Any hints from your side?

Of course I expect to be stopped doing the upload, if OBS has any problems with the number of packages. But I'm optimistic. Until I hit the wall of 10k packages in a home project everything went smoothly :-)

For a start I added Tumbleweed, SLE_15_SP2 and Leap_15.2 as targets. If you think it is sensible, I can use only Tumbleweed for a start. After filling the repo I would like to add all SuSE releases for which R-base builds.

Regards Detlef

-- Lukas Krause, Build Solutions SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org