[opensuse-factory] Policy for src.rpm's?

Adrian Schröter

8 Aug 2018 8 Aug '18

10:09

Hi, I read on the wiki that you look for some solution how to avoid additional build time due to tar ball recompression. One way to do it is like in science:unstable/FreeCAD package. It builds directly in the git tree and is just packaging the _service file in src.rpm's. It would have a > 100MB tar.xz file otherwise inside. This us speeding up the build a lot. The disadvantage of that is that you can not do an offline build. And you can't get the code anymore, if the upstream scm server disappears _and_ build.opensuse.org is not working for you anymore. Therefore the distribution source tree alone would not be enough anymore for saitisfing the GPL. Either the upstream project or the OBS instance is required as well. Not sure if this is good enough for you, but I wanted to mention it at least. How does it look in practice? _service file has only the obs_scm and set_version service active. FreeCAD.spec is just packaging the _service file instead of the tar ball. %setup is skipping tar ball extraction via %setup -q -b %_sourcedir/%name-%version -T -D Have a look in the example here, you find this also switchable via build_tar_ball rpm macro: https://build.opensuse.org/package/show/science:unstable/FreeCAD (and yes, someone broke hdf5 stuff again, but does not matter here) bye adrian -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Show replies by date

Stephan Kulow

8 Aug 8 Aug

10:20

On 08/08/2018 12:09 PM, Adrian Schröter wrote:

...

Hi,

I read on the wiki that you look for some solution how to avoid additional build time due to tar ball recompression.

One way to do it is like in science:unstable/FreeCAD package. It builds directly in the git tree and is just packaging the _service file in src.rpm's. It would have a > 100MB tar.xz file otherwise inside.

This us speeding up the build a lot.

The disadvantage of that is that you can not do an offline build.

And you can't get the code anymore, if the upstream scm server disappears _and_ build.opensuse.org is not working for you anymore.

Therefore the distribution source tree alone would not be enough anymore for saitisfing the GPL. Either the upstream project or the OBS instance is required as well. This is a clear violation of GPL version 2: https://www.gnu.org/licenses/gpl-faq#DistributeWithSourceOnInternet

and possibly also for version 3 - but the rules in it are a little more relaxed. The faq has more informations about it. Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Adrian Schröter

11:34

On Mittwoch, 8. August 2018, 12:20:46 CEST wrote Stephan Kulow:

...

On 08/08/2018 12:09 PM, Adrian Schröter wrote:

...
Hi,

I read on the wiki that you look for some solution how to avoid additional build time due to tar ball recompression.

One way to do it is like in science:unstable/FreeCAD package. It builds directly in the git tree and is just packaging the _service file in src.rpm's. It would have a > 100MB tar.xz file otherwise inside.

This us speeding up the build a lot.

The disadvantage of that is that you can not do an offline build.

And you can't get the code anymore, if the upstream scm server disappears _and_ build.opensuse.org is not working for you anymore.

Therefore the distribution source tree alone would not be enough anymore for saitisfing the GPL. Either the upstream project or the OBS instance is required as well. This is a clear violation of GPL version 2: https://www.gnu.org/licenses/gpl-faq#DistributeWithSourceOnInternet

and possibly also for version 3 - but the rules in it are a little more relaxed. The faq has more informations about it.

well, it must be available, but it is not saying that it must be in the src.rpm. We could also think about how to make archives available and rsyncable for the mirrors. -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Stephan Kulow

11:48

On 08/08/2018 01:34 PM, Adrian Schröter wrote:

...

well, it must be available, but it is not saying that it must be in the src.rpm.

We could also think about how to make archives available and rsyncable for the mirrors.

No, v2 clearly states you need to provide binaries and sources the same way. Binaries on DVD => Sources must be available on DVD. A _service file won't do. Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Adrian Schröter

11:55

On Mittwoch, 8. August 2018, 13:48:13 CEST wrote Stephan Kulow:

...

On 08/08/2018 01:34 PM, Adrian Schröter wrote:

...
well, it must be available, but it is not saying that it must be in the src.rpm.

We could also think about how to make archives available and rsyncable for the mirrors.

No, v2 clearly states you need to provide binaries and sources the same way. Binaries on DVD => Sources must be available on DVD. A _service file won't do.

yes, we would need to put the obscpio files also on the source DVD then. And keep in mind that it is also okay just to put a written offer to send the source DVD on request. No one is really enforcing anyone to put the source in the src.rpm as only option. But "the way" in the context of the internet is less clear IIRC. It is for example 100% okay that mirrors do not mirror the sources as long as our main server is still providing it and they point to it. => So as long the upstream project is providing the sources and we point to it it should be okay as well. I agree that the enforcement to create an account for OBS would be a problem as long as the binaries are downloadable without. However, as said before, this could be solved. I just wanted to mention this approach here since the wiki asked for ideas how to avoid the re-compression times. This would be one way and I think the GPL issues could be solved as well... -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Stephan Kulow

11:57

On 08/08/2018 01:55 PM, Adrian Schröter wrote:

...

On Mittwoch, 8. August 2018, 13:48:13 CEST wrote Stephan Kulow:

...
On 08/08/2018 01:34 PM, Adrian Schröter wrote:

...
well, it must be available, but it is not saying that it must be in the src.rpm.

We could also think about how to make archives available and rsyncable for the mirrors.

No, v2 clearly states you need to provide binaries and sources the same way. Binaries on DVD => Sources must be available on DVD. A _service file won't do.

yes, we would need to put the obscpio files also on the source DVD then.

If you do that, you can just as well stop compressing src.rpms and be done with the issue Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Adrian Schröter

12:02

On Mittwoch, 8. August 2018, 13:57:50 CEST wrote Stephan Kulow:

...

On 08/08/2018 01:55 PM, Adrian Schröter wrote:

...
On Mittwoch, 8. August 2018, 13:48:13 CEST wrote Stephan Kulow:

...
On 08/08/2018 01:34 PM, Adrian Schröter wrote:

...
well, it must be available, but it is not saying that it must be in the src.rpm.

We could also think about how to make archives available and rsyncable for the mirrors.

No, v2 clearly states you need to provide binaries and sources the same way. Binaries on DVD => Sources must be available on DVD. A _service file won't do.

yes, we would need to put the obscpio files also on the source DVD then.

If you do that, you can just as well stop compressing src.rpms and be done with the issue

you would still need to tar and untar them. -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Stephan Kulow

12:04

On 08/08/2018 02:02 PM, Adrian Schröter wrote:

...

On Mittwoch, 8. August 2018, 13:57:50 CEST wrote Stephan Kulow:

...
On 08/08/2018 01:55 PM, Adrian Schröter wrote:

...
On Mittwoch, 8. August 2018, 13:48:13 CEST wrote Stephan Kulow:

...
On 08/08/2018 01:34 PM, Adrian Schröter wrote:

...
well, it must be available, but it is not saying that it must be in the src.rpm.

We could also think about how to make archives available and rsyncable for the mirrors.

No, v2 clearly states you need to provide binaries and sources the same way. Binaries on DVD => Sources must be available on DVD. A _service file won't do.

yes, we would need to put the obscpio files also on the source DVD then.

If you do that, you can just as well stop compressing src.rpms and be done with the issue

you would still need to tar and untar them.

Where? I guess I lost you. My point is that if obscpio is fine as 'source deliverable', so should be an uncompressed .tar Greetings, Stephan -- Talking much about oneself can also be a means to conceal oneself. -- Friedrich Nietzsche -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Adrian Schröter

12:10

On Mittwoch, 8. August 2018, 14:04:39 CEST wrote Stephan Kulow:

...

On 08/08/2018 02:02 PM, Adrian Schröter wrote:

...
On Mittwoch, 8. August 2018, 13:57:50 CEST wrote Stephan Kulow:

...
On 08/08/2018 01:55 PM, Adrian Schröter wrote:

...
On Mittwoch, 8. August 2018, 13:48:13 CEST wrote Stephan Kulow:

...
On 08/08/2018 01:34 PM, Adrian Schröter wrote:

...
well, it must be available, but it is not saying that it must be in the src.rpm.

We could also think about how to make archives available and rsyncable for the mirrors.

No, v2 clearly states you need to provide binaries and sources the same way. Binaries on DVD => Sources must be available on DVD. A _service file won't do.

yes, we would need to put the obscpio files also on the source DVD then.

If you do that, you can just as well stop compressing src.rpms and be done with the issue

you would still need to tar and untar them.

Where? I guess I lost you. My point is that if obscpio is fine as 'source deliverable', so should be an uncompressed .tar

at build time, the sources from some SCM are provided as an extracted tree already to the build. But if you want to put them into some source rpm, you need to turn this into a file again. So you would need to tar them. And usually untaring it again as this is what %setup does usually by default. This was intended as a response to the question how to avoid these steps (and recompression). -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Jan Engelhardt

12:05

On Wednesday 2018-08-08 13:57, Stephan Kulow wrote:

...

...
yes, we would need to put the obscpio files also on the source DVD then.

If you do that, you can just as well stop compressing src.rpms and be done with the issue

I'll entertain you for that thought too: %define _source_payload w.ufdio -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Ludwig Nussel

12:35

Adrian Schröter wrote:

...

I read on the wiki that you look for some solution how to avoid additional build time due to tar ball recompression. [...] Have a look in the example here, you find this also switchable via build_tar_ball rpm macro:

https://build.opensuse.org/package/show/science:unstable/FreeCAD

It would make sense for sure nowadays to make the build process smarter wrt dealing with distributed scms. Indeed packing a git tree in a tar, just to extract it for building and then compress it again into a src.rpm adds extra steps. Not just for the cpu but also for the packager who has to figure out how to create a tar from git. Ie in case of OBS needs to learn about the _service extra mechanism. The FreeCAD example doesn't seem to make that any easier. As packager I'd want to specify Name: FreeCAD Version: 0.18 Source: https://github.com/FreeCAD/FreeCAD.git That's it basically. All the rest should be automatic. Can we get to that somehow without extra config files? cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Stephan Kulow

13:02

On 08/08/2018 02:35 PM, Ludwig Nussel wrote:

...

Adrian Schröter wrote:

...
I read on the wiki that you look for some solution how to avoid additional build time due to tar ball recompression. [...] Have a look in the example here, you find this also switchable via build_tar_ball rpm macro:

https://build.opensuse.org/package/show/science:unstable/FreeCAD

It would make sense for sure nowadays to make the build process smarter wrt dealing with distributed scms. Indeed packing a git tree in a tar, just to extract it for building and then compress it again into a src.rpm adds extra steps. Not just for the cpu but also for the packager who has to figure out how to create a tar from git. Ie in case of OBS needs to learn about the _service extra mechanism. The FreeCAD example doesn't seem to make that any easier.

As packager I'd want to specify

Name: FreeCAD Version: 0.18 Source: https://github.com/FreeCAD/FreeCAD.git

That's it basically. All the rest should be automatic. Can we get to that somehow without extra config files?

OK, that's a neat attempt to change the topic - but we were talking about compression of .src.rpm. No matter how you got there: at the end the sources need to be in the src.rpm. I just don't think they need to be compressed (or only very lightly) Greetings, Stephan -- Johnson's law: Systems resemble the organizations that create them. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Stefan Seyfried

13:16

Am 08.08.2018 um 15:02 schrieb Stephan Kulow:

...

OK, that's a neat attempt to change the topic - but we were talking about compression of .src.rpm. No matter how you got there: at the end the sources need to be in the src.rpm.

I don't read that in the GPL. They (a tarball, cpio archive, expanded git checkout, whatever) need to be on the same media set as the src.rpm, but I cannot see how a HOWTO telling "SRPMS/foo.src.rpm needs sourceballs/foo.zip for building" and both delivered on the same DVD / the same would violate the GPL. But if we need to tar it up for the Source DVD anyway, we can just as well just put it into the .src.rpm ;)

...

I just don't think they need to be compressed (or only very lightly)

The first thing I forbid my users is <service mode="buildtime" name="recompress"> <param name="file">*.tar</param> <param name="compression">xz</param> </service> :-) This is the most useless waste of CPU time ever ;-) Unfortunately it's the default when you create a _service via "osc add https://github.com/seife/foo.git" Just add an rpmlint error that chokes on compressed tarballs in Source: statement and keep the .src.rpms compressed ;-) -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Jan Engelhardt

13:25

On Wednesday 2018-08-08 15:16, Stefan Seyfried wrote:

...

Am 08.08.2018 um 15:02 schrieb Stephan Kulow:

...
OK, that's a neat attempt to change the topic - but we were talking about compression of .src.rpm. No matter how you got there: at the end the sources need to be in the src.rpm.

I don't read that in the GPL.

They (a tarball, cpio archive, expanded git checkout, whatever) need to be on the same media set as the src.rpm, but I cannot see how a HOWTO telling "SRPMS/foo.src.rpm needs sourceballs/foo.zip for building" and both delivered on the same DVD / the same would violate the GPL.

But if we need to tar it up for the Source DVD anyway, we can just as well just put it into the .src.rpm ;)

...
I just don't think they need to be compressed (or only very lightly)

The first thing I forbid my users is

<service mode="buildtime" name="recompress"> <param name="file">*.tar</param> <param name="compression">xz</param> </service>

:-) This is the most useless waste of CPU time ever ;-)

Well, for starters, we could switch to zstd. If the colorful graphics at https://raw.githubusercontent.com/facebook/zstd/master/doc/images/DCspeed5.p... [linked from https://facebook.github.io/zstd/] are to be believed, you get ~4-fold the speed for the same ratio. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Adrian Schröter

14:30

On Mittwoch, 8. August 2018, 15:16:05 CEST wrote Stefan Seyfried:

...

Am 08.08.2018 um 15:02 schrieb Stephan Kulow:

...
OK, that's a neat attempt to change the topic - but we were talking about compression of .src.rpm. No matter how you got there: at the end the sources need to be in the src.rpm.

I don't read that in the GPL.

They (a tarball, cpio archive, expanded git checkout, whatever) need to be on the same media set as the src.rpm, but I cannot see how a HOWTO telling "SRPMS/foo.src.rpm needs sourceballs/foo.zip for building" and both delivered on the same DVD / the same would violate the GPL.

But if we need to tar it up for the Source DVD anyway, we can just as well just put it into the .src.rpm ;)

...
I just don't think they need to be compressed (or only very lightly)

The first thing I forbid my users is

<service mode="buildtime" name="recompress"> <param name="file">*.tar</param> <param name="compression">xz</param> </service>

:-) This is the most useless waste of CPU time ever ;-)

well, putting plenty of uncompressed tar balls on all of our mirrors is the most useless waste of storage space ever ;) -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Stephan Kulow

9 Aug 9 Aug

10:26

On 08/08/2018 04:30 PM, Adrian Schröter wrote:

...

On Mittwoch, 8. August 2018, 15:16:05 CEST wrote Stefan Seyfried:

...
Am 08.08.2018 um 15:02 schrieb Stephan Kulow:

...
OK, that's a neat attempt to change the topic - but we were talking about compression of .src.rpm. No matter how you got there: at the end the sources need to be in the src.rpm.

I don't read that in the GPL.

They (a tarball, cpio archive, expanded git checkout, whatever) need to be on the same media set as the src.rpm, but I cannot see how a HOWTO telling "SRPMS/foo.src.rpm needs sourceballs/foo.zip for building" and both delivered on the same DVD / the same would violate the GPL.

But if we need to tar it up for the Source DVD anyway, we can just as well just put it into the .src.rpm ;)

...
I just don't think they need to be compressed (or only very lightly)

The first thing I forbid my users is

<service mode="buildtime" name="recompress"> <param name="file">*.tar</param> <param name="compression">xz</param> </service>

:-) This is the most useless waste of CPU time ever ;-)

well, putting plenty of uncompressed tar balls on all of our mirrors is the most useless waste of storage space ever ;)

There is a reason the sources are in /sources and - most mirrors don't even fetch them. As they are so seldomly downloaded, there is little interest to mirror it. Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Michal Kubecek

8 Aug 8 Aug

17:20

On Wed, Aug 08, 2018 at 03:16:05PM +0200, Stefan Seyfried wrote:

...

Am 08.08.2018 um 15:02 schrieb Stephan Kulow:

...
OK, that's a neat attempt to change the topic - but we were talking about compression of .src.rpm. No matter how you got there: at the end the sources need to be in the src.rpm.

I don't read that in the GPL.

They (a tarball, cpio archive, expanded git checkout, whatever) need to be on the same media set as the src.rpm, but I cannot see how a HOWTO telling "SRPMS/foo.src.rpm needs sourceballs/foo.zip for building" and both delivered on the same DVD / the same would violate the GPL.

Actually, .src.rpm are often of limited usefulness on their own. Our build process relies on carefully prepared build environment, both w.r.t. what is installed and what is _not_. Some of are packages won't even build when you try rpmbuild on the .src.rpm. Michal Kubecek -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Roger Oberholtzer

9 Aug 9 Aug

08:02

On Wed, Aug 8, 2018 at 7:21 PM Michal Kubecek wrote:

...

Actually, .src.rpm are often of limited usefulness on their own. Our build process relies on carefully prepared build environment, both w.r.t. what is installed and what is _not_. Some of are packages won't even build when you try rpmbuild on the .src.rpm.

I regularly use the .src RPM to build for targets that are not available on OBS. Quite often rpmbuild works out of the box. If not, I still have the complete source with patches and the spec file to make any needed modifications. I find them extremely useful! They have gotten me out of a bind more than once. -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Ludwig Nussel

8 Aug 8 Aug

14:44

Stephan Kulow wrote:

...

On 08/08/2018 02:35 PM, Ludwig Nussel wrote:

...
Adrian Schröter wrote:

...
I read on the wiki that you look for some solution how to avoid additional build time due to tar ball recompression. [...] Have a look in the example here, you find this also switchable via build_tar_ball rpm macro:

https://build.opensuse.org/package/show/science:unstable/FreeCAD

It would make sense for sure nowadays to make the build process smarter wrt dealing with distributed scms. Indeed packing a git tree in a tar, just to extract it for building and then compress it again into a src.rpm adds extra steps. Not just for the cpu but also for the packager who has to figure out how to create a tar from git. Ie in case of OBS needs to learn about the _service extra mechanism. The FreeCAD example doesn't seem to make that any easier.

As packager I'd want to specify

Name: FreeCAD Version: 0.18 Source: https://github.com/FreeCAD/FreeCAD.git

That's it basically. All the rest should be automatic. Can we get to that somehow without extra config files?

OK, that's a neat attempt to change the topic - but we were talking about compression of .src.rpm.

Oh really? I thought everyone picks a subtopic for their part of the thread. And you jumped on the GPL compliance :-)

...

No matter how you got there: at the end the sources need to be in the src.rpm.

I agree but that part could hopefully be done in a smarter way than obscpio.

...

I just don't think they need to be compressed (or only very lightly)

Sure, if the content is already compressed. If it's an uncompressed directory tree like a git checkout, compressing that one way or another would still make sense. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Adrian Schröter

13:03

On Mittwoch, 8. August 2018, 14:35:34 CEST wrote Ludwig Nussel:

...

Adrian Schröter wrote:

...
I read on the wiki that you look for some solution how to avoid additional build time due to tar ball recompression. [...] Have a look in the example here, you find this also switchable via build_tar_ball rpm macro:

https://build.opensuse.org/package/show/science:unstable/FreeCAD

It would make sense for sure nowadays to make the build process smarter wrt dealing with distributed scms. Indeed packing a git tree in a tar, just to extract it for building and then compress it again into a src.rpm adds extra steps. Not just for the cpu but also for the packager who has to figure out how to create a tar from git. Ie in case of OBS needs to learn about the _service extra mechanism. The FreeCAD example doesn't seem to make that any easier.

well, yes, but please note that this is a limitation of rpmbuild in first place. It requires files for the src.rpm's. OBS is happy to skip them... You could even go that far and say that we won't ship any src.rpm's at all anymore. Just the sources as they come out of a package checkout. That would require some support in OBS to prepare the published sources, but no big deal. It is more the matter of question how you distro people want to ship the sources....

...

As packager I'd want to specify

Name: FreeCAD Version: 0.18 Source: https://github.com/FreeCAD/FreeCAD.git

That's it basically. All the rest should be automatic. Can we get to that somehow without extra config files?

Well, you would need to enhance rpmbuild here. And you need to think about the special cases. We could of course also avoid rpmbuild changes and adapt the obs_scm service that it is parsing the spec file, downloading and rewriting the spec file. Not sure if that is a good idea though, since it brings complexity for special cases (eg. submodules yes/no), multiple spec files or other build descriptions. -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Adrian Schröter

13:17

On Mittwoch, 8. August 2018, 15:03:30 CEST wrote Adrian Schröter:

...

On Mittwoch, 8. August 2018, 14:35:34 CEST wrote Ludwig Nussel:

...
Adrian Schröter wrote:

...
I read on the wiki that you look for some solution how to avoid additional build time due to tar ball recompression. [...] Have a look in the example here, you find this also switchable via build_tar_ball rpm macro:

https://build.opensuse.org/package/show/science:unstable/FreeCAD

It would make sense for sure nowadays to make the build process smarter wrt dealing with distributed scms. Indeed packing a git tree in a tar, just to extract it for building and then compress it again into a src.rpm adds extra steps. Not just for the cpu but also for the packager who has to figure out how to create a tar from git. Ie in case of OBS needs to learn about the _service extra mechanism. The FreeCAD example doesn't seem to make that any easier.

well, yes, but please note that this is a limitation of rpmbuild in first place. It requires files for the src.rpm's. OBS is happy to skip them...

You could even go that far and say that we won't ship any src.rpm's at all anymore. Just the sources as they come out of a package checkout.

That would require some support in OBS to prepare the published sources, but no big deal. It is more the matter of question how you distro people want to ship the sources....

taking this last part a bit back ... enhancing product-builder might require quite some effort. But it may be even possible to avoid that and publish all sources independend of the products for openSUSE... -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Ludwig Nussel

14:48

Adrian Schröter wrote:

...

On Mittwoch, 8. August 2018, 14:35:34 CEST wrote Ludwig Nussel:

...
[...] As packager I'd want to specify

Name: FreeCAD Version: 0.18 Source: https://github.com/FreeCAD/FreeCAD.git

That's it basically. All the rest should be automatic. Can we get to that somehow without extra config files?

Well, you would need to enhance rpmbuild here.

Exactly. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Andreas Schwab

15:44

New subject: [opensuse-factory] Re: Policy for src.rpm's?

On Aug 08 2018, Adrian Schröter wrote:

...

FreeCAD.spec is just packaging the _service file instead of the tar ball.

%setup is skipping tar ball extraction via %setup -q -b %_sourcedir/%name-%version -T -D

Have a look in the example here, you find this also switchable via build_tar_ball rpm macro:

https://build.opensuse.org/package/show/science:unstable/FreeCAD

You have disabled debuginfo, otherwise you would have noticed that this breaks the generation of debuginfo packages. [ 1162s] + /usr/lib/rpm/find-debuginfo.sh -j8 --build-id-seed 2.28.9000.25.gb5403eca16-1632.1 --unique-debug-suffix -2.28.9000.25.gb5403eca16-1632.1.i386 --unique-debug-src-base glibc-2.28.9000.25.gb5403eca16-1632.1.i386 -S debugsourcefiles.list /home/abuild/rpmbuild/BUILD//home/abuild/rpmbuild/SOURCES/glibc-2.28.9000.25.gb5403eca16 [ 1162s] /usr/lib/rpm/find-debuginfo.sh: line 223: /home/abuild/rpmbuild/BUILD//home/abuild/rpmbuild/SOURCES/glibc-2.28.9000.25.gb5403eca16/debugsources.list: No such file or directory [ 1162s] /usr/lib/rpm/find-debuginfo.sh: line 224: /home/abuild/rpmbuild/BUILD//home/abuild/rpmbuild/SOURCES/glibc-2.28.9000.25.gb5403eca16/debugfiles.list: No such file or directory [ 1162s] /usr/lib/rpm/find-debuginfo.sh: line 225: /home/abuild/rpmbuild/BUILD//home/abuild/rpmbuild/SOURCES/glibc-2.28.9000.25.gb5403eca16/debuglinks.list: No such file or directory [ 1162s] /usr/lib/rpm/find-debuginfo.sh: line 226: /home/abuild/rpmbuild/BUILD//home/abuild/rpmbuild/SOURCES/glibc-2.28.9000.25.gb5403eca16/elfbins.list: No such file or directory [ 1221s] Processing files: glibc-debugsource-2.28.9000.25.gb5403eca16-1632.1.i586 [ 1221s] error: Could not open %files file /home/abuild/rpmbuild/BUILD/home/abuild/rpmbuild/SOURCES/glibc-2.28.9000.25.gb5403eca16/debugsourcefiles.list: No such file or directory Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Adrian Schröter

9 Aug 9 Aug

05:50

New subject: [opensuse-factory] Re: Policy for src.rpm's?

On Mittwoch, 8. August 2018, 17:44:24 CEST wrote Andreas Schwab:

...

On Aug 08 2018, Adrian Schröter wrote:

...
FreeCAD.spec is just packaging the _service file instead of the tar ball.

%setup is skipping tar ball extraction via %setup -q -b %_sourcedir/%name-%version -T -D

Have a look in the example here, you find this also switchable via build_tar_ball rpm macro:

https://build.opensuse.org/package/show/science:unstable/FreeCAD

You have disabled debuginfo, otherwise you would have noticed that this breaks the generation of debuginfo packages.

true, but can be fixed. -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Simon Lees

10 Aug 10 Aug

03:23

On 08/08/18 19:39, Adrian Schröter wrote:

...

Hi,

I read on the wiki that you look for some solution how to avoid additional build time due to tar ball recompression.

One way to do it is like in science:unstable/FreeCAD package. It builds directly in the git tree and is just packaging the _service file in src.rpm's. It would have a > 100MB tar.xz file otherwise inside.

This us speeding up the build a lot.

The disadvantage of that is that you can not do an offline build.

And you can't get the code anymore, if the upstream scm server disappears _and_ build.opensuse.org is not working for you anymore.

Therefore the distribution source tree alone would not be enough anymore for saitisfing the GPL. Either the upstream project or the OBS instance is required as well.

Not sure if this is good enough for you, but I wanted to mention it at least.

How does it look in practice?

_service file has only the obs_scm and set_version service active.

FreeCAD.spec is just packaging the _service file instead of the tar ball.

%setup is skipping tar ball extraction via %setup -q -b %_sourcedir/%name-%version -T -D

Have a look in the example here, you find this also switchable via build_tar_ball rpm macro:

https://build.opensuse.org/package/show/science:unstable/FreeCAD

(and yes, someone broke hdf5 stuff again, but does not matter here)

bye adrian

My team is also interested in faster build on obs, in the coming weeks i'll be looking at finishing a patchset that will allow rpmbuild to do all the post build steps in parallel, ie building multiple sub rpm's at once given that most of our larger packages ship multiple rpm's this should also help speed up many builds, I can't remember if the src and debug rpm's will also be built in parallel. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

2084

Age (days ago)

2086

Last active (days ago)

List overview

Download

24 comments

9 participants

participants (9)

Adrian Schröter
Andreas Schwab
Jan Engelhardt
Ludwig Nussel
Michal Kubecek
Roger Oberholtzer
Simon Lees
Stefan Seyfried
Stephan Kulow

[opensuse-factory] Policy for src.rpm's?

tags

participants (9)