[opensuse-factory] Re: [opensuse-project] opensuse source?

newer
[opensuse-factory] yast backup tmp...

Rob OpenSuSE

5 Dec 2009 5 Dec '09

09:28

Am including Factory list, perhaps in 11.3 a better way of making source available to end users, which avoids big load on Build Service and Mirrors is possible? On project list, a request was made for source by someone unfamiliar with openSUSE suggested an ISO; installing source rpm's with YaST is not working as it did in past for me, because only binary packages are held on most of the download.opensuse.org mirrors. 2009/12/4 Peter Pöml :

...

Am 04.12.2009 um 09:56 schrieb Karsten König:

...
...
All of that is not doable via offering source rpms, so, maybe we can get rid of src rpms at all later ...

Currently osc/obs webfrontend is for novell accounts only, and also a seperate tool if one wants to really see the history of the package (osc)

I agree, that's really bad.

...
So the srpms fill an important niche, people from upstream projects getting bugreports can simply go ahead, download the source rpm and check the .spec file for obvious mistakes in the building process and/or check the patches applied.

Before dumping source rpms for saving space, which is somewhat questionable looking at the huge space and traffic the home: repositories generate, people need an anonymous read access, I don't think any other distribution forces something like this on external people.

Source RPMs have a special drawback in the build service: they are published as many times as there are build targets, with slight variations only. See http://lists.opensuse.org/opensuse-buildservice/2009-06/msg00169.html

Well, that's were 30-50% of the build service disk space goes!

This is a similar percentage of space as the home: namespace. It is significant :-)

Excluding build service source RPMs from mirroring is an important instrument to find fmirrors at all.

So problems are : 1) source rpm's are changed and re-published over frequently by Build service 2) source rpm take huge diskspace and bandwidth, for something rarely required on installed system 3) obtaining the source to build and modify a package on end-user system (as required by GPL) is less obvious than it was in past Can we simply have seperate hosts in URI's for binary download, binary debug & source packages? What about a way to install all the source of used packages? Obtaining all the project source is handled by snail mail request for a (possibly chargeable) DVD. Rob -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Show replies by date

Dave Plater

5 Dec 5 Dec

09:42

On 12/05/2009 11:28 AM, Rob OpenSuSE wrote:

...

Am including Factory list, perhaps in 11.3 a better way of making source available to end users, which avoids big load on Build Service and Mirrors is possible?

On project list, a request was made for source by someone unfamiliar with openSUSE suggested an ISO; installing source rpm's with YaST is not working as it did in past for me, because only binary packages are held on most of the download.opensuse.org mirrors.

Now you mention it, I haven't been able to install source rpms with yast for ages, I think SuSE 7.3 was the last time or 9.1, this is actually a much overlooked bug or lacking feature. I normally use "zypper si" to install source rpms. Regards Dave P -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Peter Pöml

10:47

Hi, Am 05.12.2009 um 10:28 schrieb Rob OpenSuSE:

...

Am including Factory list, perhaps in 11.3 a better way of making source available to end users, which avoids big load on Build Service and Mirrors is possible?

Uhm, these are two separate issues: - installability of source RPMs on released openSUSE (which you ran into) - the multiplication of sources in the build service (which I mentioned just because Karsten mentioned the size of the home: namespace in the build service; I tried to set into perspective where most of the space goes.)

...

On project list, a request was made for source by someone unfamiliar with openSUSE suggested an ISO; installing source rpm's with YaST is not working as it did in past for me, because only binary packages are held on most of the download.opensuse.org mirrors.

Since the distribution of downloads to mirrors is transparent, well, just let it work for you. And even if all mirrors would stop mirroring source rpms - that wouldn't change their general availability, because we still offer them, and it wouldn't change the way how YaST installs them.

...

So problems are :

1) source rpm's are changed and re-published over frequently by Build service

That problem affects only the build service - their infrastructure needs more disks, but that's bout it. It shouldn't bother you at least. It's not about re-publishing by the way. And not about the fact that they change often. It's about duplication. (To fully understand the implications, and in how far they matter, you'll have to read the complete thread that I referenced.) To give you an example: each source RPMs contains a tarball (the packed sources from the upstream project), a build description, and possible bits like configuration and stuff. If you build Apache, there is a tarball called httpd-2.2.14.tar.bz2 used in the build, and it is included into the source RPM. Since the tarball is about 5MB in size, the size of the source RPM is largely dictated by that. Now, if you build Apache for openSUSE 11.0, 11.1, 11.2, CentOS5 and Factory, you'll create 5 source RPMs, and each will contain the exact same httpd tarball from upstream. That's 25MB all out of a sudden. And before you even notice it, you may have a whole GB of tarballs just from Apache builds, all containing a tarball that is 5MB in size and can be downloaded from upstream at any time. Well, and the build service file tree is consisting of such duplicated sources to a large extent (I estimated 30%-50%). That doesn't affect users, though - download.opensuse.org offers all that for download, because that's the way things should be - but of course nobody wants the stuff. So, this extreme duplication of sources in the build service is a problem for itself, but it has no effect on users. And the sources that developers are working with are not the source RPMs (which are just a fallout from the build system by automatism), but the original sources. Adrian surely has a point in mentioning that these source RPMs (those of the build service!!) are redundant and utterly useless. It is fully reasonable to suggest getting rid of them _under_ the condition that the original sources are made available anonymously (which isn't the case yet). I have raised the same issue myself in the past. But there are also some arguments against it - the other thread should clarify most of those points.

...

2) source rpm take huge diskspace and bandwidth, for something rarely required on installed system

See 1), this affects the build service-internal infrastructure, but it's built to handle it. (Which doesn't exclude thinking about optimizations of course :-))

...

3) obtaining the source to build and modify a package on end-user system (as required by GPL) is less obvious than it was in past

I can't judge - I don't use YaST or something. However the sources are all there, which we probably all agree on. They are accessible through various means (even mirrored, and free to be mirrored by everybody with no restrictions and no exceptions); there's the convenient build service for those who want to work with the sources seriously enough to bother creating a Novell account, and there is an option in YaST (of arguably operability) that makes it easy for everybody installing to install the "feel-good-factor" as well. From what people wrote, there's even a snail mailed DVD, although I cannot confirm its existance. How obvious all this is - hard to say, but I think, who he who asks will get the helpful pointers at least ;-) And if you have ideas how to make things more obvious, I'm sure they are welcome'd by everybody else. There is always somebody who will benefit from it.

...

Can we simply have seperate hosts in URI's for binary download, binary debug & source packages?

Do you think that http://download-sources.opensuse.org/ would be better than http://download.opensuse.org/source/ ? I don't think so.

...

What about a way to install all the source of used packages?

I never use it, but I thought there'd be an option for this in the YaST package manager. If that doesn't work, then check bugzilla and open a bug if needed. If the feature doesn't exist at all, you could head over to the folks at zypp-devel@opensuse.org and/or open a feature on http://features.opensuse.org/ , right?

...

Obtaining all the project source is handled by snail mail request for a (possibly chargeable) DVD.

Rob

Peter

Rob OpenSuSE

12:46

Though this has got long, I actually think very small simple things would suffice, like a "ReadmeSource" in the right place.. 2009/12/5 Peter Pöml :

...

Am 05.12.2009 um 10:28 schrieb Rob OpenSuSE:

...
Am including Factory list, perhaps in 11.3 a better way of making source available to end users, which avoids big load on Build Service and Mirrors is possible?

Uhm, these are two separate issues: - installability of source RPMs on released openSUSE (which you ran into)

- the multiplication of sources in the build service (which I mentioned just because Karsten mentioned the size of the home: namespace in the build service; I tried to set into perspective where most of the space goes.)

Yes, I mentioned as Adrian was talked about dropping source rpm's. Perhaps he meant in very specific area, where they're redundant and not useful but I had a more general impression from the context of the discussion. Personally I think a disconnected installation ought to be (at least in theory) maintainable independant without requiring external access; it is a USP of FOSS over closed source. Sometimes it gets forgotten but "End of LIfe" (recently openSUSE 10.3) does not mean that OS stops working, or unable to function as well as it did in past. The main issue is noone patches security vulnerabilities.

...

...
On project list, a request was made for source by someone unfamiliar with openSUSE suggested an ISO; installing source rpm's with YaST is not working as it did in past for me, because only binary packages are held on most of the download.opensuse.org mirrors.

Since the distribution of downloads to mirrors is transparent, well, just let it work for you. And even if all mirrors would stop mirroring source rpms - that wouldn't change their general availability, because we still offer them, and it wouldn't change the way how YaST installs them.

They are currently available, the thing is it's easy to look in the wrong place, and they are not where one expects to find them. May be binary mirrors could simply include ReadmeSource file, which contains the actual URI as a reminder.

...

...
So problems are :

1) source rpm's are changed and re-published over frequently by Build service

That problem affects only the build service - their infrastructure needs more disks, but that's bout it. It shouldn't bother you at least.

It's not about re-publishing by the way. And not about the fact that they change often. It's about duplication. (To fully understand the implications, and in how far they matter, you'll have to read the complete thread that I referenced.)

To give you an example: each source RPMs contains a tarball (the packed sources from the upstream project), a build description, and possible bits like configuration and stuff. If you build Apache, there is a tarball called httpd-2.2.14.tar.bz2 used in the build, and it is included into the source RPM. <snip>

OK, so the source rpm format could allow an external URI for the source and cryto hash checksums. So long as a local cache copy can be auto-matically retrieved if it's not present locally on a build everyone will be happy.

...

So, this extreme duplication of sources in the build service is a problem for itself, but it has no effect on users. And the sources that developers are working with are not the source RPMs (which are just a fallout from the build system by automatism), but the original sources. Adrian surely has a point in mentioning that these source RPMs (those of the build service!!) are redundant and utterly useless. It is fully reasonable to suggest getting rid of them _under_ the condition that the original sources are made available anonymously (which isn't the case yet).

Yes. But how is it intended for end user to get the source of packages they're using? As it stands, the repo system has a source repo, but using rpm build seems to be discouraged (as it builds against current system), and installing the source seems neglected in Software Manager, making it's presence misleadingly pointless.

...

...
2) source rpm take huge diskspace and bandwidth, for something rarely required on installed system

See 1), this affects the build service-internal infrastructure, but it's built to handle it. (Which doesn't exclude thinking about optimizations of course :-))

...
3) obtaining the source to build and modify a package on end-user system (as required by GPL) is less obvious than it was in past

I can't judge - I don't use YaST or something. However the sources are all there, which we probably all agree on. They are accessible through various means (even mirrored, and free to be mirrored by everybody with no restrictions and no exceptions); there's the convenient build service for those who want to work with the sources seriously enough to bother creating a Novell account, and there is an option in YaST (of arguably operability) that makes it easy for everybody installing to install the "feel-good-factor" as well. From what people wrote, there's even a snail mailed DVD, although I cannot confirm its existance.

How obvious all this is - hard to say, but I think, who he who asks will get the helpful pointers at least ;-)

Being thorough I see number of different requirements : 1) Community end user + ought be able to find out how something works and hack on a program without reading openSUSE specific documentation + not have to inquire mail lists, forums etc 2) FOSS checklist - do we have local copy of all the source we use + business wants to be sure, nothing is missing for strategic reasons + FOSS advocate out of principal 3) Convenient License Compliance + online not snail mail + available to all

...

And if you have ideas how to make things more obvious, I'm sure they are welcome'd by everybody else. There is always somebody who will benefit from it.

...
Can we simply have seperate hosts in URI's for binary download, binary debug & source packages?

Do you think that http://download-sources.opensuse.org/ would be better than http://download.opensuse.org/source/ ? I don't think so.

Actually yes, because I looked at the repo list likely 100 times, before I noticed that first level directory change. I think it's easier to spot differences at start, or end of a URI, but not in the middle. Once you KNOW the top level directory is different, it is obivous, until then it is easy to miss. Even a small html & txt format file ReadmeSources, that's visible when browsing the repo at the i586, i686, x86_64, noarch level explaining location of source, and perhaps reminder of what recommended build tools are, fully satisfies me, because I would not waste time searching in the wrong place. But, fact is, I have not for very long time needed to get & work with distro rpm, in past I would get source rpm, in order to upgrade to newer release of a software package. With Build service, I doubt I need to do this. Maintaining a package, means working within the Build service. Mostly now, it would be external source projects not supplied.

...

...
What about a way to install all the source of used packages?

I never use it, but I thought there'd be an option for this in the YaST package manager.

If that doesn't work, then check bugzilla and open a bug if needed.

Done https://bugzilla.novell.com/show_bug.cgi?id=560868 It definitely worked in SuSE 6.1!! lol -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Peter Pöml

13:24

Am 05.12.2009 um 13:46 schrieb Rob OpenSuSE:

...

...
Since the distribution of downloads to mirrors is transparent, well, just let it work for you. And even if all mirrors would stop mirroring source rpms - that wouldn't change their general availability, because we still offer them, and it wouldn't change the way how YaST installs them.

They are currently available, the thing is it's easy to look in the wrong place, and they are not where one expects to find them. May be binary mirrors could simply include ReadmeSource file, which contains the actual URI as a reminder.

Indeed, I thought about the same! I think that would help a lot - it would help all people looking at that place, which is indeed a logical place to look. In addition, since the file would be replicated to the mirrors, the information would spread to those additional places. Talk to Adrian and Coolo, they can get that arranged I'd think!

...

Yes. But how is it intended for end user to get the source of packages they're using?

As it stands, the repo system has a source repo, but using rpm build seems to be discouraged (as it builds against current system), and installing the source seems neglected in Software Manager, making it's presence misleadingly pointless.

Well, what is the use case for end users to get the sources? What do they intend to do with it - how can we help them? Is it just about having the sources installed, out of interest? That needs a functionality in YaST. Is it to rebuild things with changes? We'd point them to do what everybody else does, namely using the build service to fetch the sources and get the required build environment at the same time. Is it to check how the package was built, and with which patches? Source RPMs are fine for that, but so is the web interface to the sources. I can think of more use cases for source RPMs (e.g. package maintainer from another distribution wants to compare his work), but for end users that's about all I could think of. Help me ;) I think it would be easier for me to suggest a way for how the end user might best get the sources, when having a clearer picture what the intention is. (Not questioning the need as such, or the desire for whatever reasons.)

...

...
Do you think that http://download-sources.opensuse.org/ would be better than http://download.opensuse.org/source/ ? I don't think so.

Actually yes, because I looked at the repo list likely 100 times, before I noticed that first level directory change. I think it's easier to spot differences at start, or end of a URI, but not in the middle. Once you KNOW the top level directory is different, it is obivous, until then it is easy to miss.

I don't think that this would help you; I'm rather convinced of the opposite. If the files are on the same host (the one you are looking at), you have at least a chance to find what you look for. If the files are on a different host, you can stare at the listing as long as you want, to no avail.

...

...
If that doesn't work, then check bugzilla and open a bug if needed.

Done https://bugzilla.novell.com/show_bug.cgi?id=560868 It definitely worked in SuSE 6.1!! lol

By the way, does the functionality take care of updates as well, and keeps the installed source RPMs updated? Otherwise it'd be quite useless (even dangerous). And does it make sure not to overwrite sources that the local user may have modified? If not, it's a pain. And while I'm at asking silly questions, does installation of source RPMs work at all, without each source RPM overwriting files of another source RPM by accident, if not every name is globally unique? Did you know that all files of all source RPMs are installed into a single directory - /usr/src/packages/SOURCES? And the best of it all, you cannot query the RPM database to find out which files belong to installed source RPMs - rpm just throws them at the system, and doesn't take note of what it puts where, as it does for binary RPMs. Yes, it's really primitive :-) Nothing you could seriously work with. But if you want the above mentioned functionality, you'd probably want it working in complete. No? Let's face it, source RPMs are the undeads of a dark past, only there for archival and no developer wants to work with them, because it'd be a total pain. (Archival however is good. As good as build service hosting might become, it may as well lead into a trap discussed here: http://esr.ibiblio.org/?p=1282 , http://esr.ibiblio.org/?p=1295 , http://esr.ibiblio.org/?p=1302 . The points made there center around the thesis: "Hosting sites are data jails." Source RPMs are a least common denominator, and suitable for storage as well as for import of data elsewhere. [They completely lack history though, apart from a cropped changelog text, which is of course a major shortcoming.])

Rob OpenSuSE

18:15

2009/12/5 Peter Pöml :

...

Am 05.12.2009 um 13:46 schrieb Rob OpenSuSE:

...
They are currently available, the thing is it's easy to look in the wrong place, and they are not where one expects to find them. May be binary mirrors could simply include ReadmeSource file, which contains the actual URI as a reminder.

Indeed, I thought about the same! I think that would help a lot - it would help all people looking at that place, which is indeed a logical place to look. In addition, since the file would be replicated to the mirrors, the information would spread to those additional places.

Talk to Adrian and Coolo, they can get that arranged I'd think!

Can try knocking up a file and seeing what they say.

...

Well, what is the use case for end users to get the sources? What do they intend to do with it - how can we help them? Is it just about having the sources installed, out of interest?

As I wrote before, there's the "have all the source available" for strategic security independance reasons, and then the potential community member who's curiosity may draw them into playing with the stuff, finding out how it's put together, or who would like to make a small alteration to something.

...

...
...
Do you think that http://download-sources.opensuse.org/ would be better than http://download.opensuse.org/source/ ? I don't think so.

Actually yes, because I looked at the repo list likely 100 times, before I noticed that first level directory change. I think it's easier to spot differences at start, or end of a URI, but not in the middle. Once you KNOW the top level directory is different, it is obivous, until then it is easy to miss.

I don't think that this would help you; I'm rather convinced of the opposite. If the files are on the same host (the one you are looking at), you have at least a chance to find what you look for. If the files are on a different host, you can stare at the listing as long as you want, to no avail.

Fact is the OP and myself could not find the source rpm's despite looking, and Google was not very helpful either, this is my explanation. When you inspect the software repo's on the machine, in the "Software Repository Manager" there are a list with URLs visible (unfortunately I cannot copy & paste directly so this list is editted here to shorten irrelevant details) : Updates for openSUSE 11.2-0 http://download.opensuse.org/update/11.2/ openSUSE 11.2-0 http://download.opensuse.org/distribution/11.2/repo/oss/ openSUSE-11.2-Debug http://download.opensuse.org/debug/distribution/11.2/repo/oss/ openSUSE-11.2-Source http://download.opensuse.org/source/distribution/11.2/repo/oss/ Simply the brain see's patterns and uses the regularity as a shortcut, which usually works well, in this case I think "http://download.opensuse.org/ ... istribution/11.2/repo/oss/", so it is easy to miss the top level directory diffrerence. Same host, and directories are falling into a pattern. Like this I am sure that the difference stands out more : Updates for openSUSE 11.2-0 http://updates.opensuse.org/11.2 openSUSE 11.2-0 http://download.opensuse.org/distribution/11.2/repo/oss/ openSUSE-11.2-Debug http://debug.opensuse.org/distribution/11.2/repo/oss/ openSUSE-11.2-Source http://source.opensuse.org/distribution/11.2/repo/oss/ URLs are clearly different, web browsers train us to check the host part, the difference is not obfuscated by coming in middle of long piece of text. Technically of course, the URI can be written into the appropriate top level form.

...

By the way, does the functionality take care of updates as well, and keeps the installed source RPMs updated? Otherwise it'd be quite useless (even dangerous). And does it make sure not to overwrite sources that the local user may have modified? If not, it's a pain. And while I'm at asking silly questions, does installation of source RPMs work at all, without each source RPM overwriting files of another source RPM by accident, if not every name is globally unique? Did you know that all files of all source RPMs are installed into a single directory - /usr/src/packages/SOURCES? And the best of it all, you cannot query the RPM database to find out which files belong to installed source RPMs - rpm just throws them at the system, and doesn't take note of what it puts where, as it does for binary RPMs. Yes, it's really primitive :-) Nothing you could seriously work with. But if you want the above mentioned functionality, you'd probably want it working in complete. No?

Presumbably installing a self built RPM changes the vendor. Yes, there are limitations, which is why a tool to grab all the installed rpm's would be better than actually installing them. If updates are made patching GPL progs, the source & build tool files HAVE to be available somewhere on request or we don't have the right to distribute them!

...

Let's face it, source RPMs are the undeads of a dark past, only there for archival and no developer wants to work with them, because it'd be a total pain.

Until the distro changes to another package management system, or alter's RPM it's what we have. Using OBS and build tools still have to write a spec file, etc etc. At present, in order to package up some developed software, creating a source rpm, and installable binary rpm files, does seem to be a necessary step. Furthermore, a good beginning is often to download source rpm, and get a specfile, written for another distro, if it's present, and then patching for SuSE. Despite limitations I would think the main reason to keep them, is that it is a widely spread format, used by other distro's and with documentation available. Rob -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Peter Pöml

19:11

Am 05.12.2009 um 19:15 schrieb Rob OpenSuSE:

...

Fact is the OP and myself could not find the source rpm's despite looking, and Google was not very helpful either, this is my explanation.

Sure, as the first link on a Google search for "opensuse sources" is http://www.novell.com/products/opensuse/source_code.html which is horribly outdated. Could you please use the "feedback" link on the bottom of the page, and request an update? Third link, http://en.opensuse.org/Download, lacks to mention the sources. But fixing should be straightforward as well. It's probably a wiki page that can be edited; otherwise I'd suggest to open a bug report. Obvious solutions, or not? :-) And if you could also open a bug report about software.opensuse.org as well, that would be great I guess - because it's another obvious place where people tend to look. Peter -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Dave Plater

20:04

On 12/05/2009 09:11 PM, Peter Pöml wrote:

...

Am 05.12.2009 um 19:15 schrieb Rob OpenSuSE:

...
Fact is the OP and myself could not find the source rpm's despite looking, and Google was not very helpful either, this is my explanation.

Sure, as the first link on a Google search for "opensuse sources" is http://www.novell.com/products/opensuse/source_code.html which is horribly outdated. Could you please use the "feedback" link on the bottom of the page, and request an update?

Third link, http://en.opensuse.org/Download, lacks to mention the sources. But fixing should be straightforward as well. It's probably a wiki page that can be edited; otherwise I'd suggest to open a bug report.

Obvious solutions, or not? :-)

And if you could also open a bug report about software.opensuse.org as well, that would be great I guess - because it's another obvious place where people tend to look.

Peter

Shouldn't the sources be in the yast community repos list and be installable by yast? Regards Dave P -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Stefan Seyfried

6 Dec 6 Dec

18:54

On Sat, 5 Dec 2009 11:47:43 +0100 Peter Pöml wrote:

...

Hi,

Am 05.12.2009 um 10:28 schrieb Rob OpenSuSE:

...
Am including Factory list, perhaps in 11.3 a better way of making source available to end users, which avoids big load on Build Service and Mirrors is possible?

Uhm, these are two separate issues: - installability of source RPMs on released openSUSE (which you ran into)

- the multiplication of sources in the build service (which I mentioned just because Karsten mentioned the size of the home: namespace in the build service; I tried to set into perspective where most of the space goes.)

...
On project list, a request was made for source by someone unfamiliar with openSUSE suggested an ISO; installing source rpm's with YaST is not working as it did in past for me, because only binary packages are held on most of the download.opensuse.org mirrors.

Since the distribution of downloads to mirrors is transparent, well, just let it work for you. And even if all mirrors would stop mirroring source rpms - that wouldn't change their general availability, because we still offer them, and it wouldn't change the way how YaST installs them.

...
So problems are :

1) source rpm's are changed and re-published over frequently by Build service

That problem affects only the build service - their infrastructure needs more disks, but that's bout it. It shouldn't bother you at least.

It's not about re-publishing by the way. And not about the fact that they change often. It's about duplication. (To fully understand the implications, and in how far they matter, you'll have to read the complete thread that I referenced.)

To give you an example: each source RPMs contains a tarball (the packed sources from the upstream project), a build description, and possible bits like configuration and stuff. If you build Apache, there is a tarball called httpd-2.2.14.tar.bz2 used in the build, and it is included into the source RPM. Since the tarball is about 5MB in size, the size of the source RPM is largely dictated by that. Now, if you build Apache for openSUSE 11.0, 11.1, 11.2, CentOS5 and Factory, you'll create 5 source RPMs, and each will contain the exact same httpd tarball from upstream. That's 25MB all out of a sudden. And before you even notice it, you may have a whole GB of tarballs just from Apache builds, all containing a tarball that is 5MB in size and can be downloaded from upstream at any time. Well, and the build service file tree is consisting of such duplicated sources to a large extent (I estimated 30%-50%). That doesn't affect users, though - download.opensuse.org offers all that for download, because that's the way things should be - but of course nobody wants the stuff.

Which brings up the question: couldn't the source RPM be assembled on the fly as soon as somebody wants to download it? There is (crappy ;) version control in OBS, and you "only" need to record which revision built the binary RPM. Then if someone wants the source RPM for this revision, you just pack it up and send it to him. Should be an interesting Project to implement ;) -- Stefan Seyfried "Any ideas, John?" "Well, surrounding them's out." -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Egbert Eich

21:26

On Sun, Dec 06, 2009 at 07:54:49PM +0100, Stefan Seyfried wrote:

...

On Sat, 5 Dec 2009 11:47:43 +0100 Peter Pöml wrote:

Which brings up the question: couldn't the source RPM be assembled on the fly as soon as somebody wants to download it? There is (crappy ;) version control in OBS, and you "only" need to record which revision built the binary RPM. Then if someone wants the source RPM for this revision, you just pack it up and send it to him.

Should be an interesting Project to implement ;)

This brings back the question why we need the source RPM at all. The only reason I could think of is that people using other distros can easily download out sources. If openSUSE users would either resourt to OSC or to the software installation tools shipped by openSUSE (zypper, YaST, ...) we didn't even need SRPMS at all. Those tools could be modified to use OSC to download the sources from the build system. This would just require meta information about the location of the packages in the build system and an anonymous read only access. Implementing this would be a more worthwhile project than tools that would generate SPRMS from the build system. Additionally the sources could be installed in saner places than /usr/src/packages/SOURCES where they are currently installed by default. Which - as Peter already stated - is a dangerous palce when installing sources for more than one package. Cheers, Egbert. -- Egbert Eich (Res. & Dev.) SUSE LINUX Products GmbH X Window System Development Tel: +49 911-740 53 0 http://www.suse.de ----------------------------------------------------------------- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Peter Nikolic

22:45

On Sunday 06 Dec 2009 21:26:21 Egbert Eich wrote:

...

This brings back the question why we need the source RPM at all. The only reason I could think of is that people using other distros can easily download out sources. If openSUSE users would either resourt to OSC or to the software installation tools shipped by openSUSE (zypper, YaST, ...) we didn't even need SRPMS at all.

It may surprise you to find out that there are people that still have Dialup internet connections and things like build services that rely on an internet connection are a big no no ,

...

Those tools could be modified to use OSC to download the sources from the build system. This would just require meta information about the location of the packages in the build system and an anonymous read only access. Implementing this would be a more worthwhile project than tools that would generate SPRMS from the build system. Additionally the sources could be installed in saner places than /usr/src/packages/SOURCES where they are currently installed by default. Which - as Peter already stated - is a dangerous palce when installing sources for more than one package.

Well with the increasing use of Blueray discs we can get ALL the sources and the complete install distro on one disc so the space issue would vanish therefore removing the need of reliance on an External build system Pete . -- Powered by openSUSE 11.2 Milestone 2 (x86_64) Kernel: 2.6.30-rc6-git3-4- default KDE: 4.2.86 (KDE 4.2.86 (KDE 4.3 >= 20090514)) "release 1" 22:40 up 15 days 12:30, 3 users, load average: 0.58, 0.66, 0.73

Egbert Eich

7 Dec 7 Dec

07:27

On Sun, Dec 06, 2009 at 10:45:42PM +0000, Peter Nikolic wrote:

...

On Sunday 06 Dec 2009 21:26:21 Egbert Eich wrote:

...
This brings back the question why we need the source RPM at all. The only reason I could think of is that people using other distros can easily download out sources. If openSUSE users would either resourt to OSC or to the software installation tools shipped by openSUSE (zypper, YaST, ...) we didn't even need SRPMS at all.

It may surprise you to find out that there are people that still have Dialup internet connections and things like build services that rely on an internet connection are a big no no ,

Hrm, you are talking about something entirely different, aren't you? Indeed when the source disk had been done away with (years ago!) I regretted this decision because I was no longer able to carry everything with me. But this is an entirely different matter. Neither I nor a number of previous posts touched on this issue. Cheers, Egbert. -- Egbert Eich (Res. & Dev.) SUSE LINUX Products GmbH X Window System Development Tel: +49 911-740 53 0 http://www.suse.de ----------------------------------------------------------------- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Rob OpenSuSE

10:15

2009/12/6 Egbert Eich :

...

This brings back the question why we need the source RPM at all. The only reason I could think of is that people using other distros can easily download out sources. If openSUSE users would either resourt to OSC or to the software installation tools shipped by openSUSE (zypper, YaST, ...) we didn't even need SRPMS at all.

One use for src.rpm is after a release is not supported, at least in theory it's possible to backport important updates, or have someone look into a new bug which suddenly manifests after end of official support, or say when a program frustratingly refuses to run virtualised on a current machine because it doesn't recognise the CPU as i386 family. Also not needing to be "online" and connected with openSUSE or Novell, provides more security against future unforeseen events. My main use for src.rpm has actually been with 3rd party software, where I've needed to alter spec files, and make configure patches to build manageable packages. In the past, end users have also rebuilt distro's to use different compile options for example "RevHat". If source was to be provided another way, like build service, the project would need to make the method much clearer. We "know" how rpm works, and that generates expecations, currently the updates hierarchy has src directory in the hierarchy with ARCH dirs (like i586, x86_64 & noarch). Also what was delivered at GM and via update, ought to be archiveable by end user, so needs like the OP's who requested source ISO by download can be met. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Egbert Eich

16:35

On Mon, Dec 07, 2009 at 10:15:18AM +0000, Rob OpenSuSE wrote:

...

2009/12/6 Egbert Eich :

...
This brings back the question why we need the source RPM at all. The only reason I could think of is that people using other distros can easily download out sources. If openSUSE users would either resourt to OSC or to the software installation tools shipped by openSUSE (zypper, YaST, ...) we didn't even need SRPMS at all.

One use for src.rpm is after a release is not supported, at least in theory it's possible to backport important updates, or have someone look into a new bug which suddenly manifests after end of official support, or say when a program frustratingly refuses to run virtualised on a current machine because it doesn't recognise the CPU as i386 family. Also not needing to be "online" and connected with openSUSE or Novell, provides more security against future unforeseen events.

If this is really the plan, wouldn't it be better to create a project in ones home and build the security maintenence packages there? This way one could share the results with others instead of keeping them on the local machine.

...

My main use for src.rpm has actually been with 3rd party software, where I've needed to alter spec files, and make configure patches to build manageable packages. In the past, end users have also rebuilt distro's to use different compile options for example "RevHat".

What would be the difference if there was a read only osc access? You can download the content of the src rpm, modify it and use build on your local machine to run the entire rpm/srpm build process. For this you don't need a srpm package hosted - you could still distribute your modified srpm.

...

If source was to be provided another way, like build service, the project would need to make the method much clearer. We "know" how rpm works, and that generates expecations, currently the updates hierarchy has src directory in the hierarchy with ARCH dirs (like i586, x86_64 & noarch).

For 'build' try 'man 1 build'.

...

Also what was delivered at GM and via update, ought to be archiveable by end user, so needs like the OP's who requested source ISO by download can be met.

Hrm, end users archive things ops request ... is this a very likely scenario? Cheers, Egbert. -- Egbert Eich (Res. & Dev.) SUSE LINUX Products GmbH X Window System Development Tel: +49 911-740 53 0 http://www.suse.de ----------------------------------------------------------------- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Rob OpenSuSE

17:14

2009/12/7 Egbert Eich :

...

On Mon, Dec 07, 2009 at 10:15:18AM +0000, Rob OpenSuSE wrote:

...
2009/12/6 Egbert Eich :

One use for src.rpm is after a release is not supported, at least in theory it's possible to backport important updates, or have someone look into a new bug which suddenly manifests after end of official support, or say when a program frustratingly refuses to run virtualised on a current machine because it doesn't recognise the CPU as i386 family. Also not needing to be "online" and connected with openSUSE or Novell, provides more security against future unforeseen events.

If this is really the plan, wouldn't it be better to create a project in ones home and build the security maintenence packages there? This way one could share the results with others instead of keeping them on the local machine.

Don't think that fits in with the management type mindset, who want to "have the source", and hope for quick hack. Having the src rpm, lets you do it either with rpm build options, or using SuSE build tool that builds a clean root from the GM files. The rpm format is defined so it's easy to extract the files with tools. Actually on other distro's to. I have rescued systems via rpm, using other distro's to boot from for example, in chroot(8) environments. Any competing format, would hopefully be similarly convenient.

...

...
My main use for src.rpm has actually been with 3rd party software, where I've needed to alter spec files, and make configure patches to build manageable packages. In the past, end users have also rebuilt distro's to use different compile options for example "RevHat".

What would be the difference if there was a read only osc access? You can download the content of the src rpm, modify it and use build on your local machine to run the entire rpm/srpm build process. For this you don't need a srpm package hosted - you could still distribute your modified srpm.

I'm not an "objector", just if there is change then those are the sort of things that folk need to be able to do, even if they don't actually do them very often, the capability (in theory) matters. Not being reliant on the project, or Novell matters. Say MS bought Novell for instance in a take over. All of a sudden upstream would be untrusted, and I'd be wanting to inspect updates for tom-foolery.

...

...
If source was to be provided another way, like build service, the project would need to make the method much clearer. We "know" how rpm works, and that generates expecations, currently the updates hierarchy has src directory in the hierarchy with ARCH dirs (like i586, x86_64 & noarch).

For 'build' try 'man 1 build'.

Have used it, the version was doing stuff with rpm's and doing an rpm build for me, so it wasn't so different.

...

...
Also what was delivered at GM and via update, ought to be archiveable by end user, so needs like the OP's who requested source ISO by download can be met.

Hrm, end users archive things ops request ... is this a very likely scenario?

Well if one person like Matt wants to do it, they need to be able to. Knowing one can download the src rpm & rebuild system packages, matters to more than actually tinker with the packages. If it's not easy then the bar to being an active community contributer is higher. Even if developers themselves generally would work with make, and release a source tar ball; for sysadmins doing customisation something fairly convenient like rpm is a help. Using alien to convert packages is good sometimes as well, even if it does not appear in an use cases and work flows for "typical" end users. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

jdd

12:34

Le 06/12/2009 22:26, Egbert Eich a écrit :

...

This brings back the question why we need the source RPM at all.

let alone to comply with the licence? jdd -- http://www.dodin.net http://valerie.dodin.org http://news.opensuse.org/2009/04/13/people-of-opensuse-jean-daniel-dodin/ -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Dominique Leuenberger

12:09

...

...
...
On 12/7/2009 at 13:34, jdd wrote: Le 06/12/2009 22:26, Egbert Eich a écrit :

...
This brings back the question why we need the source RPM at all.

let alone to comply with the licence?

ot really. To comply with the license we need to provide the SOURCE. It does not need to be packaged in an RPM at all (any package in OBS would be sufficient to access it. All we'd really need is an 'anonymous' access to OBS (and most likely an access to the history of a package with a reference which binary result was built with a specific set in the history). That would comply as much as it possibly can. Dominique -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

5253

Age (days ago)

5255

Last active (days ago)

List overview

Download

16 comments

8 participants

participants (8)

Dave Plater
Dominique Leuenberger
Egbert Eich
jdd
Peter Nikolic
Peter Pöml
Rob OpenSuSE
Stefan Seyfried