Hi,
The package cdbs keeps failing on me. I try to backport it from Debian and / or Ubuntu 10.10 to Ubuntu 10.04. Was always working, but now it fails or it fails for one architecture (something one sometimes the other arch...).
https://build.opensuse.org/package/show?package=cdbs3&project=home%3Aope...
I need some help cause pretty some packages depends on a newer version of cdbs.
Thanks in advance,
~P
On 03/25/2011 10:49 AM, peer wrote:
Hi,
The package cdbs keeps failing on me. I try to backport it from Debian and / or Ubuntu 10.10 to Ubuntu 10.04. Was always working, but now it fails or it fails for one architecture (something one sometimes the other arch...).
https://build.opensuse.org/package/show?package=cdbs3&project=home%3Aope...
I need some help cause pretty some packages depends on a newer version of cdbs.
Thanks in advance,
~P
Man this works on Debian, Ubuntu, Launchpad you name it, but it fails on OBS!??
Tried to build it locally, no success :(
Would be nice if this gets fixed, pretty some packages has cdbs as dependency!
~p
PASS: autotools-1.sh PASS: autotools-2.sh PASS: autotools-3.sh PASS: autotools-4.sh PASS: autotools-5.sh PASS: autotools-lzma.sh PASS: recursive.sh PASS: udeb-1.sh PASS: hdparm.sh PASS: debhelper-1.sh PASS: debhelper-2.sh PASS: debhelper-3.sh PASS: debhelper-4.sh PASS: debhelper-5.sh PASS: debhelper-6.sh PASS: debhelper-7.sh PASS: patchsys-1.sh make[5]: warning: jobserver unavailable: using -j1. Add `+' to parent make rule. PASS: list-missing-1.sh PASS: distutils-1.sh PASS: distutils-2.sh PASS: distutils-3.sh PASS: distutils-4.sh PASS: distutils-5.sh PASS: distutils-6.sh PASS: distutils-7.sh PASS: distutils-8.sh FAIL: distutils-9.sh FAIL: distutils-10.sh FAIL: distutils-11.sh PASS: ant-1.sh ==================== 3 of 30 tests failed ==================== make[4]: *** [check-TESTS] Error 1 make[4]: Leaving directory `/usr/src/packages/BUILD/test' make[3]: *** [check-am] Error 2 make[3]: Leaving directory `/usr/src/packages/BUILD/test' make[2]: *** [check-recursive] Error 1 make[2]: Leaving directory `/usr/src/packages/BUILD' make[1]: *** [check] Error 2 make[1]: Leaving directory `/usr/src/packages/BUILD' make: *** [debian/stamp-makefile-check] Error 2 dpkg-buildpackage: error: debian/rules build gave error exit status 2
The buildroot was: /var/tmp/build-root
On Mon, 2011-03-28 at 21:18 +0200, peer wrote:
PASS: distutils-8.sh FAIL: distutils-9.sh FAIL: distutils-10.sh FAIL: distutils-11.sh PASS: ant-1.sh ==================== 3 of 30 tests failed ====================
Checking the current build failures on the linked project, the tests actually pass all fine (even though it seems to be only 28 checks)
The builds abort due to a 'build appears to be stuck' error => trying a local build shows that bash is actually dieing:
_ /usr/bin/make install-am _ /usr/bin/make install-exec-am install-data-am _ [bash] <defunct> _ [bash] <defunct> _ [bash] <defunct>
So the clue is actually to find out why bash defunts on us in our build environment.
A small verification with osc build -j 1 shows that once more, debian stuff just does not work properly in parallel builds...
There was a discussion around about being able to disable that behavior. Not sure if there went something more in this direction.
Dominique
On 03/28/2011 10:20 PM, Dimstar / Dominique Leuenberger wrote:
On Mon, 2011-03-28 at 21:18 +0200, peer wrote:
PASS: distutils-8.sh FAIL: distutils-9.sh FAIL: distutils-10.sh FAIL: distutils-11.sh PASS: ant-1.sh ==================== 3 of 30 tests failed ====================
Checking the current build failures on the linked project, the tests actually pass all fine (even though it seems to be only 28 checks)
The builds abort due to a 'build appears to be stuck' error => trying a local build shows that bash is actually dieing:
_ /usr/bin/make install-am _ /usr/bin/make install-exec-am install-data-am _ [bash]<defunct> _ [bash]<defunct> _ [bash]<defunct>
So the clue is actually to find out why bash defunts on us in our build environment.
A small verification with osc build -j 1 shows that once more, debian stuff just does not work properly in parallel builds...
There was a discussion around about being able to disable that behavior. Not sure if there went something more in this direction.
Dominique
Hi,
Thanks for your reply and efforts Dominique. Do you think that disabling parallel builds will solve this issue here?
Regards,
~P
On 03/29/2011 04:08 PM, peer wrote:
On 03/28/2011 10:20 PM, Dimstar / Dominique Leuenberger wrote:
On Mon, 2011-03-28 at 21:18 +0200, peer wrote:
PASS: distutils-8.sh FAIL: distutils-9.sh FAIL: distutils-10.sh FAIL: distutils-11.sh PASS: ant-1.sh ==================== 3 of 30 tests failed ====================
Checking the current build failures on the linked project, the tests actually pass all fine (even though it seems to be only 28 checks)
The builds abort due to a 'build appears to be stuck' error => trying a local build shows that bash is actually dieing:
_ /usr/bin/make install-am _ /usr/bin/make install-exec-am install-data-am _ [bash]<defunct> _ [bash]<defunct> _ [bash]<defunct>
So the clue is actually to find out why bash defunts on us in our build environment.
A small verification with osc build -j 1 shows that once more, debian stuff just does not work properly in parallel builds...
There was a discussion around about being able to disable that behavior. Not sure if there went something more in this direction.
Dominique
Hi,
Thanks for your reply and efforts Dominique. Do you think that disabling parallel builds will solve this issue here?
Regards,
~P
Hmm no other replies from the list/ devs? Debian / Ubuntu issues are less important then OpenSuse issues on OBS probably...
~P
Quoting peer linuxpeerjam@gmail.com:
Hmm no other replies from the list/ devs? Debian / Ubuntu issues are less important then OpenSuse issues on OBS probably...
I can't tell you much more than I did: If I build the package without parallel, it does work (osc build -j 1).
but I have no idea how you can convince your .deb not to use parallel builds. Sorry
Dominique
Hi,
Because of the failing cdbs package for Ubuntu (and Debian) I've spoken to the maintainer of cdbs which is also the maintainer of the cdbs Debian package.
He sees the situation as follows:
Debian Policy §4.9.1 says: It is up to the package maintainer to decide whether the package build times are long enough and the package build system is robust enough to make supporting parallel builds worthwhile.
"The cdbs package did *not* explicitly enable support for parallel builds, that makes the parallel building by OBS a violation of the Debian Policy in this case."
If you do not agree and think this is a bug in cdbs, please explain that clearly to me, then I will file a bugreport to the package. Otherwise please chance the build behavior of OBS for Ubuntu / Debian packages.
Thank in advance,
~P
http://www.debian.org/doc/debian-policy/ch-source.html
On 03/31/2011 11:52 AM, peer wrote:
Hi,
Because of the failing cdbs package for Ubuntu (and Debian) I've spoken to the maintainer of cdbs which is also the maintainer of the cdbs Debian package.
He sees the situation as follows:
Debian Policy §4.9.1 says: It is up to the package maintainer to decide whether the package build times are long enough and the package build system is robust enough to make supporting parallel builds worthwhile.
"The cdbs package did *not* explicitly enable support for parallel builds, that makes the parallel building by OBS a violation of the Debian Policy in this case."
If you do not agree and think this is a bug in cdbs, please explain that clearly to me, then I will file a bugreport to the package. Otherwise please chan*g*e the build behavior of OBS for Ubuntu / Debian packages.
Thank in advance,
~P
Am Donnerstag, 31. März 2011, 11:52:56 schrieb peer:
Hi,
Because of the failing cdbs package for Ubuntu (and Debian) I've spoken to the maintainer of cdbs which is also the maintainer of the cdbs Debian package.
He sees the situation as follows:
Debian Policy §4.9.1 says: It is up to the package maintainer to decide whether the package build times are long enough and the package build system is robust enough to make supporting parallel builds worthwhile.
And what would the packager do to enable parallel builds ?
bye adrian
On Thursday March 31 2011 12:22:59 Adrian Schröter wrote:
Am Donnerstag, 31. März 2011, 11:52:56 schrieb peer:
Hi,
Because of the failing cdbs package for Ubuntu (and Debian) I've spoken to the maintainer of cdbs which is also the maintainer of the cdbs Debian package.
He sees the situation as follows:
Debian Policy §4.9.1 says: It is up to the package maintainer to decide whether the package build times are long enough and the package build system is robust enough to make supporting parallel builds worthwhile.
And what would the packager do to enable parallel builds ?
He would do some bash scripting in the rules file to get the # of parallel builds to from the DEB_BUILD_OPTIONS env variable and then calls make with the parsed argument which is butt ugly to say it politely.
IMHO the debian way plain sucks and shouldn't be repeated. Instead some "DISABLE_PARALLEL_BUILD=1" in the packages meta data is much preferable.
See also http://www.debian.org/doc/debian-policy/ch-source.html section 4.9.1
(besides that parallel builds being broken _IS_ a bug in the packages build system which should be filed upstream and not at debian)
@Peer: did you file a bug about that at cdbs upstream?
regards, Stephan
On 03/31/2011 08:48 PM, Stephan Kleine wrote:
@Peer: did you file a bug about that at cdbs upstream?
Yes I did file a bugreport, this is the reply:
When building cdbs on the opensuse buildservice (obs), cdbs fails. Obs uses parallel building by default.
I believe this can only happen if obs plays tricks to enforce parallel building - e.g. directly sets make option -j - which is unsupported.
Obs should follow Debian Policy §4.9.1 and only _request_ the use of parallel building with DEB_BUILD_OPTIONS=parallel=n.
I am therefore closing this as a non-bug. You are quite welcome to continue posting to this bugreport (closed does not mean closed for debate), to try convince us that this is in fact a bug in this package, not in your build environment.
A small verification with osc build -j 1 shows that once more, debian stuff just does not work properly in parallel builds...
...or it simply shows that obs is failing in its attempts to outsmart explicitly documented behaviour of Debian.
And what would the packager do to enable parallel builds ? He would do some bash scripting in the rules file to get the # of parallel builds to from the DEB_BUILD_OPTIONS env variable and then calls make with the parsed argument which is butt ugly to say it politely.
This is butt ugly?:
DEB_BUILD_PARALLEL=yes
Please note that above deliberately was _not_ added to the CDBS packaging, because we - the package maintainers of CDBS - did not feel that "the package build times are long enough and the package build system is robust enough to make supporting parallel builds worthwhile" as it is phrased in Debian Policy §4.9.1.
On 04/02/2011 05:45 PM, peer wrote:
On 03/31/2011 08:48 PM, Stephan Kleine wrote:
@Peer: did you file a bug about that at cdbs upstream?
Yes I did file a bugreport, this is the reply:
When building cdbs on the opensuse buildservice (obs), cdbs fails. Obs uses parallel building by default.
I believe this can only happen if obs plays tricks to enforce parallel building - e.g. directly sets make option -j - which is unsupported.
Obs should follow Debian Policy §4.9.1 and only _request_ the use of parallel building with DEB_BUILD_OPTIONS=parallel=n.
I am therefore closing this as a non-bug. You are quite welcome to continue posting to this bugreport (closed does not mean closed for debate), to try convince us that this is in fact a bug in this package, not in your build environment.
A small verification with osc build -j 1 shows that once more, debian stuff just does not work properly in parallel builds...
...or it simply shows that obs is failing in its attempts to outsmart explicitly documented behaviour of Debian.
And what would the packager do to enable parallel builds ? He would do some bash scripting in the rules file to get the # of parallel builds to from the DEB_BUILD_OPTIONS env variable and then calls make with the parsed argument which is butt ugly to say it politely.
This is butt ugly?:
DEB_BUILD_PARALLEL=yes
Please note that above deliberately was _not_ added to the CDBS packaging, because we - the package maintainers of CDBS - did not feel that "the package build times are long enough and the package build system is robust enough to make supporting parallel builds worthwhile" as it is phrased in Debian Policy §4.9.1.
Here is the bug report
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=620350
This is an automatic notification regarding your Bug report which was filed against the cdbs package:
#620350: cdbs fails when building in parallel
It has been closed by Jonas Smedegaarddr@jones.dk (reply to620350@bugs.debian.org).
Their explanation is attached below along with your original report. If this explanation is unsatisfactory and you have not received a better one in a separate message then please contact Jonas Smedegaarddr@jones.dk (reply to620350@bugs.debian.org) by replying to this email.
On Saturday April 2 2011 18:02:30 peer wrote:
On 04/02/2011 05:45 PM, peer wrote:
On 03/31/2011 08:48 PM, Stephan Kleine wrote:
@Peer: did you file a bug about that at cdbs upstream?
Yes I did file a bugreport, this is the reply:
When building cdbs on the opensuse buildservice (obs), cdbs fails. Obs uses parallel building by default.
I believe this can only happen if obs plays tricks to enforce parallel building - e.g. directly sets make option -j - which is unsupported.
Obs should follow Debian Policy §4.9.1 and only _request_ the use of parallel building with DEB_BUILD_OPTIONS=parallel=n.
I am therefore closing this as a non-bug. You are quite welcome to continue posting to this bugreport (closed does not mean closed for debate), to try convince us that this is in fact a bug in this package, not in your build environment.
A small verification with osc build -j 1 shows that once more, debian stuff just does not work properly in parallel builds...
...or it simply shows that obs is failing in its attempts to outsmart explicitly documented behaviour of Debian.
And what would the packager do to enable parallel builds ? He would do some bash scripting in the rules file to get the # of parallel builds to from the DEB_BUILD_OPTIONS env variable and then calls make with the parsed argument which is butt ugly to say it politely.
This is butt ugly?:
DEB_BUILD_PARALLEL=yes
Please note that above deliberately was _not_ added to the CDBS packaging, because we - the package maintainers of CDBS - did not feel that "the package build times are long enough and the package build system is robust enough to make supporting parallel builds worthwhile" as it is phrased in Debian Policy §4.9.1.
Here is the bug report
Well, it is obvious that the debian folks don´t care about anything outside their ivory tower. I meant filing a bug at the cdbs upstream that their build scripts are broken cause being unable to build in parallel _is_ a bug IMHO, so they would perhaps fix this.
This is an automatic notification regarding your Bug report which was filed against the cdbs package:
#620350: cdbs fails when building in parallel
It has been closed by Jonas Smedegaarddr@jones.dk (reply to620350@bugs.debian.org).
Their explanation is attached below along with your original report. If this explanation is unsatisfactory and you have not received a better one in a separate message then please contact Jonas Smedegaarddr@jones.dk (reply to620350@bugs.debian.org) by replying to this email.
On Saturday April 2 2011 19:13:48 you wrote:
Here is the bug report
Well, it is obvious that the debian folks don´t care about anything outside their ivory tower. I meant filing a bug at the cdbs upstream that their build scripts are broken cause being unable to build in parallel _is_ a bug IMHO, so they would perhaps fix this.
Er, sry, I forgot what cdbs is, seems like you are screwed ;P
But more seriously, not doing the builds in parallel is just a waste of time and resources and debian & buntu builds already take much longer than rpm builds (e.g. during package install) so disabling parallel builds to please some broken packages isn't an option imho. Simply wait till a way to disable them in the packages meta data is implemented and fix it in the meantime by adding "-j1" to your make call.
On 04/02/2011 08:20 PM, Stephan Kleine wrote:
On Saturday April 2 2011 19:13:48 you wrote:
Here is the bug report
Well, it is obvious that the debian folks don´t care about anything outside their ivory tower. I meant filing a bug at the cdbs upstream that their build scripts are broken cause being unable to build in parallel _is_ a bug IMHO, so they would perhaps fix this.
Er, sry, I forgot what cdbs is, seems like you are screwed ;P
But more seriously, not doing the builds in parallel is just a waste of time and resources and debian& buntu builds already take much longer than rpm builds (e.g. during package install) so disabling parallel builds to please some broken packages isn't an option imho. Simply wait till a way to disable them in the packages meta data is implemented and fix it in the meantime by adding "-j1" to your make call.
You didn't read the reply by the author well. The package is not broken, it's a choice to have the package like it is. That's why it's not a accident that it fails on obs and not on the Debian, Ubuntu and Launchpad build systems. They doesn't violate the Debian policy and I think that's a good choice. That's also why the cdbs author argues that obs is broken, not the package and I do agree with him here.
It's fundamentally wrong imo to provide a service for Debian packages and not accept the Debian way and the Debian policy. Of course you could discuss the way of building with the Debian devs and try to convince them (at least the cdbs author seems to be open for a rational discussion). But at the end the best thing a service for Debian packages can do, it to adapt to the Debian way. Then we as users are assured that we don't need all kinds of hacks to get our Debian packages build.
~P
On Saturday April 2 2011 21:51:24 peer wrote:
On 04/02/2011 08:20 PM, Stephan Kleine wrote:
On Saturday April 2 2011 19:13:48 you wrote:
Here is the bug report
Well, it is obvious that the debian folks don´t care about anything outside their ivory tower. I meant filing a bug at the cdbs upstream that their build scripts are broken cause being unable to build in parallel _is_ a bug IMHO, so they would perhaps fix this.
Er, sry, I forgot what cdbs is, seems like you are screwed ;P
But more seriously, not doing the builds in parallel is just a waste of time and resources and debian& buntu builds already take much longer than rpm builds (e.g. during package install) so disabling parallel builds to please some broken packages isn't an option imho. Simply wait till a way to disable them in the packages meta data is implemented and fix it in the meantime by adding "-j1" to your make call.
You didn't read the reply by the author well. The package is not broken, it's a choice to have the package like it is.
You misunderstood me. The packages source _is_ broken cause it can't be built in parallel. So their (debian) packaging is fine but the packages (cdbs´) build scripts contain a bug that screw you over if you build it in parallel.
Simply fix that and send the patch upstream.
On 04/03/2011 12:55 AM, Stephan Kleine wrote:
On Saturday April 2 2011 21:51:24 peer wrote:
On 04/02/2011 08:20 PM, Stephan Kleine wrote:
On Saturday April 2 2011 19:13:48 you wrote:
Here is the bug report
Well, it is obvious that the debian folks don´t care about anything outside their ivory tower. I meant filing a bug at the cdbs upstream that their build scripts are broken cause being unable to build in parallel _is_ a bug IMHO, so they would perhaps fix this.
Er, sry, I forgot what cdbs is, seems like you are screwed ;P
But more seriously, not doing the builds in parallel is just a waste of time and resources and debian& buntu builds already take much longer than rpm builds (e.g. during package install) so disabling parallel builds to please some broken packages isn't an option imho. Simply wait till a way to disable them in the packages meta data is implemented and fix it in the meantime by adding "-j1" to your make call.
You didn't read the reply by the author well. The package is not broken, it's a choice to have the package like it is.
You misunderstood me. The packages source _is_ broken cause it can't be built in parallel. So their (debian) packaging is fine but the packages (cdbs´) build scripts contain a bug that screw you over if you build it in parallel.
Simply fix that and send the patch upstream.
You could be right on this, I'll report it back. But this doesn't change the fact that obs is broken also. Again, I think violating the Debian policy is a fundamental bug and obs should fix this imho.
~P
On 04/02/2011 09:51 PM, peer wrote:
On 04/02/2011 08:20 PM, Stephan Kleine wrote:
On Saturday April 2 2011 19:13:48 you wrote:
Here is the bug report
Well, it is obvious that the debian folks don´t care about anything outside their ivory tower. I meant filing a bug at the cdbs upstream that their build scripts are broken cause being unable to build in parallel _is_ a bug IMHO, so they would perhaps fix this.
Er, sry, I forgot what cdbs is, seems like you are screwed ;P
But more seriously, not doing the builds in parallel is just a waste of time and resources and debian& buntu builds already take much longer than rpm builds (e.g. during package install) so disabling parallel builds to please some broken packages isn't an option imho. Simply wait till a way to disable them in the packages meta data is implemented and fix it in the meantime by adding "-j1" to your make call.
You didn't read the reply by the author well. The package is not broken, it's a choice to have the package like it is. That's why it's not a accident that it fails on obs and not on the Debian, Ubuntu and Launchpad build systems. They doesn't violate the Debian policy and I think that's a good choice. That's also why the cdbs author argues that obs is broken, not the package and I do agree with him here.
It's fundamentally wrong imo to provide a service for Debian packages and not accept the Debian way and the Debian policy. Of course you could discuss the way of building with the Debian devs and try to convince them (at least the cdbs author seems to be open for a rational discussion). But at the end the best thing a service for Debian packages can do, it to adapt to the Debian way. Then we as users are assured that we don't need all kinds of hacks to get our Debian packages build.
~P
I read the Debian policy carefully and nothing there it seems _prohibits_ parallel building.
Quote:
"parallel=n
This tag means that the package should be built using up to n parallel processes if the package build system supports this.[25] If the package build system does not support parallel builds, this string must be ignored. If the package build system only supports a lower level of concurrency than n, the package should be built using as many parallel processes as the package build system supports. It is up to the package maintainer to decide whether the package build times are long enough and the package build system is robust enough to make supporting parallel builds worthwhile."
So, reading that it seems then that in the case of OBS, it can provide a robust build system to parallel builds. Therefore, perhaps it might be worthwhile for Debian to reconsider what might be an out of date policy - especially in light of the prevalence of multi-core CPUS.
Moreover, from my long time experience with OBS as a user/packager, the statement that OBS is broken is fundamentally wrong. It is one of the true gems of openSUSE. I'd sense more NIH syndrome :(
Peter
Am Dienstag, 12. April 2011, 21:54:56 schrieb Peter Linnell:
On 04/02/2011 09:51 PM, peer wrote:
On 04/02/2011 08:20 PM, Stephan Kleine wrote:
On Saturday April 2 2011 19:13:48 you wrote:
Here is the bug report
Well, it is obvious that the debian folks don´t care about anything outside their ivory tower. I meant filing a bug at the cdbs upstream that their build scripts are broken cause being unable to build in parallel _is_ a bug IMHO, so they would perhaps fix this.
Er, sry, I forgot what cdbs is, seems like you are screwed ;P
But more seriously, not doing the builds in parallel is just a waste of time and resources and debian& buntu builds already take much longer than rpm builds (e.g. during package install) so disabling parallel builds to please some broken packages isn't an option imho. Simply wait till a way to disable them in the packages meta data is implemented and fix it in the meantime by adding "-j1" to your make call.
You didn't read the reply by the author well. The package is not broken, it's a choice to have the package like it is. That's why it's not a accident that it fails on obs and not on the Debian, Ubuntu and Launchpad build systems. They doesn't violate the Debian policy and I think that's a good choice. That's also why the cdbs author argues that obs is broken, not the package and I do agree with him here.
It's fundamentally wrong imo to provide a service for Debian packages and not accept the Debian way and the Debian policy. Of course you could discuss the way of building with the Debian devs and try to convince them (at least the cdbs author seems to be open for a rational discussion). But at the end the best thing a service for Debian packages can do, it to adapt to the Debian way. Then we as users are assured that we don't need all kinds of hacks to get our Debian packages build.
~P
I read the Debian policy carefully and nothing there it seems _prohibits_ parallel building.
Quote:
"parallel=n
Where is this set exactly ?
We can parse that maybe in build script and limit it to "n". Or take the dpkg tools already care of that ?
On 04/12/2011 10:19 PM, Adrian Schröter wrote:
Am Dienstag, 12. April 2011, 21:54:56 schrieb Peter Linnell:
On 04/02/2011 09:51 PM, peer wrote:
On 04/02/2011 08:20 PM, Stephan Kleine wrote:
On Saturday April 2 2011 19:13:48 you wrote:
Here is the bug report
Well, it is obvious that the debian folks don´t care about anything outside their ivory tower. I meant filing a bug at the cdbs upstream that their build scripts are broken cause being unable to build in parallel _is_ a bug IMHO, so they would perhaps fix this.
Er, sry, I forgot what cdbs is, seems like you are screwed ;P
But more seriously, not doing the builds in parallel is just a waste of time and resources and debian& buntu builds already take much longer than rpm builds (e.g. during package install) so disabling parallel builds to please some broken packages isn't an option imho. Simply wait till a way to disable them in the packages meta data is implemented and fix it in the meantime by adding "-j1" to your make call.
You didn't read the reply by the author well. The package is not broken, it's a choice to have the package like it is. That's why it's not a accident that it fails on obs and not on the Debian, Ubuntu and Launchpad build systems. They doesn't violate the Debian policy and I think that's a good choice. That's also why the cdbs author argues that obs is broken, not the package and I do agree with him here.
It's fundamentally wrong imo to provide a service for Debian packages and not accept the Debian way and the Debian policy. Of course you could discuss the way of building with the Debian devs and try to convince them (at least the cdbs author seems to be open for a rational discussion). But at the end the best thing a service for Debian packages can do, it to adapt to the Debian way. Then we as users are assured that we don't need all kinds of hacks to get our Debian packages build.
~P
I read the Debian policy carefully and nothing there it seems _prohibits_ parallel building.
Quote:
"parallel=n
Where is this set exactly ?
We can parse that maybe in build script and limit it to "n". Or take the dpkg tools already care of that ?
http://www.debian.org/doc/debian-policy/ch-source.html#s-debianrules-options Section 4.9.1
HTH, Peter
Am Dienstag, 12. April 2011, 22:38:17 schrieb Peter Linnell:
On 04/12/2011 10:19 PM, Adrian Schröter wrote:
Am Dienstag, 12. April 2011, 21:54:56 schrieb Peter Linnell:
On 04/02/2011 09:51 PM, peer wrote:
On 04/02/2011 08:20 PM, Stephan Kleine wrote:
On Saturday April 2 2011 19:13:48 you wrote:
> Here is the bug report > > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=620350 > Well, it is obvious that the debian folks don´t care about anything outside their ivory tower. I meant filing a bug at the cdbs upstream that their build scripts are broken cause being unable to build in parallel _is_ a bug IMHO, so they would perhaps fix this.
Er, sry, I forgot what cdbs is, seems like you are screwed ;P
But more seriously, not doing the builds in parallel is just a waste of time and resources and debian& buntu builds already take much longer than rpm builds (e.g. during package install) so disabling parallel builds to please some broken packages isn't an option imho. Simply wait till a way to disable them in the packages meta data is implemented and fix it in the meantime by adding "-j1" to your make call.
You didn't read the reply by the author well. The package is not broken, it's a choice to have the package like it is. That's why it's not a accident that it fails on obs and not on the Debian, Ubuntu and Launchpad build systems. They doesn't violate the Debian policy and I think that's a good choice. That's also why the cdbs author argues that obs is broken, not the package and I do agree with him here.
It's fundamentally wrong imo to provide a service for Debian packages and not accept the Debian way and the Debian policy. Of course you could discuss the way of building with the Debian devs and try to convince them (at least the cdbs author seems to be open for a rational discussion). But at the end the best thing a service for Debian packages can do, it to adapt to the Debian way. Then we as users are assured that we don't need all kinds of hacks to get our Debian packages build.
~P
I read the Debian policy carefully and nothing there it seems _prohibits_ parallel building.
Quote:
"parallel=n
Where is this set exactly ?
We can parse that maybe in build script and limit it to "n". Or take the dpkg tools already care of that ?
http://www.debian.org/doc/debian-policy/ch-source.html#s-debianrules-options Section 4.9.1
okay, so it seems that the packages really need to get fixed debian/rules file to avoid the usage of parallel builds too me. (similar as in spec files where the macro must not be used).
Thanks for digging into it. adrian
On 04/12/2011 10:56 PM, Adrian Schröter wrote:
Am Dienstag, 12. April 2011, 22:38:17 schrieb Peter Linnell:
On 04/12/2011 10:19 PM, Adrian Schröter wrote:
Am Dienstag, 12. April 2011, 21:54:56 schrieb Peter Linnell:
On 04/02/2011 09:51 PM, peer wrote:
On 04/02/2011 08:20 PM, Stephan Kleine wrote:
On Saturday April 2 2011 19:13:48 you wrote:
>> Here is the bug report >> >> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=620350 >> >> > Well, it is obvious that the debian folks don´t care about anything > outside > their ivory tower. I meant filing a bug at the cdbs upstream that their > build scripts are broken cause being unable to build in parallel _is_ a > bug IMHO, so they would perhaps fix this. > > Er, sry, I forgot what cdbs is, seems like you are screwed ;P
But more seriously, not doing the builds in parallel is just a waste of time and resources and debian& buntu builds already take much longer than rpm builds (e.g. during package install) so disabling parallel builds to please some broken packages isn't an option imho. Simply wait till a way to disable them in the packages meta data is implemented and fix it in the meantime by adding "-j1" to your make call.
You didn't read the reply by the author well. The package is not broken, it's a choice to have the package like it is. That's why it's not a accident that it fails on obs and not on the Debian, Ubuntu and Launchpad build systems. They doesn't violate the Debian policy and I think that's a good choice. That's also why the cdbs author argues that obs is broken, not the package and I do agree with him here.
It's fundamentally wrong imo to provide a service for Debian packages and not accept the Debian way and the Debian policy. Of course you could discuss the way of building with the Debian devs and try to convince them (at least the cdbs author seems to be open for a rational discussion). But at the end the best thing a service for Debian packages can do, it to adapt to the Debian way. Then we as users are assured that we don't need all kinds of hacks to get our Debian packages build.
~P
I read the Debian policy carefully and nothing there it seems _prohibits_ parallel building.
Quote:
"parallel=n
Where is this set exactly ?
We can parse that maybe in build script and limit it to "n". Or take the dpkg tools already care of that ?
http://www.debian.org/doc/debian-policy/ch-source.html#s-debianrules-options Section 4.9.1
okay, so it seems that the packages really need to get fixed debian/rules file to avoid the usage of parallel builds too me. (similar as in spec files where the macro must not be used).
Thanks for digging into it. adrian
Enabling parallel building via DEB_BUILD_OPTIONS, parallel=n seems to be the proper way to enable parallel building on Debian. But if this is not set by the package maintainer, obs shouldn't build the package in parallel, because that would violate the Debian policy.
~P
On Tuesday April 12 2011 23:36:45 peer wrote:
On 04/12/2011 10:56 PM, Adrian Schröter wrote:
Am Dienstag, 12. April 2011, 22:38:17 schrieb Peter Linnell:
On 04/12/2011 10:19 PM, Adrian Schröter wrote:
Am Dienstag, 12. April 2011, 21:54:56 schrieb Peter Linnell:
On 04/02/2011 09:51 PM, peer wrote:
On 04/02/2011 08:20 PM, Stephan Kleine wrote: > On Saturday April 2 2011 19:13:48 you wrote: >>> Here is the bug report >>> >>> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=620350 >> >> Well, it is obvious that the debian folks don´t care about anything >> outside >> their ivory tower. I meant filing a bug at the cdbs upstream that >> their build scripts are broken cause being unable to build in >> parallel _is_ a bug IMHO, so they would perhaps fix this. > > Er, sry, I forgot what cdbs is, seems like you are screwed ;P > > But more seriously, not doing the builds in parallel is just a waste > of time > and resources and debian& buntu builds already take much longer > than rpm builds (e.g. during package install) so disabling parallel > builds to please > some broken packages isn't an option imho. Simply wait till a way to > disable > them in the packages meta data is implemented and fix it in the > meantime by > adding "-j1" to your make call.
You didn't read the reply by the author well. The package is not broken, it's a choice to have the package like it is. That's why it's not a accident that it fails on obs and not on the Debian, Ubuntu and Launchpad build systems. They doesn't violate the Debian policy and I think that's a good choice. That's also why the cdbs author argues that obs is broken, not the package and I do agree with him here.
It's fundamentally wrong imo to provide a service for Debian packages and not accept the Debian way and the Debian policy. Of course you could discuss the way of building with the Debian devs and try to convince them (at least the cdbs author seems to be open for a rational discussion). But at the end the best thing a service for Debian packages can do, it to adapt to the Debian way. Then we as users are assured that we don't need all kinds of hacks to get our Debian packages build.
~P
I read the Debian policy carefully and nothing there it seems _prohibits_ parallel building.
Quote:
"parallel=n
Where is this set exactly ?
We can parse that maybe in build script and limit it to "n". Or take the dpkg tools already care of that ?
http://www.debian.org/doc/debian-policy/ch-source.html#s-debianrules-opt ions Section 4.9.1
okay, so it seems that the packages really need to get fixed debian/rules file to avoid the usage of parallel builds too me. (similar as in spec files where the macro must not be used).
Thanks for digging into it. adrian
Enabling parallel building via DEB_BUILD_OPTIONS, parallel=n seems to be the proper way to enable parallel building on Debian. But if this is not set by the package maintainer, obs shouldn't build the package in parallel, because that would violate the Debian policy.
~P
No, make it the other way round. Until explicitly disabled in the packages meta data build the stuff in parallel since .deb builds already waste enough time during package installation and most people probably will never know or care to enable parallel builds for working packages.
On Wednesday 13 April 2011, 19:18:03 Stephan Kleine wrote:
On Tuesday April 12 2011 23:36:45 peer wrote:
On 04/12/2011 10:56 PM, Adrian Schröter wrote:
Am Dienstag, 12. April 2011, 22:38:17 schrieb Peter Linnell:
On 04/12/2011 10:19 PM, Adrian Schröter wrote:
Am Dienstag, 12. April 2011, 21:54:56 schrieb Peter Linnell:
On 04/02/2011 09:51 PM, peer wrote: > On 04/02/2011 08:20 PM, Stephan Kleine wrote: >> On Saturday April 2 2011 19:13:48 you wrote: >>>> Here is the bug report >>>> >>>> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=620350 >>> >>> Well, it is obvious that the debian folks don´t care about >>> anything outside >>> their ivory tower. I meant filing a bug at the cdbs >>> upstream that their build scripts are broken cause being >>> unable to build in parallel _is_ a bug IMHO, so they would >>> perhaps fix this. >> >> Er, sry, I forgot what cdbs is, seems like you are screwed >> ;P >> >> But more seriously, not doing the builds in parallel is just >> a waste of time >> and resources and debian& buntu builds already take much >> longer than rpm builds (e.g. during package install) so >> disabling parallel builds to please >> some broken packages isn't an option imho. Simply wait till >> a way to disable >> them in the packages meta data is implemented and fix it in >> the meantime by >> adding "-j1" to your make call. > > You didn't read the reply by the author well. The package is > not broken, it's a choice to have the package like it is. > That's why it's not a accident that it fails on obs and not > on the Debian, Ubuntu and Launchpad build systems. They > doesn't violate the Debian policy and I think that's a good > choice. That's also why the cdbs author argues that obs is > broken, not the package and I do agree with him here. > > It's fundamentally wrong imo to provide a service for Debian > packages and not accept the Debian way and the Debian policy. > Of course you could discuss the way of building with the > Debian devs and try to convince them (at least the cdbs > author seems to be open for a rational discussion). But at > the end the best thing a service for Debian packages can do, > it to adapt to the Debian way. Then we as users are assured > that we don't need all kinds of hacks to get our Debian > packages build. > > > ~P
I read the Debian policy carefully and nothing there it seems _prohibits_ parallel building.
Quote:
"parallel=n
Where is this set exactly ?
We can parse that maybe in build script and limit it to "n". Or take the dpkg tools already care of that ?
http://www.debian.org/doc/debian-policy/ch-source.html#s-debianr ules-opt ions Section 4.9.1
okay, so it seems that the packages really need to get fixed debian/rules file to avoid the usage of parallel builds too me. (similar as in spec files where the macro must not be used).
Thanks for digging into it. adrian
Enabling parallel building via DEB_BUILD_OPTIONS, parallel=n seems to be the proper way to enable parallel building on Debian. But if this is not set by the package maintainer, obs shouldn't build the package in parallel, because that would violate the Debian policy.
~P
No, make it the other way round. Until explicitly disabled in the packages meta data build the stuff in parallel since .deb builds already waste enough time during package installation and most people probably will never know or care to enable parallel builds for working packages.
And if they choose to punish themself, why not meet their policy, but simply penalize those non parallel builds in the scheduler adequately in order to lower burden of other BS users.
Guess, how fast the self-healing power will change this silly situation.
Pete
On Wednesday April 13 2011 20:52:09 Hans-Peter Jansen wrote:
On Wednesday 13 April 2011, 19:18:03 Stephan Kleine wrote:
On Tuesday April 12 2011 23:36:45 peer wrote:
On 04/12/2011 10:56 PM, Adrian Schröter wrote:
Am Dienstag, 12. April 2011, 22:38:17 schrieb Peter Linnell:
On 04/12/2011 10:19 PM, Adrian Schröter wrote:
Am Dienstag, 12. April 2011, 21:54:56 schrieb Peter Linnell: > On 04/02/2011 09:51 PM, peer wrote: >> On 04/02/2011 08:20 PM, Stephan Kleine wrote: >>> On Saturday April 2 2011 19:13:48 you wrote: >>>>> Here is the bug report >>>>> >>>>> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=620350 >>>> >>>> Well, it is obvious that the debian folks don´t care about >>>> anything outside >>>> their ivory tower. I meant filing a bug at the cdbs >>>> upstream that their build scripts are broken cause being >>>> unable to build in parallel _is_ a bug IMHO, so they would >>>> perhaps fix this. >>> >>> Er, sry, I forgot what cdbs is, seems like you are screwed >>> ;P >>> >>> But more seriously, not doing the builds in parallel is just >>> a waste of time >>> and resources and debian& buntu builds already take much >>> longer than rpm builds (e.g. during package install) so >>> disabling parallel builds to please >>> some broken packages isn't an option imho. Simply wait till >>> a way to disable >>> them in the packages meta data is implemented and fix it in >>> the meantime by >>> adding "-j1" to your make call. >> >> You didn't read the reply by the author well. The package is >> not broken, it's a choice to have the package like it is. >> That's why it's not a accident that it fails on obs and not >> on the Debian, Ubuntu and Launchpad build systems. They >> doesn't violate the Debian policy and I think that's a good >> choice. That's also why the cdbs author argues that obs is >> broken, not the package and I do agree with him here. >> >> It's fundamentally wrong imo to provide a service for Debian >> packages and not accept the Debian way and the Debian policy. >> Of course you could discuss the way of building with the >> Debian devs and try to convince them (at least the cdbs >> author seems to be open for a rational discussion). But at >> the end the best thing a service for Debian packages can do, >> it to adapt to the Debian way. Then we as users are assured >> that we don't need all kinds of hacks to get our Debian >> packages build. >> >> >> ~P > > I read the Debian policy carefully and nothing there it seems > _prohibits_ parallel building. > > Quote: > > "parallel=n
Where is this set exactly ?
We can parse that maybe in build script and limit it to "n". Or take the dpkg tools already care of that ?
http://www.debian.org/doc/debian-policy/ch-source.html#s-debianr
ules-opt ions Section 4.9.1
okay, so it seems that the packages really need to get fixed debian/rules file to avoid the usage of parallel builds too me. (similar as in spec files where the macro must not be used).
Thanks for digging into it. adrian
Enabling parallel building via DEB_BUILD_OPTIONS, parallel=n seems to be the proper way to enable parallel building on Debian. But if this is not set by the package maintainer, obs shouldn't build the package in parallel, because that would violate the Debian policy.
~P
No, make it the other way round. Until explicitly disabled in the packages meta data build the stuff in parallel since .deb builds already waste enough time during package installation and most people probably will never know or care to enable parallel builds for working packages.
And if they choose to punish themself, why not meet their policy, but simply penalize those non parallel builds in the scheduler adequately in order to lower burden of other BS users.
Guess, how fast the self-healing power will change this silly situation.
Again, because that "policy" is screwing over everyone else cause the .deb builds already waste loads of time (e.g. the package installation takes considerable longer than on rpm based distros) so anything that makes .deb builds faster is plain awesome in my book.
Also the majority of the packages should "just work" when build in parallel because - again - the only reason that wouldn't work is because the packages build scripts have a bug in the first place.
So please explain me why everyone else should get punished and time get wasted just cause a handful of people happen to want to build broken packages they refuse to fix?
So simply fix your packages build scripts and send the patch upstream or implement some option to turn of parallel builds in the packages meta data or wait till someone else does it for you but stop punishing everyone cause your stuff is broken and you refuse to fix it.
regards, Stephan
On Thursday 14 April 2011, 18:53:28 Stephan Kleine wrote:
On Wednesday April 13 2011 20:52:09 Hans-Peter Jansen wrote:
On Wednesday 13 April 2011, 19:18:03 Stephan Kleine wrote:
On Tuesday April 12 2011 23:36:45 peer wrote:
On 04/12/2011 10:56 PM, Adrian Schröter wrote:
Am Dienstag, 12. April 2011, 22:38:17 schrieb Peter Linnell:
On 04/12/2011 10:19 PM, Adrian Schröter wrote: > Am Dienstag, 12. April 2011, 21:54:56 schrieb Peter Linnell: >> On 04/02/2011 09:51 PM, peer wrote: >>> On 04/02/2011 08:20 PM, Stephan Kleine wrote: >>>> On Saturday April 2 2011 19:13:48 you wrote: >>>>>> Here is the bug report >>>>>> >>>>>> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=62035 >>>>>>0 >>>>> >>>>> Well, it is obvious that the debian folks don´t care >>>>> about anything outside >>>>> their ivory tower. I meant filing a bug at the cdbs >>>>> upstream that their build scripts are broken cause >>>>> being unable to build in parallel _is_ a bug IMHO, so >>>>> they would perhaps fix this. >>>> >>>> Er, sry, I forgot what cdbs is, seems like you are >>>> screwed ;P >>>> >>>> But more seriously, not doing the builds in parallel is >>>> just a waste of time >>>> and resources and debian& buntu builds already take >>>> much longer than rpm builds (e.g. during package >>>> install) so disabling parallel builds to please >>>> some broken packages isn't an option imho. Simply wait >>>> till a way to disable >>>> them in the packages meta data is implemented and fix it >>>> in the meantime by >>>> adding "-j1" to your make call. >>> >>> You didn't read the reply by the author well. The package >>> is not broken, it's a choice to have the package like it >>> is. That's why it's not a accident that it fails on obs >>> and not on the Debian, Ubuntu and Launchpad build >>> systems. They doesn't violate the Debian policy and I >>> think that's a good choice. That's also why the cdbs >>> author argues that obs is broken, not the package and I >>> do agree with him here. >>> >>> It's fundamentally wrong imo to provide a service for >>> Debian packages and not accept the Debian way and the >>> Debian policy. Of course you could discuss the way of >>> building with the Debian devs and try to convince them >>> (at least the cdbs author seems to be open for a rational >>> discussion). But at the end the best thing a service for >>> Debian packages can do, it to adapt to the Debian way. >>> Then we as users are assured that we don't need all kinds >>> of hacks to get our Debian packages build. >>> >>> >>> ~P >> >> I read the Debian policy carefully and nothing there it >> seems _prohibits_ parallel building. >> >> Quote: >> >> "parallel=n > > Where is this set exactly ? > > We can parse that maybe in build script and limit it to > "n". Or take the dpkg tools already care of that ?
http://www.debian.org/doc/debian-policy/ch-source.html#s-deb ianr
ules-opt ions Section 4.9.1
okay, so it seems that the packages really need to get fixed debian/rules file to avoid the usage of parallel builds too me. (similar as in spec files where the macro must not be used).
Thanks for digging into it. adrian
Enabling parallel building via DEB_BUILD_OPTIONS, parallel=n seems to be the proper way to enable parallel building on Debian. But if this is not set by the package maintainer, obs shouldn't build the package in parallel, because that would violate the Debian policy.
~P
No, make it the other way round. Until explicitly disabled in the packages meta data build the stuff in parallel since .deb builds already waste enough time during package installation and most people probably will never know or care to enable parallel builds for working packages.
And if they choose to punish themself, why not meet their policy, but simply penalize those non parallel builds in the scheduler adequately in order to lower burden of other BS users.
Guess, how fast the self-healing power will change this silly situation.
Again, because that "policy" is screwing over everyone else cause the .deb builds already waste loads of time (e.g. the package installation takes considerable longer than on rpm based distros) so anything that makes .deb builds faster is plain awesome in my book.
Also the majority of the packages should "just work" when build in parallel because - again - the only reason that wouldn't work is because the packages build scripts have a bug in the first place.
So please explain me why everyone else should get punished and time get wasted just cause a handful of people happen to want to build broken packages they refuse to fix?
So simply fix your packages build scripts and send the patch upstream or implement some option to turn of parallel builds in the packages meta data or wait till someone else does it for you but stop punishing everyone cause your stuff is broken and you refuse to fix it.
Stephan, you probably misunderstood me. Given, debian require BS to build their stuff with a single CPU by default, no problem, just penalize these builds in the scheduler by _lowering_ their priority with _factor_ 10 (est. avg. 6 CPUs, e.g. estimated to hog a build node 10 times longer than other projects packages, iow. 10 RPM builds and then one "single CPU" debian build).
Result: bigger debian builds will take ages, but this is exactly, what they _demand_¹. debian packagers, that run off their patience will try to solve this by enabling parallel builds in their packages.
The idea is, that debian packagers stop being jobworths, and start fixing their issues for their own sake and the rest of us doesn't suffer from debian project policy silliness too much.
This is not about punishment, it's about fairness and raising the attraction to fix this with self-healing power.
Hopefully, I expressed myself better this time, Pete
¹) Well, not exactly, but OTOH this scheduler penalization does work pretty well: before Stephan Kulow fixed this, I remember having packages hanging around in scheduled state for several _days_! Enough time for them to think about solutions to the problem.
On 04/14/2011 10:04 PM, Hans-Peter Jansen wrote:
On Thursday 14 April 2011, 18:53:28 Stephan Kleine wrote:
On Wednesday April 13 2011 20:52:09 Hans-Peter Jansen wrote:
On Wednesday 13 April 2011, 19:18:03 Stephan Kleine wrote:
On Tuesday April 12 2011 23:36:45 peer wrote:
On 04/12/2011 10:56 PM, Adrian Schröter wrote:
Am Dienstag, 12. April 2011, 22:38:17 schrieb Peter Linnell:
> On 04/12/2011 10:19 PM, Adrian Schröter wrote: > >> Am Dienstag, 12. April 2011, 21:54:56 schrieb Peter Linnell: >> >>> On 04/02/2011 09:51 PM, peer wrote: >>> >>>> On 04/02/2011 08:20 PM, Stephan Kleine wrote: >>>> >>>>> On Saturday April 2 2011 19:13:48 you wrote: >>>>> >>>>>>> Here is the bug report >>>>>>> >>>>>>> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=62035 >>>>>>> 0 >>>>>>> >>>>>> Well, it is obvious that the debian folks don´t care >>>>>> about anything outside >>>>>> their ivory tower. I meant filing a bug at the cdbs >>>>>> upstream that their build scripts are broken cause >>>>>> being unable to build in parallel _is_ a bug IMHO, so >>>>>> they would perhaps fix this. >>>>>> >>>>> Er, sry, I forgot what cdbs is, seems like you are >>>>> screwed ;P >>>>> >>>>> But more seriously, not doing the builds in parallel is >>>>> just a waste of time >>>>> and resources and debian& buntu builds already take >>>>> much longer than rpm builds (e.g. during package >>>>> install) so disabling parallel builds to please >>>>> some broken packages isn't an option imho. Simply wait >>>>> till a way to disable >>>>> them in the packages meta data is implemented and fix it >>>>> in the meantime by >>>>> adding "-j1" to your make call. >>>>> >>>> You didn't read the reply by the author well. The package >>>> is not broken, it's a choice to have the package like it >>>> is. That's why it's not a accident that it fails on obs >>>> and not on the Debian, Ubuntu and Launchpad build >>>> systems. They doesn't violate the Debian policy and I >>>> think that's a good choice. That's also why the cdbs >>>> author argues that obs is broken, not the package and I >>>> do agree with him here. >>>> >>>> It's fundamentally wrong imo to provide a service for >>>> Debian packages and not accept the Debian way and the >>>> Debian policy. Of course you could discuss the way of >>>> building with the Debian devs and try to convince them >>>> (at least the cdbs author seems to be open for a rational >>>> discussion). But at the end the best thing a service for >>>> Debian packages can do, it to adapt to the Debian way. >>>> Then we as users are assured that we don't need all kinds >>>> of hacks to get our Debian packages build. >>>> >>>> >>>> ~P >>>> >>> I read the Debian policy carefully and nothing there it >>> seems _prohibits_ parallel building. >>> >>> Quote: >>> >>> "parallel=n >>> >> Where is this set exactly ? >> >> We can parse that maybe in build script and limit it to >> "n". Or take the dpkg tools already care of that ? >> > http://www.debian.org/doc/debian-policy/ch-source.html#s-deb > ianr > > ules-opt ions Section 4.9.1 > okay, so it seems that the packages really need to get fixed debian/rules file to avoid the usage of parallel builds too me. (similar as in spec files where the macro must not be used).
Thanks for digging into it. adrian
Enabling parallel building via DEB_BUILD_OPTIONS, parallel=n seems to be the proper way to enable parallel building on Debian. But if this is not set by the package maintainer, obs shouldn't build the package in parallel, because that would violate the Debian policy.
~P
No, make it the other way round. Until explicitly disabled in the packages meta data build the stuff in parallel since .deb builds already waste enough time during package installation and most people probably will never know or care to enable parallel builds for working packages.
And if they choose to punish themself, why not meet their policy, but simply penalize those non parallel builds in the scheduler adequately in order to lower burden of other BS users.
Guess, how fast the self-healing power will change this silly situation.
Again, because that "policy" is screwing over everyone else cause the .deb builds already waste loads of time (e.g. the package installation takes considerable longer than on rpm based distros) so anything that makes .deb builds faster is plain awesome in my book.
Also the majority of the packages should "just work" when build in parallel because - again - the only reason that wouldn't work is because the packages build scripts have a bug in the first place.
So please explain me why everyone else should get punished and time get wasted just cause a handful of people happen to want to build broken packages they refuse to fix?
So simply fix your packages build scripts and send the patch upstream or implement some option to turn of parallel builds in the packages meta data or wait till someone else does it for you but stop punishing everyone cause your stuff is broken and you refuse to fix it.
Stephan, you probably misunderstood me. Given, debian require BS to build their stuff with a single CPU by default, no problem, just penalize these builds in the scheduler by _lowering_ their priority with _factor_ 10 (est. avg. 6 CPUs, e.g. estimated to hog a build node 10 times longer than other projects packages, iow. 10 RPM builds and then one "single CPU" debian build).
Result: bigger debian builds will take ages, but this is exactly, what they _demand_¹. debian packagers, that run off their patience will try to solve this by enabling parallel builds in their packages.
The idea is, that debian packagers stop being jobworths, and start fixing their issues for their own sake and the rest of us doesn't suffer from debian project policy silliness too much.
This is not about punishment, it's about fairness and raising the attraction to fix this with self-healing power.
Hopefully, I expressed myself better this time, Pete
¹) Well, not exactly, but OTOH this scheduler penalization does work pretty well: before Stephan Kulow fixed this, I remember having packages hanging around in scheduled state for several _days_! Enough time for them to think about solutions to the problem.
This doesn't sounds like an constructive and adult way to solve these issues. You harm the end-users because of something they have little influence on. People use obs just to publish packages for their projects. They aren't necessarily Debian developers themselves. The real Debian developers do prefer other services instead of obs, so they won't be impressed or forced to change their policy at all.
At the end you only hurt the end-user of obs who likes to publish Ubuntu | Debian packages via obs. If you slow down the build process of Debian packages heavily, the only thing you achieve is that people like me are forced to choose other services like Launchpad. I personally prefer obs, but this discussion doesn't make me feel that obs really cares about the Debian based projects and it makes me wonder why it even provide the option to build for Debian.
Best regards,
~P
On 14.4.2011 22:04, Hans-Peter Jansen wrote:
Stephan, you probably misunderstood me. Given, debian require BS to build their stuff with a single CPU by default, no problem, just penalize these builds in the scheduler by _lowering_ their priority with _factor_ 10 (est. avg. 6 CPUs, e.g. estimated to hog a build node 10 times longer than other projects packages, iow. 10 RPM builds and then one "single CPU" debian build).
It's 2011. So how about just passing -j1 to the one or two packages that still don't work, instead of implementing elaborate solutions that needlessly penalize all deb builds? Mindlessly sticking to questionable policies kills any progress.
Michal
On 04/14/2011 10:46 PM, Michal Marek wrote:
On 14.4.2011 22:04, Hans-Peter Jansen wrote:
Stephan, you probably misunderstood me. Given, debian require BS to build their stuff with a single CPU by default, no problem, just penalize these builds in the scheduler by _lowering_ their priority with _factor_ 10 (est. avg. 6 CPUs, e.g. estimated to hog a build node 10 times longer than other projects packages, iow. 10 RPM builds and then one "single CPU" debian build).
It's 2011. So how about just passing -j1 to the one or two packages that still don't work,
Is that something I should do as end-user? Via local build? How do I publish that package then?
Best regards,
~P
On Thu, Apr 14, 2011 at 10:56:09PM +0200, peer wrote:
On 04/14/2011 10:46 PM, Michal Marek wrote:
On 14.4.2011 22:04, Hans-Peter Jansen wrote:
Stephan, you probably misunderstood me. Given, debian require BS to build their stuff with a single CPU by default, no problem, just penalize these builds in the scheduler by _lowering_ their priority with _factor_ 10 (est. avg. 6 CPUs, e.g. estimated to hog a build node 10 times longer than other projects packages, iow. 10 RPM builds and then one "single CPU" debian build).
It's 2011. So how about just passing -j1 to the one or two packages that still don't work,
Is that something I should do as end-user? Via local build? How do I publish that package then?
You should patch the build scripts to pass -j1 to make. Looking at the package in question,
diff --git a/1/class/makefile-vars.mk.in b/1/class/makefile-vars.mk.in index 4cdc2c1..9154fbc 100644 --- a/1/class/makefile-vars.mk.in +++ b/1/class/makefile-vars.mk.in @@ -31,7 +31,7 @@ DEB_MAKE_PARALLEL ?= $(and $(DEB_BUILD_PARALLEL),$(DEB_PARALLEL_JOBS),-j$(DEB_PA # should set this variable to empty. DEB_MAKE_EXTRA_ARGS ?= CFLAGS="$(or $(CFLAGS_$(cdbs_curpkg)),$(CFLAGS))" CXXFLAGS="$(or $(CXXFLAGS_$(cdbs_curpkg)),$(CXXFLAGS))" CPPFLAGS="$(or $(CPPFLAGS_$(cdbs_curpkg)),$(CPPFLAGS))" LDFLAGS="$(or $(LDFLAGS_$(cdbs_curpkg)),$(LDFLAGS))" $(DEB_MAKE_PARALLEL)
-DEB_MAKE_INVOKE ?= $(DEB_MAKE_ENVVARS) $(MAKE) $(if $(DEB_MAKE_MAKEFILE), -f $(DEB_MAKE_MAKEFILE),) -C $(cdbs_make_curbuilddir) $(DEB_MAKE_EXTRA_ARGS) +DEB_MAKE_INVOKE ?= $(DEB_MAKE_ENVVARS) $(MAKE) $(if $(DEB_MAKE_MAKEFILE), -f $(DEB_MAKE_MAKEFILE),) -C $(cdbs_make_curbuilddir) $(DEB_MAKE_EXTRA_ARGS) -j1
DEB_MAKE_BUILD_TARGET ?= $(DEB_BUILD_MAKE_TARGET)
On Thursday 14 April 2011, 22:46:07 Michal Marek wrote:
On 14.4.2011 22:04, Hans-Peter Jansen wrote:
Stephan, you probably misunderstood me. Given, debian require BS to build their stuff with a single CPU by default, no problem, just penalize these builds in the scheduler by _lowering_ their priority with _factor_ 10 (est. avg. 6 CPUs, e.g. estimated to hog a build node 10 times longer than other projects packages, iow. 10 RPM builds and then one "single CPU" debian build).
It's 2011. So how about just passing -j1 to the one or two packages that still don't work, instead of implementing elaborate solutions that needlessly penalize all deb builds? Mindlessly sticking to questionable policies kills any progress.
Michal, debian policy _requires_ BS to build debian packages with a single CPU, if not stated _otherwise_.
Your suggestion is fine, but it doesn't fit the the problem in question. Please note the $subject, and what both debian packagers in this thread say.
As long as they stick to their _policy_ and some of their developers _insist_ in complying to it, how can you avoid to "still violating the debian policy §4.9.1", or take the whole burden of fixing _their_ sillyness.
Probably, I should consult an attorney before further commenting to this thread.
Pete
Am Freitag, 15. April 2011, 00:21:37 schrieb Hans-Peter Jansen:
On Thursday 14 April 2011, 22:46:07 Michal Marek wrote:
On 14.4.2011 22:04, Hans-Peter Jansen wrote:
Stephan, you probably misunderstood me. Given, debian require BS to build their stuff with a single CPU by default, no problem, just penalize these builds in the scheduler by _lowering_ their priority with _factor_ 10 (est. avg. 6 CPUs, e.g. estimated to hog a build node 10 times longer than other projects packages, iow. 10 RPM builds and then one "single CPU" debian build).
It's 2011. So how about just passing -j1 to the one or two packages that still don't work, instead of implementing elaborate solutions that needlessly penalize all deb builds? Mindlessly sticking to questionable policies kills any progress.
Michal, debian policy _requires_ BS to build debian packages with a single CPU, if not stated _otherwise_.
understanding this, I have commented out the parallel build parameter in build script again.
However, we have no switch to enable it again so far.
On 04/15/2011 08:23 AM, Adrian Schröter wrote:
Am Freitag, 15. April 2011, 00:21:37 schrieb Hans-Peter Jansen:
On Thursday 14 April 2011, 22:46:07 Michal Marek wrote:
On 14.4.2011 22:04, Hans-Peter Jansen wrote:
Stephan, you probably misunderstood me. Given, debian require BS to build their stuff with a single CPU by default, no problem, just penalize these builds in the scheduler by _lowering_ their priority with _factor_ 10 (est. avg. 6 CPUs, e.g. estimated to hog a build node 10 times longer than other projects packages, iow. 10 RPM builds and then one "single CPU" debian build).
It's 2011. So how about just passing -j1 to the one or two packages that still don't work, instead of implementing elaborate solutions that needlessly penalize all deb builds? Mindlessly sticking to questionable policies kills any progress.
Michal, debian policy _requires_ BS to build debian packages with a single CPU, if not stated _otherwise_.
understanding this, I have commented out the parallel build parameter in build script again.
However, we have no switch to enable it again so far.
Ok, thanks.
However the package still seems to fail https://build.opensuse.org/package/live_build_log?arch=i586&package=cdbs...
~P
On 04/21/2011 11:03 AM, peer wrote:
On 04/15/2011 08:23 AM, Adrian Schröter wrote:
Am Freitag, 15. April 2011, 00:21:37 schrieb Hans-Peter Jansen:
On Thursday 14 April 2011, 22:46:07 Michal Marek wrote:
On 14.4.2011 22:04, Hans-Peter Jansen wrote:
Stephan, you probably misunderstood me. Given, debian require BS to build their stuff with a single CPU by default, no problem, just penalize these builds in the scheduler by _lowering_ their priority with _factor_ 10 (est. avg. 6 CPUs, e.g. estimated to hog a build node 10 times longer than other projects packages, iow. 10 RPM builds and then one "single CPU" debian build).
It's 2011. So how about just passing -j1 to the one or two packages that still don't work, instead of implementing elaborate solutions that needlessly penalize all deb builds? Mindlessly sticking to questionable policies kills any progress.
Michal, debian policy _requires_ BS to build debian packages with a single CPU, if not stated _otherwise_.
understanding this, I have commented out the parallel build parameter in build script again.
However, we have no switch to enable it again so far.
Ok, thanks.
However the package still seems to fail https://build.opensuse.org/package/live_build_log?arch=i586&package=cdbs...
This is on a Ubuntu based project. I do assume the parallel build parameter is also commented out on the Debian based projects, e.g. Ubuntu.
Regards, ~P
On 04/21/2011 11:07 AM, peer wrote:
On 04/21/2011 11:03 AM, peer wrote:
On 04/15/2011 08:23 AM, Adrian Schröter wrote:
Am Freitag, 15. April 2011, 00:21:37 schrieb Hans-Peter Jansen:
On Thursday 14 April 2011, 22:46:07 Michal Marek wrote:
On 14.4.2011 22:04, Hans-Peter Jansen wrote:
Stephan, you probably misunderstood me. Given, debian require BS to build their stuff with a single CPU by default, no problem, just penalize these builds in the scheduler by _lowering_ their priority with _factor_ 10 (est. avg. 6 CPUs, e.g. estimated to hog a build node 10 times longer than other projects packages, iow. 10 RPM builds and then one "single CPU" debian build).
It's 2011. So how about just passing -j1 to the one or two packages that still don't work, instead of implementing elaborate solutions that needlessly penalize all deb builds? Mindlessly sticking to questionable policies kills any progress.
Michal, debian policy _requires_ BS to build debian packages with a single CPU, if not stated _otherwise_.
understanding this, I have commented out the parallel build parameter in build script again.
However, we have no switch to enable it again so far.
Ok, thanks.
However the package still seems to fail https://build.opensuse.org/package/live_build_log?arch=i586&package=cdbs...
This is on a Ubuntu based project. I do assume the parallel build parameter is also commented out on the Debian based projects, e.g. Ubuntu.
The package builds fine on a Debian based project. So it seems that the parallel build parameter is not commented out for Ubuntu. Could you fix that please?
Thanks in advance, ~P
On 04/21/2011 12:09 PM, peer wrote:
On 04/21/2011 11:07 AM, peer wrote:
On 04/21/2011 11:03 AM, peer wrote:
On 04/15/2011 08:23 AM, Adrian Schröter wrote:
Am Freitag, 15. April 2011, 00:21:37 schrieb Hans-Peter Jansen:
On Thursday 14 April 2011, 22:46:07 Michal Marek wrote:
On 14.4.2011 22:04, Hans-Peter Jansen wrote: > Stephan, you probably misunderstood me. Given, debian require BS to > build their stuff with a single CPU by default, no problem, just > penalize these builds in the scheduler by _lowering_ their priority > with _factor_ 10 (est. avg. 6 CPUs, e.g. estimated to hog a build > node 10 times longer than other projects packages, iow. 10 RPM > builds and then one "single CPU" debian build). It's 2011. So how about just passing -j1 to the one or two packages that still don't work, instead of implementing elaborate solutions that needlessly penalize all deb builds? Mindlessly sticking to questionable policies kills any progress.
Michal, debian policy _requires_ BS to build debian packages with a single CPU, if not stated _otherwise_.
understanding this, I have commented out the parallel build parameter in build script again.
However, we have no switch to enable it again so far.
Ok, thanks.
However the package still seems to fail https://build.opensuse.org/package/live_build_log?arch=i586&package=cdbs...
This is on a Ubuntu based project. I do assume the parallel build parameter is also commented out on the Debian based projects, e.g. Ubuntu.
The package builds fine on a Debian based project. So it seems that the parallel build parameter is not commented out for Ubuntu. Could you fix that please?
Thanks in advance, ~P
Hi,
Is this fixed in Ubuntu already?
Thanks.
~P
On 15.4.2011 00:21, Hans-Peter Jansen wrote:
It's 2011. So how about just passing -j1 to the one or two packages that still don't work, instead of implementing elaborate solutions that needlessly penalize all deb builds? Mindlessly sticking to questionable policies kills any progress.
Michal, debian policy _requires_ BS to build debian packages with a single CPU, if not stated _otherwise_.
I wouldn't be surprised it the policy also required any package to fit on a 3.5" floppy. But most of the world uses neither floppies nor UP machines nowadays, so makefiles are expected to work fine with -j. If not, it's a bug.
But now you now got what you wanted, so the BS is super compliant, just a little bit unusable for building deb packages. *sigh*
Probably, I should consult an attorney before further commenting to this thread.
Huh? :-)
Michal
On 04/12/2011 09:54 PM, Peter Linnell wrote:
On 04/02/2011 09:51 PM, peer wrote:
On 04/02/2011 08:20 PM, Stephan Kleine wrote:
On Saturday April 2 2011 19:13:48 you wrote:
Here is the bug report
Well, it is obvious that the debian folks don´t care about anything outside their ivory tower. I meant filing a bug at the cdbs upstream that their build scripts are broken cause being unable to build in parallel _is_ a bug IMHO, so they would perhaps fix this.
Er, sry, I forgot what cdbs is, seems like you are screwed ;P
But more seriously, not doing the builds in parallel is just a waste of time and resources and debian& buntu builds already take much longer than rpm builds (e.g. during package install) so disabling parallel builds to please some broken packages isn't an option imho. Simply wait till a way to disable them in the packages meta data is implemented and fix it in the meantime by adding "-j1" to your make call.
You didn't read the reply by the author well. The package is not broken, it's a choice to have the package like it is. That's why it's not a accident that it fails on obs and not on the Debian, Ubuntu and Launchpad build systems. They doesn't violate the Debian policy and I think that's a good choice. That's also why the cdbs author argues that obs is broken, not the package and I do agree with him here.
It's fundamentally wrong imo to provide a service for Debian packages and not accept the Debian way and the Debian policy. Of course you could discuss the way of building with the Debian devs and try to convince them (at least the cdbs author seems to be open for a rational discussion). But at the end the best thing a service for Debian packages can do, it to adapt to the Debian way. Then we as users are assured that we don't need all kinds of hacks to get our Debian packages build.
~P
I read the Debian policy carefully and nothing there it seems _prohibits_ parallel building.
Quote:
"parallel=n
This tag means that the package should be built using up to nparallel processes if the package build system supports this.[25] If the package build system does not support parallel builds, this string must be ignored. If the package build system only supports a lower level of concurrency than n, the package should be built using as many parallel processes as the package build system supports. It is up to the package maintainer to decide whether the package build times are long enough and the package build system is robust enough to make supporting parallel builds worthwhile."
So, reading that it seems then that in the case of OBS, it can provide a robust build system to parallel builds. Therefore, perhaps it might be worthwhile for Debian to reconsider what might be an out of date policy
- especially in light of the prevalence of multi-core CPUS.
Moreover, from my long time experience with OBS as a user/packager, the statement that OBS is broken is fundamentally wrong. It is one of the true gems of openSUSE. I'd sense more NIH syndrome :(
Peter
QUOTE:
"It is up to the package maintainer to decide whether the package build times are long enough and the package build system is robust enough to make supporting parallel builds worthwhile."
It's up to the package maintainer to decide, not to obs. That's the point.
~P
On Tue, Apr 12, 2011 at 10:36:04PM +0200, peer wrote:
So, reading that it seems then that in the case of OBS, it can provide a robust build system to parallel builds. Therefore, perhaps it might be worthwhile for Debian to reconsider what might be an out of date policy
- especially in light of the prevalence of multi-core CPUS.
Indeed, and I belive that DEB_BUILD_OPTIONS is meant to be a progression towards this. I may even suggest it as a goal for the next release, to have -jN safe compiles for all packages. It'd certainly improve package quality.
"It is up to the package maintainer to decide whether the package build times are long enough and the package build system is robust enough to make supporting parallel builds worthwhile."
It's up to the package maintainer to decide, not to obs. That's the point.
Well, yes. I would agree that by default, Debian packages should be built with -j1.
Mind you, getting the data as to which packages *don't* build with N > 1 would also be very useful.
Neil (aka, neilm@debian.org, Debian Release Manager)
On 04/13/2011 09:38 AM, Neil McGovern wrote:
"It is up to the package maintainer to decide whether the package
build times are long enough and the package build system is robust enough to make supporting parallel builds worthwhile."
It's up to the package maintainer to decide, not to obs. That's the point.
Well, yes. I would agree that by default, Debian packages should be built with -j1. Neil (aka, neilm@debian.org, Debian Release Manager)
Right. And packagers which are set to build in parallel by the Debian package maintainer should be build in parallel.
~P
On Wed, Apr 13, 2011 at 07:21:49PM +0200, peer wrote:
On 04/13/2011 09:38 AM, Neil McGovern wrote:
"It is up to the package maintainer to decide whether the package
build times are long enough and the package build system is robust enough to make supporting parallel builds worthwhile."
It's up to the package maintainer to decide, not to obs. That's the point.
Well, yes. I would agree that by default, Debian packages should be built with -j1. Neil (aka, neilm@debian.org, Debian Release Manager)
Right. And packagers which are set to build in parallel by the Debian package maintainer should be build in parallel.
That would be a choice for OpenSUSE build service authors, so I won't comment about what it *should* do, but I belive that it *could* do sucessfully.
Neil
On 04/13/2011 07:55 PM, Neil McGovern wrote:
On Wed, Apr 13, 2011 at 07:21:49PM +0200, peer wrote:
On 04/13/2011 09:38 AM, Neil McGovern wrote:
"It is up to the package maintainer to decide whether the package
build times are long enough and the package build system is robust enough to make supporting parallel builds worthwhile."
It's up to the package maintainer to decide, not to obs. That's the point.
Well, yes. I would agree that by default, Debian packages should be built with -j1. Neil (aka, neilm@debian.org, Debian Release Manager)
Right. And packagers which are set to build in parallel by the Debian package maintainer should be build in parallel.
That would be a choice for OpenSUSE build service authors, so I won't comment about what it *should* do, but I belive that it *could* do sucessfully.
You're right and *could* might fit better then *should* here.
~p
buildservice@lists.opensuse.org
-
Adrian Schröter -
Dimstar / Dominique Leuenberger -
Dominique Leuenberger a.k.a DimStar
-
Hans-Peter Jansen -
Michal Marek -
Neil McGovern -
peer -
Peter Linnell -
Stephan Kleine