[opensuse-buildservice] question about package aggregate
Hi, In file BSXML.pm: our $aggregatelist = [ 'aggregatelist' => [[ 'aggregate' => 'project', [ 'package' ], [ 'binary' ], [[ 'repository' => 'target', 'source', ]], ]], ]; According to the code in bs_sched, the binary item is treated as a filter to ignore a binary file when copying files in rebuildaggregate. But there seems no user interface to set value for the item. Does it mean this feature is only partially implemented? And there is a feature in openFate -- https://features.opensuse.org/309350, which is to skip source files when aggregating. Does it mean that an extra item is needed, such as "source", for feature 309350? ---------------- Best Regards, Brook Hong -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Hello Adrian Schröter, Could you help to clarify this? ---------------- Best Regards, Brook Hong
-----Original Message----- From: Hong Brook (Nokia-D/Beijing) Sent: Thursday, July 01, 2010 4:20 PM To: opensuse-buildservice@opensuse.org Subject: [opensuse-buildservice] question about package aggregate
Hi,
In file BSXML.pm:
our $aggregatelist = [ 'aggregatelist' => [[ 'aggregate' => 'project', [ 'package' ], [ 'binary' ], [[ 'repository' => 'target', 'source', ]], ]], ];
According to the code in bs_sched, the binary item is treated as a filter to ignore a binary file when copying files in rebuildaggregate. But there seems no user interface to set value for the item. Does it mean this feature is only partially implemented?
And there is a feature in openFate -- https://features.opensuse.org/309350, which is to skip source files when aggregating. Does it mean that an extra item is needed, such as "source", for feature 309350?
---------------- Best Regards, Brook Hong
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Fri, Jul 02, 2010 at 08:21:26AM +0200, brook.hong@nokia.com wrote:
our $aggregatelist = [ 'aggregatelist' => [[ 'aggregate' => 'project', [ 'package' ], [ 'binary' ], [[ 'repository' => 'target', 'source', ]], ]], ];
According to the code in bs_sched, the binary item is treated as a filter to ignore a binary file when copying files in rebuildaggregate. But there seems no user interface to set value for the item. Does it mean this feature is only partially implemented?
No, you just have to write the xml yourself.
And there is a feature in openFate -- https://features.opensuse.org/309350, which is to skip source files when aggregating. Does it mean that an extra item is needed, such as "source", for feature 309350?
What do you mean, there simply is no way to filter out source packages (.src.rpm) yet. (If you want to implement some access control, please note that aggregates are currently linkable, i.e. you can modify them with a patch.) Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Markus Rex, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
According to the code in bs_sched, the binary item is treated as a filter to ignore a binary file when copying files in rebuildaggregate. But there seems no user interface to set value for the item. Does it mean this feature is only partially implemented?
No, you just have to write the xml yourself.
That's okay for this.
And there is a feature in openFate -- https://features.opensuse.org/309350, which is to skip source files when aggregating. Does it mean that an extra item is needed, such as "source", for feature 309350?
What do you mean, there simply is no way to filter out source packages (.src.rpm) yet.
I mean to add "source" item like current "binary" as a filter to skip source packages (.src.rpm).
(If you want to implement some access control, please note that aggregates are currently linkable, i.e. you can modify them with a patch.)
Yes, I want to implement feature 309350. If I'm not misunderstanding, whose purpose is to skip source packages (.src.rpm) when copying build results during aggregating.
Cheers, Michael.
-- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Markus Rex, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Monday 05 July 2010 04:12:22 brook.hong@nokia.com wrote: ...
And there is a feature in openFate -- https://features.opensuse.org/309350, which is to skip source files when aggregating. Does it mean that an extra item is needed, such as "source", for feature 309350?
What do you mean, there simply is no way to filter out source packages (.src.rpm) yet.
I mean to add "source" item like current "binary" as a filter to skip source packages (.src.rpm).
(If you want to implement some access control, please note that aggregates are currently linkable, i.e. you can modify them with a patch.)
Yes, I want to implement feature 309350. If I'm not misunderstanding, whose purpose is to skip source packages (.src.rpm) when copying build results during aggregating.
In the end it is done by the scheduler, have a look in the "sub rebuildaggregate" function. Yes, you need some kind of filter or a flag to the aggregate xml to support this. For this particular use case a "nosources" flag might make sense to keep the logic to decide about source packages in the scheduler. (but a filter might make also sense in other use cases). -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Hello, I have added an option for "aggregatepac" as osc aggregatepac --nosources devel libzip home:Admin It will generate the _aggregate xml file as <aggregatelist> <aggregate project="devel"> <package>libzip</package> <nosources/> </aggregate> </aggregatelist> Then bs_shed will check if there is nosources flag, copying source packages only if the nosources flag is NOT set. This patch is to implement feature https://features.opensuse.org/309350. Any comments? ---------------- Best Regards, Brook Hong
-----Original Message----- From: ext Adrian Schröter [mailto:adrian@suse.de] Sent: Monday, July 05, 2010 3:04 PM To: opensuse-buildservice@opensuse.org Cc: Hong Brook (Nokia-MS/Beijing); mls@suse.de Subject: Re: [opensuse-buildservice] RE: question about package aggregate
And there is a feature in openFate -- https://features.opensuse.org/309350, which is to skip source
files
when
aggregating. Does it mean that an extra item is needed, such as "source", for feature 309350?
What do you mean, there simply is no way to filter out source
(.src.rpm) yet.
I mean to add "source" item like current "binary" as a filter to skip
On Monday 05 July 2010 04:12:22 brook.hong@nokia.com wrote: ... packages source packages (.src.rpm).
(If you want to implement some access control, please note that aggregates are currently linkable, i.e. you can modify them with a patch.)
Yes, I want to implement feature 309350. If I'm not misunderstanding,
whose purpose is to skip source packages (.src.rpm) when copying build results during aggregating.
In the end it is done by the scheduler, have a look in the "sub rebuildaggregate" function.
Yes, you need some kind of filter or a flag to the aggregate xml to support this. For this particular use case a "nosources" flag might make sense to keep the logic to decide about source packages in the scheduler. (but a filter might make also sense in other use cases).
--
Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Monday 19 July 2010 05:06:00 brook.hong@nokia.com wrote:
Hello,
I have added an option for "aggregatepac" as osc aggregatepac --nosources devel libzip home:Admin
It will generate the _aggregate xml file as <aggregatelist> <aggregate project="devel"> <package>libzip</package> <nosources/> </aggregate> </aggregatelist>
Then bs_shed will check if there is nosources flag, copying source packages only if the nosources flag is NOT set.
This patch is to implement feature https://features.opensuse.org/309350.
Any comments?
Great, I have imported it into OBS master (for OBS 2.1) and osc for 0.129 release. Just one comment, it would be nicer to have seperate patch files, because OBS and osc are hosted in different git repos. And if you provide submissions via git repos, your commit id is not getting lost (in case we do automatic submission statistics later you would become visible ;) But I have no problem with the patch by mail way either ;) thanks again adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Tuesday 20 July 2010 14:52:52 Adrian Schröter wrote:
On Monday 19 July 2010 05:06:00 brook.hong@nokia.com wrote:
Hello,
I have added an option for "aggregatepac" as osc aggregatepac --nosources devel libzip home:Admin
It will generate the _aggregate xml file as <aggregatelist> <aggregate project="devel"> <package>libzip</package> <nosources/> </aggregate> </aggregatelist>
Then bs_shed will check if there is nosources flag, copying source packages only if the nosources flag is NOT set.
This patch is to implement feature https://features.opensuse.org/309350.
Any comments?
Great, I have imported it into OBS master (for OBS 2.1) and osc for 0.129 release.
Hm, not that I think about it, are the -debugsource packages anyway exported ? (I have not tested that). We may need to skip these and maybe even -debuginfo in this situation as well ? Michael, any comment from you ? bye adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Hello, http://www.redhat.com/archives/shrike-list/2003-April/msg00069.html According to release notes of rpmbuild, rpmbuild has a new feature in that it will automatically create a <name>-debuginfo package when you (re)build a package. But I've never seen a debuginfo package generated under openSUSE, regardless of the way to start rpmbuild, triggered by OBS or started from shell. Is there any special macro defined for rpmbuild under openSUSE? ---------------- Best Regards, Brook Hong
-----Original Message----- From: ext Adrian Schröter [mailto:adrian@suse.de] Sent: Tuesday, July 20, 2010 8:56 PM To: opensuse-buildservice@opensuse.org Cc: Hong Brook (Nokia-MS/Beijing); mls@suse.de Subject: Re: [opensuse-buildservice] RE: question about package aggregate
On Monday 19 July 2010 05:06:00 brook.hong@nokia.com wrote:
Hello,
I have added an option for "aggregatepac" as osc aggregatepac --nosources devel libzip home:Admin
It will generate the _aggregate xml file as <aggregatelist> <aggregate project="devel"> <package>libzip</package> <nosources/> </aggregate> </aggregatelist>
Then bs_shed will check if there is nosources flag, copying source
On Tuesday 20 July 2010 14:52:52 Adrian Schröter wrote: packages only if the nosources flag is NOT set.
This patch is to implement feature
https://features.opensuse.org/309350.
Any comments?
Great, I have imported it into OBS master (for OBS 2.1) and osc for 0.129 release.
Hm, not that I think about it, are the -debugsource packages anyway exported ? (I have not tested that).
We may need to skip these and maybe even -debuginfo in this situation as well ?
Michael, any comment from you ?
bye adrian
-- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Thursday July 22 2010 05:12:55 brook.hong@nokia.com wrote:
Hello,
http://www.redhat.com/archives/shrike-list/2003-April/msg00069.html According to release notes of rpmbuild, rpmbuild has a new feature in that it will automatically create a <name>-debuginfo package when you (re)build a package.
But I've never seen a debuginfo package generated under openSUSE, regardless of the way to start rpmbuild, triggered by OBS or started from shell.
Is there any special macro defined for rpmbuild under openSUSE?
Nope, I have plenty of them ;P e.g. at http://download.opensuse.org/repositories/security:/OpenVAS:/STABLE:/v3/open... You don't need to do a thing besides enabling the "Debuginfo Flag" for your repo which then will trigger the building of any debug packages. https://build.opensuse.org/project/repositories?project=security%3AOpenVAS%3... for the above repo. Just note that you have to disable that flag for Mandriva because they build debug packages by default but they don't get build which prolly is a bug in OBS.
---------------- Best Regards, Brook Hong -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Thanks for pointing out the setting, stephan. But I still wonder why no debuginfo package gets generated when running rpmbuild from shell. BTW, anyone know how to get debugsource package generated? ---------------- Best Regards, Brook Hong
-----Original Message----- From: ext Stephan Kleine [mailto:bitdealer@gmail.com] Sent: Thursday, July 22, 2010 11:37 AM To: opensuse-buildservice@opensuse.org Subject: Re: [opensuse-buildservice] RE: question about package aggregate
Hello,
http://www.redhat.com/archives/shrike-list/2003-April/msg00069.html According to release notes of rpmbuild, rpmbuild has a new feature in
On Thursday July 22 2010 05:12:55 brook.hong@nokia.com wrote: that
it will automatically create a <name>-debuginfo package when you (re)build a package.
But I've never seen a debuginfo package generated under openSUSE, regardless of the way to start rpmbuild, triggered by OBS or started from shell.
Is there any special macro defined for rpmbuild under openSUSE?
Nope, I have plenty of them ;P e.g. at http://download.opensuse.org/repositories/security:/OpenVAS:/STABLE:/v3/ openSUSE_Factory/x86_64/
You don't need to do a thing besides enabling the "Debuginfo Flag" for your repo which then will trigger the building of any debug packages.
https://build.opensuse.org/project/repositories?project=security%3AOpenV AS%3ASTABLE%3Av3 for the above repo. Just note that you have to disable that flag for Mandriva because they build debug packages by default but they don't get build which prolly is a bug in OBS.
---------------- Best Regards, Brook Hong -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On 07/22/2010 07:43 AM, brook.hong@nokia.com wrote:
Thanks for pointing out the setting, stephan.
But I still wonder why no debuginfo package gets generated when running rpmbuild from shell. BTW, anyone know how to get debugsource package generated?
osc build --debuginfo -- Best Regards / S pozdravom, Pavol RUSNAK SUSE LINUX, s.r.o openSUSE Boosters Team Lihovarska 1060/12 PGP 0xA6917144 19000 Praha 9 prusnak[at]opensuse.org Czech Republic -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Thanks Pavol. I mean the command "rpmbuild -ba somepackage.spec". Either webui or osc passes a -debug option to the build script, which generates relevant macros for rpmbuild. Then rpmbuild generates those debuginfo and debugsource packages. Adrian, the aggregatepac operation does copy the debuginfo and debugsource packages to target, here I added one line to skip them when nosources flag is set. diff --git a/src/backend/bs_sched b/src/backend/bs_sched index 5ef13bb..8392049 100755 --- a/src/backend/bs_sched +++ b/src/backend/bs_sched @@ -2906,6 +2906,7 @@ sub rebuildaggregate { next unless $r->{'source'}; next if $abinfilter && !$abinfilter->{$r->{'name'}}; next if $jobbins{$r->{'name'}}; + next if ($copysources == 0) && ($r->{'name'} =~ /-debug(info|source)/); $jobbins{$r->{'name'}} = 1; my $basename = $abin; $basename =~ s/.*\///; ---------------- Best Regards, Brook Hong
-----Original Message----- From: ext Pavol Rusnak [mailto:prusnak@opensuse.org] Sent: Thursday, July 22, 2010 11:50 PM Cc: opensuse-buildservice@opensuse.org Subject: Re: [opensuse-buildservice] RE: question about package aggregate
On 07/22/2010 07:43 AM, brook.hong@nokia.com wrote:
Thanks for pointing out the setting, stephan.
But I still wonder why no debuginfo package gets generated when running rpmbuild from shell. BTW, anyone know how to get debugsource package generated?
osc build --debuginfo
-- Best Regards / S pozdravom,
Pavol RUSNAK SUSE LINUX, s.r.o openSUSE Boosters Team Lihovarska 1060/12 PGP 0xA6917144 19000 Praha 9 prusnak[at]opensuse.org Czech Republic -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Friday 23 July 2010 04:57:26 brook.hong@nokia.com wrote:
Thanks Pavol. I mean the command "rpmbuild -ba somepackage.spec". Either webui or osc passes a -debug option to the build script, which generates relevant macros for rpmbuild. Then rpmbuild generates those debuginfo and debugsource packages.
Adrian, the aggregatepac operation does copy the debuginfo and debugsource packages to target, here I added one line to skip them when nosources flag is set.
okay, I have submitted a slightly different version with an exact match for rpm packages. I dunno how debian is handling debug packages at all. Does anyone know how we can detect debug packages on debian ? thanks adrian
diff --git a/src/backend/bs_sched b/src/backend/bs_sched index 5ef13bb..8392049 100755 --- a/src/backend/bs_sched +++ b/src/backend/bs_sched @@ -2906,6 +2906,7 @@ sub rebuildaggregate { next unless $r->{'source'}; next if $abinfilter && !$abinfilter->{$r->{'name'}}; next if $jobbins{$r->{'name'}}; + next if ($copysources == 0) && ($r->{'name'} =~ /-debug(info|source)/); $jobbins{$r->{'name'}} = 1; my $basename = $abin; $basename =~ s/.*\///;
-- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
I found you added on gitorious, like this " next if ($copysources == 0) && ($r->{'name'} =~ /-debug(info|source)-[^-]+-[^-]+\.([a-zA-Z][^\.\-]*)\.rpm$/)" But from my debugging log, I found $r->{'name'} doesn't end with version number and rpm suffix. For example, "libzip-util-debuginfo-0.9-24.3.i586.rpm" has a value as "libzip-util-debuginfo" for $r->{'name'}. It's also the same for debugsource package. ---------------- Best Regards, Brook Hong
-----Original Message----- From: ext Adrian Schröter [mailto:adrian@suse.de] Sent: Friday, July 23, 2010 2:37 PM To: opensuse-buildservice@opensuse.org Cc: Hong Brook (Nokia-MS/Beijing); prusnak@opensuse.org Subject: Re: [opensuse-buildservice] RE: question about package aggregate
On Friday 23 July 2010 04:57:26 brook.hong@nokia.com wrote:
Thanks Pavol. I mean the command "rpmbuild -ba somepackage.spec". Either webui or osc passes a -debug option to the build script, which generates relevant macros for rpmbuild. Then rpmbuild generates those debuginfo and debugsource packages.
Adrian, the aggregatepac operation does copy the debuginfo and debugsource packages to target, here I added one line to skip them when nosources flag is set.
okay, I have submitted a slightly different version with an exact match for rpm packages. I dunno how debian is handling debug packages at all.
Does anyone know how we can detect debug packages on debian ?
thanks adrian
diff --git a/src/backend/bs_sched b/src/backend/bs_sched index 5ef13bb..8392049 100755 --- a/src/backend/bs_sched +++ b/src/backend/bs_sched @@ -2906,6 +2906,7 @@ sub rebuildaggregate { next unless $r->{'source'}; next if $abinfilter && !$abinfilter->{$r->{'name'}}; next if $jobbins{$r->{'name'}}; + next if ($copysources == 0) && ($r->{'name'} =~ /- debug(info|source)/); $jobbins{$r->{'name'}} = 1; my $basename = $abin; $basename =~ s/.*\///;
-- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Fri, Jul 23, 2010 at 08:36:33AM +0200, Adrian Schröter wrote:
On Friday 23 July 2010 04:57:26 brook.hong@nokia.com wrote:
Thanks Pavol. I mean the command "rpmbuild -ba somepackage.spec". Either webui or osc passes a -debug option to the build script, which generates relevant macros for rpmbuild. Then rpmbuild generates those debuginfo and debugsource packages.
Adrian, the aggregatepac operation does copy the debuginfo and debugsource packages to target, here I added one line to skip them when nosources flag is set.
okay, I have submitted a slightly different version with an exact match for rpm packages.
And I also did some changes: - changed the definition in BSXML so that it is a string, not an array - added a verifyer so that people do not set it to "0" and are surprised that they still do not get sources - changed the code so that it tests the existence of the string, not if it is true, as an empty element is false - plus I also repaired the copysources handling again so that it still works correctly if "nosource" is not set Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Markus Rex, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
participants (5)
-
Adrian Schröter -
brook.hong＠nokia.com -
Michael Schroeder -
Pavol Rusnak -
Stephan Kleine