[opensuse-buildservice] _product and ISO generation
I've got a local OBS installation that's been working out pretty well for me; however I have a new requirement to add ISO generation. I've been reading up on this, and found: http://wiki.opensuse.org/openSUSE:Build_Service_Concept_Product_Definition http://en.opensuse.org/Build_Service/Product_Definition/products_file I also checked out (from build.opensuse.org) the _project 'package' in openSUSE:Tools and openSUSE:11.2 (so I have examples to look at): osc co openSUSE:11.2 _product osc co openSUSE:Tools _product The documentation and the _product information I've downloaded seems to make sense. I made sure I have obs- productconverter installed (I do; 1.7.5, built & downloaded yesterday). I already have openSUSE:11.2 in my local OBS installation, so I created a "new" project named "ISO_TEST", with the package _product. I copied the files from _project (which I had checked out from build.opensuse.org's openSUSE:11.2). Next, I tried to commit them (to my local OBS). I get an HTTP error 403: Forbidden error from osc when I try to check in the files: my backend_access.log has the following: 20100527T152504 GET obsapi.americas.sgi.com:5352/source/ISO_TEST/_product?rev=latest 200 20100527T152504 PUT obsapi.americas.sgi.com:5352/source/ISO_TEST/_product/DVD5-i586.group?user=ttelford&rev=upload 200 20100527T152504 PUT obsapi.americas.sgi.com:5352/source/ISO_TEST/_product/DVD5-lang.group?user=ttelford&rev=upload 200 20100527T152504 PUT obsapi.americas.sgi.com:5352/source/ISO_TEST/_product/DVD5-nonoss.group?user=ttelford&rev=upload 200 20100527T152504 PUT obsapi.americas.sgi.com:5352/source/ISO_TEST/_product/DVD5-promo- i386.group?user=ttelford&rev=upload 200 20100527T152505 PUT obsapi.americas.sgi.com:5352/source/ISO_TEST/_product/DVD5-promo- x86_64.group?user=ttelford&rev=upload 200 20100527T152505 PUT obsapi.americas.sgi.com:5352/source/ISO_TEST/_product/DVD5-x86_64.group?user=ttelford&rev=upload 200 20100527T152505 PUT obsapi.americas.sgi.com:5352/source/ISO_TEST/_product/FROZEN.group?user=ttelford&rev=upload 200 20100527T152505 PUT obsapi.americas.sgi.com:5352/source/ISO_TEST/_product/NON_FTP_PACKAGES.group?user=ttelford&rev=upload 200 20100527T152505 POST obsapi.americas.sgi.com:5352/source/ISO_TEST/_product?cmd=deleteuploadrev 200 20100527T152534 GET obsapi.americas.sgi.com:5352/build/Management_Center/_result?view=status&code=succeeded&code=failed&code=expansion+error&code=broken&code=blocked&code=dispatching&code=scheduled&code=building&code=finished&code=disabled&code=excluded&code=unknown&arch=x86_64 200 So I'm wondering what I am doing wrong; as far as I can tell, it should work. Does anybody have experience using the Product Definitions with OBS? -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Donnerstag, 27. Mai 2010 23:29:55 Troy Telford wrote:
I've got a local OBS installation that's been working out pretty well for me; however I have a new requirement to add ISO generation.
I've been reading up on this, and found: http://wiki.opensuse.org/openSUSE:Build_Service_Concept_Product_Definition http://en.opensuse.org/Build_Service/Product_Definition/products_file
I also checked out (from build.opensuse.org) the _project 'package' in openSUSE:Tools and openSUSE:11.2 (so I have examples to look at):
osc co openSUSE:11.2 _product osc co openSUSE:Tools _product
The documentation and the _product information I've downloaded seems to make sense. I made sure I have obs- productconverter installed (I do; 1.7.5, built & downloaded yesterday).
I already have openSUSE:11.2 in my local OBS installation, so I created a "new" project named "ISO_TEST", with the package _product.
I copied the files from _project (which I had checked out from build.opensuse.org's openSUSE:11.2). Next, I tried to commit them (to my local OBS).
I get an HTTP error 403: Forbidden error from osc when I try to check in the files:
Do you try to edit the _product:* packages ? if not please send the "osc -d ci" output. These are generated, you can only change the "_product" package, the other packages will get re-generated based by this. Are you aware that you need really a full build of the distribution to create products ? Just having the :full directory filled is not enough, you need also all the binaries in /obs/build/$PROJECT/$standard/$arch/ directory. bye adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Donnerstag, 27. Mai 2010 23:29:55 Troy Telford wrote:
I already have openSUSE:11.2 in my local OBS installation, so I created a "new" project named "ISO_TEST", with the package _product.
I copied the files from _project (which I had checked out from build.opensuse.org's openSUSE:11.2). Next, I tried to commit them (to my local OBS).
I get an HTTP error 403: Forbidden error from osc when I try to check in
On Friday, May 28, 2010 02:16:45 am Adrian Schröter wrote: the files:
Do you try to edit the _product:* packages ?
No; I only copied in the _product package (with no :*)
if not please send the "osc -d ci" output.
~/src/obs/ISO_TEST/_product$ osc -d ci GET http://obsapi.americas.sgi.com/source/ISO_TEST GET http://obsapi.americas.sgi.com/source/ISO_TEST/_product/_meta Sending meta data... PUT http://obsapi.americas.sgi.com/source/ISO_TEST/_product/_meta Done. Sending _product GET http://obsapi.americas.sgi.com/source/ISO_TEST/_product?rev=latest Sending _product/DVD5-i586.group Sending _product/DVD5-lang.group Sending _product/DVD5-nonoss.group Sending _product/DVD5-promo-i386.group Sending _product/DVD5-promo-x86_64.group Sending _product/DVD5-x86_64.group Sending _product/FROZEN.group Sending _product/NON_FTP_PACKAGES.group Sending _product/archsets.inc Sending _product/conditionals.inc Sending _product/i686_list.group Sending _product/obsoletepackages.inc Sending _product/openSUSE-Addon-Lang.product Sending _product/openSUSE-Addon-NonOss.product Sending _product/openSUSE-LiveCD.product Sending _product/openSUSE-Promo.product Sending _product/openSUSE-release.spec Sending _product/openSUSE.product Sending _product/update-drops Transmitting file data .PUT http://obsapi.americas.sgi.com/source/ISO_TEST/_product/DVD5- i586.group?rev=upload .PUT http://obsapi.americas.sgi.com/source/ISO_TEST/_product/DVD5- lang.group?rev=upload .PUT http://obsapi.americas.sgi.com/source/ISO_TEST/_product/DVD5- nonoss.group?rev=upload .PUT http://obsapi.americas.sgi.com/source/ISO_TEST/_product/DVD5-promo- i386.group?rev=upload .PUT http://obsapi.americas.sgi.com/source/ISO_TEST/_product/DVD5-promo- x86_64.group?rev=upload .PUT http://obsapi.americas.sgi.com/source/ISO_TEST/_product/DVD5- x86_64.group?rev=upload .PUT http://obsapi.americas.sgi.com/source/ISO_TEST/_product/FROZEN.group?rev=upl... .PUT http://obsapi.americas.sgi.com/source/ISO_TEST/_product/NON_FTP_PACKAGES.gro... .PUT http://obsapi.americas.sgi.com/source/ISO_TEST/_product/archsets.inc?rev=upl... POST http://obsapi.americas.sgi.com/source/ISO_TEST/_product?cmd=deleteuploadrev Server returned an error: HTTP Error 403: Forbidden Connection: close Content-Type: text/html Content-Length: 345 Date: Fri, 28 May 2010 17:51:42 GMT Server: lighttpd/1.4.20 <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title>403 - Forbidden</title> </head> <body> <h1>403 - Forbidden</h1> </body> </html>
These are generated, you can only change the "_product" package, the other packages will get re-generated based by this.
Yeah; I had noticed that.
Are you aware that you need really a full build of the distribution to create products ? Just having the :full directory filled is not enough, you need also all the binaries in /obs/build/$PROJECT/$standard/$arch/ directory.
Let's not get ahead of ourselves. I'm not able to check in _product. Thus,
no kiwi files are being generated, _package:* are also not being generated.
I've found (through trial and error) that 'osc ci' doesn't like _any_ files
with the extension *.inc. I can even create a 'junk' package (named anything;
it doesn't have to be _product), and if I try to add a file named foo.inc and
check it in, I see the same error:
ttelford@obs:~/src/obs/TEST/$ osc mkpac foo
ttelford@obs:~/src/obs/TEST/foo$ touch foo.inc
ttelford@obs:~/src/obs/TEST/foo$ osc add foo.inc
A foo.inc
ttelford@obs:~/src/obs/TEST/foo$ osc -d ci
GET http://obsapi.foo.bar.com/source/TEST
GET http://obsapi.foo.bar.com/source/TEST/foo?rev=latest
Sending foo.inc
Transmitting file data .PUT
http://obsapi.foo.bar.com/source/TEST/foo/foo.inc?rev=upload
POST http://obsapi.foo.bar.com/source/TEST/foo?cmd=deleteuploadrev
Server returned an error: HTTP Error 403: Forbidden
Connection: close
Content-Type: text/html
Content-Length: 345
Date: Fri, 28 May 2010 18:04:58 GMT
Server: lighttpd/1.4.20
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>403 - Forbidden</title>
</head>
<body>
<h1>403 - Forbidden</h1>
</body>
</html>
If I rename the '*.inc' files to '*.xml', and modify the .product files to
match, then I can successfully run 'osc ci'. At that point, the _product:*
packages are auto-generated and I get .kiwi files. Why OBS is worried about
the '*.inc' extension, I'm not sure.
Then the build fails in kiwi, probably for the reason you stated above...
(Kiwi output:)
[I] STEP 3 [collectPackages]
[I] Handle scripts for metafiles and metapackages
[E] Metapackage <skelcd-openSUSE> not available for architecure
On Freitag, 28. Mai 2010 20:13:44 Troy Telford wrote: ...
http://obsapi.americas.sgi.com/source/ISO_TEST/_product?cmd=deleteuploadrev Server returned an error: HTTP Error 403: Forbidden
k, definitive a bug. But I can not reproduce it here with OBS 2.0 beta 2. And I am quite sure we would noticed this one with 1.7.x quite soon as well. Can you check the api server log file ? Usually /srv/www/obs/api/log/production.log Maybe you need to switch on debugging via config.log_level = :debug to see more. ...
Are you aware that you need really a full build of the distribution to create products ? Just having the :full directory filled is not enough, you need also all the binaries in /obs/build/$PROJECT/$standard/$arch/ directory.
Let's not get ahead of ourselves. I'm not able to check in _product. Thus, no kiwi files are being generated, _package:* are also not being generated.
I've found (through trial and error) that 'osc ci' doesn't like _any_ files with the extension *.inc. I can even create a 'junk' package (named anything; it doesn't have to be _product), and if I try to add a file named foo.inc and check it in, I see the same error:
Interessting, either our filename validation code wents wrong or this may be a rails bug. Can you tell which OBS version and which base system do you use ? Do you use our packages or is it installed from git ? As a workaround you can rename the file to any other name. You just need to adapt the include statements in the .product files afterwards. ...
If I rename the '*.inc' files to '*.xml', and modify the .product files to match, then I can successfully run 'osc ci'. At that point, the _product:* packages are auto-generated and I get .kiwi files. Why OBS is worried about the '*.inc' extension, I'm not sure.
oh, you found out ;)
Then the build fails in kiwi, probably for the reason you stated above...
(Kiwi output:) [I] STEP 3 [collectPackages] [I] Handle scripts for metafiles and metapackages [E] Metapackage <skelcd-openSUSE> not available for architecure
!
yep.
In the end, what I want to do is have something more like the OpenSUSE:Tools _product -- an add-on package to a distro. (OpenSUSE would be nice, but SLES 11+ is mandatory.) Fortunately, I can look at the OpenSUSE:Tools _product for guidance.
okay, when you build all your packages on the media in your OBS, you will not have this problem. have fun! adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Friday, May 28, 2010 01:14:12 pm Adrian Schröter wrote:
On Freitag, 28. Mai 2010 20:13:44 Troy Telford wrote: ...
http://obsapi.americas.sgi.com/source/ISO_TEST/_product?cmd=deleteuploadr ev Server returned an error: HTTP Error 403: Forbidden
k, definitive a bug. But I can not reproduce it here with OBS 2.0 beta 2. And I am quite sure we would noticed this one with 1.7.x quite soon as well.
Can you check the api server log file ? Usually
/srv/www/obs/api/log/production.log
Maybe you need to switch on debugging via
config.log_level = :debug
to see more.
$ sudo tail -f production.log
[D|#15949] add validation mapping: "tag", :tags_by_user_and_object => :tags
[D|#15949] add validation mapping: "tag", :project_tags => :tags
[D|#15949] add validation mapping: "tag", :package_tags => :tags
[D|#15949] default_server for :rest models: obsapi.foo.bar.com:5352
[D|#15949] setting up transport for model project
[D|#15949] [BSSQL] initialize( "bssql", {} )
[D|#15949] setting up transport for model package
[D|#15949] setting up transport for model bsrequest
[D|#15949] [REST] initialize( "rest", {:all=>#
I've found (through trial and error) that 'osc ci' doesn't like _any_ files with the extension *.inc. I can even create a 'junk' package (named anything; it doesn't have to be _product), and if I try to add a file named foo.inc and
check it in, I see the same error: Interessting, either our filename validation code wents wrong or this may be a rails bug. Can you tell which OBS version and which base system do you use ? Do you use our packages or is it installed from git ?
I'm using your packages (for openSUSE 11.2). (http://download.opensuse.org/repositories/openSUSE:/Tools/openSUSE_11.2/) I keep them updated via zypper. $ rpm -qa | grep obs obs-service-download_url-0.1-7.1.noarch obs-utils-1.7.5-1.1.x86_64 obs-common-1.7.0-1.1.x86_64 obs-api-1.7.5-1.1.x86_64 obs-worker-1.7.5-1.1.x86_64 obs-server-1.7.2-18.8.x86_64 obs-signd-1.7.0-5.1.x86_64 obs-source_service-1.7.5-1.1.x86_64 obs-productconverter-1.7.5-1.1.x86_64
have fun!
I managed to get an ISO image created; so I am having fun. Thanks. -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Freitag, 28. Mai 2010 22:44:55 Troy Telford wrote:
On Friday, May 28, 2010 01:14:12 pm Adrian Schröter wrote:
On Freitag, 28. Mai 2010 20:13:44 Troy Telford wrote: ...
http://obsapi.americas.sgi.com/source/ISO_TEST/_product?cmd=deleteuploadr ev Server returned an error: HTTP Error 403: Forbidden
k, definitive a bug. But I can not reproduce it here with OBS 2.0 beta 2. And I am quite sure we would noticed this one with 1.7.x quite soon as well.
Can you check the api server log file ? Usually
/srv/www/obs/api/log/production.log
Maybe you need to switch on debugging via
config.log_level = :debug
to see more.
$ sudo tail -f production.log ... [D|#15950] dispatch_command: trying to call method 'index_package_deleteuploadrev' [D|#15950] [backend] POST: /source/ISO_TEST/_product?cmd=deleteuploadrev [I|#15950] Sending data [I|#15950] Completed in 19ms (View: 0, DB: 3) | 200 OK [http://obsapi.foo.bar.com/source/ISO_TEST/_product?cmd=deleteuploadrev]
So api reports a 200 OK and osc sees a 403 ? Time to check the lighttpd logs ...
...
have fun!
I managed to get an ISO image created; so I am having fun. Thanks.
great :) bye adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
participants (2)
-
Adrian Schröter
-
Troy Telford