[opensuse-buildservice] Cannonical names for debs
While testing osc 0.123, I ran into an issue with debianutils not being detected in the cache because it has not release, so the local file:// mirror fails to find the file since something adds -0 as a default release. But, when I was in the code I noticed that osc.fetch.Fetcher.fetch was constructing a canonical name for debs as canonname = '%s-%s.%s.%s' % (pkgq.name(), pkgq.version(), arch, pkgq.filename_suffix) which would give debianutils-2.30ubuntu3.amd64.deb but, the naming scheme for debs uses _ has a separator for verision and arch. The actual deb is called /srv/obs/build/Ubuntu:9.10/standard/x86_64/:full/debianutils_2.30ubuntu3_amd64.deb Is this just an error in expanding osc build to handle deb based bundles and should be fixed or was a decision made to use . has a separator for the canonical names for both rpms and debs? In which case, when does this canonical name get translated to the correct name since the debs in the download url use the '_' separator? -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
The bad format for the name actually leads to the download URL never being used and the api always being called because it drops the version and arch: pac.urllist ['file:///var/tmp/osbuild-packagecache/Ubuntu:9.10/standard/amd64/debianutils-2.30ubuntu3.amd64.deb/', 'http://download.buildservice.us.cray.com/repositories/Ubuntu:/9.10/standard/...', 'http://api.buildservice.us.cray.com/build/Ubuntu:9.10/standard/x86_64/_repos...'] http://download.buildservice.us.cray.com/repositories/Ubuntu:/9.10/standard/... Would never exist because of the format is wrong. Here's an example for the correct name with osc being built on our server: http://download.buildservice.us.cray.com/openSUSE:/Tools/xUbuntu_9.10/amd64/... So, this is a bug and it causes extra load on the frontend. Any debian based local build is always going to hit the frontend even if the deb exists on the download servers because the name calculated for the downoadload server is incorrect. On Fri, 2009-12-04 at 09:14 -0600, Luke Imhoff wrote:
While testing osc 0.123, I ran into an issue with debianutils not being detected in the cache because it has not release, so the local file:// mirror fails to find the file since something adds -0 as a default release. But, when I was in the code I noticed that osc.fetch.Fetcher.fetch was constructing a canonical name for debs as
canonname = '%s-%s.%s.%s' % (pkgq.name(), pkgq.version(), arch, pkgq.filename_suffix)
which would give
debianutils-2.30ubuntu3.amd64.deb
but, the naming scheme for debs uses _ has a separator for verision and arch. The actual deb is called
/srv/obs/build/Ubuntu:9.10/standard/x86_64/:full/debianutils_2.30ubuntu3_amd64.deb
Is this just an error in expanding osc build to handle deb based bundles and should be fixed or was a decision made to use . has a separator for the canonical names for both rpms and debs? In which case, when does this canonical name get translated to the correct name since the debs in the download url use the '_' separator?
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Seems there's a parsing error on the server too: <bdep arch="amd64" name="bash" notmeta="1" preinstall="1" project="Ubuntu:9.10" release="5ubuntu2" repository="standard" version="4.0" /> debs never have releases, it's all in version, but bash's version is 4.0-5ubuntu2, so the server is miss parsing the version as a version and release. osc assumes anything with a release is an rpm and so handles the name wrong. The server shouldn't be parsing the "release" out of the deb versions, since not all debs have "releases" embedded in the versions, debianutils is such an example. On Fri, 2009-12-04 at 09:35 -0600, Luke Imhoff wrote:
The bad format for the name actually leads to the download URL never being used and the api always being called because it drops the version and arch:
pac.urllist ['file:///var/tmp/osbuild-packagecache/Ubuntu:9.10/standard/amd64/debianutils-2.30ubuntu3.amd64.deb/', 'http://download.buildservice.us.cray.com/repositories/Ubuntu:/9.10/standard/...', 'http://api.buildservice.us.cray.com/build/Ubuntu:9.10/standard/x86_64/_repos...']
http://download.buildservice.us.cray.com/repositories/Ubuntu:/9.10/standard/...
Would never exist because of the format is wrong. Here's an example for the correct name with osc being built on our server:
http://download.buildservice.us.cray.com/openSUSE:/Tools/xUbuntu_9.10/amd64/...
So, this is a bug and it causes extra load on the frontend. Any debian based local build is always going to hit the frontend even if the deb exists on the download servers because the name calculated for the downoadload server is incorrect.
On Fri, 2009-12-04 at 09:14 -0600, Luke Imhoff wrote:
While testing osc 0.123, I ran into an issue with debianutils not being detected in the cache because it has not release, so the local file:// mirror fails to find the file since something adds -0 as a default release. But, when I was in the code I noticed that osc.fetch.Fetcher.fetch was constructing a canonical name for debs as
canonname = '%s-%s.%s.%s' % (pkgq.name(), pkgq.version(), arch, pkgq.filename_suffix)
which would give
debianutils-2.30ubuntu3.amd64.deb
but, the naming scheme for debs uses _ has a separator for verision and arch. The actual deb is called
/srv/obs/build/Ubuntu:9.10/standard/x86_64/:full/debianutils_2.30ubuntu3_amd64.deb
Is this just an error in expanding osc build to handle deb based bundles and should be fixed or was a decision made to use . has a separator for the canonical names for both rpms and debs? In which case, when does this canonical name get translated to the correct name since the debs in the download url use the '_' separator?
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Fri, Dec 04, 2009 at 10:21:39AM -0600, Luke Imhoff wrote:
Seems there's a parsing error on the server too:
<bdep arch="amd64" name="bash" notmeta="1" preinstall="1" project="Ubuntu:9.10" release="5ubuntu2" repository="standard" version="4.0" />
debs never have releases, it's all in version, but bash's version is 4.0-5ubuntu2, so the server is miss parsing the version as a version and release.
The server splits it into version-release if the string contains a '-', otherwise it sets only the version. I think this makes sense, the upstream bash version is "4.0", "5ubuntu2" is some release scheme from ubuntu. This is pretty much in line with the Debian Policy Manual, which states: 5.6.12 Version The version number of a package. The format is: [epoch:]upstream_version[-debian_revision] Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Markus Rex, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Ok, if that's the policy, that's fine then. osc still needs to be changed to not key off of the 'release' attribute to determine the whether to use rpm or deb format then. On Fri, 2009-12-04 at 11:27 -0600, Michael Schroeder wrote:
On Fri, Dec 04, 2009 at 10:21:39AM -0600, Luke Imhoff wrote:
Seems there's a parsing error on the server too:
<bdep arch="amd64" name="bash" notmeta="1" preinstall="1" project="Ubuntu:9.10" release="5ubuntu2" repository="standard" version="4.0" />
debs never have releases, it's all in version, but bash's version is 4.0-5ubuntu2, so the server is miss parsing the version as a version and release.
The server splits it into version-release if the string contains a '-', otherwise it sets only the version. I think this makes sense, the upstream bash version is "4.0", "5ubuntu2" is some release scheme from ubuntu.
This is pretty much in line with the Debian Policy Manual, which states:
5.6.12 Version The version number of a package. The format is: [epoch:]upstream_version[-debian_revision]
Cheers, Michael.
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Ok, I think this is a real bug on the server, but just let me know if it's some policy I'm unaware of (again ;-)): libattr1's version in a bdep tag is 1:2.4.43 the full bdep is <bdep arch="amd64" name="libattr1" notmeta="1" preinstall="1" project="Ubuntu:9.10" release="3" repository="standard" version="1:2.4.43" /> The full deb name on the server is libattr1_2.4.43-3_amd64.deb The same thing happens with other debs that have numbers at the end of their names like kdelibs4-*. Somehow it's grabbing both numerical pieces and then joining them with ':'. Where in the obs-server or build code is the version parsed out? I can fix it, I just don't know off hand where in the perl the parse is located. On Fri, 2009-12-04 at 11:32 -0600, Luke Imhoff wrote:
Ok, if that's the policy, that's fine then. osc still needs to be changed to not key off of the 'release' attribute to determine the whether to use rpm or deb format then.
On Fri, 2009-12-04 at 11:27 -0600, Michael Schroeder wrote:
On Fri, Dec 04, 2009 at 10:21:39AM -0600, Luke Imhoff wrote:
Seems there's a parsing error on the server too:
<bdep arch="amd64" name="bash" notmeta="1" preinstall="1" project="Ubuntu:9.10" release="5ubuntu2" repository="standard" version="4.0" />
debs never have releases, it's all in version, but bash's version is 4.0-5ubuntu2, so the server is miss parsing the version as a version and release.
The server splits it into version-release if the string contains a '-', otherwise it sets only the version. I think this makes sense, the upstream bash version is "4.0", "5ubuntu2" is some release scheme from ubuntu.
This is pretty much in line with the Debian Policy Manual, which states:
5.6.12 Version The version number of a package. The format is: [epoch:]upstream_version[-debian_revision]
Cheers, Michael.
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Fri, 2009-12-04 at 13:21 -0600, Luke Imhoff wrote:
Ok, I think this is a real bug on the server, but just let me know if it's some policy I'm unaware of (again ;-)):
libattr1's version in a bdep tag is 1:2.4.43
the full bdep is
<bdep arch="amd64" name="libattr1" notmeta="1" preinstall="1" project="Ubuntu:9.10" release="3" repository="standard" version="1:2.4.43" />
The full deb name on the server is
libattr1_2.4.43-3_amd64.deb
The same thing happens with other debs that have numbers at the end of their names like kdelibs4-*. Somehow it's grabbing both numerical pieces and then joining them with ':'.
Where in the obs-server or build code is the version parsed out? I can fix it, I just don't know off hand where in the perl the parse is located.
Once again, the data is correct, it just doesn't match the naming scheme for the deb package. If you look at the Version reported by dpkg --info libattr1_2.4.43-3_amd64.deb it says: Version: 1:2.4.43-3 I guess that means if version has a ':' in it, then the file name calculation needs to be special cased so that the bit before the ':' gets added to the name, while the bit after gets created as the normal version. Michael, do you know if this is a standard for Versions with ':' in them that can be safely assumed? I'm not sure how else to derive the package filename correctly from the bdeps.
On Fri, 2009-12-04 at 11:32 -0600, Luke Imhoff wrote:
Ok, if that's the policy, that's fine then. osc still needs to be changed to not key off of the 'release' attribute to determine the whether to use rpm or deb format then.
On Fri, 2009-12-04 at 11:27 -0600, Michael Schroeder wrote:
On Fri, Dec 04, 2009 at 10:21:39AM -0600, Luke Imhoff wrote:
Seems there's a parsing error on the server too:
<bdep arch="amd64" name="bash" notmeta="1" preinstall="1" project="Ubuntu:9.10" release="5ubuntu2" repository="standard" version="4.0" />
debs never have releases, it's all in version, but bash's version is 4.0-5ubuntu2, so the server is miss parsing the version as a version and release.
The server splits it into version-release if the string contains a '-', otherwise it sets only the version. I think this makes sense, the upstream bash version is "4.0", "5ubuntu2" is some release scheme from ubuntu.
This is pretty much in line with the Debian Policy Manual, which states:
5.6.12 Version The version number of a package. The format is: [epoch:]upstream_version[-debian_revision]
Cheers, Michael.
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Ok, one more correction. I looked at dpkg --info kdelibs4-dev_3.5.10.dfsg.1-2ubuntu7_amd64.deb Version: 4:3.5.10.dfsg.1-2ubuntu7 epoch matches major version number in name But, that doesn't work for kdelibs5-dev_3.5.10.dfsg.1-2ubuntu7_amd64.deb, where dpkg --info kdelibs5-dev_4.3.2-0ubuntu7_amd64.deb Version: 4:4.3.2-0ubuntu7 So, I'm going to say that the epoch is just dropped from filename of the deb, at least for Ubuntu 9.10 as I can't find any debs that have ':' in the actual filename. On Fri, 2009-12-04 at 14:25 -0600, Luke Imhoff wrote:
On Fri, 2009-12-04 at 13:21 -0600, Luke Imhoff wrote:
Ok, I think this is a real bug on the server, but just let me know if it's some policy I'm unaware of (again ;-)):
libattr1's version in a bdep tag is 1:2.4.43
the full bdep is
<bdep arch="amd64" name="libattr1" notmeta="1" preinstall="1" project="Ubuntu:9.10" release="3" repository="standard" version="1:2.4.43" />
The full deb name on the server is
libattr1_2.4.43-3_amd64.deb
The same thing happens with other debs that have numbers at the end of their names like kdelibs4-*. Somehow it's grabbing both numerical pieces and then joining them with ':'.
Where in the obs-server or build code is the version parsed out? I can fix it, I just don't know off hand where in the perl the parse is located.
Once again, the data is correct, it just doesn't match the naming scheme for the deb package. If you look at the Version reported by dpkg --info libattr1_2.4.43-3_amd64.deb it says:
Version: 1:2.4.43-3
I guess that means if version has a ':' in it, then the file name calculation needs to be special cased so that the bit before the ':' gets added to the name, while the bit after gets created as the normal version.
Michael, do you know if this is a standard for Versions with ':' in them that can be safely assumed? I'm not sure how else to derive the package filename correctly from the bdeps.
On Fri, 2009-12-04 at 11:32 -0600, Luke Imhoff wrote:
Ok, if that's the policy, that's fine then. osc still needs to be changed to not key off of the 'release' attribute to determine the whether to use rpm or deb format then.
On Fri, 2009-12-04 at 11:27 -0600, Michael Schroeder wrote:
On Fri, Dec 04, 2009 at 10:21:39AM -0600, Luke Imhoff wrote:
Seems there's a parsing error on the server too:
<bdep arch="amd64" name="bash" notmeta="1" preinstall="1" project="Ubuntu:9.10" release="5ubuntu2" repository="standard" version="4.0" />
debs never have releases, it's all in version, but bash's version is 4.0-5ubuntu2, so the server is miss parsing the version as a version and release.
The server splits it into version-release if the string contains a '-', otherwise it sets only the version. I think this makes sense, the upstream bash version is "4.0", "5ubuntu2" is some release scheme from ubuntu.
This is pretty much in line with the Debian Policy Manual, which states:
5.6.12 Version The version number of a package. The format is: [epoch:]upstream_version[-debian_revision]
Cheers, Michael.
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Fri, Dec 04, 2009 at 02:25:08PM -0600, Luke Imhoff wrote:
On Fri, 2009-12-04 at 13:21 -0600, Luke Imhoff wrote:
Ok, I think this is a real bug on the server, but just let me know if it's some policy I'm unaware of (again ;-)):
libattr1's version in a bdep tag is 1:2.4.43
the full bdep is
<bdep arch="amd64" name="libattr1" notmeta="1" preinstall="1" project="Ubuntu:9.10" release="3" repository="standard" version="1:2.4.43" />
The full deb name on the server is
libattr1_2.4.43-3_amd64.deb
The same thing happens with other debs that have numbers at the end of their names like kdelibs4-*. Somehow it's grabbing both numerical pieces and then joining them with ':'.
Where in the obs-server or build code is the version parsed out? I can fix it, I just don't know off hand where in the perl the parse is located.
Once again, the data is correct, it just doesn't match the naming scheme for the deb package.
No, it's not correct. Epoch handling is missing from Build/Deb.pm in the build package. Fixing... Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Markus Rex, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Fri, Dec 04, 2009 at 02:35:21PM -0600, Luke Imhoff wrote:
So, I'm going to say that the epoch is just dropped from filename of the deb, at least for Ubuntu 9.10 as I can't find any debs that have ':' in the actual filename.
Same as in the rpm world, actually. Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Markus Rex, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
participants (2)
-
Luke Imhoff -
Michael Schroeder