[zypp-devel] zypper/libsolv broken @System/solv file
Hi, we got some trouble with getting zypper up and running. We like to use it for a linux system from scratch. We already got the OBS working. I now set up a system and am trying to implement zypper. I installed all rpms from a local directory with "zypper install *.rpm". After that, I got an error message from zypper which said, "illegal rpmdbid" and that there was an error with caching the rpm db. So far so good (zypper installed the packages anyway), the error is gone (created /etc/zypp/*.d and /etc/products.d directories, and rebuild the solv files/rpmdb). I added our OBS repository, called "standard", to zypper. And if I'm searching for a package from it, all I got is:
# zypper search qemu Loading repository data... Reading installed packages... Problem occured while reading the installed packages: Error reading solv-file: /var/cache/zypp/solv/@System/solv Please see the above error message for a hint. No packages found.
So, now I'm trying to solv the warning message - and here, I would appreciate every hint :-) I tried to dump the @System/solv db and got an error. Dumping the repository's standard/solv works as expected. Here is, how a clean dumpsolv behaves:
bash# zypper clean -m All repositories have been cleaned up. bash# zypper ref Retrieving repository 'standard' metadata ..............................................................[done] Building repository 'standard' cache ...................................................................[done] All repositories have been refreshed. bash# dumpsolv -a /var/cache/zypp/solv/\@System/solv data_read_id_max: id too large (2566914048/0) unexpected EOF, depth = 3 could not read repository: unexpected EOF, depth = 3 pool contains 3666 strings, 944 rels, string size is 59784 repo 1 contains 0 solvables repo start: 2 end: 2 bash-4.2# rpm -qa | wc -l 122
strace dumpsolv while reading /var/cache/zypp/solv/@System/solv:
open("@System/solv", O_RDONLY) = 3 dup3(3, 0, 0) = 0 close(3) = 0 gettimeofday({1367501109, 466110}, NULL) = 0 fstat(0, {st_mode=S_IFREG|0644, st_size=307418, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f5401de4000 read(0, "SOLV\0\0\0\10\0\0\r\332\0\0\3\257\0\0\f\357\0\0\0{\0\0\0$\0\0\0\n"..., 4096) = 4096 brk(0x254e000) = 0x254e000 read(0, "1.1-5.4\0\0031-4.4\0\0036.1\0\6-7.3\0\00250\0\0029"..., 32768) = 32768 read(0, "o()(64bit)\0\3uct\0\2yles\0\1ubprocess"..., 4096) = 4096 brk(0x2545000) = 0x2545000 read(0, "=\2\230j\212r\2\230l\213\36\2\230m\202<\2\211a\212q\2\230n\213\25\2\230o)\2\211"..., 4096) = 4096 read(0, "\203\30\203\31\203\37\203 \203\"\203&\203\10\203\t\250V\203\f\203\32\203\35\203!\250B\203\n\203\v"..., 4096) = 4096 read(0, "\200\0the bash\0\205!\201\20\v\fhttp://www.gnu."..., 4096) = 4096 read(0, "rg/software/grep/\0\2\n\205\213\326\254n\205\214\201\241G\201\214"..., 4096) = 4096 write(2, "data_read_id_max: id too large ("..., 46data_read_id_max: id too large (2298478592/0) ) = 46 write(2, "unexpected EOF, depth = 3\n", 26unexpected EOF, depth = 3 ) = 26 fstat(1, {st_mode=S_IFREG|0644, st_size=12973, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f5401de0000 brk(0x2529000) = 0x2529000 write(1, "could not read repository: unexp"..., 161could not read repository: unexpected EOF, depth = 3 pool contains 3666 strings, 944 rels, string size is 59784 repo 1 contains 0 solvables repo start: 2 end: 2 ) = 161 exit_group(0)
The "data_read_id_max" error message is from libsolv/src/repo_solv.c line 190 to 202:
static inline unsigned char * data_read_id_max(unsigned char *dp, Id *ret, Id *map, int max, Repodata *data) { Id x; dp = data_read_id(dp, &x); if (x < 0 || (max && x >= max)) { data->error = pool_error(data->repo->pool, SOLV_ERROR_ID_RANGE, "data_read_id_max: id too large (%u/%u)", x, max); x = 0; } *ret = map ? map[x] : x; return dp; }
So, rpm sees all installed packages; but the @System/solv files seems to be broken. We use the following versions: * zypper 1.8.9 (+ bugfix of swig BoolSave) * libzypp 13.1.0 * libsolv latest snapshot from git/origin * boost 1.53.0 Do you have a hint, where I may take a look at, to solve the problem? Thank you, for your time! Kind Regards Tony Wolf -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org To contact the owner, e-mail: zypp-devel+owner@opensuse.org
On Thursday 02 May 2013 16:21:45 Tony Wolf wrote:
So, rpm sees all installed packages; but the @System/solv files seems to be broken.
We use the following versions: * zypper 1.8.9 (+ bugfix of swig BoolSave) * libzypp 13.1.0 * libsolv latest snapshot from git/origin * boost 1.53.0
Do you have a hint, where I may take a look at, to solve the problem?
In general: In openSUSE we use a static libsolv. This way it is important that libzypp (staticaly linked against libsolv) and the libsolv-tools (dumpsoplv and tools to create the solv files) are based on the same libsolv version. If you are also building the libs on this machine, be sure that you do not have conflicting versions installed in /usr and /usr/local. libzypp follows $PATH when calling the libsolv-tools. `rpmdb2solv | dumpsolv` should give you a reasonable dump of the rpm database. If it does not, try `rpm --rebuilddb` to be sure the rpm database is ok. If the above works, then simply remove the @System solv directory. `zypper ref` will recreate it. If the above does not work, we need to ask Michael Schroeder. -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres SUSE LINUX Products GmbH, Development, ma@suse.de GF:Jeff Hawn,Jennifer Guild,Felix Imendörffer, HRB16746(AG Nürnberg) Maxfeldstrasse 5, D-90409 Nuernberg, Germany, ++49 (0)911 - 740 53-0 +------------------------------------------------------------------+ -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org To contact the owner, e-mail: zypp-devel+owner@opensuse.org
Hi, On 03/05/13 12:44, Michael Andres wrote:
In general: In openSUSE we use a static libsolv. This way it is important that libzypp (staticaly linked against libsolv) and the libsolv-tools (dumpsoplv and tools to create the solv files) are based on the same libsolv version. Thanks for sharing this! It answered a question, I asked myself "what, if I had to update libsolv/zypper with zypper" :-).
`rpmdb2solv | dumpsolv` should give you a reasonable dump of the rpm database. If it does not, try `rpm --rebuilddb` to be sure the rpm database is ok. A rebuilddb helped getting the parsing error fixed, But the dbid (too large) isn't gone. But this seems to be another error, not related to this ML :-)
If the above works, then simply remove the @System solv directory. `zypper ref` will recreate it.
Thank you! Tony -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org To contact the owner, e-mail: zypp-devel+owner@opensuse.org
Hi Tony, On Thu, May 02, 2013 at 04:21:45PM +0200, Tony Wolf wrote:
So, rpm sees all installed packages; but the @System/solv files seems to be broken.
We use the following versions: * zypper 1.8.9 (+ bugfix of swig BoolSave) * libzypp 13.1.0 * libsolv latest snapshot from git/origin * boost 1.53.0
Do you have a hint, where I may take a look at, to solve the problem?
What version of rpm do you use? Thanks, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org To contact the owner, e-mail: zypp-devel+owner@opensuse.org
On Fri, May 03, 2013 at 02:42:57PM +0200, Michael Schroeder wrote:
On Thu, May 02, 2013 at 04:21:45PM +0200, Tony Wolf wrote:
So, rpm sees all installed packages; but the @System/solv files seems to be broken.
We use the following versions: * zypper 1.8.9 (+ bugfix of swig BoolSave) * libzypp 13.1.0 * libsolv latest snapshot from git/origin * boost 1.53.0
Do you have a hint, where I may take a look at, to solve the problem?
What version of rpm do you use?
Does the little 'solv' example program that comes with libsolv work correctly? M. -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org To contact the owner, e-mail: zypp-devel+owner@opensuse.org
Hi Michael, I'm currently checking the rpm db. As far as I know, libsolv supports 32Bit IDs only. That should be okay, because we won't need that much IDs :-) But for any reason, our IDs within our rpmdb are much higher... (there is no small number anywhere, really strange :-)) So it does not seem to be a libsolv problem at all, sorry for that! The rpm version is 4.11.0.1 with berkeley db 4.8.30. Now I'll recreate the rpm db and will track which tool causes this behavior. Thank you, Tony On 03/05/13 14:42, Michael Schroeder wrote:
Hi Tony,
On Thu, May 02, 2013 at 04:21:45PM +0200, Tony Wolf wrote:
So, rpm sees all installed packages; but the @System/solv files seems to be broken.
We use the following versions: * zypper 1.8.9 (+ bugfix of swig BoolSave) * libzypp 13.1.0 * libsolv latest snapshot from git/origin * boost 1.53.0
Do you have a hint, where I may take a look at, to solve the problem?
What version of rpm do you use?
Thanks, Michael.
-- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org To contact the owner, e-mail: zypp-devel+owner@opensuse.org
Hi, all IDs seems to be a multiple of 16777216 (first one is 16777216, second 33554432 and so on). An "rpm -i" already inserted those IDs. 16777216 is a defined value within berkeleydb/common/db_shash.c So it's not related to libsolv/zypper. Sorry for that! I'll try to figure out, what causes that issue (berkelydb or rpm itself?). Thank you, for your time! Kind Regards Tony On 03/05/13 14:54, Tony Wolf wrote:
Hi Michael,
I'm currently checking the rpm db. As far as I know, libsolv supports 32Bit IDs only. That should be okay, because we won't need that much IDs :-)
But for any reason, our IDs within our rpmdb are much higher... (there is no small number anywhere, really strange :-))
So it does not seem to be a libsolv problem at all, sorry for that!
The rpm version is 4.11.0.1 with berkeley db 4.8.30.
Now I'll recreate the rpm db and will track which tool causes this behavior.
Thank you, Tony
On 03/05/13 14:42, Michael Schroeder wrote:
Hi Tony,
On Thu, May 02, 2013 at 04:21:45PM +0200, Tony Wolf wrote:
So, rpm sees all installed packages; but the @System/solv files seems to be broken.
We use the following versions: * zypper 1.8.9 (+ bugfix of swig BoolSave) * libzypp 13.1.0 * libsolv latest snapshot from git/origin * boost 1.53.0
Do you have a hint, where I may take a look at, to solve the problem?
What version of rpm do you use?
Thanks, Michael.
-- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org To contact the owner, e-mail: zypp-devel+owner@opensuse.org
On Fri, May 03, 2013 at 03:28:49PM +0200, Tony Wolf wrote:
all IDs seems to be a multiple of 16777216 (first one is 16777216, second 33554432 and so on). An "rpm -i" already inserted those IDs.
16777216 is a defined value within berkeleydb/common/db_shash.c
Sounds like an endian issue. "rpmdb2solv | dumpsolv | grep rpm:dbid" should show little numbers as ids, if that's not the case, libsolv is somehow misconfigured or has a bug. Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org To contact the owner, e-mail: zypp-devel+owner@opensuse.org
Hi Michael, Thank you a lot! It was the -DRPM5 cmake flag. I replaced it with ENABLE_RPMDB and ENABLE_RPMMD, and now everything works as expected. Thanks again, you saved my weekend :-) Have a nice weekend! Kind Regards Tony On 03/05/13 16:07, Michael Schroeder wrote:
On Fri, May 03, 2013 at 03:28:49PM +0200, Tony Wolf wrote:
all IDs seems to be a multiple of 16777216 (first one is 16777216, second 33554432 and so on). An "rpm -i" already inserted those IDs.
16777216 is a defined value within berkeleydb/common/db_shash.c
Sounds like an endian issue.
"rpmdb2solv | dumpsolv | grep rpm:dbid" should show little numbers as ids, if that's not the case, libsolv is somehow misconfigured or has a bug.
Cheers, Michael.
-- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org To contact the owner, e-mail: zypp-devel+owner@opensuse.org
On Fri, May 03, 2013 at 02:54:15PM +0200, Tony Wolf wrote:
I'm currently checking the rpm db. As far as I know, libsolv supports 32Bit IDs only. That should be okay, because we won't need that much IDs :-)
But for any reason, our IDs within our rpmdb are much higher... (there is no small number anywhere, really strange :-))
So it does not seem to be a libsolv problem at all, sorry for that!
The rpm version is 4.11.0.1 with berkeley db 4.8.30.
Hmm, you don't happen to have defined RPM5 when configuring libsolv by mistake? RPM5 has switched the index from little to bih endian. Also: what arch is that? Is that some standard little endian Intel/AMD platform? M. -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org To contact the owner, e-mail: zypp-devel+owner@opensuse.org
On 03/05/13 16:05, Michael Schroeder wrote:
Hmm, you don't happen to have defined RPM5 when configuring libsolv by mistake? RPM5 has switched the index from little to bih endian.
Yes, I've enabled the RPM5 switch, damn, this happed while experimenting with libsolv... :-) I'll disable it and will try it again.
Also: what arch is that? Is that some standard little endian Intel/AMD platform?
arch is x86_64 I'll be back with the new result soon. Thank you a lot! Tony -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org To contact the owner, e-mail: zypp-devel+owner@opensuse.org
participants (3)
-
Michael Andres -
Michael Schroeder -
Tony Wolf