[zypp-devel] Preliminary zypper port
Hi! I have merged zypper changes from trunk into the sat branch, and then ported zypper to compile with libzypp. I had some porting steps in wiki Libzypp/SAT_Porting but a nvidia crash sent it to hell before I could save it :-( Some notes: - I submited the packages to zypp:svn project. llibzypp does not build because a weird error after rpm checks for unpackaged files: |RPM build errors: cannot open Pubkeys index using db3 - No such file or directory (2) File not found: /var/tmp/libzypp-4.1.6-build/usr/lib/zypp (anyone knows about this) - the first try, zypper hangs and eats all ram just after loading proxy settings. I unittested the parser and it was OK. So the codepath that comes next, which is something curl proxy related might be causing problems. Renaming sysconfig/proxy away solved this issue. - zypper loads the repos fine, reads rpm database, very fast (9.5 seconds for 18 repos _cold_ time including rpm, that is 10x faster, if I am not lying myself reading the logs in a wrong way), but it can't find the object to install in the pool because the pool iterator is not connected to the SAT pool, this is something Michael is working on. - Right now there is no store for saving patterns and patches. This will be tricky: current store: store --> rpmmd2solv (xml convertor) --> /var/lib/zypp/store.solv -> zypp repo (pool) rpm db ---> rpmdb2solv -----> /var/cache/zypp/_rpm.solv -> zypp repo (pool) The first step of converting the store from xml happens only once, and from rpm every time the rpm db changes. Now, when target reads, we have to load both solvs in the same repo, because sat solver takes only one "installed repo", but later we need to dump the patterns and patches to one solv, while the packages are not dumped (because the primary source is the rpm database ). This requires 2 AIs - integrating repo_write_solv into the library - allowing to save by kind or better: some generic filter) so a solv. This part is _very important_ |- zypper search has to be redone, basically using sat-solver/Repodata search, perhaps we can keep the same interface for now. This is not urgent for the first submit though. | Cheers Duncan | -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Hi, On Sun, 3 Feb 2008, Duncan Mac-Vicar P. wrote:
I have merged zypper changes from trunk into the sat branch, and then ported zypper to compile with libzypp.
Excellent!
- Right now there is no store for saving patterns and patches. This will be tricky:
current store: store --> rpmmd2solv (xml convertor) --> /var/lib/zypp/store.solv -> zypp repo (pool) rpm db ---> rpmdb2solv -----> /var/cache/zypp/_rpm.solv -> zypp repo (pool)
The first step of converting the store from xml happens only once, and from rpm every time the rpm db changes.
Now, when target reads, we have to load both solvs in the same repo, because sat solver takes only one "installed repo", but later we need to dump the patterns and patches to one solv, while the packages are not dumped (because the primary source is the rpm database ). This requires 2 AIs
- integrating repo_write_solv into the library - allowing to save by kind or better: some generic filter) so a solv.
We can already filter per attribute. I could implement also a filter per solvable. Ciao, Michael. -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Michael Matz wrote:
We can already filter per attribute. I could implement also a filter per solvable.
It will not be needed at all, Klaus explained us "patches and patterns" and we wont install them anymore in the system, so we can continue without this. The functionality is still "nice to have" Duncan -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Hi, On Mon, 4 Feb 2008, Duncan Mac-Vicar Prett wrote:
Michael Matz wrote:
We can already filter per attribute. I could implement also a filter per solvable.
It will not be needed at all, Klaus explained us "patches and patterns" and we wont install them anymore in the system, so we can continue without this.
Oha. Explain to me too please... :-) (So we won't ever know if the system had a pattern so-and-so installed?) Ciao, Michael. -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
* Michael Matz
It will not be needed at all, Klaus explained us "patches and patterns" and we wont install them anymore in the system, so we can continue without this.
Oha. Explain to me too please... :-) Just come to my office[1].
(So we won't ever know if the system had a pattern so-and-so installed?)
Exactly. Klaus [1] The details will be made available for non-Novell employees during hack-week (Feb 11. - 15.) --- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Michael Matz wrote:
Oha. Explain to me too please... :-) (So we won't ever know if the system had a pattern so-and-so installed?)
It will be infered bottom up, so if you have certain packages, the pattern will go green, same with patches. The other way around, if you mark a pattern for installation, it will be solved and packages selected, but the pattern itself will not live with the packages in the same pool. Klaus can explain more. Duncan -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Duncan Mac-Vicar P. wrote:
- the first try, zypper hangs and eats all ram just after loading proxy settings. I unittested the parser and it was OK. So the codepath that comes next, which is something curl proxy related might be causing problems. Renaming sysconfig/proxy away solved this issue hm. might be a bug in my new .curlrc reader code (media/CurlConfig.cc). I 'll try to reproduce. Can you pls. send me your ~root/.curlrc and sysconfig/proxy file? (change the passwords if there are any).
jano -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Jano Kupec wrote:
- the first try, zypper hangs and eats all ram just after loading proxy settings. I unittested the parser and it was OK. So the codepath that comes next, which is something curl proxy related might be causing problems. Renaming sysconfig/proxy away solved this issue hm. might be a bug in my new .curlrc reader code (media/CurlConfig.cc). I 'll try to reproduce. Can you pls. send me your ~root/.curlrc and sysconfig/proxy file? (change the passwords if
Duncan Mac-Vicar P. wrote: there are any).
jano does that class has a unit test? :-) -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Duncan Mac-Vicar Prett wrote:
Jano Kupec wrote:
- the first try, zypper hangs and eats all ram just after loading proxy settings. I unittested the parser and it was OK. So the codepath that comes next, which is something curl proxy related might be causing problems. Renaming sysconfig/proxy away solved this issue hm. might be a bug in my new .curlrc reader code (media/CurlConfig.cc). I 'll try to reproduce. Can you pls. send me your ~root/.curlrc and sysconfig/proxy file? (change the passwords if
Duncan Mac-Vicar P. wrote: there are any).
jano does that class has a unit test? :-) yes, in my TODO :O) i did test it quite much however, this will be a new entry to the unit test then :O)
j. -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Sun, Feb 03, Duncan Mac-Vicar P. wrote:
- the first try, zypper hangs and eats all ram just after loading proxy settings. I unittested the parser and it was OK. So the codepath that
Mea culpa. fixed in revision 8459. (str::split running wild)
- zypper loads the repos fine, reads rpm database, very fast (9.5 ...
I know we're in a hurry, but we should try to remove created files/directories in case of an error. It's quite easy to achieve this: +#include "zypp/ManagedFile.h" switch ( repokind.toEnum() ) { case RepoType::RPMMD_e : case RepoType::YAST2_e : { MIL << "Executing solv converter" << endl; + // Take care we unlink the solvfile on exception + ManagedFile guard( solvfile, filesystem::erase ); - string cmd( str::form( "repo2solv.sh \"%s\" > %s", + string cmd( str::form( "repo2solv.sh \"%s\" > '%s'", rawpath.c_str(), solvfile.c_str() ) ); ExternalProgram prog( cmd, ExternalProgram::Stderr_To_Stdout ); for ( string output( prog.receiveLine() ); output.length(); output = prog.receiveLine() ) { MIL << " " << output; } int ret = prog.close(); if ( ret != 0 ) ZYPP_THROW(RepoUnknownTypeException()); + // We keep it. + guard.resetDispose(); } break; default: ZYPP_THROW(Exception("Unhandled repostory type")); break; } -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres YaST Development ma@novell.com SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg) Maxfeldstrasse 5, D-90409 Nuernberg, Germany, ++49 (0)911 - 740 53-0 +------------------------------------------------------------------+ -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Michael Andres wrote:
On Sun, Feb 03, Duncan Mac-Vicar P. wrote:
- the first try, zypper hangs and eats all ram just after loading proxy settings. I unittested the parser and it was OK. So the codepath that
Mea culpa. fixed in revision 8459.
(str::split running wild)
heh: freenode_#yast.log:[Sat Feb 2 2008] [20:56:27] <duncanmv> coolo: I found the problem, michael andres broke all string utility functions some days ago So this time you committed a unit test for String.h ? :-)
+#include "zypp/ManagedFile.h"
switch ( repokind.toEnum() ) { case RepoType::RPMMD_e : case RepoType::YAST2_e : { MIL << "Executing solv converter" << endl; + // Take care we unlink the solvfile on exception + ManagedFile guard( solvfile, filesystem::erase );
- string cmd( str::form( "repo2solv.sh \"%s\" > %s", + string cmd( str::form( "repo2solv.sh \"%s\" > '%s'", rawpath.c_str(), solvfile.c_str() ) ); ExternalProgram prog( cmd, ExternalProgram::Stderr_To_Stdout ); for ( string output( prog.receiveLine() ); output.length(); output = prog.receiveLine() ) { MIL << " " << output; } int ret = prog.close(); if ( ret != 0 ) ZYPP_THROW(RepoUnknownTypeException());
+ // We keep it. + guard.resetDispose(); } break; default: ZYPP_THROW(Exception("Unhandled repostory type")); break; }
thanks! I forgot about this class Duncan -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Mon, Feb 04, Duncan Mac-Vicar P. wrote:
Michael Andres wrote:
On Sun, Feb 03, Duncan Mac-Vicar P. wrote:
- the first try, zypper hangs and eats all ram just after loading proxy settings. I unittested the parser and it was OK. So the codepath that
Mea culpa. fixed in revision 8459.
(str::split running wild)
heh: freenode_#yast.log:[Sat Feb 2 2008] [20:56:27] <duncanmv> coolo: I found the problem, michael andres broke all string utility functions some days ago
If you recognized this on saturday, why didn't you tell me? We will finally run into trouble if we, instead of fixing things, try to circumvent them. -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres YaST Development ma@novell.com SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg) Maxfeldstrasse 5, D-90409 Nuernberg, Germany, ++49 (0)911 - 740 53-0 +------------------------------------------------------------------+ -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Michael Andres wrote:
If you recognized this on saturday, why didn't you tell me?
We will finally run into trouble if we, instead of fixing things, try to circumvent them.
Because when I examined later, I did not have enough evidence, and Jano's looked a better victim ;-) If we had unit tests for every single class, they would break in cruisecontrol server and you would be informed 3 minutes after commit. Duncan -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
participants (6)
-
Duncan Mac-Vicar P. -
Duncan Mac-Vicar Prett
-
Jano Kupec -
Klaus Kaempf -
Michael Andres -
Michael Matz