February 2008 - Zypp Development - openSUSE Mailing Lists

[zypp-devel] Re: [zypp-commit] r8586 - /trunk/sat-solver/tools/repo_rpmmd.c
by Michael Matz 10 Feb '08

10 Feb '08

Hi Duncan, On Sun, 10 Feb 2008, dmacvicar(a)svn.opensuse.org wrote: > @@ -451,18 +463,18 @@ > { > case STATE_METADATA: > for (; *atts; atts += 2) > - { > - if (!strcmp(*atts, "packages")) > - { > - pd->numpacks = atoi(atts[1]); > - if (pd->numpacks < 0) > - pd->numpacks = 0; > + { > + if (!strcmp(*atts, "packages")) > + { > + pd->numpacks = atoi(atts[1]); > + if (pd->numpacks < 0) > + pd->numpacks = 0; Please leave the sat-solver files in GNU indentation style. See http://www.gnu.org/prep/standards/standards.html.gz#Formatting . mls doesn't like the space before open-parentheses in function calls (boo, hiss!) but apart from that the style is fairly homogenous right now. It should stay that way. Thanks, Michael. -- To unsubscribe, e-mail: zypp-devel+unsubscribe(a)opensuse.org For additional commands, e-mail: zypp-devel+help(a)opensuse.org

2 2

[zypp-devel] Re: [zypp-commit] r8592 - in /branches/tmp/ma/jump_sat/libzypp/zypp: Package.cc sat/Solvable.cc
by Michael Matz 10 Feb '08

10 Feb '08

Hi, On Sun, 10 Feb 2008, dmacvicar(a)svn.opensuse.org wrote: > Author: dmacvicar > Date: Sun Feb 10 23:34:15 2008 > New Revision: 8592 > > URL: http://svn.opensuse.org/viewcvs/zypp?rev=8592&view=rev > Log: > - this is wrong, suse/ dir is a datadir told in the content file so this uggly append is > not valid for yum repos. > > so for now, as the optimization of mediafile only happens in susetags repos > append suse/ only for this case. As I said, someone needs to store the datadir into Repoinfo (for rpm-md it's for instance usually "repodata"). Then we can use it to form full rpm filepaths. Ciao, Michael. -- To unsubscribe, e-mail: zypp-devel+unsubscribe(a)opensuse.org For additional commands, e-mail: zypp-devel+help(a)opensuse.org

2 2

[zypp-devel] testing jump_sat zypper
by Stephan Kulow 08 Feb '08

08 Feb '08

Hi, I found two problems: 1. "zypper dup" takes _FOREVER_ (and creates again MBs of zypper.log) 2. "zypper search gnome" crashes in ResolvableQuery.cc ==12994== Invalid read of size 4 ==12994== at 0x8076E48: std::basic_string<char, std::char_traits<char>, std::allocator<char> > std::operator+<char, std::char_traits<char>, std::all ocator<char> >(char const*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (basic_string.tcc:675) ==12994== by 0x4113764: _ZN4zypp5cache15ResolvableQuery4Impl43iterateResolvablesByKindsAndStringsAndReposERKSt6vectorINS_7ResKindESaIS4_EERKS3_ISsSa ISsEEiSA_N5boost8functionIFbRKNS_15DefaultIntegralIxLxn1EEENSD_13intrusive_ptrINS_4data9ResObjectEEEESaIvEEE (ResolvableQuery.cc:209) ==12994== by 0x41127DA: _ZN4zypp5cache15ResolvableQuery43iterateResolvablesByKindsAndStringsAndReposERKSt6vectorINS_7ResKindESaIS3_EERKS2_ISsSaISsEE iSB_N5boost8functionIFbRKNS_15DefaultIntegralIxLxn1EEENSC_13intrusive_ptrINS_4data9ResObjectEEEESaIvEEE (ResolvableQuery.cc:525) ==12994== by 0x80D031B: int ZyppSearch::invokeOnEachSearched<Match, zypp::functor::FunctorRef<InstalledCache, bool, zypp::PoolItem const&, zypp::fun ctor::functor_detail::nil>, zypp::functor::FunctorRef<InstalledCache, bool, zypp::DefaultIntegral<long long, (long long)-1> const&, boost::intrusive_pt r<zypp::data::ResObject> > >(Match, zypp::functor::FunctorRef<InstalledCache, bool, zypp::PoolItem const&, zypp::functor::functor_detail::nil>, zypp::f unctor::FunctorRef<InstalledCache, bool, zypp::DefaultIntegral<long long, (long long)-1> const&, boost::intrusive_ptr<zypp::data::ResObject> >) (zypper -search.cc:211) ==12994== by 0x80CA359: ZyppSearch::cacheInstalled() (zypper-search.cc:243) ==12994== by 0x80CC49F: ZyppSearch::ZyppSearch(boost::intrusive_ptr<zypp::ZYpp>&, ZyppSearchOptions const&, std::vector<std::string, std::allocator< std::string> >) (zypper-search.cc:95) ==12994== by 0x806DDA3: Zypper::doCommand() (zypper.cc:1930) ==12994== by 0x80712A3: Zypper::safeDoCommand() (zypper.cc:601) ==12994== by 0x8075F9E: Zypper::main(int, char**) (zypper.cc:113) ==12994== by 0x80872B7: main (zypper-main.cc:43) ==12994== Address 0x8 is not stack'd, malloc'd or (recently) free'd -- To unsubscribe, e-mail: zypp-devel+unsubscribe(a)opensuse.org For additional commands, e-mail: zypp-devel+help(a)opensuse.org

4 9

[zypp-devel] RFC: ZYpp application layer comments
by Duncan Mac-Vicar P. 08 Feb '08

08 Feb '08

I would like others (especially Michael) to review it. I am quoting the blocks of texts from the Wiki, which I would like to focus the discussion on. >Communication between the YCP application and the C++ package selector widgets is very limited (almost nonexistent). In particular, the YCP application cannot create a pool and pass a handle to that pool to the C++ package selector or vice versa. >Thus, for all important objects like pool or similar, there must be a singleton object for each that can easily be accessed from both the YCP code (via package bindings) and from the package selectors. Those singleton objects should live in the app layer. >Selectables have a higher level status that is the result of the useful permutations of the status flags of their resolvables and the fact whether or not one resolvable from this selectable (one instance of that package) is already installed: >Matching Selectable and Resolvable Status and Candidate Handling >The package selectors use the selectable status. They never store any selectable's status; they always retrieve the status when it is needed. When the user changes a selectable's status, they set new status and retrieve it for display so the display and the selectable status are always in sync. >Since selectable don't exist in the libzypp layer, matching a selectable status to the individual resolvable status flags is done by the app layer and vice versa. The app layer also keeps track of the candidate. This is important when the selectable status is used since some operations (install, update) implicitly operate on the candidate. I think this is one of the central parts of the problem we need to solve. >As long as the user did not explicitly choose a specific candidate, the app layer automatically chooses one from the available resolvables. That automatic choice takes matching architectures and most recent version number into account. This is not trivial. Right now the UIs do this choices in some random code somewhere in the UI, or using the selectable candidate function, which is now incomplete. This kind of decision is the same as the solver local policy. So we have to connect them trough the API somehow. Imagine this something like: bestCandidate( list_of_candidates, task ) Of course this has to be in Selectable, but it has to be connected to the local policy, and it depends on the task you want to execute. Otherwise the UIs will continue breaking things like vendor (the UIs iterate over a package list and select the higher version instead of the best candidate according to the system policies ). >Patterns (collections of packages to fulfil a certain task) This is where the UI <-> library problems starts. What the wiki describes there is the description of a Selection. >Technically, none of those higher level objects has a package content: They all just have dependencies that typically (but not necessarily) are satisfied by packages. Exactly. >From a user's point of view, however, those higher level objects are nothing else than containers for packages. Though this is a somewhat simplistic view, users typically want to see what is behind a pattern, a patch, or a language resolvable. This is why each of them should have a mechanism to enumerate the packages that belong to them. This is true, but only if you can live with the true: the mechanism to enumerate the list of packages is just an algorithm, not a list. It can be false information. That is why I tried to start using the term "related packages" instead of "content". >Enumerate all packages that belong to an RPM group (not just for leaf RPM groups, also those that belong to a higher level like "Productivity") I think the best way to solve this problems is to introduce categories for all resolvabes (tags), in the case of packages it can be implemented using rpm groups, and implement iterators to match full tag, part of it, a collection of them, etc. >Changing the status of a pattern, patch, or language typically affects the status of other resolvables, most notably the packages that belong to this pattern, patch, or language. Those changes should be propagated immediately to those packages so the user can instantly see the effect. >In the past there were calls to just propagate those changes, which worked in most cases, but not in certain pathological ones. But those calls were quick, so for the user there was an instant response. >Recent changes rely on a full solver run to propagate the changes, which gives the correct result in all cases, but which on the other hand is much slower. As some users disable the automatic dependency check because of this, some of them get confused because no change is propagated. There have been bug reports with lively discussions about that. I read this as, we need to do a solver run without doing a solver run? This is a technical limitation, which could be solved of we could for example cheap-ly clone a pool , do something and see the changes, without making the main one dirty. I have no idea yet if that undocumented dirpool has something to do with that ;-) >The user can select any one of those search modes: > > * Contains > * Begins with > * Exact match > * Use wildcards (*, ?) > * Use regular expression > >Case sensitivity in all searches can be switched on or off. > >A query engine to replace this functionality in the UI should enumerate >(return an iterator to) the selectables that match the given criteria. > Update Problems Filter View >This filter view enumerates those packages that for one reason or the other >could not automatically be updated during a system update. Adding packages >to this list is done by the system update application layer. And what is the criteria for searching those? >Libzypp/Application Layer/API Proposal >enum Type { > PACKAGE, > PATTERN, > PATCH, > LANGUAGE > } Why resolvable kind can't be used here? Why the need to duplicate it? > * Create pool iterator (based on the above query as a parameter) > > Pool pool = new Pool (q); So this is like a pool proxy based on a query? Why not the iterator taking the query as a parameter (Michael could comment more?) I have one concern with the design of the pool based on queries. It means we have to load the pool data, this will not be bad with the sat solver, but this is not the same zypper does, as zypper only gets the resolvable data to be displayed on the screen, without any pool. Appart of that the query api seems fine. Other API that could be part of the zypp layer is a command oriented api. If you look what API smart offers to the UIs is a much simpler API, and it is the same the command line tool offers, and this is quite similar to all zypper code. It defines a Command class (install, search, etc) it defines a abstract interface for callbacks (confirm dialogs, progress, questions, etc) The UI simply creates a command and executes it. This is a much simpler approach, but I think it is needed in addition to the layer our selector need. Both packagekit and any app wanting to install a package could benefit from this. Duncan -- To unsubscribe, e-mail: zypp-devel+unsubscribe(a)opensuse.org For additional commands, e-mail: zypp-devel+help(a)opensuse.org

5 9

[zypp-devel] status
by Duncan Mac-Vicar Prett 07 Feb '08

07 Feb '08

- pkg-bndings are ported and they are in build service, it is a _link against 10.3 one plus a patch, the patch does not apply, if anyone knows how to fix this please do it. I am not expert in these OBS features. - TODO: mergesolv and yum attributes (at least package path) - libzypp, zypper submited to obs, libzypp does not build (can't understand the error) - pool now is fast (ma fixed it ;-) ) - zypper in foo is really fast... - added coolo and Klaus to OBS zypp:svn project so only having the attribute medafile in the solv file while commit, is what we need to install a package (unless other minor bug appears) Cheers! Duncan -- To unsubscribe, e-mail: zypp-devel+unsubscribe(a)opensuse.org For additional commands, e-mail: zypp-devel+help(a)opensuse.org

3 4

[zypp-devel] missing stuff
by Duncan Mac-Vicar P. 06 Feb '08

06 Feb '08

We are close guys, but still things don't work together. After Michael connected the pool with the sat pool, zypper now works just before commit, it now marks the packages to install and they get solved as expected. Commit fails then because resolvables have no repo information. URGENT STUFF (for "tomorrows" deadline) - repo2solv.sh was not using -a by default, and ignoring the packages translations, I added packages.en for now. but later, it uses mergesolv, which makes the attributes to dissapear, so it works if I manually cat packages.en and packages and use susetags2solv -a, but not with repo2solv.sh which uses mergesolv. Can we fix this? libzypp forks repo2solv.sh - I tried to implement something similar like repo_lookup_str which does not require a callback, but for nums, and it did not work :-( Can aynone check it? - Commit fails because Resolvables created by the pool iterator have a dummy Repository class. We need a way to associate the sat::Repo (which can be constructed just by using an id), this should be easy, but it is tricky, before all resolvables were created when creating a repository object, and this was constructed from the info, so every resolvable had a link to it. I think the right place to do it is when calling RepoManager::loadFromCache, we get the repoinfo and we should store in some place, where? no idea, as the Repository class has no sense anymore. - pkg-bindings are not ported yet :-( but from zypper porting this should be straightforward OTHER STUFF - zypper iterates the pool to show a summary of installation, also to mark a candidate for install. This is really slow with the current (non-optimized) pool adaptor. Is there another way we can speed up this? This also slows commit calculation which also heavily relies on pool iterations. For now it looks like all time moved from reading the cache is now here ;-) Appart of that things look promising. Speed and memory usage is nice. -- To unsubscribe, e-mail: zypp-devel+unsubscribe(a)opensuse.org For additional commands, e-mail: zypp-devel+help(a)opensuse.org

5 5

[zypp-devel] Re: [zypp-commit] r8486 - /trunk/sat-solver/tools/repo_rpmdb.c
by Michael Matz 06 Feb '08

06 Feb '08

Hi Duncan, On Wed, 6 Feb 2008, dmacvicar(a)svn.opensuse.org wrote: > Author: dmacvicar > Date: Wed Feb 6 03:08:04 2008 > New Revision: 8486 > > URL: http://svn.opensuse.org/viewcvs/zypp?rev=8486&view=rev > Log: > avoid segfault. FIXME please Can't reproduce with current trunk. There was a segfault before in stringpool_str2id (from calling strlen on NULL), but that was fixed by ma some time ago. Are you sure you had current src/stringpool.c ? (I'm commenting this in again, if you still experience segfault, please tell me where I can take a look). Ciao, Michael. -- To unsubscribe, e-mail: zypp-devel+unsubscribe(a)opensuse.org For additional commands, e-mail: zypp-devel+help(a)opensuse.org

1 0

[zypp-devel] Preliminary zypper port
by Duncan Mac-Vicar P. 05 Feb '08

05 Feb '08

Hi! I have merged zypper changes from trunk into the sat branch, and then ported zypper to compile with libzypp. I had some porting steps in wiki Libzypp/SAT_Porting but a nvidia crash sent it to hell before I could save it :-( Some notes: - I submited the packages to zypp:svn project. llibzypp does not build because a weird error after rpm checks for unpackaged files: |RPM build errors: cannot open Pubkeys index using db3 - No such file or directory (2) File not found: /var/tmp/libzypp-4.1.6-build/usr/lib/zypp (anyone knows about this) - the first try, zypper hangs and eats all ram just after loading proxy settings. I unittested the parser and it was OK. So the codepath that comes next, which is something curl proxy related might be causing problems. Renaming sysconfig/proxy away solved this issue. - zypper loads the repos fine, reads rpm database, very fast (9.5 seconds for 18 repos _cold_ time including rpm, that is 10x faster, if I am not lying myself reading the logs in a wrong way), but it can't find the object to install in the pool because the pool iterator is not connected to the SAT pool, this is something Michael is working on. - Right now there is no store for saving patterns and patches. This will be tricky: current store: store --> rpmmd2solv (xml convertor) --> /var/lib/zypp/store.solv -> zypp repo (pool) rpm db ---> rpmdb2solv -----> /var/cache/zypp/_rpm.solv -> zypp repo (pool) The first step of converting the store from xml happens only once, and from rpm every time the rpm db changes. Now, when target reads, we have to load both solvs in the same repo, because sat solver takes only one "installed repo", but later we need to dump the patterns and patches to one solv, while the packages are not dumped (because the primary source is the rpm database ). This requires 2 AIs - integrating repo_write_solv into the library - allowing to save by kind or better: some generic filter) so a solv. This part is _very important_ |- zypper search has to be redone, basically using sat-solver/Repodata search, perhaps we can keep the same interface for now. This is not urgent for the first submit though. | Cheers Duncan | -- To unsubscribe, e-mail: zypp-devel+unsubscribe(a)opensuse.org For additional commands, e-mail: zypp-devel+help(a)opensuse.org

6 12

[zypp-devel] Re: [zypp-commit] r8464 - in /branches/tmp/ma/jump_sat/libzypp/zypp: ./ cache/ parser/ parser/xmlstore/ parser/xmlstore/schema/ solver/detail/ target/ target/store/
by Stanislav Visnovsky 05 Feb '08

05 Feb '08

Dňa Monday 04 February 2008 22:58:14 dmacvicar(a)svn.opensuse.org ste napísal: > Author: dmacvicar > Date: Mon Feb 4 22:58:14 2008 > New Revision: 8464 > > URL: http://svn.opensuse.org/viewcvs/zypp?rev=8464&view=rev > Log: > - remove xml store > > snif, this is emotional for me, it was the first piece of c++ code I wrote > when joining SUSE. > > But the replacement is... nothing! what can be better than removing code by > simplifying the concept. Where is the concept documented when you started to implement it already? Stano -- To unsubscribe, e-mail: zypp-devel+unsubscribe(a)opensuse.org For additional commands, e-mail: zypp-devel+help(a)opensuse.org

1 0

[zypp-devel] zypp::ResolverProblemsList->details()
by Katarina Machalkova 04 Feb '08

04 Feb '08

Hi guys, does anyone know what happened to $subject? While testing package selector, I found out that details describing current solver problem (messages such as "A conflict over foo-1.2.3 requires removal of bar-3.2.1 which is scheduled to be kept etc. etc.") have suddenly disappeared. In other words, zypp::ResolverProblemsList->details() keeps returning empty strings. Or, at least I've been unable to find some case where it would actually return something (but maybe I have obsoleted zypp headers). Are these problem details gone for good with the new sat solver? Whether they are or not, I would probably consider adjusting UI (nc) not to display them at all, it has always been overly long string, hardly fitting the screen and confusing user more than bringing some actual useful info. Afaik, Qt package manager does not display solver problem details either (in the new Qt4 port). What do you think? B. -- \\\\\ Katarina Machalkova \\\\\\\__o YaST developer __\\\\\\\'/_ & hedgehog painter

2 2