[zypp-devel] Things TODO in CacheStore
Duncan, what i was talking about is storing all the rest of the data other than NVRAD and kind in these methods: consumePatch() consumePackageAtom() consumeMessage() consumeScript() consumePattern() consumeProduct() Do you want to code it, or shell i? There is some work on the YUMParser stuff yet, but after polishing the code i'm free to do other things. FILELISTS AND CHANGELOGS of YUM Also, what has to be solved (with low priority, though) whether we can support interface like: void CacheStore::consumeFilelist( const data::RecordId &catalog_id, const data::Resolvable_Ptr & resolvable, const data::Filenames & filenames ) This would mean querying the DB for the resolvable of given kind and NVRA in given catalog and appending the changelog/filelist to it. You said that it is not possible since the kind and NVRA are not unique within one catalog? If this would be possible, the YUM parser will have very low need for memory (suse tags will probably not benefit from it(?)). If we don't agree on this way (or if it turns out to be slow), then i would like to have API like: void CacheStore::consumeFilelist( data::RecordId &resolvable_id, const data::Filenames & filenames ) With such API i will be remembering a map of yum package IDs to resolvable record IDs and use it to insert the filelist and changelog or whatever with still low memory foot-print. Otherwise i will need to read all the package/filelist/changelog data of all packages into memory. What do you think? Jano -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Wed, May 23, Jan Kupec wrote:
Duncan, what i was talking about is storing all the rest of the data other than NVRAD and kind in these methods:
consumePatch() consumePackageAtom() consumeMessage() consumeScript() consumePattern() consumeProduct()
JFYI: I add: CacheStore::consumeSourcePackage() -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres YaST Development ma@novell.com SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg) Maxfeldstrasse 5, D-90409 Nuernberg, Germany, ++49 (0)911 - 740 53-0 +------------------------------------------------------------------+ -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Wednesday 23 May 2007 16:59:52 Jan Kupec wrote:
Duncan, what i was talking about is storing all the rest of the data other than NVRAD and kind in these methods:
consumePatch() consumePackageAtom() consumeMessage() consumeScript() consumePattern() consumeProduct()
Do you want to code it, or shell i? There is some work on the YUMParser stuff yet, but after polishing the code i'm free to do other things.
Go ahead. I have to update the testsuite later, now that the parsers are ready, I can use the YUM or YaST parser for the cache store testcase, beter yum, as it has patches.
FILELISTS AND CHANGELOGS of YUM void CacheStore::consumeFilelist( const data::RecordId &catalog_id, const data::Resolvable_Ptr & resolvable, const data::Filenames & filenames )
This would mean querying the DB for the resolvable of given kind and NVRA in given catalog and appending the changelog/filelist to it. You said that it is not possible since the kind and NVRA are not unique within one catalog?
Yes, you have to pass the record id. The problem lays in another part. When I first thought about the ResolvableConsumer interface, I only expected to have void consumeSomething() methods that accept a struct, but not method that returns ids, because this means the producer (parser) has to keep some status. If you need to keep status, then it would not be feasible to implement consumers like a logger. However, YaST metadata needs to fill complete package structs, and then add descriptons latter, so it needs to get the id of a consumePackage call. Perhaps we should move the complete interface of CacheStore to the ResolvableConsumer call. What do you think? So in your case, the consumePackage would return the id, and then you can fill the filelists using this id, then you only need a map of nvra to id, which is pretty cheap. -- Duncan Mac-Vicar Prett Novell :: SUSE R&D, Maxfeldstr. 5, 90409 Nürnberg GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Thu, May 24, Duncan Mac-Vicar Prett wrote:
Yes, you have to pass the record id.
The problem lays in another part. When I first thought about the ResolvableConsumer interface, I only expected to have void consumeSomething() methods that accept a struct, but not method that returns ids, because this means the producer (parser) has to keep some status. If you need to keep status, then it would not be feasible to implement consumers like a logger.
Parsers will hapily keep the ids, because otherwise they had to keep the complete data untill they are complete. A (dummy/test)consumer like a logger can probabely use a counter that's used as Id.
However, YaST metadata needs to fill complete package structs, and then add descriptons latter, so it needs to get the id of a consumePackage call.
In case YUM evaluates a 2nd file (besides primary), it will require the same.
Perhaps we should move the complete interface of CacheStore to the ResolvableConsumer call.
The way you designed it, ResolvableConsumer is the interface for the parser. What's not in ResolvableConsumer can't be used by the parsers. CacheStore is one implementation of a ResolvableConsumer. The application/workflow decides, which ResolvableConsumer to used, by creating a CacheStore or DummLogStore or whatever kind of ResolvableConsumer we provide. After creating the concrete object, it's just a ResolvableConsumer and used like a ResolvableConsumer can be used. In case CacheStore extends the public interface, it is kind a of 'back door', hidden from the parser.
What do you think?
Define a consistent basic way to feed data into the database (full data structs). Offer shortcuts for parsers that will benfit from beig able to feed data in pieces (provide Id).
So in your case, the consumePackage would return the id, and then you can fill the filelists using this id, then you only need a map of nvra to id, which is pretty cheap.
PLEASE if you touche the interface,fix this as well: void consumePackageAtom ( const data::RecordId &catalog_id, const data::PackageAtom_Ptr & ) = 0; void consumeMessage ( const data::RecordId &catalog_id, data::Message_Ptr ) = 0; ... All methods doing the same, just with different kinds of data pointer should use the same signature (for the 2nd arg). If you extend an existing interface, try to adapt it's style. Pass either by "value" or by "const &". -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres YaST Development ma@novell.com SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg) Maxfeldstrasse 5, D-90409 Nuernberg, Germany, ++49 (0)911 - 740 53-0 +------------------------------------------------------------------+ -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Michael Andres wrote:
PLEASE if you touche the interface,fix this as well:
void consumePackageAtom ( const data::RecordId &catalog_id, const data::PackageAtom_Ptr & ) = 0;
void consumeMessage ( const data::RecordId &catalog_id, data::Message_Ptr ) = 0;
...
All methods doing the same, just with different kinds of data pointer should use the same signature (for the 2nd arg).
If you extend an existing interface, try to adapt it's style. Pass either by "value" or by "const &".
This is my fault. I introduced those const &. What's better here? Passing the pointer by value or reference? I used const & to declare constness (shell we possibly modify the object?) and to avoid copying. j. -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Fri, May 25, Jan Kupec wrote:
by "value" or by "const &".
What's better here? Passing the pointer by value or reference? I used const & to declare constness (shell we possibly modify the object?) and to avoid copying.
I'm a fan of "const &", unless it is an integral type. No matter how cheap it is to copy class, it's hardly cheaper than a "const &". Even a class like a smart pointer (or any other class using PIMPL, a std::string e.g. ) has to adjust at least a refcount (2times+1check for counter==0), if a copy is created. And be aware that a copy of a container like list/set/map is not cheap! It is right, that the copies share the payload data, but the list/tree structure is duplicated. That's one reason why one should consider providing iterators, instead of passing back containers. class Foo { NOT: std::list<int> getlist() const; OR: const std::list<int> & getlist() const; BUT: typedef std::list<int> ContainerType; typedef ContainerType::size_type size_type; typedef ContainerType::value_type value_type; typedef ContainerType::iterator iterator; typedef ContainerType::const_iterator const_iterator; bool empty() const { return container.empty(); } size_type size() const { return container.size(); } const_iterator begin() const { return container.begin(); } const_iterator end() const { return container.end(); } // in most classes you will NOT provide nonconst iterators! iterator begin() { return container.begin(); } iterator end() { return container.end(); } }; One benefit for the user: He's not tied to a certain container: Foo foo; std::list<int> mLists( foo.begin(), foo.end() ); std::set<int> mSet ( foo.begin(), foo.end() ); std::vector<int> mVec ( foo.begin(), foo.end() ); // or std::for_each( foo.begin(), foo.end(), &doSomething ); -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres YaST Development ma@novell.com SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg) Maxfeldstrasse 5, D-90409 Nuernberg, Germany, ++49 (0)911 - 740 53-0 +------------------------------------------------------------------+ -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
participants (3)
-
Duncan Mac-Vicar Prett
-
Jan Kupec
-
Michael Andres