[zypp-devel] Playing with sqlite cache
The cached RepoImpl sets the cache to something bigger, but CacheStore just used the default. For a sets of small repos plus 10.2 repo, playing with the sqlite3 cache size changed writing (done during parsing) times: default (2000) 30.122s 8000 0m25.415s 10000 24.443s 15000 23.567 20000 23.529 You can see the curve has an optimum at 10 M or 15 M (is it worth to use 5 mb to save 1 more second?) but this value should depend on the amount of data you write. It would be cool we could optimize this value on runtime depending on the situation. Duncan -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Den Sunday 01 July 2007 18:43:45 skrev Duncan Mac-Vicar P.:
You can see the curve has an optimum at 10 M or 15 M (is it worth to use 5 mb to save 1 more second?)
Speaking as a normal home user I'd say it's worth it. Think time is more valuable to most than an insignificant amount of disk space. Might be just 1 sec, but after all it's 4-5% -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Sunday 01 July 2007 09:23:20 pm Martin Schlander wrote:
Den Sunday 01 July 2007 18:43:45 skrev Duncan Mac-Vicar P.:
You can see the curve has an optimum at 10 M or 15 M (is it worth to use 5 mb to save 1 more second?)
Speaking as a normal home user I'd say it's worth it. Think time is more valuable to most than an insignificant amount of disk space. Might be just 1 sec, but after all it's 4-5%
First: that is memory: it is the ram sqlite cache, not hard disk space. Sqlite uses this to cache accessed pages. I think you are missing one scenario: installation: We could reduce a lot the time of writing the cache, without normalizing names, that is what makes writing slow: we have to select each time before we insert (well not really always, we have some memory caches), but this increases your on-dik cache size by 3 mb per-factory repository: In order to query FAST by names, you need an index, and if the columns are not normalized, the index is big. On installation, there is a ramdisk so everything you have on disk affects ram usage. Still, if we are not in installation, we could default to bigger ram caches, or setup the value depending on the amoun of memory available. You can see more numbers here: http://en.opensuse.org/Libzypp/Refactoring/CacheSchema Duncan -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
Den Monday 02 July 2007 11:40:59 skrev Duncan Mac-Vicar P.:
First: that is memory: it is the ram sqlite cache, not hard disk space. Sqlite uses this to cache accessed pages.
It occured to me after I sent that I might have misunderstood.
I think you are missing one scenario: installation:
On installation, there is a ramdisk so everything you have on disk affects ram usage.
Still, if we are not in installation, we could default to bigger ram caches, or setup the value depending on the amoun of memory available.
Hmm.. couple of questions: #1 Are these few megs of RAM that we're talking about crucial for running the graphical installer on 256 megs of RAM without needing swap or not? If yes, I think saving RAM should be high priority. #2 Are we near the dangerzone wrt. RAM during installation on 384 MB or 512 MB systems? If not, why not use away and proritize daily use performance? (assuming #1 doesn't come into play) -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
participants (2)
-
Duncan Mac-Vicar P.
-
Martin Schlander