[Bug 953130] New: zypper up "checking for file conflicts" takes 3.5 cpu-minutes for 32 packages
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 Bug ID: 953130 Summary: zypper up "checking for file conflicts" takes 3.5 cpu-minutes for 32 packages Classification: openSUSE Product: openSUSE Distribution Version: Leap 42.1 Hardware: x86-64 OS: Other Status: NEW Severity: Normal Priority: P5 - None Component: Basesystem Assignee: bnc-team-screening@forge.provo.novell.com Reporter: koenig@linux.de QA Contact: qa-bugs@suse.de Found By: --- Blocker: --- User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:41.0) Gecko/20100101 Firefox/41.0 Build Identifier: just run "zypper up" again, and for 32 RPMs the Checking for file conflicts: ..................................<48%>===========================================[|] took > 3.5 minutes cpu time (mostly at the 48% 'location'. that sucks! comparing even a huge list of files shouldn't take that long -- maybe do some intelligent sorting/indexing of filename lists ?! this is the update summary from zypper, more details on request: Reading installed packages... The following 13 NEW packages are going to be installed: crash-kmp-default-7.1.3_k4.1.12_1-4.14 crash-kmp-xen-7.1.3_k4.1.12_1-4.14 hdjmod-kmp-xen-1.28_k4.1.12_1-23.17 kernel-default-4.1.12-1.1 kernel-default-devel-4.1.12-1.1 kernel-devel-4.1.12-1.1 kernel-pv-devel-4.1.12-1.1 kernel-source-4.1.12-1.1 kernel-syms-4.1.12-1.1 kernel-xen-4.1.12-1.1 kernel-xen-devel-4.1.12-1.1 lttng-modules-kmp-default-2.7.0_k4.1.12_1-1.6 xen-kmp-default-4.5.1_10_k4.1.12_1-1.4 The following 19 packages are going to be upgraded: crash install-initrd-openSUSE kernel-macros lttng-modules package-translations virtualbox virtualbox-guest-kmp-default virtualbox-guest-tools virtualbox-guest-x11 virtualbox-host-kmp-default virtualbox-qt xen xen-doc-html xen-libs xen-tools yast2-kdump yast2-ntp-client yast2-storage yast2-storage-devel 19 packages to upgrade, 13 new. Overall download size: 251.4 MiB. Already cached: 0 B. After the operation, additional 980.8 MiB will be used. Continue? [y/n/? shows all options] (y): Reproducible: Always Actual Results: takes too much time, too slow Expected Results: should be *much* faster ;-) you ask, I'll provide... -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 Dominique Leuenberger <dimstar@opensuse.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dimstar@opensuse.org Component|Basesystem |libzypp Assignee|bnc-team-screening@forge.pr |zypp-maintainers@forge.prov |ovo.novell.com |o.novell.com -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c1 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |koenig@linux.de Flags| |needinfo?(koenig@linux.de) --- Comment #1 from Michael Andres <ma@suse.com> --- Please attach the zypper logfile /var/log/zypper.log (or an older and compressed /var/log/zypper.log-YYYYMMDD.xz) that shows the reported behavior. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c2 --- Comment #2 from Harald Koenig <koenig@linux.de> --- Created attachment 654254 --> http://bugzilla.opensuse.org/attachment.cgi?id=654254&action=edit zypper.log -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c3 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mls@suse.com Summary|zypper up "checking for |[libsolv] zypper up |file conflicts" takes 3.5 |"checking for file |cpu-minutes for 32 packages |conflicts" takes 3.5 | |cpu-minutes for 32 packages Flags|needinfo?(koenig@linux.de) |needinfo?(mls@suse.com) --- Comment #3 from Michael Andres <ma@suse.com> ---
11:30:34 [zypp] TargetImpl.commitFindFileConflicts.cc(commitFindFileConflicts):127 Checking for file conflicts in 32 new packages... 11:30:34 [libsolv++] PoolImpl.cc(logSat):114 searching for file conflicts 11:30:34 [libsolv++] PoolImpl.cc(logSat):114 packages: 8577, cutoff 32 . 11:30:48 [libsolv++] PoolImpl.cc(logSat):114 filemap size: 131072, used 44568 11:30:48 [libsolv++] PoolImpl.cc(logSat):114 filemap memory usage: 1024 K 11:30:48 [libsolv++] PoolImpl.cc(logSat):114 filemap creation took 14006 ms 11:30:48 [libsolv++] PoolImpl.cc(logSat):114 lookat_dir size: 206004 . 11:31:54 [libsolv++] PoolImpl.cc(logSat):114 normap size: 262144, used 85084 11:31:54 [libsolv++] PoolImpl.cc(logSat):114 normap memory usage: 2048 K 11:31:54 [libsolv++] PoolImpl.cc(logSat):114 stats made: 85084 11:31:54 [libsolv++] PoolImpl.cc(logSat):114 statmap size: 262144, used 68329 11:31:54 [libsolv++] PoolImpl.cc(logSat):114 statmap memory usage: 2048 K 11:31:54 [libsolv++] PoolImpl.cc(logSat):114 alias processing took 66379 ms 11:31:54 [libsolv++] PoolImpl.cc(logSat):114 raw candidates: 688248, pruning 11:31:55 [libsolv++] PoolImpl.cc(logSat):114 candidates now: 95267 . 11:33:51 [libsolv++] PoolImpl.cc(logSat):114 filespace size: 15430 K 11:33:51 [libsolv++] PoolImpl.cc(logSat):114 candidate check took 117017 ms 11:33:51 [libsolv++] PoolImpl.cc(logSat):114 found 0 file conflicts 11:33:51 [libsolv++] PoolImpl.cc(logSat):114 file conflict detection took 197407 ms 11:33:51 [zypp] TargetImpl.commitFindFileConflicts.cc(commitFindFileConflicts):160 Found 0 file conflicts.
@mls: Retrieving the RPM headers seems to take some time, but the last pass seems to be very slow. Any idea how to improve it? -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 H. Hansen <cyberbeat@gmx.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |cyberbeat@gmx.de -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c4 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(koenig@linux.de) --- Comment #4 from Michael Andres <ma@suse.com> --- @Harald: if the problems persists, could you please add a solver testcase.
# zypper in --debug-solver nopackage ... Generating solver test case... Solver test case generated successfully at /var/log/zypper.solverTestCase. Then pack the output directory together with /var/log/zypper.log and attach it to the bugreport.
In case the testcase exceeds the 10MB limit for bugzilla uploads, you can use 'split' to divide the file into smaller pieces:
$ cd /tmp $ tar cvjf zypper.solverTestCase.tar.bz2 /var/log/zypper.solverTestCase/ $ split -b 9M --additional-suffix=-zypper.solverTestCase.tar.bz2 \ zypper.solverTestCase.tar.bz2 The created pieces will be named like this: xaa-zypper.solverTestCase.tar.bz2 xab-zypper.solverTestCase.tar.bz2 xac-zypper.solverTestCase.tar.bz2 ...
-- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c8 Stefan Quandt <squan@web.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |squan@web.de --- Comment #8 from Stefan Quandt <squan@web.de> --- For me "Checking for file conflicts" just stalled at 57% for >5 minutes without finishing, hence I needed to kill zypper. I could resolve this situation by deleting *lots* of different installed versions of package kernel-source and package kernel-devel. I don't know why these package versions accumulate over time (and I also deleted their files once when running out of disk space (which could also have contributed to the "checking for file conflicts" issue)). -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c9 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ma@suse.com Assignee|zypp-maintainers@forge.prov |trenn@suse.com |o.novell.com | Flags| |needinfo?(squan@web.de) --- Comment #9 from Michael Andres <ma@suse.com> --- @Stefan: Maybe you can attach your /var/log/zypp/history file. MAybe it can help us to reproduce the issue. @Thomas: As dracut maintainer, maybe you can figure out why /sbin/purge-kernels left so many kernel-source and package kernel-devel on Stefans system (comment#8). -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c10 Daniel Faust <dfaust@mailbox.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dfaust@mailbox.org --- Comment #10 from Daniel Faust <dfaust@mailbox.org> --- I had the same problem, I didn't stop the time but I gave up waiting only after quite a few minutes. Like Stefan suggested, I checked for old kernel versions and found a lot. I removed them and after that "checking for file conflicts" took less than a second. I wrote down all the packages I removed/installed to fix the problem: kernel-default, kernel-default-devel in 4.8.4-1.2-x86_64 rm 4.8.4-1.1-x86_64 rm 4.8.3-1.1-x86_64 rm 4.7.6-1.1-x86_64 rm 4.7.5-1.1-x86_64 rm 4.7.4-2.1-x86_64 rm 4.7.4-1.2-x86_64 rm 4.7.3-1.1-x86_64 rm 4.7.2-2.2-x86_64 rm 4.7.2-1.2-x86_64 kernel-devel, kernel-source in 4.8.4-1.2-noarch rm 4.8.4-1.1-noarch rm 4.8.3-1.1-noarch rm 4.7.6-1.1-noarch rm 4.7.5-1.2-noarch rm 4.7.5-1.1-noarch rm 4.7.4-2.1-noarch rm 4.7.4-1.2-noarch rm 4.7.3-1.1-noarch rm 4.7.2-2.2-noarch rm 4.7.2-1.3-noarch rm 4.7.2-1.2-noarch kernel-syms in 4.8.4-1.2-x86_64 rm 4.8.4-1.1-x86_64 rm 4.8.3-1.1-x86_64 rm 4.7.6-1.1-x86_64 rm 4.7.5-1.2-x86_64 rm 4.7.5-1.1-x86_64 rm 4.7.4-2.1-x86_64 rm 4.7.4-1.2-x86_64 rm 4.7.3-1.1-x86_64 rm 4.7.2-2.2-x86_64 rm 4.7.2-1.3-x86_64 virtualbox, virtualbox-qt in 5.1.6-1.18-x86_64 rm 5.1.4-3.1-x86_64 virtualbox-host-kmp-default in 5.1.6_k4.8.4_1.18-x86_64 rm 5.1.4_k4.7.2_1-1.3-x86_64 virtualbox-guest-kmp-default rm 5.1.4_k4.7.2_1-1.3-x86_64 For some reason I had virtualbox-guest-kmp-default installed, which I couldn't upgrade to the newest version because it had a conflict with virtualbox-host-kmp-default. I'll also attach the part of /var/log/zypp/history with today's messages. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c11 --- Comment #11 from Daniel Faust <dfaust@mailbox.org> --- Created attachment 700057 --> http://bugzilla.opensuse.org/attachment.cgi?id=700057&action=edit /var/log/zypp/history of removing old kernels and then running "zypper up" -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c12 Stefan Brüns <stefan.bruens@rwth-aachen.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |stefan.bruens@rwth-aachen.d | |e --- Comment #12 from Stefan Brüns <stefan.bruens@rwth-aachen.de> --- It seems to be sufficient to install either kernel-devel or kernel-default-devel while having another version thereof installed: $> LANG=C zypper se -si kernel Loading repository data... Reading installed packages... S | Name | Type | Version | Arch | Repository --+----------------------+---------+--------------+--------+------------------ i | kernel-default | package | 4.8.4-1.2 | x86_64 | (System Packages) i | kernel-default | package | 4.8.3-1.1 | x86_64 | (System Packages) i | kernel-default | package | 4.8.6-1.1 | x86_64 | repo-oss i | kernel-default-devel | package | 4.8.4-1.2 | x86_64 | (System Packages) i | kernel-default-devel | package | 4.8.3-1.1 | x86_64 | (System Packages) i | kernel-default-devel | package | 4.8.6-1.1 | x86_64 | repo-oss i | kernel-devel | package | 4.8.4-1.2 | noarch | (System Packages) i | kernel-devel | package | 4.8.4-1.1 | noarch | (System Packages) i | kernel-devel | package | 4.8.3-1.1 | noarch | (System Packages) i | kernel-devel | package | 4.8.6-1.1 | noarch | repo-oss i | kernel-firmware | package | 20161005-1.1 | noarch | repo-oss i | kernel-macros | package | 4.8.6-1.1 | noarch | repo-oss i | kernel-source | package | 4.8.4-1.2 | noarch | (System Packages) i | kernel-source | package | 4.8.4-1.1 | noarch | (System Packages) i | kernel-source | package | 4.8.3-1.1 | noarch | (System Packages) i | kernel-source | package | 4.8.6-1.1 | noarch | repo-oss i | nfs-kernel-server | package | 1.3.4-3.1 | x86_64 | repo-oss for p in kernel-default kernel-default-devel ; do zypper in --force $p ; done $> sudo cat /var/log/zypper.log | egrep 'Checking for file|took [0-9]{3,9}|candidates now|\.rpm from' 2016-11-06 22:31:05 <1> pebbles.site(20719) [zypper] Summary.cc(readPool):281 package update candidates: 0 2016-11-06 22:31:05 <1> pebbles.site(20719) [zypper] Summary.cc(readPool):281 product update candidates: 0 2016-11-06 22:31:07 <1> pebbles.site(20719) [zypp++] MediaSetAccess.cc(provide):203 Going to try to provide file ./suse/x86_64/kernel-default-4.8.6-1.1.x86_64.rpm from media number 1 2016-11-06 22:31:54 <1> pebbles.site(20719) [zypp++] MediaSetAccess.cc(releaseFile):85 Going to release file ./suse/x86_64/kernel-default-4.8.6-1.1.x86_64.rpm from media number 1 2016-11-06 22:31:54 <1> pebbles.site(20719) [zypp] TargetImpl.commitFindFileConflicts.cc(commitFindFileConflicts):127 Checking for file conflicts in 1 new packages... 2016-11-06 22:31:55 <1> pebbles.site(20719) [libsolv++] PoolImpl.cc(logSat):119 filemap creation took 175 ms 2016-11-06 22:31:55 <1> pebbles.site(20719) [libsolv++] PoolImpl.cc(logSat):119 alias processing took 147 ms 2016-11-06 22:31:55 <1> pebbles.site(20719) [libsolv++] PoolImpl.cc(logSat):119 raw candidates: 15615, pruning 2016-11-06 22:31:55 <1> pebbles.site(20719) [libsolv++] PoolImpl.cc(logSat):119 candidates now: 26 2016-11-06 22:31:55 <1> pebbles.site(20719) [libsolv++] PoolImpl.cc(logSat):119 file conflict detection took 398 ms 2016-11-06 22:33:26 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 addfileprovides took 113 ms 2016-11-06 22:33:26 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 pkg rule creation took 139 ms 2016-11-06 22:33:26 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 solver_solve took 171 ms 2016-11-06 22:33:27 <1> pebbles.site(3976) [zypper] Summary.cc(readPool):281 package update candidates: 0 2016-11-06 22:33:27 <1> pebbles.site(3976) [zypper] Summary.cc(readPool):281 product update candidates: 0 2016-11-06 22:33:30 <1> pebbles.site(3976) [zypp++] MediaSetAccess.cc(provide):203 Going to try to provide file ./suse/x86_64/kernel-default-devel-4.8.6-1.1.x86_64.rpm from media number 1 2016-11-06 22:33:33 <1> pebbles.site(3976) [zypp++] MediaSetAccess.cc(releaseFile):85 Going to release file ./suse/x86_64/kernel-default-devel-4.8.6-1.1.x86_64.rpm from media number 1 2016-11-06 22:33:33 <1> pebbles.site(3976) [zypp] TargetImpl.commitFindFileConflicts.cc(commitFindFileConflicts):127 Checking for file conflicts in 1 new packages... 2016-11-06 22:33:34 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 filemap creation took 111 ms 2016-11-06 22:33:34 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 alias processing took 165 ms 2016-11-06 22:33:34 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 raw candidates: 51391, pruning 2016-11-06 22:33:34 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 candidates now: 12964 2016-11-06 22:33:56 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 file info fetching took 22310 ms 2016-11-06 22:33:56 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 file conflict detection took 22594 ms So after the "pruning" step (whatever that is), there are a lot of candidates left, each triggering a call to rpm_iterate_filelist. The calls to rpm_iterate_filelist are the actually expensive part. Maybe there should be an rpm_iterate_dirlist there, fetching directory names is significantly less expensive than fetching info for all files (assuming there are far less directories than files), and file conflicts can only happen if the sets of directories have a non-empty intersection. Another point that may be relevant here, at least the kernel-default-devel packages actually *have* a conflicting file - /usr/src/linux-obj/x86_64/default is a symlink to ./default, which gets modified in the postinstall scriptlet. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c13 Peter Sütterlin <P.Suetterlin@royac.iac.es> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |P.Suetterlin@royac.iac.es --- Comment #13 from Peter Sütterlin <P.Suetterlin@royac.iac.es> --- Just wanted to add: - I'm having the same issue with Tumbleweed (just spent 4min on a 3.5GHz i7) - I only do have two sets of kernel source/devel installed (obviously three after an zypper up, until the next boot when the oldest is purged) - It just happened today with a zypper up (which indeed did install new kernel), but not for quite some updates that also installed new kernels. - grepping for the time for conflict check in zypper.log shows the last time it happened was last year November 17. In case it matters: This were the packages installed during both updates: bbswitch bbswitch-kmp-default bumblebee kernel-devel kernel-macros kernel-source kernel-syms openSUSE-release openSUSE-release-ftp plasma5-defaults-openSUSE plasma5-theme-openSUSE plasma5-workspace-branding-openSUSE sddm-theme-openSUSE subsurface VirtualGL VirtualGL-32bit -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 Thomas Renninger <trenn@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |daniel.molkentin@suse.com -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c22 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(mls@suse.com) --- Comment #22 from Michael Andres <ma@suse.com> --- @mls: how to speed this up? -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 - - <suse.pasi@xoxy.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |suse.pasi@xoxy.net -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 Vadim Krevs <vkrevs@yahoo.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |vkrevs@yahoo.com -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c29 --- Comment #29 from Vadim Krevs <vkrevs@yahoo.com> --- Same issue with a brand spanking new tumbleweed install on an SSD. First ran "zypper -v dup -d", then "zypper -v dup" and the latter hangs. "Workaround" - delete everything under /var/cache/zypp and rerun with --download-as-needed - this seems to skip the file conflict check. Kinda defeats the whole point of downloading in advance. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c30 --- Comment #30 from Vadim Krevs <vkrevs@yahoo.com> --- And not an issue on Leap 42.3 with multiple kernel-source packages. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c31 Michael Hanscho <reset11@gmx.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |reset11@gmx.net --- Comment #31 from Michael Hanscho <reset11@gmx.net> --- Heavily delayed checking for file conflicts found also on my machine. If needed I could provide a zypper debug logfile for an update - updating to openSUSE-release-ftp-20171125-1.2.x86_64. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c32 --- Comment #32 from Michael Andres <ma@suse.com> --- (In reply to Michael Hanscho from comment #31)
..I could provide a zypper debug logfile. Yes, maybe it reveals some detail that helps us to reproduce it.
-- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c33 --- Comment #33 from Michael Hanscho <reset11@gmx.net> --- Created attachment 750248 --> http://bugzilla.opensuse.org/attachment.cgi?id=750248&action=edit zypper.log zypper.log emptied - so only logging from the following zypper command rm -rf /var/cache/zypp export ZYPP_LIBSAT_FULLLOG=1 export ZYPP_FULLOG=1 zypper -v dup --no-allow-vendor-change openSUSE Tumbleweed 20171123-0 -> 20171125-0 -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 Michiel Janssens <michiel@nexigon.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |michiel@nexigon.net -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 Andrei Borzenkov <arvidjaar@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |arvidjaar@gmail.com -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c34 Freek de Kruijf <freek@opensuse.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |freek@opensuse.org --- Comment #34 from Freek de Kruijf <freek@opensuse.org> --- My solution is to zypper rm all versions of kernel-devel and kernel-source and "zypper in kernel-devel kernel-source". This went like a charm. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c35 --- Comment #35 from Michiel Janssens <michiel@nexigon.net> --- Solution from Freek in comment #34 works for me too. Time needed checking files was 5-10 minuten, now 1-2 seconds. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c36 --- Comment #36 from H. Hansen <cyberbeat@gmx.de> --- I don't have kernel-devel and kernel-source installed, and checking still takes more than a minute. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c37 Qi Fan <fanqi1234@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fanqi1234@gmail.com --- Comment #37 from Qi Fan <fanqi1234@gmail.com> --- Tumbleweed upgraded from 42.3. Additional repo: skypeforlinux Each time I do zypper dup I get stuck at "Checking for file conflicts" for several minutes. During the really long "Checking for file conflicts" step, System Load Viewer shows 100% CPU on one core, and the disk activity LED is OFF during most (>99%) of the waiting time. So I do think something is running more loops than necessary. Will information from "valgrind --tool=callgrind" help if I run it next time I zypper dup? -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c38 --- Comment #38 from Stefan Brüns <stefan.bruens@rwth-aachen.de> --- (In reply to Michael Andres from comment #26)
$ zypper se -s kernel-default-devel S | Name | Type | Version | Arch | Repository ---+----------------------+---------+-------------+--------+------------------ v | kernel-default-devel | package | 4.4.85-22.1 | x86_64 | update-oss (42.3) i+ | kernel-default-devel | package | 4.4.79-19.1 | x86_64 | update-oss (42.3) i+ | kernel-default-devel | package | 4.4.79-4.2 | x86_64 | update-oss (42.3) i+ | kernel-default-devel | package | 4.4.76-1.1 | x86_64 | repo-oss (42.3)
$ time zypper in kernel-default-devel-4.4.85-22.1 real 0m13.329s user 0m3.534s sys 0m1.048s
[zypper] main.cc(main):75 ===== '/Local/ma/zypp/BUILD/zypper/src/zypper' 'in' 'kernel-default-devel-4.4.85-22.1' ===== [libsolv++] PoolImpl.cc(logSat):119 file conflict detection took 328 ms
Similar on my other machines (Leap/TW). I don't experience these kind of delays. Something is missing...
What happens if you reinstall (zypper in --force ..) one of the already installed kernel-default-devel packages? -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c39 --- Comment #39 from Stefan Brüns <stefan.bruens@rwth-aachen.de> --- (In reply to Qi Fan from comment #37)
Will information from "valgrind --tool=callgrind" help if I run it next time I zypper dup?
It spends most of the time in rpm_iterate_filelist, as already said. rpm_iterate_filelist itself could be speed up by a factor of ~2 if it were better optimized (it does e.g. multiple mallocs per entry, where one would suffice), but this helps only a little. The big problem still is the O(n^2) behaviour (for each candidate, check each file in each package if it conflicts). -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c40 --- Comment #40 from Michael Andres <ma@suse.com> --- (In reply to Stefan Brüns from comment #38)
What happens if you reinstall (zypper in --force ..) one of the already installed kernel-default-devel packages?
The largest amount of time seems to be spent reading data from disk:
1.) real 0m41.645s 2.) real 0m5.820s sync; echo 3 > /proc/sys/vm/drop_caches 3.) real 0m53.244s 4.) real 0m6.750s
-- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c41 --- Comment #41 from Qi Fan <fanqi1234@gmail.com> --- (In reply to Stefan Brüns from comment #39)
(In reply to Qi Fan from comment #37)
Will information from "valgrind --tool=callgrind" help if I run it next time I zypper dup?
It spends most of the time in rpm_iterate_filelist, as already said.
rpm_iterate_filelist itself could be speed up by a factor of ~2 if it were better optimized (it does e.g. multiple mallocs per entry, where one would suffice), but this helps only a little.
The big problem still is the O(n^2) behaviour (for each candidate, check each file in each package if it conflicts).
I think you are right. The comment like this: https://github.com/openSUSE/libsolv/blob/bef46d75f6d2537060114eea0f3a892e958... does seem to have assumed that n is pretty small. Some O(n) algorithm should be used instead. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c48 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |cy6erGn0m@gmail.com --- Comment #48 from Michael Andres <ma@suse.com> --- *** Bug 1058242 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c53 --- Comment #53 from Adam Spiers <aspiers@suse.com> --- I still see this check being extremely slow on Leap 15.0 - should I reopen this or file a separate bug? -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c54 --- Comment #54 from H. Hansen <cyberbeat@gmx.de> --- I also think its still too slow (you need to test this on a normal HD (no ssd)) -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c56 --- Comment #56 from Harald Koenig <koenig@linux.de> --- (In reply to H. Hansen from comment #54)
(you need to test this on a normal HD (no ssd))
at least for my original issue that's not true, for me it was 100% cpu bound... -- You are receiving this mail because: You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠novell.com