http://bugzilla.opensuse.org/show_bug.cgi?id=953130 Dominique Leuenberger changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dimstar@opensuse.org Component|Basesystem |libzypp Assignee|bnc-team-screening@forge.pr |zypp-maintainers@forge.prov |ovo.novell.com |o.novell.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c2 --- Comment #2 from Harald Koenig --- Created attachment 654254 --> http://bugzilla.opensuse.org/attachment.cgi?id=654254&action=edit zypper.log -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 H. Hansen changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |cyberbeat@gmx.de -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c8 Stefan Quandt changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |squan@web.de --- Comment #8 from Stefan Quandt --- For me "Checking for file conflicts" just stalled at 57% for >5 minutes without finishing, hence I needed to kill zypper. I could resolve this situation by deleting *lots* of different installed versions of package kernel-source and package kernel-devel. I don't know why these package versions accumulate over time (and I also deleted their files once when running out of disk space (which could also have contributed to the "checking for file conflicts" issue)). -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c10 Daniel Faust changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dfaust@mailbox.org --- Comment #10 from Daniel Faust --- I had the same problem, I didn't stop the time but I gave up waiting only after quite a few minutes. Like Stefan suggested, I checked for old kernel versions and found a lot. I removed them and after that "checking for file conflicts" took less than a second. I wrote down all the packages I removed/installed to fix the problem: kernel-default, kernel-default-devel in 4.8.4-1.2-x86_64 rm 4.8.4-1.1-x86_64 rm 4.8.3-1.1-x86_64 rm 4.7.6-1.1-x86_64 rm 4.7.5-1.1-x86_64 rm 4.7.4-2.1-x86_64 rm 4.7.4-1.2-x86_64 rm 4.7.3-1.1-x86_64 rm 4.7.2-2.2-x86_64 rm 4.7.2-1.2-x86_64 kernel-devel, kernel-source in 4.8.4-1.2-noarch rm 4.8.4-1.1-noarch rm 4.8.3-1.1-noarch rm 4.7.6-1.1-noarch rm 4.7.5-1.2-noarch rm 4.7.5-1.1-noarch rm 4.7.4-2.1-noarch rm 4.7.4-1.2-noarch rm 4.7.3-1.1-noarch rm 4.7.2-2.2-noarch rm 4.7.2-1.3-noarch rm 4.7.2-1.2-noarch kernel-syms in 4.8.4-1.2-x86_64 rm 4.8.4-1.1-x86_64 rm 4.8.3-1.1-x86_64 rm 4.7.6-1.1-x86_64 rm 4.7.5-1.2-x86_64 rm 4.7.5-1.1-x86_64 rm 4.7.4-2.1-x86_64 rm 4.7.4-1.2-x86_64 rm 4.7.3-1.1-x86_64 rm 4.7.2-2.2-x86_64 rm 4.7.2-1.3-x86_64 virtualbox, virtualbox-qt in 5.1.6-1.18-x86_64 rm 5.1.4-3.1-x86_64 virtualbox-host-kmp-default in 5.1.6_k4.8.4_1.18-x86_64 rm 5.1.4_k4.7.2_1-1.3-x86_64 virtualbox-guest-kmp-default rm 5.1.4_k4.7.2_1-1.3-x86_64 For some reason I had virtualbox-guest-kmp-default installed, which I couldn't upgrade to the newest version because it had a conflict with virtualbox-host-kmp-default. I'll also attach the part of /var/log/zypp/history with today's messages. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c12 Stefan Brüns changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |stefan.bruens@rwth-aachen.d | |e --- Comment #12 from Stefan Brüns --- It seems to be sufficient to install either kernel-devel or kernel-default-devel while having another version thereof installed: $> LANG=C zypper se -si kernel Loading repository data... Reading installed packages... S | Name | Type | Version | Arch | Repository --+----------------------+---------+--------------+--------+------------------ i | kernel-default | package | 4.8.4-1.2 | x86_64 | (System Packages) i | kernel-default | package | 4.8.3-1.1 | x86_64 | (System Packages) i | kernel-default | package | 4.8.6-1.1 | x86_64 | repo-oss i | kernel-default-devel | package | 4.8.4-1.2 | x86_64 | (System Packages) i | kernel-default-devel | package | 4.8.3-1.1 | x86_64 | (System Packages) i | kernel-default-devel | package | 4.8.6-1.1 | x86_64 | repo-oss i | kernel-devel | package | 4.8.4-1.2 | noarch | (System Packages) i | kernel-devel | package | 4.8.4-1.1 | noarch | (System Packages) i | kernel-devel | package | 4.8.3-1.1 | noarch | (System Packages) i | kernel-devel | package | 4.8.6-1.1 | noarch | repo-oss i | kernel-firmware | package | 20161005-1.1 | noarch | repo-oss i | kernel-macros | package | 4.8.6-1.1 | noarch | repo-oss i | kernel-source | package | 4.8.4-1.2 | noarch | (System Packages) i | kernel-source | package | 4.8.4-1.1 | noarch | (System Packages) i | kernel-source | package | 4.8.3-1.1 | noarch | (System Packages) i | kernel-source | package | 4.8.6-1.1 | noarch | repo-oss i | nfs-kernel-server | package | 1.3.4-3.1 | x86_64 | repo-oss for p in kernel-default kernel-default-devel ; do zypper in --force $p ; done $> sudo cat /var/log/zypper.log | egrep 'Checking for file|took [0-9]{3,9}|candidates now|\.rpm from' 2016-11-06 22:31:05 <1> pebbles.site(20719) [zypper] Summary.cc(readPool):281 package update candidates: 0 2016-11-06 22:31:05 <1> pebbles.site(20719) [zypper] Summary.cc(readPool):281 product update candidates: 0 2016-11-06 22:31:07 <1> pebbles.site(20719) [zypp++] MediaSetAccess.cc(provide):203 Going to try to provide file ./suse/x86_64/kernel-default-4.8.6-1.1.x86_64.rpm from media number 1 2016-11-06 22:31:54 <1> pebbles.site(20719) [zypp++] MediaSetAccess.cc(releaseFile):85 Going to release file ./suse/x86_64/kernel-default-4.8.6-1.1.x86_64.rpm from media number 1 2016-11-06 22:31:54 <1> pebbles.site(20719) [zypp] TargetImpl.commitFindFileConflicts.cc(commitFindFileConflicts):127 Checking for file conflicts in 1 new packages... 2016-11-06 22:31:55 <1> pebbles.site(20719) [libsolv++] PoolImpl.cc(logSat):119 filemap creation took 175 ms 2016-11-06 22:31:55 <1> pebbles.site(20719) [libsolv++] PoolImpl.cc(logSat):119 alias processing took 147 ms 2016-11-06 22:31:55 <1> pebbles.site(20719) [libsolv++] PoolImpl.cc(logSat):119 raw candidates: 15615, pruning 2016-11-06 22:31:55 <1> pebbles.site(20719) [libsolv++] PoolImpl.cc(logSat):119 candidates now: 26 2016-11-06 22:31:55 <1> pebbles.site(20719) [libsolv++] PoolImpl.cc(logSat):119 file conflict detection took 398 ms 2016-11-06 22:33:26 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 addfileprovides took 113 ms 2016-11-06 22:33:26 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 pkg rule creation took 139 ms 2016-11-06 22:33:26 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 solver_solve took 171 ms 2016-11-06 22:33:27 <1> pebbles.site(3976) [zypper] Summary.cc(readPool):281 package update candidates: 0 2016-11-06 22:33:27 <1> pebbles.site(3976) [zypper] Summary.cc(readPool):281 product update candidates: 0 2016-11-06 22:33:30 <1> pebbles.site(3976) [zypp++] MediaSetAccess.cc(provide):203 Going to try to provide file ./suse/x86_64/kernel-default-devel-4.8.6-1.1.x86_64.rpm from media number 1 2016-11-06 22:33:33 <1> pebbles.site(3976) [zypp++] MediaSetAccess.cc(releaseFile):85 Going to release file ./suse/x86_64/kernel-default-devel-4.8.6-1.1.x86_64.rpm from media number 1 2016-11-06 22:33:33 <1> pebbles.site(3976) [zypp] TargetImpl.commitFindFileConflicts.cc(commitFindFileConflicts):127 Checking for file conflicts in 1 new packages... 2016-11-06 22:33:34 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 filemap creation took 111 ms 2016-11-06 22:33:34 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 alias processing took 165 ms 2016-11-06 22:33:34 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 raw candidates: 51391, pruning 2016-11-06 22:33:34 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 candidates now: 12964 2016-11-06 22:33:56 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 file info fetching took 22310 ms 2016-11-06 22:33:56 <1> pebbles.site(3976) [libsolv++] PoolImpl.cc(logSat):119 file conflict detection took 22594 ms So after the "pruning" step (whatever that is), there are a lot of candidates left, each triggering a call to rpm_iterate_filelist. The calls to rpm_iterate_filelist are the actually expensive part. Maybe there should be an rpm_iterate_dirlist there, fetching directory names is significantly less expensive than fetching info for all files (assuming there are far less directories than files), and file conflicts can only happen if the sets of directories have a non-empty intersection. Another point that may be relevant here, at least the kernel-default-devel packages actually *have* a conflicting file - /usr/src/linux-obj/x86_64/default is a symlink to ./default, which gets modified in the postinstall scriptlet. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 Thomas Renninger changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |daniel.molkentin@suse.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 - - changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |suse.pasi@xoxy.net -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c29 --- Comment #29 from Vadim Krevs --- Same issue with a brand spanking new tumbleweed install on an SSD. First ran "zypper -v dup -d", then "zypper -v dup" and the latter hangs. "Workaround" - delete everything under /var/cache/zypp and rerun with --download-as-needed - this seems to skip the file conflict check. Kinda defeats the whole point of downloading in advance. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c31 Michael Hanscho changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |reset11@gmx.net --- Comment #31 from Michael Hanscho --- Heavily delayed checking for file conflicts found also on my machine. If needed I could provide a zypper debug logfile for an update - updating to openSUSE-release-ftp-20171125-1.2.x86_64. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c33 --- Comment #33 from Michael Hanscho --- Created attachment 750248 --> http://bugzilla.opensuse.org/attachment.cgi?id=750248&action=edit zypper.log zypper.log emptied - so only logging from the following zypper command rm -rf /var/cache/zypp export ZYPP_LIBSAT_FULLLOG=1 export ZYPP_FULLOG=1 zypper -v dup --no-allow-vendor-change openSUSE Tumbleweed 20171123-0 -> 20171125-0 -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 Andrei Borzenkov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |arvidjaar@gmail.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c35 --- Comment #35 from Michiel Janssens --- Solution from Freek in comment #34 works for me too. Time needed checking files was 5-10 minuten, now 1-2 seconds. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c37 Qi Fan changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fanqi1234@gmail.com --- Comment #37 from Qi Fan --- Tumbleweed upgraded from 42.3. Additional repo: skypeforlinux Each time I do zypper dup I get stuck at "Checking for file conflicts" for several minutes. During the really long "Checking for file conflicts" step, System Load Viewer shows 100% CPU on one core, and the disk activity LED is OFF during most (>99%) of the waiting time. So I do think something is running more loops than necessary. Will information from "valgrind --tool=callgrind" help if I run it next time I zypper dup? -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c39 --- Comment #39 from Stefan Brüns --- (In reply to Qi Fan from comment #37)

Will information from "valgrind --tool=callgrind" help if I run it next time I zypper dup?

It spends most of the time in rpm_iterate_filelist, as already said. rpm_iterate_filelist itself could be speed up by a factor of ~2 if it were better optimized (it does e.g. multiple mallocs per entry, where one would suffice), but this helps only a little. The big problem still is the O(n^2) behaviour (for each candidate, check each file in each package if it conflicts). -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c41 --- Comment #41 from Qi Fan --- (In reply to Stefan Brüns from comment #39)

(In reply to Qi Fan from comment #37)

...

Will information from "valgrind --tool=callgrind" help if I run it next time I zypper dup?

It spends most of the time in rpm_iterate_filelist, as already said.

rpm_iterate_filelist itself could be speed up by a factor of ~2 if it were better optimized (it does e.g. multiple mallocs per entry, where one would suffice), but this helps only a little.

The big problem still is the O(n^2) behaviour (for each candidate, check each file in each package if it conflicts).

I think you are right. The comment like this: https://github.com/openSUSE/libsolv/blob/bef46d75f6d2537060114eea0f3a892e958... does seem to have assumed that n is pretty small. Some O(n) algorithm should be used instead. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c53 --- Comment #53 from Adam Spiers --- I still see this check being extremely slow on Leap 15.0 - should I reopen this or file a separate bug? -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=953130 http://bugzilla.opensuse.org/show_bug.cgi?id=953130#c56 --- Comment #56 from Harald Koenig --- (In reply to H. Hansen from comment #54)

(you need to test this on a normal HD (no ssd))

at least for my original issue that's not true, for me it was 100% cpu bound... -- You are receiving this mail because: You are on the CC list for the bug.