[Bug 1061208] New: zypper segfaults upon refresh
http://bugzilla.suse.com/show_bug.cgi?id=1061208 Bug ID: 1061208 Summary: zypper segfaults upon refresh Classification: openSUSE Product: openSUSE Distribution Version: Leap 42.3 Hardware: Other OS: Other Status: NEW Severity: Normal Priority: P5 - None Component: libzypp Assignee: zypp-maintainers@forge.provo.novell.com Reporter: richard@nod.at QA Contact: qa-bugs@suse.de Found By: --- Blocker: --- Created attachment 742668 --> http://bugzilla.suse.com/attachment.cgi?id=742668&action=edit zypper core After adding the mozilla repo zypper crashes every time upon refresh. e.g. zypper> ref Repository 'go' is up to date. Repository 'libdvdcss repository' is up to date. Repository 'Packman Repository' is up to date. Retrieving repository 'mozilla' metadata -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------[/]Segmentation fault (core dumped) Core file is attached. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c1 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |richard@nod.at Flags| |needinfo?(richard@nod.at) --- Comment #1 from Michael Andres <ma@suse.com> --- Please attach the zypper logfile /var/log/zypper.log (or an older and compressed /var/log/zypper.log-YYYYMMDD.xz) that shows the reported behavior. To see the execution dates and zypper commands included in a logfile you can install the zypper-log package (available since zypper-1.6.11) and execute: zypper-log To extract the log of a specific command use zypper-log pid See the manual page zypper-log(8) for details on how to read older (rotated) zypper-log files. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c2 Richard Weinberger <richard@nod.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(richard@nod.at) | --- Comment #2 from Richard Weinberger <richard@nod.at> --- Created attachment 742705 --> http://bugzilla.suse.com/attachment.cgi?id=742705&action=edit zypper log file -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c3 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(richard@nod.at) --- Comment #3 from Michael Andres <ma@suse.com> --- @Richard: If the crash is reproducible, please execute (as root)
ZYPP_LOGFILE=/tmp/zypp.1061208.log ZYPP_MEDIA_CURL_DEBUG=2 zypper ref -f mozilla (or any other zypper command that crashes) and attach the logfile /tmp/zypp.1061208.log.
As the crash happens inside libcurl (libssl/libcrypto) it would be interesting to now if
curl https://download.opensuse.org/repositories/mozilla/openSUSE_Leap_42.3/repoda... succeeds?
-- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c4 Richard Weinberger <richard@nod.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(richard@nod.at) | --- Comment #4 from Richard Weinberger <richard@nod.at> --- (In reply to Michael Andres from comment #3)
@Richard: If the crash is reproducible, please execute (as root)
ZYPP_LOGFILE=/tmp/zypp.1061208.log ZYPP_MEDIA_CURL_DEBUG=2 zypper ref -f mozilla (or any other zypper command that crashes) and attach the logfile /tmp/zypp.1061208.log.
As the crash happens inside libcurl (libssl/libcrypto) it would be interesting to now if
curl https://download.opensuse.org/repositories/mozilla/openSUSE_Leap_42.3/repoda... succeeds?
Yes, this succeeds. I tried that already yesterday before filing this bug against zypper. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c5 --- Comment #5 from Michael Andres <ma@suse.com> --- (In reply to Michael Andres from comment #3)
@Richard: If the crash is reproducible, please execute (as root)
ZYPP_LOGFILE=/tmp/zypp.1061208.log ZYPP_MEDIA_CURL_DEBUG=2 zypper ref -f mozilla (or any other zypper command that crashes) and attach the logfile /tmp/zypp.1061208.log.
Are you able to such a log? As I can't reproduce this, a log with CURL debug output could reveal some ore details. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c6 --- Comment #6 from Richard Weinberger <richard@nod.at> --- (In reply to Michael Andres from comment #5)
(In reply to Michael Andres from comment #3)
@Richard: If the crash is reproducible, please execute (as root)
ZYPP_LOGFILE=/tmp/zypp.1061208.log ZYPP_MEDIA_CURL_DEBUG=2 zypper ref -f mozilla (or any other zypper command that crashes) and attach the logfile /tmp/zypp.1061208.log.
Are you able to such a log? As I can't reproduce this, a log with CURL debug output could reveal some ore details.
Sorry, I missed that question. Log is attached. I can reproduce, also after reboots (thought about a memory corruption at first). -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c7 --- Comment #7 from Richard Weinberger <richard@nod.at> --- Created attachment 742726 --> http://bugzilla.suse.com/attachment.cgi?id=742726&action=edit zypper debug log -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c8 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |meissner@suse.com Flags| |needinfo?(meissner@suse.com | |) --- Comment #8 from Michael Andres <ma@suse.com> ---
MediaCurl.cc(doGetDoesFileExist):1195 URL: https://download.opensuse.org/repositories/mozilla/openSUSE_Leap_42.3/repoda... MediaCurl.cc(log_curl):111 * Trying 195.135.221.134... MediaCurl.cc(log_curl):111 * TCP_NODELAY set MediaCurl.cc(log_curl):111 * Connected to download.opensuse.org (195.135.221.134) port 443 (#0) ZYppFactory.cc(sigsegvHandler):55 Error: signal 11 ZYppFactory.cc(sigsegvHandler):55 [bt]: (0) /lib64/libc.so.6 : +0x90e5a [0x7f873f0c6e5a] ZYppFactory.cc(sigsegvHandler):55 [bt]: (1) /lib64/libcrypto.so.1.0.0 : +0x120869 [0x7f873e067869] ZYppFactory.cc(sigsegvHandler):55 [bt]: (2) /lib64/libcrypto.so.1.0.0 : lh_insert+0x42 [0x7f873e067b42] ZYppFactory.cc(sigsegvHandler):55 [bt]: (3) /lib64/libcrypto.so.1.0.0 : OBJ_NAME_add+0x6f [0x7f873dfb7eef] ZYppFactory.cc(sigsegvHandler):55 [bt]: (4) /usr/lib64/libssl.so.1.1 : +0x2beb4 [0x7f873b3c7eb4] ZYppFactory.cc(sigsegvHandler):55 [bt]: (5) /lib64/libpthread.so.0 : +0x6c13 [0x7f873beb4c13] ZYppFactory.cc(sigsegvHandler):55 [bt]: (6) /usr/lib64/libcrypto.so.1.1 : CRYPTO_THREAD_run_once+0x9 [0x7f873b0c6c49] ZYppFactory.cc(sigsegvHandler):55 [bt]: (7) /usr/lib64/libssl.so.1.1 : OPENSSL_init_ssl+0x73 [0x7f873b3c7fc3] ZYppFactory.cc(sigsegvHandler):55 [bt]: (8) /usr/lib64/libssl.so.1.1 : SSL_CTX_new+0x20 [0x7f873b3cb040] ZYppFactory.cc(sigsegvHandler):55 [bt]: (9) /usr/lib64/libcurl.so.4 : +0x5c531 [0x7f873e83f531] ZYppFactory.cc(sigsegvHandler):55 [bt]: (10) /usr/lib64/libcurl.so.4 : +0x5f833 [0x7f873e842833] ZYppFactory.cc(sigsegvHandler):55 [bt]: (11) /usr/lib64/libcurl.so.4 : +0x60bcb [0x7f873e843bcb] ZYppFactory.cc(sigsegvHandler):55 [bt]: (12) /usr/lib64/libcurl.so.4 : +0x14562 [0x7f873e7f7562] ZYppFactory.cc(sigsegvHandler):55 [bt]: (13) /usr/lib64/libcurl.so.4 : +0x26341 [0x7f873e809341] ZYppFactory.cc(sigsegvHandler):55 [bt]: (14) /usr/lib64/libcurl.so.4 : +0x3a53e [0x7f873e81d53e] ZYppFactory.cc(sigsegvHandler):55 [bt]: (15) /usr/lib64/libcurl.so.4 : curl_multi_perform+0x81 [0x7f873e81e1c1] ZYppFactory.cc(sigsegvHandler):55 [bt]: (16) /usr/lib64/libcurl.so.4 : curl_easy_perform+0x102 [0x7f873e814ca2] ZYppFactory.cc(sigsegvHandler):55 [bt]: (17) /usr/lib64/libzypp.so.1600 : zypp::media::MediaCurl::doGetDoesFileExist(zypp::filesystem::Pathname const&) ZYppFactory.cc(sigsegvHandler):55 [bt]: (18) /usr/lib64/libzypp.so.1600 : zypp::media::MediaCurl::getDoesFileExist(zypp::filesystem::Pathname const&) c ZYppFactory.cc(sigsegvHandler):55 [bt]: (19) /usr/lib64/libzypp.so.1600 : zypp::media::MediaHandler::doesFileExist(zypp::filesystem::Pathname const&) c ZYppFactory.cc(sigsegvHandler):55 [bt]: (20) /usr/lib64/libzypp.so.1600 : zypp::media::MediaAccess::doesFileExist(zypp::filesystem::Pathname const&) co ZYppFactory.cc(sigsegvHandler):55 [bt]: (21) /usr/lib64/libzypp.so.1600 : zypp::media::MediaManager::doesFileExist(unsigned int, zypp::filesystem::Path ZYppFactory.cc(sigsegvHandler):55 [bt]: (22) /usr/lib64/libzypp.so.1600 : +0x29d61e [0x7f874041661e] ZYppFactory.cc(sigsegvHandler):55 [bt]: (23) /usr/lib64/libzypp.so.1600 : zypp::MediaSetAccess::provide(boost::function<void (unsigned int, zypp::files ZYppFactory.cc(sigsegvHandler):55 [bt]: (24) /usr/lib64/libzypp.so.1600 : zypp::MediaSetAccess::doesFileExist(zypp::filesystem::Pathname const&, unsign ZYppFactory.cc(sigsegvHandler):55 [bt]: (25) /usr/lib64/libzypp.so.1600 : zypp::RepoManager::Impl::probe(zypp::Url const&, zypp::filesystem::Pathname c ZYppFactory.cc(sigsegvHandler):55 [bt]: (26) /usr/lib64/libzypp.so.1600 : zypp::RepoManager::Impl::refreshMetadata(zypp::RepoInfo const&, zypp::RepoMan ZYppFactory.cc(sigsegvHandler):55 [bt]: (27) zypper() [0x4abf53] ZYppFactory.cc(sigsegvHandler):55 [bt]: (28) zypper : refresh_repo(Zypper&, zypp::RepoInfo const&)+0x247 [0x4af607] ZYppFactory.cc(sigsegvHandler):55 [bt]: (29) zypper : refresh_repos(Zypper&)+0x6d3 [0x4aff03] ZYppFactory.cc(sigsegvHandler):55 [bt]: (30) zypper : Zypper::doCommand()+0x8edb [0x475cab] ZYppFactory.cc(sigsegvHandler):55 [bt]: (31) zypper : Zypper::safeDoCommand()+0x7d [0x48308d] ZYppFactory.cc(sigsegvHandler):55 [bt]: (32) zypper : Zypper::main(int, char**)+0x44 [0x43e894] ZYppFactory.cc(sigsegvHandler):55 [bt]: (33) zypper : main+0x3ad [0x43dabd] ZYppFactory.cc(sigsegvHandler):55 [bt]: (34) /lib64/libc.so.6 : __libc_start_main+0xf5 [0x7f873f0566e5] ZYppFactory.cc(sigsegvHandler):55 [bt]: (35) zypper : _start+0x29 [0x43f8e9]
@Marcus It seems to crash pretty early during OPENSSL_init_ssl. Not even CApath etc. is reported. Any idea what could cause this? -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c9 Marcus Meissner <meissner@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(meissner@suse.com | |) | --- Comment #9 from Marcus Meissner <meissner@suse.com> --- i think seperate versions of openssl are mixed together. rpm -qif /lib64/libcrypto.so.1.0.0 rpm -qif /usr/lib64/libssl.so.1.1 rpm -qif /usr/lib64/libcrypto.so.1.1 (There should only be 1 libcrypto.so.1 loaded, so there are two installed ... one is from Leap, the other one is from somewhere else) -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c10 --- Comment #10 from Richard Weinberger <richard@nod.at> --- (In reply to Marcus Meissner from comment #9)
i think seperate versions of openssl are mixed together.
rpm -qif /lib64/libcrypto.so.1.0.0 rpm -qif /usr/lib64/libssl.so.1.1 rpm -qif /usr/lib64/libcrypto.so.1.1
(There should only be 1 libcrypto.so.1 loaded, so there are two installed ... one is from Leap, the other one is from somewhere else)
You are right, one version came from repositories/security:/tls/openSUSE_Leap_42.3/. This repo is present in my zypper config since I needed a package from there. dup --from security_tls fixed the issue. Feel free to close as invalid. ;-\ -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c11 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |INVALID --- Comment #11 from Michael Andres <ma@suse.com> --- . -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |1146027 -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c15 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(maninredd@yahoo.c | |om) --- Comment #15 from Michael Andres <ma@suse.com> --- Patrick: Thanks. Your issue is different than the one reported here, but it's something that worries us most ATM. It's seems to be a duplicate of bug#1146027, but for now I'd like to continue here. Let's stay with your server: (In reply to Patrick Finie from comment #13)
Now to a different and third machine, My server.
instead of doing a update before changing over the repositories i did this method. # cp -Rv /etc/zypp/repos.d /etc/zypp/repos.d.Old ># sed -i 's/15.0/15.1/g' /etc/zypp/repos.d/* > # zypper --gpg-auto-import-keys ref > zypper dup
dealt with all the package that needed to be changed over and rebooted.
went to use zypper and segfault.
I'm most interested in the 'zypper dup' before you rebooted: - 'zypper dup' creates a resolver testcase at /var/log/updateTestcase-YYYY-MM-DD-hh_mm_ss (the date the dup command was executed). Please pack the output directory and attach it to THIS bugreport. (In case the testcase exceeds the 10MB limit for bugzilla uploads, you can use 'split' to divide the file into smaller pieces) - If the zypper.log showing the refresh command and the dup are still available (maybe backuped) please attach them as well. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c19 --- Comment #19 from Michael Andres <ma@suse.com> --- (In reply to Patrick Finie from comment #18)
In this instance the version of libzypp is a higher number in 15.0 than the 15.1 which may be where the zypper dup issue stems.
As I tried to explain: 'dup' will not look at the installed packages version. The 'dup' job requires an installed libzypp to be replaced by a version provided by the enabled repos. If this can not be done, a conflict is raised. (You want the packages to come from the new distro, even if the distro decided to downgrade some package) If your repos provided lp15.1 and no lp15.0 packages, a 15.0 libzypp should not have stayed in the system (unless no lp15.1 provided a libzypp). As it nevertheless happened, the update testcase could have revealed why. It contains enough information about the repos and systems content to be able to replay 'dup's decisions. ! As there is a small chance that the old libzypp was seen due to a broken rpm database index, you should run 'rpm --rebuilddb' (as root). Just to make sure your database is ok. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c20 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|REOPENED |RESOLVED Resolution|--- |INVALID --- Comment #20 from Michael Andres <ma@suse.com> --- Patrick: The SEGV backtrace visible in your logs is most probably caused bug#1146027. This issue will be fixed in libzypp-17.14.1. I'll close the bug again, restoring the original INVALID resolution. If you again experience an incomplete dup, please open a bug for it and attach the zypper.log and the updateTestcase (see comment #15). Thanks. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(maninredd@yahoo.c | |om) | -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c21 --- Comment #21 from Patrick Finie <maninredd@yahoo.com> --- Created attachment 823558 --> http://bugzilla.suse.com/attachment.cgi?id=823558&action=edit updatetestcase WSL (In reply to Michael Andres from comment #20)
Patrick: The SEGV backtrace visible in your logs is most probably caused bug#1146027. This issue will be fixed in libzypp-17.14.1. I'll close the bug again, restoring the original INVALID resolution.
If you again experience an incomplete dup, please open a bug for it and attach the zypper.log and the updateTestcase (see comment #15). Thanks.
I did a zypper dup on a WSL instance and ran into the same issue with 15.0 and 15.1. What is the best means of uploading the 25 megabyte updatetestcase directory? Seems the option is greyed out on most of the archive managers i use. *several hours later with a lot of yelling and swearing and preying to Geeko* Looks like it got it. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c22 --- Comment #22 from Patrick Finie <maninredd@yahoo.com> --- Created attachment 823559 --> http://bugzilla.suse.com/attachment.cgi?id=823559&action=edit updatetestcase WSL 2 here is the second one. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c23 --- Comment #23 from Patrick Finie <maninredd@yahoo.com> --- Created attachment 823560 --> http://bugzilla.suse.com/attachment.cgi?id=823560&action=edit updatetestcase WSL 3 and the third. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c24 Patrick Finie <maninredd@yahoo.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |REOPENED Resolution|INVALID |--- --- Comment #24 from Patrick Finie <maninredd@yahoo.com> --- Reopening. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c25 --- Comment #25 from Michael Andres <ma@suse.com> --- (In reply to Patrick Finie from comment #21)
updatetestcase directory? Seems the option is greyed out on most of the archive managers i use.
You mean some split-archive option? There's a generic 'split' command. If you have some `big.tar.bz2`, then
split -b 10M --additional-suffix=-big.tar.bz2 big.tar.bz2 will split it into 10MB parts named xaa-big.tar.bz2 xab-big.tar.bz2 xac-big.tar.bz2 ...
Later 'cat' can be used to restore the original file
cat x??-big.tar.bz2 >big.tar.bz2
-- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.suse.com/show_bug.cgi?id=1061208 http://bugzilla.suse.com/show_bug.cgi?id=1061208#c26 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(maninredd@yahoo.c | |om) --- Comment #26 from Michael Andres <ma@suse.com> --- May it be that updateTestcase-2019-11-07-01-46-39 is _not_ from the zypper dup, that brought the system into the inconsistent state? In the testcase I see 348 packages being installed, but just 28 are lp150. The rest is already lp151. libsolv-tools-0.7.5-lp150.7.1.x86_64 libzypp-17.12.0-lp150.2.13.1.x86_64 zypper-1.14.28-lp150.2.13.1.x86_64 The zypp stack is still lp150, but running the dup, the resolver would try to update them as expected:
!> install libsolv-tools-0.7.6-lp151.2.3.2.x86_64[repo-update] !> install libzypp-17.15.0-lp151.2.3.2.x86_64[repo-update] !> install zypper-1.14.30-lp151.2.3.1.x86_64[repo-update]
If you install `zypper-log`, the command will print you a summary of the commands found in a zypper.log file (or a rotated one, see the man page). This may help to figure out what was executed before 2019-11-07-01-46-39. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1061208 https://bugzilla.suse.com/show_bug.cgi?id=1061208#c28 Patrick Finie <maninredd@yahoo.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|REOPENED |RESOLVED Resolution|--- |NORESPONSE --- Comment #28 from Patrick Finie <maninredd@yahoo.com> --- There are now better workaround out. Going to 15.2 doesnt produce this issue, if there is a zypper issue then dnf is a backup. Detailed a procedure that eliminates most of the risk involved with this old bug and thus it is being closed. Never was resolved but not enough information was produced and by the time responses where made the systems where already brute forced to upgrade. -- You are receiving this mail because: You are on the CC list for the bug.
participants (2)
-
bugzilla_noreply＠novell.com -
bugzilla_noreply＠suse.com