[Bug 1230912] New: zypper hangs if no network connection
https://bugzilla.suse.com/show_bug.cgi?id=1230912 Bug ID: 1230912 Summary: zypper hangs if no network connection Classification: openSUSE Product: openSUSE Tumbleweed Version: Current Hardware: Other OS: Other Status: NEW Severity: Normal Priority: P5 - None Component: libzypp Assignee: zypp-maintainers@suse.de Reporter: fvogt@suse.com QA Contact: qa-bugs@suse.de Target Milestone: --- Found By: --- Blocker: --- Created attachment 877505 --> https://bugzilla.suse.com/attachment.cgi?id=877505&action=edit log of combustion and zypper.log Recently, the disk-encryption-tool CI started failing reproducibly because the image no longer boots within 5min. Some logging showed that it's stuck in transactional-update's selfupdate during the combustion phase in the initrd. I was able to collect zypper.log of such a hang. Last few lines: 2024-09-23 12:02:15 <1> localhost(840) [zypp::media] MediaHandler.cc(attach):654 Attached: https://download.opensuse.org/ attached; localRoot "/var/tmp/AP_0xMm8N8v" 2024-09-23 12:02:15 <1> localhost(840) [zypp::media++] MediaManager.cc(checkDesired):155 checkDesired(1): desired (report by zypp::media::NoVerifier) 2024-09-23 12:02:15 <1> localhost(840) [zypp::media++] MediaManager.cc(checkDesired):157 checkDesired(1): desired (cached) 2024-09-23 12:02:15 <1> localhost(840) [zypp::media++] MediaMultiCurl.cc(doGetFileCopy):1502 dest: /var/tmp/AP_0xMm8N8v/geoip 2024-09-23 12:02:15 <1> localhost(840) [zypp::media++] MediaMultiCurl.cc(doGetFileCopy):1503 temp: /var/tmp/AP_0xMm8N8v/geoip.new.zypp.p9Et4x 2024-09-23 12:02:15 <1> localhost(840) [zypp::media++] MediaCurl.cc(doGetFileCopyFile):1204 /geoip 2024-09-23 12:02:15 <1> localhost(840) [zypp::media++] MediaCurl.cc(doGetFileCopyFile):1214 URL: https://download.opensuse.org/geoip There is no /etc/resolv.conf, so I'd expect this to fail immediately. Maybe this is a race condition, where curl fails before zypper sets up waiting? I wasn't able to reproduce this locally in a container. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1230912 https://bugzilla.suse.com/show_bug.cgi?id=1230912#c1 Benjamin Zeller <bzeller@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bzeller@suse.com --- Comment #1 from Benjamin Zeller <bzeller@suse.com> --- Could this be due to timeout settings in curl/zypp? E.G. CURLOPT_CONNECTTIMEOUT .. but iirc the default setting in zypp should be 60 seconds for a connect timeout. Multicurl uses fd based waiting... so even if something fails before we start to poll() it should be no issue because as long as the event is not read from the fd poll() should return immediately... -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1230912 https://bugzilla.suse.com/show_bug.cgi?id=1230912#c2 --- Comment #2 from Benjamin Zeller <bzeller@suse.com> --- https://curl.se/libcurl/c/CURLOPT_CONNECTTIMEOUT.html -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1230912 https://bugzilla.suse.com/show_bug.cgi?id=1230912#c3 --- Comment #3 from Fabian Vogt <fvogt@suse.com> --- (In reply to Benjamin Zeller from comment #1)
Could this be due to timeout settings in curl/zypp? E.G. CURLOPT_CONNECTTIMEOUT .. but iirc the default setting in zypp should be 60 seconds for a connect timeout.
Multicurl uses fd based waiting... so even if something fails before we start to poll() it should be no issue because as long as the event is not read from the fd poll() should return immediately...
FTR, it doesn't look like it would ever return, it's stuck for ~5min before it gets terminated by the test timeout handler here. I can reproduce the issue easily, but in a context which doesn't allow for easy debugging... I'll try to figure out something. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1230912 https://bugzilla.suse.com/show_bug.cgi?id=1230912#c4 Fabian Vogt <fvogt@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |IN_PROGRESS --- Comment #4 from Fabian Vogt <fvogt@suse.com> --- I managed to set up some debugging environment and had a debugging session with bzeller. In the working case, curl_multi_socket_action adds a socket and sets a timeout, then returns to libzypp's event loop. In the broken case, curl_multi_socket_action adds a socket to the wait list and sets a timeout, but also removes the socket and clears the timeout before returning to the event loop: 2024-09-27 13:12:43 <1> localhost(844) [zypp-curl++] curlhelper.cc(log_curl):120 0x556b6be79148 * !!! WARNING !!! 2024-09-27 13:12:43 <1> localhost(844) [zypp-curl++] curlhelper.cc(log_curl):120 0x556b6be79148 * This is a debug build of libcurl, do not use in production. 2024-09-27 13:12:43 <1> localhost(844) [zypp::media++] MediaCurl.cc(executeCurl):1442 Calling _curlHelper.handleTimout 2024-09-27 13:12:43 <1> localhost(844) [zypp-curl++] curlhelper.cc(log_curl):120 0x556b6be79148 * STATE: INIT => SETUP handle 0x556b6be79148; line 1884 2024-09-27 13:12:43 <1> localhost(844) [zypp-curl++] curlhelper.cc(log_curl):120 0x556b6be79148 * STATE: SETUP => CONNECT handle 0x556b6be79148; line 1900 2024-09-27 13:12:43 <1> localhost(844) [zypp-curl++] curlhelper.cc(log_curl):120 0x556b6be79148 * Added connection 0. The cache now contains 1 members 2024-09-27 13:12:43 <1> localhost(844) [zypp-curl++] curlhelper.cc(log_curl):120 0x556b6be79148 * STATE: CONNECT => RESOLVING handle 0x556b6be79148; line 1925 2024-09-27 13:12:43 <1> localhost(844) [zypp-curl++] curlhelper.cc(socketcb):431 CurlPollHelper::socketcb easy 0x556b6be79148 s: 16 what: 1 userp: 0x7ffc1b6c5dc0 2024-09-27 13:12:43 <1> localhost(844) [zypp-curl++] curlhelper.cc(log_curl):120 0x556b6be79148 * Could not resolve host: download.opensuse.org 2024-09-27 13:12:43 <1> localhost(844) [zypp-curl++] curlhelper.cc(log_curl):120 0x556b6be79148 * Curl_multi_closed, fd=16 multi is 0x556b6be7ceb8 2024-09-27 13:12:43 <1> localhost(844) [zypp-curl++] curlhelper.cc(log_curl):120 0x556b6be79148 * Curl_multi_closed, fd=16 entry is 0x556b6be7bbd8 2024-09-27 13:12:43 <1> localhost(844) [zypp-curl++] curlhelper.cc(socketcb):431 CurlPollHelper::socketcb easy 0x556b6be79148 s: 16 what: 4 userp: 0x7ffc1b6c5dc0 Without sockets to poll and an infinite timeout (-1), libzypp waits forever. The curl behaviour happens because some earlier curl call added a negative cache entry for "download.opensuse.org", so it could reach the RESOLVING stage and fail before performing any wait? -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1230912 https://bugzilla.suse.com/show_bug.cgi?id=1230912#c7 --- Comment #7 from Benjamin Zeller <bzeller@suse.com> --- https://github.com/openSUSE/libzypp/pull/582 -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1230912 https://bugzilla.suse.com/show_bug.cgi?id=1230912#c8 Benjamin Zeller <bzeller@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|IN_PROGRESS |RESOLVED Resolution|--- |FIXED --- Comment #8 from Benjamin Zeller <bzeller@suse.com> --- Fixed in libzypp >= 17.35.12 (35) -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1230912 https://bugzilla.suse.com/show_bug.cgi?id=1230912#c9 Michael Andres <ma@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |locilka@suse.com --- Comment #9 from Michael Andres <ma@suse.com> --- *** Bug 1231643 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1230912 https://bugzilla.suse.com/show_bug.cgi?id=1230912#c10 --- Comment #10 from Maintenance Automation <maint-coord+maintenance-robot@suse.de> --- SUSE-RU-2024:3681-1: An update that has two fixes can now be installed. URL: https://www.suse.com/support/update/announcement/2024/suse-ru-20243681-1 Category: recommended (important) Bug References: 1230912, 1231043 Maintenance Incident: [SUSE:Maintenance:36059](https://smelt.suse.de/incident/36059/) Sources used: openSUSE Leap 15.6 (src): libzypp-17.35.12-150600.3.27.1 SUSE Linux Enterprise High Performance Computing 15 SP6 (src): libzypp-17.35.12-150600.3.27.1 SUSE Linux Enterprise Server 15 SP6 (src): libzypp-17.35.12-150600.3.27.1 SUSE Linux Enterprise Server for SAP Applications 15 SP6 (src): libzypp-17.35.12-150600.3.27.1 SUSE Linux Enterprise Desktop 15 SP6 (src): libzypp-17.35.12-150600.3.27.1 Basesystem Module 15-SP6 (src): libzypp-17.35.12-150600.3.27.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1230912 https://bugzilla.suse.com/show_bug.cgi?id=1230912#c11 --- Comment #11 from Maintenance Automation <maint-coord+maintenance-robot@suse.de> --- SUSE-RU-2024:3718-1: An update that has two fixes can now be installed. URL: https://www.suse.com/support/update/announcement/2024/suse-ru-20243718-1 Category: recommended (important) Bug References: 1230912, 1231043 Maintenance Incident: [SUSE:Maintenance:36058](https://smelt.suse.de/incident/36058/) Sources used: openSUSE Leap 15.5 (src): libzypp-17.35.12-150500.6.21.1 openSUSE Leap Micro 5.5 (src): libzypp-17.35.12-150500.6.21.1 SUSE Linux Enterprise High Performance Computing 15 SP5 (src): libzypp-17.35.12-150500.6.21.1 SUSE Linux Enterprise Server 15 SP5 (src): libzypp-17.35.12-150500.6.21.1 SUSE Linux Enterprise Server for SAP Applications 15 SP5 (src): libzypp-17.35.12-150500.6.21.1 SUSE Linux Enterprise Desktop 15 SP5 (src): libzypp-17.35.12-150500.6.21.1 SUSE Linux Enterprise Micro 5.5 (src): libzypp-17.35.12-150500.6.21.1 Basesystem Module 15-SP5 (src): libzypp-17.35.12-150500.6.21.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1230912 https://bugzilla.suse.com/show_bug.cgi?id=1230912#c12 --- Comment #12 from Maintenance Automation <maint-coord+maintenance-robot@suse.de> --- SUSE-RU-2024:3730-1: An update that has two fixes can now be installed. URL: https://www.suse.com/support/update/announcement/2024/suse-ru-20243730-1 Category: recommended (important) Bug References: 1230912, 1231043 Maintenance Incident: [SUSE:Maintenance:36056](https://smelt.suse.de/incident/36056/) Sources used: SUSE Linux Enterprise Server 15 SP2 (src): libzypp-17.35.12-150200.129.1 SUSE Linux Enterprise Server 15 SP3 (src): libzypp-17.35.12-150200.129.1 SUSE Linux Enterprise High Performance Computing 15 SP2 LTSS 15-SP2 (src): libzypp-17.35.12-150200.129.1 SUSE Linux Enterprise High Performance Computing LTSS 15 SP3 (src): libzypp-17.35.12-150200.129.1 SUSE Linux Enterprise Server 15 SP2 LTSS 15-SP2 (src): libzypp-17.35.12-150200.129.1 SUSE Linux Enterprise Server 15 SP3 LTSS 15-SP3 (src): libzypp-17.35.12-150200.129.1 SUSE Linux Enterprise Server for SAP Applications 15 SP2 (src): libzypp-17.35.12-150200.129.1 SUSE Linux Enterprise Server for SAP Applications 15 SP3 (src): libzypp-17.35.12-150200.129.1 SUSE Enterprise Storage 7.1 (src): libzypp-17.35.12-150200.129.1 SUSE Linux Enterprise Micro 5.1 (src): libzypp-17.35.12-150200.129.1 SUSE Linux Enterprise Micro 5.2 (src): libzypp-17.35.12-150200.129.1 SUSE Linux Enterprise Micro for Rancher 5.2 (src): libzypp-17.35.12-150200.129.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1230912 https://bugzilla.suse.com/show_bug.cgi?id=1230912#c13 --- Comment #13 from Maintenance Automation <maint-coord+maintenance-robot@suse.de> --- SUSE-RU-2024:3727-1: An update that has two fixes can now be installed. URL: https://www.suse.com/support/update/announcement/2024/suse-ru-20243727-1 Category: recommended (important) Bug References: 1230912, 1231043 Maintenance Incident: [SUSE:Maintenance:36057](https://smelt.suse.de/incident/36057/) Sources used: openSUSE Leap 15.4 (src): libzypp-17.35.12-150400.3.93.1 SUSE Linux Enterprise High Performance Computing 15 SP4 (src): libzypp-17.35.12-150400.3.93.1 SUSE Linux Enterprise Server 15 SP4 (src): libzypp-17.35.12-150400.3.93.1 SUSE Manager Server 4.3 (src): libzypp-17.35.12-150400.3.93.1 SUSE Linux Enterprise Server for SAP Applications 15 SP4 (src): libzypp-17.35.12-150400.3.93.1 SUSE Linux Enterprise Desktop 15 SP4 (src): libzypp-17.35.12-150400.3.93.1 SUSE Manager Retail Branch Server 4.3 (src): libzypp-17.35.12-150400.3.93.1 SUSE Manager Proxy 4.3 (src): libzypp-17.35.12-150400.3.93.1 SUSE Linux Enterprise Micro for Rancher 5.3 (src): libzypp-17.35.12-150400.3.93.1 SUSE Linux Enterprise Micro 5.3 (src): libzypp-17.35.12-150400.3.93.1 SUSE Linux Enterprise Micro for Rancher 5.4 (src): libzypp-17.35.12-150400.3.93.1 SUSE Linux Enterprise Micro 5.4 (src): libzypp-17.35.12-150400.3.93.1 SUSE Linux Enterprise High Performance Computing ESPOS 15 SP4 (src): libzypp-17.35.12-150400.3.93.1 SUSE Linux Enterprise High Performance Computing LTSS 15 SP4 (src): libzypp-17.35.12-150400.3.93.1 SUSE Linux Enterprise Desktop 15 SP4 LTSS 15-SP4 (src): libzypp-17.35.12-150400.3.93.1 SUSE Linux Enterprise Server 15 SP4 LTSS 15-SP4 (src): libzypp-17.35.12-150400.3.93.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination. -- You are receiving this mail because: You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠suse.com