http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 http://bugzilla.opensuse.org/show_bug.cgi?id=1068470#c1 Sebastian Parschauer changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|P5 - None |P3 - Medium CC| |tchvatal@suse.com, | |vcizek@suse.com --- Comment #1 from Sebastian Parschauer --- I just found a solution by adding the repo

https://download.opensuse.org/repositories/security:/tls/openSUSE_Tumbleweed... and upgrading to openssl 1.1.0g from there.

-- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 http://bugzilla.opensuse.org/show_bug.cgi?id=1068470#c5 --- Comment #5 from Sebastian Parschauer --- Hmm, I always have entropy > 3700 when the command is stalled. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 http://bugzilla.opensuse.org/show_bug.cgi?id=1068470#c7 Jan Matejek changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(jmatejek@suse.com | |) | --- Comment #7 from Jan Matejek --- We have no patches relevant to this, and I don't have advice on debugging, apart from using GDB to see what's up. From a very quick read of the M2crypto source of `ssl_read`, i see some possibility of entering an infinite loop, given a specific set of error conditions. However, your report first catches the problem in `ssl_connect` and later in `ssl_read` ? I can't reproduce it on my current TW system in any case. Will try again when the update run finishes. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 http://bugzilla.opensuse.org/show_bug.cgi?id=1068470#c9 --- Comment #9 from Sebastian Parschauer --- I just triggered the hanging at osc co with tracing. There it is indeed ssl_connect which is hanging. Command:

bash -c "while [ 1 ]; do python -m trace --trace /usr/bin/osc co Base:System/parted; sleep 1; rm -Rf Base:System; done"

--- modulename: Connection, funcname: connect Connection.py(298): self.socket.connect(addr) --- modulename: socket, funcname: meth socket.py(228): return getattr(self._sock,name)(*args) Connection.py(299): self.addr = addr Connection.py(300): self.setup_ssl() --- modulename: Connection, funcname: setup_ssl Connection.py(209): self.sockbio = m2.bio_new_socket(self.socket.fileno(), 0) --- modulename: socket, funcname: meth socket.py(228): return getattr(self._sock,name)(*args) Connection.py(211): m2.ssl_set_bio(self.ssl, self.sockbio, self.sockbio) Connection.py(213): self.sslbio = m2.bio_new(m2.bio_f_ssl()) Connection.py(215): m2.bio_set_ssl(self.sslbio, self.ssl, m2.bio_noclose) Connection.py(301): self.set_connect_state() --- modulename: Connection, funcname: set_connect_state Connection.py(284): m2.ssl_set_connect_state(self.ssl) Connection.py(302): ret = self.connect_ssl() --- modulename: Connection, funcname: connect_ssl Connection.py(288): return m2.ssl_connect(self.ssl, self._timeout) <-- hangs here for several minutes -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 http://bugzilla.opensuse.org/show_bug.cgi?id=1068470#c11 --- Comment #11 from Sebastian Parschauer --- (In reply to Jan Matejek from comment #10)

So, are you sure that the connection to api.opensuse.org in particular is not faulty?

Yes, as wget for the same files with same user name + pw works when osc co is hanging and as it works if timing is different. Of cause it also works with different OS.

wget --user sparschauer --ask-password https://api.opensuse.org/source/Base:System/parted/0005-libparted-Remove-fda...

I've set "http_debug = 1" in .oscrc here. -- GET https://api.opensuse.org/source/Base:System/parted/0004-fdasd.c-Safeguard-ag... send: 'GET /source/Base:System/parted/0004-fdasd.c-Safeguard-against-geometry-misprobing.patch?rev=135 HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: api.opensuse.org\r\nConnection: close\r\nUser-agent: osc/0.161.1\r\n\r\n' reply: 'HTTP/1.1 200 OK\r\n' header: Date: Wed, 22 Nov 2017 15:22:21 GMT header: Server: Apache/2.4.23 (Linux/SUSE) header: Strict-Transport-Security: max-age=31536000 header: Cache-Control: no-transform header: X-XSS-Protection: 1; mode=block header: X-Opensuse-Runtimes: {"view":null,"db":6.0924879999999995,"backend":0,"xml":0} header: X-Request-Id: 7aeeeab1-b6e2-4d65-94c4-16f2a3dc985b header: X-Opensuse-APIVersion: 2.8.51.git20171106T173515.61f8121be header: X-Runtime: 0.027189 header: X-Frame-Options: SAMEORIGIN header: X-Content-Type-Options: nosniff header: X-Powered-By: Phusion Passenger 5.0.30 header: Content-Type: application/octet-stream header: Content-Length: 1568 header: Cache-Control: no-cache header: Connection: close A Base:System/parted/0004-fdasd.c-Safeguard-against-geometry-misprobing.patch -- GET https://api.opensuse.org/source/Base:System/parted/0005-libparted-Remove-fda... In this example it can get everything until patch 0004 but patch 0005 can't be loaded. It hangs here for minutes. It is random at which file it hangs. Adding "debug = 1" to .oscrc also doesn't show more information. This is why I've done Python tracing.

You could use wireshark to analyze the TLS handshake. If the hang is actually happening in ssl_connect, that should be in the part that is not yet encrypted and you could see what is going on.

Will try this next, thanks. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 http://bugzilla.opensuse.org/show_bug.cgi?id=1068470#c13 --- Comment #13 from Sebastian Parschauer --- I've also noticed that api.suse.de doesn't have:

header: Strict-Transport-Security: max-age=31536000

The max-age value seems to be extremely high to me. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 http://bugzilla.opensuse.org/show_bug.cgi?id=1068470#c15 --- Comment #15 from Sebastian Parschauer --- Looks like the trigger is receiving

Application Data, Application Data, Encrypted Alert in the same TLSv1.2 packet. The client doesn't sent an ACK for this. Then api.opensuse.org complains about "TCP ACKed unseen segment".

-- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 http://bugzilla.opensuse.org/show_bug.cgi?id=1068470#c17 --- Comment #17 from Sebastian Parschauer --- Same issue showing "TCP ACKed unseen segment" with kernel 4.14.0-1 and 4.4.92-31 from Leap but also at regular "Encrypted Alert" packets. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 http://bugzilla.opensuse.org/show_bug.cgi?id=1068470#c20 Sebastian Parschauer changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|P3 - Medium |P2 - High Summary|Tumbleweed openssl 1.0.2m: |openssl: osc co command |osc co command hangs in |hangs in libcrypto with |libcrypto with |https://api.opensuse.org |https://api.opensuse.org | Severity|Normal |Major --- Comment #20 from Sebastian Parschauer --- This is even much easier reproducible with WiFi. Leap 42.3 is affected as well this way. This blocks me from doing my work efficiently and likely multiple other developers are affected as well. So raising priority here. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 http://bugzilla.opensuse.org/show_bug.cgi?id=1068470#c22 --- Comment #22 from Sebastian Parschauer --- It is strange that I don't hit it with zypper. openssl 1.1.0g-3.1 is also affected. I've just updated Tumbleweed. In the osc source class mySSLContext I've found this:

#self.set_info_callback() # debug

When uncommenting it, I see the following osc co output:

A Base:System/parted/lib-fs-resize-prevent-crash-resizing-FAT16.patch LOOP: SSL connect: before SSL initialization LOOP: SSL connect: SSLv3/TLS write client hello LOOP: SSL connect: SSLv3/TLS write client hello LOOP: SSL connect: SSLv3/TLS read server hello LOOP: SSL connect: SSLv3/TLS read change cipher spec LOOP: SSL connect: SSLv3/TLS read finished LOOP: SSL connect: SSLv3/TLS write change cipher spec LOOP: SSL connect: SSLv3/TLS write finished INFO: SSL connect: SSL negotiation finished successfully ALERT: write: warning: close notify A Base:System/parted/libparted-Add-support-for-NVDIMM-devices.patch LOOP: SSL connect: before SSL initialization LOOP: SSL connect: SSLv3/TLS write client hello

It always hangs at the first client hello. It seems to be a bug to me that there are two client hellos. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 http://bugzilla.opensuse.org/show_bug.cgi?id=1068470#c24 --- Comment #24 from Sebastian Parschauer --- I've debugged this further with gdb. The sequence

LOOP: SSL connect: SSLv3/TLS write client hello LOOP: SSL connect: SSLv3/TLS write client hello

is correct. The first one is printed before sending the client hello and the second one is printed after sending it. So the bug is located in the write_state_machine() of libcrypto in ssl/statem/statem.c. It returns before sending the client hello and waits for the server hello in the read_state_machine() too early this way. I'll try to catch it with breakpoints next. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 Adrian Schröter changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(adrian@suse.com) | -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 http://bugzilla.opensuse.org/show_bug.cgi?id=1068470#c31 Sebastian Parschauer changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |IN_PROGRESS Flags| |needinfo?(jmatejek@suse.com | |) --- Comment #31 from Sebastian Parschauer --- The Python tracing already indicated that the situation gets better if delaying some Python code. osc configures the https connection to be closed in M2Crypto upon every request.

headers["Connection"] = "close"

M2Crypto closes the socket too early while closing the connection. I've finally found a way how to work around this bug. I've edited:

/usr/lib64/python2.7/site-packages/M2Crypto/SSL/Connection.py

+from time import sleep Connection::__del__: if self.ssl_close_flag == m2.bio_noclose and \ getattr(self, 'ssl', None): self.m2_ssl_free(self.ssl) + sleep(0.1) self.socket.close() 50ms was too short but 100ms seems to work fine for me. But one RST per downloaded file is still in the tcpdump. No difference if putting the sleep in front of the self.m2_ssl_free() call. @Jan: Is there a more elegant way than using a sleep to detect that the socket should not be closed yet or does the kernel have to detect that? TIA -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 Tomáš Chvátal changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(tchvatal@suse.com | |) | -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 http://bugzilla.opensuse.org/show_bug.cgi?id=1068470#c62 --- Comment #62 from Marcus Hüwe --- (In reply to Matej Cepl from comment #61)

(In reply to Marcus Hüwe from comment #60)

...

@Sebastian: is there any chance to make bug 1113787 public?

??? It has nothing to do with this bug.

Ok, if it is unrelated, I don't care:) I was just curious because #1113787 was added to the "See also" list of this bug... (but I have no permission to view that bug). -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1068470 http://bugzilla.opensuse.org/show_bug.cgi?id=1068470#c70 --- Comment #70 from Matej Cepl --- (In reply to Sebastian Parschauer from comment #69)

Moving fix packages to personal OBS account "TuxCheater".

This bug won't be fixed and I have to keep a custom patch/package. So closing now.

I hoped (and still I would like) that somebody finishes https://gitlab.com/m2crypto/m2crypto/merge_requests/188 (resp. now rebased as !219), but alas it didn't happen, and I haven't found enough stamina to finish it myself. I am sorry. -- You are receiving this mail because: You are on the CC list for the bug.