https://bugzilla.novell.com/show_bug.cgi?id=761501 https://bugzilla.novell.com/show_bug.cgi?id=761501#c1 --- Comment #1 from James Oakley 2012-05-09 22:42:14 UTC --- Unfortunately, the requirement of a bundle file goes down to core Python: http://docs.python.org/library/ssl.html The SSL support in requests is implemented using ssl.wrap_socket(). It takes the certs via the ca_certs parameter, which is expected to point to a file. I checked to ensure this was the case:

...

...

sock = ssl.wrap_socket(sock, cert_reqs=ssl.CERT_REQUIRED, ca_certs="/etc/ssl/certs") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python2.7/ssl.py", line 372, in wrap_socket ciphers=ciphers) File "/usr/lib64/python2.7/ssl.py", line 132, in __init__ ciphers) ssl.SSLError: [Errno 0] _ssl.c:340: error:00000000:lib(0):func(0):reason(0)

It works fine with the bundle provided by python-certifi. which is currently required by python-requests. To make it work on a directory, the ssl module would need to be patched. Here is the code in Python-2.7.3/Modules/_ssl.c line 335: ret = SSL_CTX_load_verify_locations(self->ctx, cacerts_file, NULL); The easiest way is probably to check to see if the cacerts_file parameter is a directory, and call it with (self->ctx, NULL, cacerts_file) instead. I can certainly submit a patch for that, if that's the way you want to go. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=761501 https://bugzilla.novell.com/show_bug.cgi?id=761501#c3 James Oakley changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|Major |Critical --- Comment #3 from James Oakley 2012-05-10 17:55:43 UTC --- I made a patch to allow for passing a directory and tested it. It seems to work fine, but I didn't submit it yet since I'm not sure whether it would be considered a good idea to deviate from the documented behaviour. I do like the idea of calling SSL_CTX_set_default_verify_paths() too. I will generate a patch for that as well. I can submit either or both patches to Factory if you want. Meanwhile, I will raise the issue on the Python tracker. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=761501 https://bugzilla.novell.com/show_bug.cgi?id=761501#c5 --- Comment #5 from James Oakley 2012-05-11 00:16:05 UTC --- I made and tested patches for Python 2.7.3 and 3.2. I submitted them to the Python issue tracker: http://bugs.python.org/issue14780 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=761501 https://bugzilla.novell.com/show_bug.cgi?id=761501#c7 --- Comment #7 from Ludwig Nussel 2012-05-14 09:26:49 CEST --- I'd prefer to have the patch that calls SSL_CTX_set_default_verify_paths() to avoid applications having to hardcode any paths. SSL_CTX_set_default_verify_paths() doesn't cost much anyways as openssl does not need to load a certificate directory into memory but can access it on demand. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=761501 https://bugzilla.novell.com/show_bug.cgi?id=761501#c9 --- Comment #9 from Ludwig Nussel 2012-05-15 13:40:17 CEST --- Well, I find it more suprising that I as application developer have to care where the trusted certificates are. Not using any doesn't make sense as the connection can't be verified then. Having to patch each and every application doesn't make sense either. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=761501 https://bugzilla.novell.com/show_bug.cgi?id=761501#c11 --- Comment #11 from Marcus Meissner 2012-05-16 14:06:34 UTC --- Missing SSL certificate checking is a exploitable security issue (man in the middle attack) which we see more often these days. Everyone knows that python code does this very badly however, but nethertheless it should be considered also to approach the old codebase which is still in wide use. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=761501 https://bugzilla.novell.com/show_bug.cgi?id=761501#c13 --- Comment #13 from James Oakley 2012-05-17 19:40:57 UTC --- I just submitted the patch to support directory-based certificate stores. It's SR 121341 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=761501 https://bugzilla.novell.com/show_bug.cgi?id=761501#c15 --- Comment #15 from James Oakley 2012-05-18 18:13:19 UTC --- My SR to the python package was accepted. I'm currently waiting on a couple of SRs in d:l:p for new requirements in the latest version of python-requests. Once those are in, I will patch it to behave like I proposed in comment 12 and remove the python-certifi requirement for the upcoming openSUSE release. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=761501 https://bugzilla.novell.com/show_bug.cgi?id=761501#c17 --- Comment #17 from Ludwig Nussel 2012-05-21 10:50:52 CEST --- (In reply to comment #12)

So let's assume we patch Python for openSUSE. If we make it load the store by default, module authors will have a hard time distinguishing between our patched version and other versions. There is no way to really check if it was successful, especially since the OpenSSL call fails silently.

However, if we go with my original suggestion and patch to allow loading directory stores, it will be obvious when it doesn't work.

There's nothing the application author needs to know. The situation doesn't get worse. Right now if one doesn't pass a path for a CA bundle two things might happen depending on how modules interact with openssl: a) no ssl checks at all, connection succeeds but is in fact insecure b) ssl connections always fail due to lack of trust anchors Neither is desirable. By patching the layer above openssl to always load the default store if no bundle/dir was given explicitly connections will be safe by default. There won't be a disadvantage for applications. Connections that previously worked but were insecure now correctly fail. Connections that didn't work before because of missing trust store start to work. I don't think the patch is inappropriate or too intrusive for python2. The alternative of patching potentially dozends of modules and applications to hardcode the CA path is worse. Esp since we might decide to use a different default location or even format in the future. Fedora for example has an extra location with certificates in openssl's "TRUSTED CERTIFICATE" format which cannot be used in /etc/ssl/certs for compatibility reasons. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=761501 https://bugzilla.novell.com/show_bug.cgi?id=761501#c19 --- Comment #19 from Ludwig Nussel 2012-05-22 09:02:52 CEST --- (In reply to comment #18)

...

There's nothing the application author needs to know. The situation doesn't get worse. Right now if one doesn't pass a path for a CA bundle two things might happen depending on how modules interact with openssl: a) no ssl checks at all, connection succeeds but is in fact insecure b) ssl connections always fail due to lack of trust anchors

Please consider the following:

1. In the requests module, the subject of this bug report, the code specifies a ca path. It checks for various distribution-specific locations, and failing that, loads the certifi bundle. Your proposed change doesn't affect that in the slightest

Just patch the code away that does a fallback to some bundle. That's better than patchin in yet another path.

2. If the certificate store is loaded automatically, but only in openSUSE (remember that upstream rejected this), and there is no feedback to indicate whether the store actually loaded or not, how can a module author be sure whether the certificates are verified or not? Should they be checking to see

Doesn't matter whether the system certificate store was loaded successfully as long as certificate checking is guaranteed to be on always. If loading the store fails (which is basically impossible with the CA directory) all certificate validations would fail. Ie fail-safe behavior.

3. If certificates were verified by default, what if I were calling something on a server with a self-signed certificate? What if I wanted to communicate with a private OBS instance? It would raise exceptions and I would be stuck and unable to use it

You need to handle that anyways. If self-signed certs "work" without any extra handling by an application/module it's pretty obvious that no certificate checking was done ie the connection is unsafe. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=761501 https://bugzilla.novell.com/show_bug.cgi?id=761501#c21 --- Comment #21 from Ludwig Nussel 2012-05-23 09:14:31 CEST --- (In reply to comment #20)

(In reply to comment #19)

...

Just patch the code away that does a fallback to some bundle. That's better than patchin in yet another path.

Except that won't work on other distros. The current requests patch can be upstreamed.

At a certain point we have do to what makes sense for us. Hardcoding some path may be fine for certain individual upstreams, for a distro it's insane though.

...

Doesn't matter whether the system certificate store was loaded successfully as long as certificate checking is guaranteed to be on always. If loading the store fails (which is basically impossible with the CA directory) all certificate validations would fail. Ie fail-safe behavior.

Feel free to look at the code in the requests module. The actual validation occurs in a totally different place in the code that than the determination of the cert store. The code needs some way to figure out if the store is actually correct, and fall back if necessary.

Are you referring to the code in models.py? if not cert_loc: cert_loc = DEFAULT_CA_BUNDLE_PATH if not cert_loc: raise Exception("Could not find a suitable SSL CA certificate bundle.") I suppose the cert_loc set here ends up as the ca_certs parameter of ssl.wrap_socket() in packages/urllib3/connectionpool.py. So if ssl.wrap_socket would just use the system store if ca_certs=None the lines above could be removed without replacement and without loss of any feature.

...

You need to handle that anyways. If self-signed certs "work" without any extra handling by an application/module it's pretty obvious that no certificate checking was done ie the connection is unsafe.

But if you make it always on, then you can't use self-signed certs at all. That will break a LOT of things.

Well, if you turn off verification the 'secure' in SSL wouldn't deserve it's name. You could just as well use http then. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=761501 https://bugzilla.novell.com/show_bug.cgi?id=761501#c23 --- Comment #23 from Ludwig Nussel 2012-05-24 15:07:59 CEST --- (In reply to comment #22)

This doesn't matter one way or the other. The real problem is that certificate validation is *turned off by default* - the cert_reqs argument is set to ssl.CERT_NONE. Unless explicitly set to CERT_REQUIRED or CERT_OPTIONAL, the cert store is completely ignored - and if it is set, ca_certs must also be set to a correct path.

IOW, literally no packages are affected in any way by whether we load the default cert store. Either they are insecure, and will continue to be insecure, or they are already supplying their own cert bundles.

Correct. We're trying to address the latter for a start.

Only thing we can do for the insecure packages is change the default value of cert_reqs argument, and only _then_ load the default cert store automatically. But that is a bad idea because it is in direct contradiction with the official docs. I mean, yes, this default is a bad default, but that doesn't mean we're in any position to change this unilaterally.

Yes we are. It's free software after all. I'd be happy with getting rid of hardcoding the ca path everywhere as first step though :-) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.