Mailinglist Archive: opensuse-buildservice (145 mails)

< Previous Next >
[opensuse-buildservice] Two patches for osc: API URLs with pathname and TLS SNI.
Hello everybody!

I'm sending two patches to osc which implement the following useful features.

1. Support for API URLs with pathname.

Currently osc supports API URLs without path only, like
https://api.example.com (if there is pathname, it's just ignored).
With this change API URLS with path, like https://example.com/api are
supported correctly.

This is useful for those who can't have OBS api, webui and main site on
different domains and buy separate X.509 certificates for them, or just can't
afford to have separate IP addresses for them (please note that currently osc
doesn't support TLS SNI).

You may say that OBS API doesn't work correctly if installed with non-root
path. But I took care about that already. I'll prepare and send a patch little
bit later.

2. Support for TLS SNI (if M2Crypto supports it).

Currently osc can't access API URLs which share the same IP address with other
SSL-enabled sites, complaining about certificate not matching hostname.

This change solves this problem by instructing M2Crypto.SSL.Connection to send
the desired hostname to https server using TLS SNI extension, thus allowing
the server to present the right certificate and choose the right virtual site.

This is useful for those who can't afford to have a separate IP address for
OBS API.

For TLS SNI to work correctly, M2Crypto should be patched:
https://bugzilla.osafoundation.org/show_bug.cgi?id=13073
For unpatched M2Crypto osc degrades to operation without TLS SNI.

I've tested these changes with my OBS server, which has API installed on a
virtual HTTPS server with non-root path.

Patches are attached to this message.

Please reply with cc to me, I'm not subscribed to opensuse-buildservice
mailing list.

Best regards,
-- Oleg Girko, http://www.infoserver.lv/~ol/
From cbd028d2d5f1c35ea648e95ffc49070cfde5aeb0 Mon Sep 17 00:00:00 2001
From: Oleg Girko <ol@xxxxxxxxxxxxx>
Date: Sat, 26 Oct 2013 00:15:18 +0100
Subject: [PATCH 1/2] Add support for non-root paths in API URLs.

Currently osc supports API URLs without path only,
like https://api.example.com (if there is pathname, it's just ignored).
With this change API URLS with path, like https://example.com/api
are supported correctly.

This is useful for those who can't have OBS api, webui and main site
on different domains and buy separate X.509 certificates for them,
or just can't afford to have separate IP addresses for them
(please note that currently osc doesn't support TLS SNI).

Signed-off-by: Oleg Girko <ol@xxxxxxxxxxxxx>
---
osc/conf.py | 39 ++++++++++++++++++++-------------------
osc/core.py | 19 ++++++++++++++-----
2 files changed, 34 insertions(+), 24 deletions(-)

diff --git a/osc/conf.py b/osc/conf.py
index 6a72313..768c111 100644
--- a/osc/conf.py
+++ b/osc/conf.py
@@ -371,17 +371,17 @@ cookiejar = None

def parse_apisrv_url(scheme, apisrv):
if apisrv.startswith('http://') or apisrv.startswith('https://'):
- return urlsplit(apisrv)[0:2]
+ return urlsplit(apisrv)[0:3]
elif scheme != None:
# the split/join is needed to get a proper url (e.g. without a
trailing slash)
- return urlsplit(urljoin(scheme, apisrv))[0:2]
+ return urlsplit(urljoin(scheme, apisrv))[0:3]
else:
msg = 'invalid apiurl \'%s\' (specify the protocol (http:// or
https://))' % apisrv
raise URLError(msg)


-def urljoin(scheme, apisrv):
- return '://'.join([scheme, apisrv])
+def urljoin(scheme, apisrv, path=''):
+ return '://'.join([scheme, apisrv]) + path


def is_known_apiurl(url):
@@ -430,10 +430,9 @@ def get_apiurl_usr(apiurl):
# So we need to build a new opener everytime we switch the
# apiurl (because different apiurls may have different
# cafile/capath locations)
-def _build_opener(url):
+def _build_opener(apiurl):
from osc.core import __version__
global config
- apiurl = urljoin(*parse_apisrv_url(None, url))
if 'last_opener' not in _build_opener.__dict__:
_build_opener.last_opener = (None, None)
if apiurl == _build_opener.last_opener[0]:
@@ -624,18 +623,18 @@ def config_set_option(section, opt, val=None,
delete=False, update=True, **kwarg
general_opts = [i for i in DEFAULTS.keys() if not i in ['user', 'pass',
'passx']]
if section != 'general':
section = config['apiurl_aliases'].get(section, section)
- scheme, host = \
+ scheme, host, path = \
parse_apisrv_url(config.get('scheme', 'https'), section)
- section = urljoin(scheme, host)
+ section = urljoin(scheme, host, path)

sections = {}
for url in cp.sections():
if url == 'general':
sections[url] = url
else:
- scheme, host = \
+ scheme, host,path = \
parse_apisrv_url(config.get('scheme', 'https'), url)
- apiurl = urljoin(scheme, host)
+ apiurl = urljoin(scheme, host, path)
sections[apiurl] = url

section = sections.get(section.rstrip('/'), section)
@@ -675,19 +674,20 @@ def write_initial_config(conffile, entries,
custom_template=''):
config.update(entries)
# at this point use_keyring and gnome_keyring are str objects
if config['use_keyring'] == '1' and GENERIC_KEYRING:
- protocol, host = \
+ protocol, host, path = \
parse_apisrv_url(None, config['apiurl'])
keyring.set_password(host, config['user'], config['pass'])
config['pass'] = ''
config['passx'] = ''
elif config['gnome_keyring'] == '1' and GNOME_KEYRING:
- protocol, host = \
+ protocol, host, path = \
parse_apisrv_url(None, config['apiurl'])
gnomekeyring.set_network_password_sync(
user=config['user'],
password=config['pass'],
protocol=protocol,
- server=host)
+ server=host,
+ object=path)
config['user'] = ''
config['pass'] = ''
config['passx'] = ''
@@ -714,19 +714,20 @@ def add_section(filename, url, user, passwd):
# Section might have existed, but was empty
pass
if config['use_keyring'] and GENERIC_KEYRING:
- protocol, host = parse_apisrv_url(None, url)
+ protocol, host, path = parse_apisrv_url(None, url)
keyring.set_password(host, user, passwd)
cp.set(url, 'keyring', '1')
cp.set(url, 'user', user)
cp.remove_option(url, 'pass')
cp.remove_option(url, 'passx')
elif config['gnome_keyring'] and GNOME_KEYRING:
- protocol, host = parse_apisrv_url(None, url)
+ protocol, host, path = parse_apisrv_url(None, url)
gnomekeyring.set_network_password_sync(
user=user,
password=passwd,
protocol=protocol,
- server=host)
+ server=host,
+ object=path)
cp.set(url, 'keyring', '1')
cp.remove_option(url, 'pass')
cp.remove_option(url, 'passx')
@@ -809,8 +810,8 @@ def get_config(override_conffile=None,
aliases = {}
for url in [x for x in cp.sections() if x != 'general']:
# backward compatiblity
- scheme, host = parse_apisrv_url(config.get('scheme', 'https'), url)
- apiurl = urljoin(scheme, host)
+ scheme, host, path = parse_apisrv_url(config.get('scheme', 'https'),
url)
+ apiurl = urljoin(scheme, host, path)
user = None
password = None
if config['use_keyring'] and GENERIC_KEYRING:
@@ -824,7 +825,7 @@ def get_config(override_conffile=None,
elif config['gnome_keyring'] and GNOME_KEYRING:
# Read from gnome keyring if available
try:
- gk_data =
gnomekeyring.find_network_password_sync(protocol=scheme, server=host)
+ gk_data =
gnomekeyring.find_network_password_sync(protocol=scheme, server=host,
object=path)
if not 'user' in gk_data[0]:
raise oscerr.ConfigError('no user found in keyring',
conffile)
user = gk_data[0]['user']
diff --git a/osc/core.py b/osc/core.py
index c9ee57d..7f1ae8e 100644
--- a/osc/core.py
+++ b/osc/core.py
@@ -2897,8 +2897,8 @@ def makeurl(baseurl, l, query=[]):
elif isinstance(query, type(dict())):
query = urlencode(query)

- scheme, netloc = urlsplit(baseurl)[0:2]
- return urlunsplit((scheme, netloc, '/'.join(l), query, ''))
+ scheme, netloc, path = urlsplit(baseurl)[0:3]
+ return urlunsplit((scheme, netloc, '/'.join([path] + l), query, ''))


def http_request(method, url, headers={}, data=None, file=None, timeout=0):
@@ -2923,10 +2923,19 @@ def http_request(method, url, headers={}, data=None,
file=None, timeout=0):

req = URLRequest(url)
api_host_options = {}
- if conf.is_known_apiurl(url):
+ scheme, host, path = conf.parse_apisrv_url(None, url)
+ p = path.split('/')
+ while p:
+ apiurl = conf.urljoin(scheme, host, '/'.join(p))
+ if apiurl in conf.config['api_host_options']:
+ break
+ p.pop()
+ else:
+ apiurl = None
+ if apiurl is not None:
# ok no external request
- install_opener(conf._build_opener(url))
- api_host_options = conf.get_apiurl_api_host_options(url)
+ install_opener(conf._build_opener(apiurl))
+ api_host_options = conf.get_apiurl_api_host_options(apiurl)
for header, value in api_host_options['http_headers']:
req.add_header(header, value)

--
1.8.3.1

From e9ea0e71f1e37f845dc0ba9aa189de2aacf81d36 Mon Sep 17 00:00:00 2001
From: Oleg Girko <ol@xxxxxxxxxxxxx>
Date: Sat, 26 Oct 2013 02:11:46 +0100
Subject: [PATCH 2/2] Add support for TLS SNI if M2Crypto supports it.

Currently osc can't access API URLs which share the same IP address
with other SSL-enabled sites, complaining about certificate
not matching hostname.

This change solves this problem by instructing M2Crypto.SSL.Connection
to send the desired hostname to https server using TLS SNI extension,
thus allowing the server to present the right certificate and choose
the right virtual site.

This is useful for those who can't afford to have a separate IP address
for OBS API.

For TLS SNI to work correctly, M2Crypto should be patched:
https://bugzilla.osafoundation.org/show_bug.cgi?id=13073
For unpatched M2Crypto osc degrades to operation without TLS SNI.

Signed-off-by: Oleg Girko <ol@xxxxxxxxxxxxx>
---
osc/oscssl.py | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/osc/oscssl.py b/osc/oscssl.py
index 72a0dde..2bf1358 100644
--- a/osc/oscssl.py
+++ b/osc/oscssl.py
@@ -250,7 +250,12 @@ class myHTTPSConnection(M2Crypto.httpslib.HTTPSConnection):
M2Crypto.httpslib.HTTPSConnection.__init__(self, *args, **kwargs)

def connect(self, *args):
- M2Crypto.httpslib.HTTPSConnection.connect(self, *args)
+ self.sock = SSL.Connection(self.ssl_ctx)
+ if self.session:
+ self.sock.set_session(self.session)
+ if hasattr(self.sock, 'set_tlsext_host_name'):
+ self.sock.set_tlsext_host_name(self.host)
+ self.sock.connect((self.host, self.port))
verify_certificate(self)

def getHost(self):
--
1.8.3.1

< Previous Next >
This Thread
Follow Ups