[mirror] HTTPS support for Mirrors
Hi folks, We are currently working on more https enablement for the openSUSE download tooling. Background is that safe repository usage is possible for the main and update repositories, but not for the rest buildservice repositories, which have unknown keys. software.opensuse.org and download.opensuse.org can provide https URLs already. The problem we are facing is that download.opensuse.org redirects from HTTPS to HTTP, which is caught by various clients as a security violation. So to avoid making clients less secure, we would like to know if some of you can offer https serving mirrors of openSUSE. With the availability of letsencrypt this can even be done without any additional costs. A low footprint client for letsencrypt is dehydrated (https://github.com/lukas2511/dehydrated), which can also be obtained as packages for openSUSE, SLES, CentOS and RHEL from https://software.opensuse.org/download.html?project=security:dehydrated&package=dehydrated. Ciao, Marcus
Am 10.10.2017 um 14:19 schrieb Marcus Meissner:
So to avoid making clients less secure, we would like to know if some of you can offer https serving mirrors of openSUSE.
How do you want to know? As a reply on this list? In this case: yes we can ;) or +1 respectively But as I write this.. AFAIK you do already have a probing mechanism in place. Why dont you generally check https (with a deep URL) availability and set a flag in mirrorbrain or whatever? Matthias -- Dipl.-Inf. Matthias Hunstock UniRZ der TU Ilmenau, Raum 07 Tel.: +49 3677 69-1289 -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
On Tue, Oct 10, 2017 at 02:27:01PM +0200, Matthias Hunstock wrote:
Am 10.10.2017 um 14:19 schrieb Marcus Meissner:
So to avoid making clients less secure, we would like to know if some of you can offer https serving mirrors of openSUSE.
How do you want to know? As a reply on this list? In this case: yes we can ;) or +1 respectively
But as I write this.. AFAIK you do already have a probing mechanism in place. Why dont you generally check https (with a deep URL) availability and set a flag in mirrorbrain or whatever?
Hmm, yes, I think we could do that. Not sure how much work mirrorbrain needs to handle this new distinction ;) Ciao, Marcus -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
On Tue, 10 Oct 2017 14:27:01 +0200
Matthias Hunstock
Am 10.10.2017 um 14:19 schrieb Marcus Meissner:
So to avoid making clients less secure, we would like to know if some of you can offer https serving mirrors of openSUSE.
How do you want to know? As a reply on this list? In this case: yes we can ;) or +1 respectively
But as I write this.. AFAIK you do already have a probing mechanism in place. Why dont you generally check https (with a deep URL) availability and set a flag in mirrorbrain or whatever?
Well finding out *if* a mirror has https we can do ourself sure. but ... If more and more people start using download.o.o via https then we also need to migrate a lot of mirrors to https, to make this useful. So see this as a reminder to enable https if you dont have already and that you can use letsencrypt for a low cost solution to the problem. darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
On Tue, 10 Oct 2017, Marcus Meissner wrote:
On Tue, Oct 10, 2017 at 02:27:01PM +0200, Matthias Hunstock wrote:
Am 10.10.2017 um 14:19 schrieb Marcus Meissner:
So to avoid making clients less secure, we would like to know if some of you can offer https serving mirrors of openSUSE.
How do you want to know? As a reply on this list? In this case: yes we can ;) or +1 respectively
But as I write this.. AFAIK you do already have a probing mechanism in place. Why dont you generally check https (with a deep URL) availability and set a flag in mirrorbrain or whatever?
Hmm, yes, I think we could do that.
Not sure how much work mirrorbrain needs to handle this new distinction ;)
https://mirrors.opensuse.org/ indicates that mirrorbrain is clueless wrt HTTPS as there is only a HTTP column in the list... In any case, ftp.acc.umu.se speaks https since a while back... I think this was communicated to OpenSUSE as well. In fact, we seem to be listed twice as "Academic Computer Club" and "Academic Computer Club Umeå University", which is interesting. Also, ftp.acc.umu.se is highly optimized for http/https. We prefer not to have clickable ftp:// links published if possible. Those really needing ftp can usually figure it out from the host name ;) /Nikke -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | nikke@acc.umu.se --------------------------------------------------------------------------- I am Hamlet of Borg: prepare to be, or not to be... =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
On 10/10/2017 03:00 PM, Marcus Rückert wrote:
Well finding out *if* a mirror has https we can do ourself sure. but ... If more and more people start using download.o.o via https then we also need to migrate a lot of mirrors to https, to make this useful.
Well - did you already try and make some statistics about how many of your existing mirrors already do support HTTPS? I would expect a significant percentage to do so, as other open-source-projects have supported HTTPS for a long time, and many mirror admins rolled it out when letsencrypt became available. We have been offering HTTPS for many years now (with a proper certificate though, not letsencrypt). Regards, -- Michael Meier, FTP-Admin Friedrich-Alexander-Universitaet Erlangen-Nuernberg Regionales Rechenzentrum Erlangen Martensstrasse 1, 91058 Erlangen, Germany Tel.: +49 9131 85-28973, Fax: +49 9131 302941 rrze-ftp-admins@fau.de blogs.fau.de/ftp/ -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
Marcus Meissner (meissner@suse.de) wrote on Tue, Oct 10, 2017 at 09:19:08AM -03:
With the availability of letsencrypt this can even be done without any additional costs.
There's a higher overhead on the machine, so "no additional costs" is an illusion... Further, there's no security bennefit because clients are able to check for hash mismatches using the distribution keys. There's also no privacy protection because if a man-in-the-middle watches your traffic they'll discover what you pulled from the IPs and sizes of the transfers. That's why we (opensuse.c3sl.ufpr.br) are against https for mirrors. However some recent developments are reducing the https burden, notably TLS in the kernel. It's not yet finished but since not all of the distribuitions will change immediately, we can already offer https for some now. This means we can offer https for opensuse (starting next week) if you officially decide that this is your policy and request it from mirrors. -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
On Tue, 10 Oct 2017, Carlos Carvalho wrote:
Marcus Meissner (meissner@suse.de) wrote on Tue, Oct 10, 2017 at 09:19:08AM -03:
With the availability of letsencrypt this can even be done without any additional costs.
There's a higher overhead on the machine, so "no additional costs" is an illusion... Further, there's no security bennefit because clients are able to check for hash mismatches using the distribution keys. There's also no privacy protection because if a man-in-the-middle watches your traffic they'll discover what you pulled from the IPs and sizes of the transfers. That's why we (opensuse.c3sl.ufpr.br) are against https for mirrors.
Our experience is that for mirrors (ie, file-serving) performance is only an issue if you have very good connectivity and/or very old CPUs. As a case in point, ftp.acc.umu.se HP DL180G6 hosts had to be upgraded to 2xE5606 CPU:s (3 EUR each on ebay) in order to get first-generation TLS acceleration. With this upgrade, 10 Gbps https (AES-128-GCM) per host is achieved with lots of spare CPU cycles. Yes, we need 3-4 streams to fill up 10 Gbps but since most users are on GigE or slower in Sweden that's not a big issue... openssl speed -evp aes-128-gcm is the command you want to use to check your TLS CPU performance. Note that you really want to use the gcm cryptos, since that's where the CPU accelleration is really good. I get around 3-4 GiB/s per core on modern CPUs...
However some recent developments are reducing the https burden, notably TLS in the kernel. It's not yet finished but since not all of the distribuitions will change immediately, we can already offer https for some now. This means we can offer https for opensuse (starting next week) if you officially decide that this is your policy and request it from mirrors.
We see this as promising to solve the single-stream/core problem, in order to achieve 100 Gbps single-stream https. For multiple stream workloads there is a benefit in avoiding unnecessary memory copying (think https enabled sendfile()), but it's not really solving any hard bottlenecks for the open source mirror usecase (assuming 10 Gbps connectivity)... The situation is likely different for hosts seeing lots of connections and small transfers, but since we're not in that problem space I don't have any insights there. /Nikke -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | nikke@acc.umu.se --------------------------------------------------------------------------- Water + Malt + Hops + Yeast = Satisfaction =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
Niklas Edmundsson (nikke@acc.umu.se) wrote on Tue, Oct 10, 2017 at 11:31:46AM -03:
For multiple stream workloads there is a benefit in avoiding unnecessary memory copying (think https enabled sendfile()),
Exactly, that's the main problem, not cpu for doing crypto.
The situation is likely different for hosts seeing lots of connections and small transfers, but since we're not in that problem space I don't have any insights there.
Not for an individual distro but the total load in our machine is several thousands continuously, and several tens of thousands in peaks. That's why it's a concern for big mirrors if everybody goes https together without avoiding the dreadful kernel->userspace->kernel memory copy. -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
Hi. Our mirror started serving https since... a few minutes :-) https://espejito.fder.edu.uy/opensuse/ We plan to offer http also. Please let us know about any problem, misconfiguration or recommendation. best regards. ariel El 10/10/17 a las 09:19, Marcus Meissner escribió:
Hi folks,
We are currently working on more https enablement for the openSUSE download tooling.
Background is that safe repository usage is possible for the main and update repositories, but not for the rest buildservice repositories, which have unknown keys.
software.opensuse.org and download.opensuse.org can provide https URLs already.
The problem we are facing is that download.opensuse.org redirects from HTTPS to HTTP, which is caught by various clients as a security violation.
So to avoid making clients less secure, we would like to know if some of you can offer https serving mirrors of openSUSE.
With the availability of letsencrypt this can even be done without any additional costs. A low footprint client for letsencrypt is dehydrated (https://github.com/lukas2511/dehydrated), which can also be obtained as packages for openSUSE, SLES, CentOS and RHEL from https://software.opensuse.org/download.html?project=security:dehydrated&package=dehydrated.
Ciao, Marcus
-- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
Hi all, We've been offering HTTPS access for a long time: https://mirrors.ustc.edu.cn/opensuse/ Another mirror site I know which offers stable HTTPS service: https://mirrors.tuna.tsinghua.edu.cn/opensuse/ Thanks, Boyuan Yang 在 2017年10月10日星期二 CST 下午1:30:10，ariel sabiguero yawelak 写道：
Hi.
Our mirror started serving https since... a few minutes :-)
https://espejito.fder.edu.uy/opensuse/
We plan to offer http also. Please let us know about any problem, misconfiguration or recommendation.
best regards.
ariel
El 10/10/17 a las 09:19, Marcus Meissner escribió:
Hi folks,
We are currently working on more https enablement for the openSUSE download tooling.
Background is that safe repository usage is possible for the main and update repositories, but not for the rest buildservice repositories, which have unknown keys.
software.opensuse.org and download.opensuse.org can provide https URLs already.
The problem we are facing is that download.opensuse.org redirects from HTTPS to HTTP, which is caught by various clients as a security violation.
So to avoid making clients less secure, we would like to know if some of you can offer https serving mirrors of openSUSE.
With the availability of letsencrypt this can even be done without any additional costs. A low footprint client for letsencrypt is dehydrated (https://github.com/lukas2511/dehydrated), which can also be obtained as packages for openSUSE, SLES, CentOS and RHEL from https://software.opensuse.org/download.html?project=security:dehydrated&p ackage=dehydrated.
Ciao, Marcus
Hi all. Only 850 lines on /var/log/apache2/ssl_request_log since it was enabled. Any news on this issue? regards ariel El 10/10/17 a las 13:30, ariel sabiguero yawelak escribió:
Hi.
Our mirror started serving https since... a few minutes :-)
https://espejito.fder.edu.uy/opensuse/
We plan to offer http also. Please let us know about any problem, misconfiguration or recommendation.
best regards.
ariel
El 10/10/17 a las 09:19, Marcus Meissner escribió:
Hi folks,
We are currently working on more https enablement for the openSUSE download tooling.
Background is that safe repository usage is possible for the main and update repositories, but not for the rest buildservice repositories, which have unknown keys.
software.opensuse.org and download.opensuse.org can provide https URLs already.
The problem we are facing is that download.opensuse.org redirects from HTTPS to HTTP, which is caught by various clients as a security violation.
So to avoid making clients less secure, we would like to know if some of you can offer https serving mirrors of openSUSE.
With the availability of letsencrypt this can even be done without any additional costs. A low footprint client for letsencrypt is dehydrated (https://github.com/lukas2511/dehydrated), which can also be obtained as packages for openSUSE, SLES, CentOS and RHEL from https://software.opensuse.org/download.html?project=security:dehydrated&package=dehydrated.
Ciao, Marcus
-- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
ariel sabiguero yawelak wrote:
Hi all.
Only 850 lines on /var/log/apache2/ssl_request_log since it was enabled.
Any news on this issue?
I thought the matter had been settled already. Because many mirrors are unlikely to support SSL, the mirrorbrain software would need updating. Once mirrorbrain has become ssl / non-ssl aware, we can continue the discussion. -- Per Jessen, Zürich (0.1°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
Per Jessen wrote:
ariel sabiguero yawelak wrote:
Hi all.
Only 850 lines on /var/log/apache2/ssl_request_log since it was enabled.
Any news on this issue?
I thought the matter had been settled already. Because many mirrors are unlikely to support SSL, the mirrorbrain software would need updating. Once mirrorbrain has become ssl / non-ssl aware, we can continue the discussion.
See also https://progress.opensuse.org/news/40 for a more detailed explanation. -- Per Jessen, Zürich (1.1°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
participants (9)
-
ariel sabiguero yawelak -
Boyuan Yang -
Carlos Carvalho -
Marcus Meissner -
Marcus Rückert -
Matthias Hunstock -
Michael Meier (FTP Admin) -
Niklas Edmundsson -
Per Jessen