On Wed, Nov 18, 2015 at 10:07 AM, Roger Oberholtzer wrote:

Perhaps bad form to reply to one's own post. I might add that google was no help with a search limited to ZYPP_MULTICURL and leap. I don't mind looking for this in the source to see what is going on. But I do not know where to look. I doubt it is in zypper, as it applies to yast as well.

libzypp? https://github.com/openSUSE/libzypp -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Wed, Nov 18, 2015 at 8:28 AM, Per Jessen wrote:

Roger Oberholtzer wrote:

...

Perhaps bad form to reply to one's own post. I might add that google was no help with a search limited to ZYPP_MULTICURL and leap.

It's probably a little early to expect any useful results on that search. When I googled "ZYPP_MULTICURL", the first hit was

http://lists.opensuse.org/opensuse/2015-11/msg00442.html :-)

The 2nd hit weas about libzypp, as Andrei suggested.

I see that ZYPP_MULTICURL is still checked, and it seems that if it is true (not set to 0), the connection is made via MediaMultiCurl::MediaMultiCurl, which I would suspect is the same as it has been. The problem I am having is that I have installed Leap in a VirtualBox. I have done this before with other versions of openSUSE, and used zypper without issue. We always make one of these as the 'official' build environment of the various releases we support. When I try to install some packages that we want in this VBox, the status when downloading is crazy. It goes up and down in percent. In larger packages, it never to completes, and zypper fails. This looks like what I saw when we did not use ZYPP_MULTICURL=0. Our systems are behind a Cisco firewall (called something like ironhorse, if memory serves). It wants to download the entire item on it's own and check for virus before passing the bits to the client. When libzypp makes multiple connections, this causes problems. Earlier discussions here came to the conclusion that ZYPP_MULTICURL=0 could help. And, up until Leap it has. -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Wed, Nov 18, 2015 at 9:21 AM, Andrei Borzenkov wrote:

On Wed, Nov 18, 2015 at 11:17 AM, Roger Oberholtzer wrote:

...

multiple connections, this causes problems. Earlier discussions here came to the conclusion that ZYPP_MULTICURL=0 could help. And, up until Leap it has.

If you want to check Leap version it is here: https://build.opensuse.org/package/show/openSUSE:Leap:42.1/libzypp

That is the version I checked. And as I wrote, it looks like ZYPP_MULTICURL=0 is still accepted. So perhaps the issue is elsewhere. It is just that the behavior looked so similar to what I saw when not using ZYPP_MULTICURL=0 -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Wed, Nov 18, 2015 at 9:31 AM, Per Jessen wrote:

Wouldn't this also happen during a normal installation? (it sounds to me as if ZYPP_MULTICURL=0 cures the symptom, but not the problem).

I did the install from an ISO image. I did not choose to do updates at that time because I knew I could not specify ZYPP_MULTICURL=0 to the installer. So I thought I would do the updates after install. It is big RPMs that seem to be most effected. For example, my update wants texlive-cm-super-fonts. It gets to +90% percent, and then immediately fails, retires to 90%, fails, etc. If I tell it to ignore the failed RPM it continues to download the others. But of course the install will fail because an RPM is missing... -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Roger Oberholtzer wrote:

On Wed, Nov 18, 2015 at 9:31 AM, Per Jessen wrote:

...

Wouldn't this also happen during a normal installation? (it sounds to me as if ZYPP_MULTICURL=0 cures the symptom, but not the problem).

I did the install from an ISO image. I did not choose to do updates at that time because I knew I could not specify ZYPP_MULTICURL=0 to the installer. So I thought I would do the updates after install.

Ah, I see - I am so unaccustomed to installing from physical media that I forget they exist :-(

It is big RPMs that seem to be most effected.

Big files are often broken into multiple downloads, one segment each. For instance, if an rpm is 16Mb, it could be downloaded as 16 x 1Mb segments from 16 different mirrors. It played merry hell with our squid cache until I figured out what was going on.

For example, my update wants texlive-cm-super-fonts. It gets to +90% percent, and then immediately fails, retires to 90%, fails, etc.

I'll have to check one of my machines, but others should be seeing this too. -- Per Jessen, Zürich (15.2°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Roger Oberholtzer wrote:

On Wed, Nov 18, 2015 at 11:30 AM, Per Jessen wrote:

...

Big files are often broken into multiple downloads, one segment each. For instance, if an rpm is 16Mb, it could be downloaded as 16 x 1Mb segments from 16 different mirrors. It played merry hell with our squid cache until I figured out what was going on.

But the Cisco Ironport firewall gets the full file on it's own and then repackages it for the client as needed.

I'm surprised Ironport can work that out, it's fairly complex. You're certain it does that? Ah, maybe it does indeed fetch the whole file first, but that would be ridiculous. If a 16Mb file is split into 16 segments, that could potentially cause 16 full downloads :-)

IIRC, the Ironport does not deal correctly with some HTTP directives correctly set by zypper/curl when this is happening. So it breaks things. zypper/curl are not the only victims of the Cisco Ironport bug.

Aha, that's interesting. zypper is certainly not the only one to use the segmented download, but the only other one I can think of right now is youtube. Anyway, yes, it sounds like the ZYPP_MULTICURL option is in fact the solution you need. -- Per Jessen, Zürich (15.8°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Per Jessen wrote:

Roger Oberholtzer wrote:

...

On Wed, Nov 18, 2015 at 11:30 AM, Per Jessen wrote:

...

Big files are often broken into multiple downloads, one segment each. For instance, if an rpm is 16Mb, it could be downloaded as 16 x 1Mb segments from 16 different mirrors. It played merry hell with our squid cache until I figured out what was going on.

But the Cisco Ironport firewall gets the full file on it's own and then repackages it for the client as needed.

I'm surprised Ironport can work that out, it's fairly complex. You're certain it does that? Ah, maybe it does indeed fetch the whole file first, but that would be ridiculous. If a 16Mb file is split into 16 segments, that could potentially cause 16 full downloads :-)

...

IIRC, the Ironport does not deal correctly with some HTTP directives correctly set by zypper/curl when this is happening. So it breaks things. zypper/curl are not the only victims of the Cisco Ironport bug.

Aha, that's interesting. zypper is certainly not the only one to use the segmented download, but the only other one I can think of right now is youtube.

Anyway, yes, it sounds like the ZYPP_MULTICURL option is in fact the solution you need.

To test whether it works, if you can get to run tcpdump/wireshark, you should be able to see the http requests and the 206 responses. The 206 response is a clear indicator that segmenting is active. -- Per Jessen, Zürich (15.9°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org