[opensuse] zypper and a couple RPMs

Roger Oberholtzer

4 Jan 2017 4 Jan '17

11:48

I have a strange issue with zypper and RPMS from OBS. At first I thought it was related to the aarch64 repos I was using. Now I see the same files have problems on other openSUSE versions when zypper is trying to retrieve them. What happens is that most all of the file is downloaded. But the download status stays at 99% for a bit, and then zypper fails and wants to to retry the download. It never succeeds. It is always the same files that fail from different zypper clients: http://download.opensuse.org/tumbleweed/repo/oss/suse/noarch/kernel-firmware... http://download.opensuse.org/tumbleweed/repo/oss/suse/noarch/php5-ZendFramew... So, I downloaded them in my browser (on the same computer where zypper is failing) and that works. The files are probably okay. So, I thought I would put the files in /var/cache/zypp/packages/repo-oss/suse/noarch so the files would not need to be retrieved again. All the other files waiting to be installed are there, and zypper does not try to get those again. Seems just adding these files here does not make zypper see them. I guess that's too easy. It tries to retrieve them again. Is there any way to get zypper to use these files instead of trying to download them again? In one case I am doing a zypper dup from 13.1 to Tumbleweed. So there are thousands of files involved. Getting the dup to ignore needing these files seems like it would cause other issues. Help! -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Show replies by date

Lew Wolfgang

4 Jan 4 Jan

15:29

On 01/04/2017 03:48 AM, Roger Oberholtzer wrote:

...

I have a strange issue with zypper and RPMS from OBS. At first I thought it was related to the aarch64 repos I was using. Now I see the same files have problems on other openSUSE versions when zypper is trying to retrieve them.

What happens is that most all of the file is downloaded. But the download status stays at 99% for a bit, and then zypper fails and wants to to retry the download. It never succeeds.

It is always the same files that fail from different zypper clients:

http://download.opensuse.org/tumbleweed/repo/oss/suse/noarch/kernel-firmware...

http://download.opensuse.org/tumbleweed/repo/oss/suse/noarch/php5-ZendFramew...

I've seen this too, on multiple non-TW boxes, but with different rpms. It makes installs/upgrades real ordeals. I've noticed that after zypper stalls at 99% for a while, a single ^C will back it out to an abort/retry dialog. Selecting "retry" at this point will frequently allow zypper to complete. But not always. I've seen updates after fresh installs take hours because of this issue, which started about one-month ago. I've been thinking it's something in the network security stack where I'm working, but if others are seeing it? I'll take notes the next time it happens. Regards, Lew -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R.

16:02

On 2017-01-04 12:48, Roger Oberholtzer wrote:

...

I have a strange issue with zypper and RPMS from OBS. At first I thought it was related to the aarch64 repos I was using. Now I see the same files have problems on other openSUSE versions when zypper is trying to retrieve them.

What happens is that most all of the file is downloaded. But the download status stays at 99% for a bit, and then zypper fails and wants to to retry the download. It never succeeds.

One cause for it would be that you are on a corporate network, which does antivir analysis and these files trigger a positive.

...

So, I downloaded them in my browser (on the same computer where zypper is failing) and that works. The files are probably okay.

There is a checksum. Check them.

...

So, I thought I would put the files in /var/cache/zypp/packages/repo-oss/suse/noarch so the files would not need to be retrieved again. All the other files waiting to be installed are there, and zypper does not try to get those again.

Should work. Maybe the checksum fails?

...

Seems just adding these files here does not make zypper see them. I guess that's too easy. It tries to retrieve them again.

Is there any way to get zypper to use these files instead of trying to download them again?

Try to install/update them with "rpm" manually. Or create a local directory as repository, tell YaST, and put the files there. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)

Roger Oberholtzer

5 Jan 5 Jan

09:50

On Wed, Jan 4, 2017 at 5:02 PM, Carlos E. R. wrote:

...

One cause for it would be that you are on a corporate network, which does antivir analysis and these files trigger a positive.

Indeed. There is a Cisco Ironport sniffing everything. I can get the files elsewhere.

...

...
So, I thought I would put the files in /var/cache/zypp/packages/repo-oss/suse/noarch so the files would not need to be retrieved again. All the other files waiting to be installed are there, and zypper does not try to get those again.

Should work. Maybe the checksum fails?

...
Seems just adding these files here does not make zypper see them. I guess that's too easy. It tries to retrieve them again.

Is there any way to get zypper to use these files instead of trying to download them again?

Try to install/update them with "rpm" manually.

Not possible because they require other things be installed. Which is what I am trying to do...

...

Or create a local directory as repository, tell YaST, and put the files there.

I put the files (obtained outside the corporate network - sneakernet prevailed!) in a local directory, and added that as a repo. Then I 'downloaded' them with zypper -d. After that, the zypper dup completed as the files were now happily present. My 13.1 system is now Tumbleweed. -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Per Jessen

10:10

Roger Oberholtzer wrote:

...

On Wed, Jan 4, 2017 at 5:02 PM, Carlos E. R. wrote:

...
One cause for it would be that you are on a corporate network, which does antivir analysis and these files trigger a positive.

Indeed. There is a Cisco Ironport sniffing everything. I can get the files elsewhere.

Ironport used to be for email (anti-spam/virus) only, but maybe they've expanded it since Cisco took over. -- Per Jessen, Zürich (0.1°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R.

6 Jan 6 Jan

03:30

On 2017-01-05 10:50, Roger Oberholtzer wrote:

...

On Wed, Jan 4, 2017 at 5:02 PM, Carlos E. R. wrote:

...
One cause for it would be that you are on a corporate network, which does antivir analysis and these files trigger a positive.

Indeed. There is a Cisco Ironport sniffing everything. I can get the files elsewhere.

Ah. Ironport. Rings a bell.

...

...
...
Is there any way to get zypper to use these files instead of trying to download them again?

Try to install/update them with "rpm" manually.

Not possible because they require other things be installed. Which is what I am trying to do...

Oh, with "rpm" you can always force it.

...

...
Or create a local directory as repository, tell YaST, and put the files there.

I put the files (obtained outside the corporate network - sneakernet prevailed!) in a local directory, and added that as a repo. Then I 'downloaded' them with zypper -d. After that, the zypper dup completed as the files were now happily present. My 13.1 system is now Tumbleweed.

He. Good :-) Did you compare those rpms with those that you downloaded according to the "rules"? -- Cheers/Saludos Carlos E. R. (testing openSUSE Leap 42.2, at Minas-Anor) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Roger Oberholtzer

9 Jan 9 Jan

07:15

On Fri, Jan 6, 2017 at 4:30 AM, Carlos E. R. wrote:

...

Did you compare those rpms with those that you downloaded according to the "rules"?

I tried wget internally. It also failed. But it left the file for me to play with. They were different. No idea why. -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R.

14:44

On 2017-01-09 08:15, Roger Oberholtzer wrote:

...

On Fri, Jan 6, 2017 at 4:30 AM, Carlos E. R. <> wrote:

...
Did you compare those rpms with those that you downloaded according to the "rules"?

I tried wget internally. It also failed. But it left the file for me to play with. They were different. No idea why.

Different size, or same size but different content? First case, an antivirus blocked the full file (or timed out). Second case, an antivirus tried to "repair" the virus in the file. Another test you can do is download the file using aria2c and the metalink url. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)

Roger Oberholtzer

14:58

Slightly smaller. Bigger files get downloaded ok. Odd that it picked on these two so consistently and from different mirrors and by different clients. But all in the company network. On Mon, Jan 9, 2017 at 3:44 PM, Carlos E. R. wrote:

...

On 2017-01-09 08:15, Roger Oberholtzer wrote:

...
On Fri, Jan 6, 2017 at 4:30 AM, Carlos E. R. <> wrote:

...
Did you compare those rpms with those that you downloaded according to the "rules"?

I tried wget internally. It also failed. But it left the file for me to play with. They were different. No idea why.

Different size, or same size but different content?

First case, an antivirus blocked the full file (or timed out). Second case, an antivirus tried to "repair" the virus in the file.

Another test you can do is download the file using aria2c and the metalink url.

-- Cheers / Saludos,

Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)

-- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R.

18:46

On 2017-01-09 15:58, Roger Oberholtzer wrote:

...

Slightly smaller. Bigger files get downloaded ok. Odd that it picked on these two so consistently and from different mirrors and by different clients. But all in the company network.

No, not odd at all. Those files trigger a positive in the company antivirus, or trigger "analyze in depth". I mentioned metalinks because it can download chunks from different servers, and compares the checksum to download if it sees an error. This could play havoc with the proxy, and never finish or abort in error. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)

Roger Oberholtzer

10 Jan 10 Jan

09:14

I have to disable this kind of thing with ZYPP_MULTICURL. Otherwise nothing works. So I think it is getting each file from a single server. On Mon, Jan 9, 2017 at 7:46 PM, Carlos E. R. wrote:

...

On 2017-01-09 15:58, Roger Oberholtzer wrote:

...
Slightly smaller. Bigger files get downloaded ok. Odd that it picked on these two so consistently and from different mirrors and by different clients. But all in the company network.

No, not odd at all. Those files trigger a positive in the company antivirus, or trigger "analyze in depth".

I mentioned metalinks because it can download chunks from different servers, and compares the checksum to download if it sees an error. This could play havoc with the proxy, and never finish or abort in error.

-- Cheers / Saludos,

Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)

-- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R.

09:25

On 2017-01-10 10:14, Roger Oberholtzer wrote:

...

I have to disable this kind of thing with ZYPP_MULTICURL. Otherwise nothing works. So I think it is getting each file from a single server.

Yes, but my idea is that you run the aria2c command yourself, not zypper, with a metalink url, to find out how it copes in that network of yours. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)

Roger Oberholtzer

09:37

On Tue, Jan 10, 2017 at 10:25 AM, Carlos E. R. wrote:

...

On 2017-01-10 10:14, Roger Oberholtzer wrote:

...
I have to disable this kind of thing with ZYPP_MULTICURL. Otherwise nothing works. So I think it is getting each file from a single server.

Yes, but my idea is that you run the aria2c command yourself, not zypper, with a metalink url, to find out how it copes in that network of yours.

I could do this. I just need to figure what the command is. I'll report back when I get a chance. Odd that no one else has report a similar problem. I cannot think our company has any special virus detection in place. If that is even what it is. -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R.

09:46

On 2017-01-10 10:37, Roger Oberholtzer wrote:

...

On Tue, Jan 10, 2017 at 10:25 AM, Carlos E. R. wrote:

...
On 2017-01-10 10:14, Roger Oberholtzer wrote:

...
I have to disable this kind of thing with ZYPP_MULTICURL. Otherwise nothing works. So I think it is getting each file from a single server.

Yes, but my idea is that you run the aria2c command yourself, not zypper, with a metalink url, to find out how it copes in that network of yours.

I could do this. I just need to figure what the command is. I'll report back when I get a chance.

Just "aria2c URL". To find out the URL, you go to the download page for the rpm, click on "details", and search for "Metalinks for reliable downloads: ". Pick the ".meta4" one, the first line probably.

...

Odd that no one else has report a similar problem. I cannot think our company has any special virus detection in place. If that is even what it is.

The ironclad thing. Ironport, sorry. Not the first time I hear of this problem related to that thing. https://en.wikipedia.org/wiki/IronPort -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)

2671

Age (days ago)

2677

Last active (days ago)

List overview

Download

13 comments

4 participants

participants (4)

Carlos E. R.
Lew Wolfgang
Per Jessen
Roger Oberholtzer

[opensuse] zypper and a couple RPMs

tags

participants (4)