[opensuse] update repo very slow
This morning trying to refresh my repositories, the refresh was getting hung up on updating the repo at http://download.opensuse.org/update/leap/42.2/oss/repodata/. I switched from zypper to yast, and started monitoring the update, and it turns out this file was the one that was going very slow, in the repodata subdirectory. http://download.opensuse.org/update/leap/42.2/oss/repodata/cfa171ff1eeec497b... So why would that be? The file is listed at a size of 13M, and another file listed at 9.2M seemed to take only the normal time of a few seconds to download. Is there some problem with the update repo? -- George Box: 42.2 | KDE Plasma 5.8 | AMD Phenom IIX4 | 64 | 32GB Laptop #1: 42.2 | KDE Plasma 5.8 | AMD FX 7TH GEN | 64 | 32GB Laptop #2: 42.2 | KDE Plasma 5.8 | Core i5 | 64 | 8GB -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 07/25/2017 11:26 AM, George from the tribe wrote:
This morning trying to refresh my repositories, the refresh was getting hung up on updating the repo at http://download.opensuse.org/update/leap/42.2/oss/repodata/.
I switched from zypper to yast, and started monitoring the update, and it turns out this file was the one that was going very slow, in the repodata subdirectory.
http://download.opensuse.org/update/leap/42.2/oss/repodata/cfa171ff1eeec497b...
So why would that be? The file is listed at a size of 13M, and another file listed at 9.2M seemed to take only the normal time of a few seconds to download. Is there some problem with the update repo?
To be more clear, it took around 45 minutes just to update that single xml.gz file, while the other xml.gz file (the 9.2M) that was in that same repo could not have taken more than 5 minutes, probably more like 2 minutes. -- George Box: 42.2 | KDE Plasma 5.8 | AMD Phenom IIX4 | 64 | 32GB Laptop #1: 42.2 | KDE Plasma 5.8 | AMD FX 7TH GEN | 64 | 32GB Laptop #2: 42.2 | KDE Plasma 5.8 | Core i5 | 64 | 8GB -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 24/07/17 11:32 PM, George from the tribe wrote:
To be more clear, it took around 45 minutes just to update that single xml.gz file, while the other xml.gz file (the 9.2M) that was in that same repo could not have taken more than 5 minutes, probably more like 2 minutes.
I can normally download the larger xml files, the larger RPMs at speeds that reach as much as 600kB/s. Of course that's not going to happen for a smaller file :-) But today I got speeds that never went over 100kB/s, in fact rarely over 50kB/s. The fooloing is scraped and reformatted, but the numbers are preserved. Retrieving: libMagickCore-7_Q16HDRI2-7.0.6.0-303.3.x86_64.rpm[done (23.1 KiB/s)] Retrieving: rawtherapee-5.2-38.1.x86_64.rpm .................[done (20.2 KiB/s)] Retrieving: libMagickWand-7_Q16HDRI0-7.0.6.0-303.3.x86_64.rpm[done (21.2 KiB/s)] Retrieving: ImageMagick-7.0.6.0-303.3.x86_64.rpm ............[done (29.4 KiB/s)] Retrieving: transcode-1.1.7-4.1.x86_64.rpm ..................[done (36.4 KiB/s)] Retrieving: youtube-dl-2017.07.23-1.1.noarch.rpm ............[done (32.9 KiB/s)] Retrieving: libmpg123-0-1.25.3-8.1.x86_64.rpm ...............[done (26.4 KiB/s)] Retrieving: libout123-0-1.25.3-8.1.x86_64.rpm ...............[done (6.7 KiB/s)] Retrieving: mpg123-1.25.3-8.1.x86_64.rpm ....................[done (27.2 KiB/s)] Retrieving: kernel-default-4.12.3-2.1.g880dbd7.x86_64.rpm ...[done (39.7 KiB/s)] Retrieving: kernel-macros-4.12.3-2.1.g880dbd7.noarch.rpm ....[done (97.0 KiB/s)] Retrieving: release-notes-openSUSE-42.2.20170721-5.3.1.noarch.rpm [done (58.6 KiB/s)] Retrieving: ucode-intel-20170511_20170707-7.3.1_7.6.1.x86_64.drpm [done (125.6 KiB/s)] Retrieving: libevdocument3-4-3.20.1-2.3.1.x86_64.rpm ........[done (67.4 KiB/s)] Retrieving: libevview3-3-3.20.1-1.5_2.3.1.x86_64.drpm .......[done (47.6 KiB/s)] Retrieving: evince-3.20.1-1.5_2.3.1.x86_64.drpm .............[done (48.5 KiB/s)] Retrieving: evince-plugin-psdocument-3.20.1-2.3.1.x86_64.rpm [done (32.5 KiB/s)] Retrieving: evince-lang-3.20.1-1.5_2.3.1.noarch.drpm ........[done (51.9 KiB/s)] This succeeded this morning. It failed over the weekend, dropped connections & timeouts, DNS problms. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 07/24/2017 10:32 PM, George from the tribe wrote:
On 07/25/2017 11:26 AM, George from the tribe wrote:
This morning trying to refresh my repositories, the refresh was getting hung up on updating the repo at http://download.opensuse.org/update/leap/42.2/oss/repodata/.
I switched from zypper to yast, and started monitoring the update, and it turns out this file was the one that was going very slow, in the repodata subdirectory.
http://download.opensuse.org/update/leap/42.2/oss/repodata/cfa171ff1eeec497b...
So why would that be? The file is listed at a size of 13M, and another file listed at 9.2M seemed to take only the normal time of a few seconds to download. Is there some problem with the update repo?
To be more clear, it took around 45 minutes just to update that single xml.gz file, while the other xml.gz file (the 9.2M) that was in that same repo could not have taken more than 5 minutes, probably more like 2 minutes.
George, There is a US mirror problem that I hit the other night as well. (it took over 42 minutes to download what normally would occur in 2 minutes). I haven't drilled down which mirror, but I suspect it is the Rochester Inst. of Tech. mirror. RIT has been having hosting issues lately. (I ran into that with missing links to works by one of the RIT faculty a couple of days ago on a C matter). In the interim, pick one of the update mirrors that works for you and just create a new update_whatever.repo file in /etc/zypp/repos.d, then just call it by name (e.g. whatever you put between [....]) with zypper up -r .... (where .... is the name for the repo) and use that mirror directly. Pick your mirror from one of: http://download.opensuse.org/update/leap/42.2/oss/openSUSE:Leap:42.2:Update.... Good luck.... -- David C. Rankin, J.D.,P.E. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 25/07/17 03:49 AM, David C. Rankin wrote:
In the interim, pick one of the update mirrors that works for you and just create a new update_whatever.repo file in /etc/zypp/repos.d, then just call it by name (e.g. whatever you put between [....]) with zypper up -r .... (where .... is the name for the repo) and use that mirror directly.
Pick your mirror from one of:
http://download.opensuse.org/update/leap/42.2/oss/openSUSE:Leap:42.2:Update....
"I'm sorry, Dave ..." I'm well aware of the other mirrors As another thread mentioned, I am capable of going to a repository and downloading by hand a specific RPM using wget or cURL, but that's not the way most of us work. and not a healthy way to update. It's tedious and a 'step and repeat process. Most ff us use Yast or Zypper. Sadly, that means mirrorbrain makes the selection for us. I'm unaware of any way I can load up /etc/zypp/<somenthing> to blacklist a specific repository or prioritize others. I hope there is some mechanism available. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 07/25/2017 06:03 AM, Anton Aylward wrote:
Sadly, that means mirrorbrain makes the selection for us.
Is there no way to influence this mirrorbrain thingie? (Isn't mirrorbrain only used for Downloading Opensuse ISO's? Isn't something else used for zypper/yast repository ramdomization?) I know in prior releases of opensuse, you could pick one or a few mirrors to use. I know other distros still have this capability. I believe some countries had (maybe still) laws requiring in-country mirrors. So did opensuse abandon the ability to pick a country, pick a mirror, and are we locked into random mirror engine that can't detect slow mirrors? -- After all is said and done, more is said than done. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-07-25 19:26, John Andersen wrote:
On 07/25/2017 06:03 AM, Anton Aylward wrote:
Sadly, that means mirrorbrain makes the selection for us.
Is there no way to influence this mirrorbrain thingie?
(Isn't mirrorbrain only used for Downloading Opensuse ISO's? Isn't something else used for zypper/yast repository ramdomization?)
I know in prior releases of opensuse, you could pick one or a few mirrors to use. I know other distros still have this capability. I believe some countries had (maybe still) laws requiring in-country mirrors.
So did opensuse abandon the ability to pick a country, pick a mirror, and are we locked into random mirror engine that can't detect slow mirrors?
You can write a mirror in the config. The easiest manner (IMO) is editing the file: /etc/zypp/repos.d/download.opensuse.org-oss.repo: [download.opensuse.org-oss] name=Main Repository (OSS) enabled=1 autorefresh=1 baseurl=http://download.opensuse.org/distribution/leap/42.2/repo/oss/ path=/ type=yast2 keeppackages=0 Just comment out the "baseurl" line and write your own. This is not detected as a repo change by yast/zypper. I knew of a way to alter that and use your own local redirector, using a local apache server - but I don't recall how, I used that two years ago, I think, for packman. I may be able to find that out another day if you are interested. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" (Minas Tirith))
On 25/07/17 01:59 PM, Carlos E. R. wrote:
Just comment out the "baseurl" line and write your own. This is not detected as a repo change by yast/zypper.
Yes, I understand. That locks it in to one and only one specific repository. If that fails, if that performs badly, if there's network problems getting there, then you're screwed. it does NOT alter the priorities over the set with which mirrorbrain works and does not offer you, dynamically, a choice of alternatives if the one selected by mirrorbrain performs poorly. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-07-25 21:25, Anton Aylward wrote:
On 25/07/17 01:59 PM, Carlos E. R. wrote:
Just comment out the "baseurl" line and write your own. This is not detected as a repo change by yast/zypper.
Yes, I understand. That locks it in to one and only one specific repository. If that fails, if that performs badly, if there's network problems getting there, then you're screwed.
it does NOT alter the priorities over the set with which mirrorbrain works and does not offer you, dynamically, a choice of alternatives if the one selected by mirrorbrain performs poorly.
Yes, of course. In that case, you can do this - I used it with packman: /etc/zypp/repos.d/ftp.gwdg.de-suse.repo: [EXT_Packman] name=EXT: Packman Repository enabled=1 autorefresh=1 path=/ mirrorlist=http://localhost/packmanmirrorlist.file type=rpm-md priority=95 keeppackages=1 Notice that I use "mirrorlist" instead, and it has to be an URL. At the time I did it, it had to be http://, there was talk of accepting file:///, but I don't know if that feature addition has been implemented or not. On /srv/www/htdocs/packmanmirrorlist.file I had: http://mirror.karneval.cz/pub/linux/packman/suse/openSUSE_13.1/ http://ftp.gwdg.de/pub/linux/packman/suse/openSUSE_13.1/ http://packman.jacobs-university.de/suse/openSUSE_13.1/ which acted as a random redirector if I recall correctly. And of course, you need one file per repo, on apache. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
Anton Aylward composed on 2017-07-25 15:25 (UTC-0400):
Carlos E. R. wrote:
Just comment out the "baseurl" line and write your own. This is not detected as a repo change by yast/zypper.
Yes, I understand. That locks it in to one and only one specific repository. If that fails, if that performs badly, if there's network problems getting there, then you're screwed.
Only until you change it to something else that works as expected. -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 26/07/17 12:29 AM, Felix Miata wrote:
Anton Aylward composed on 2017-07-25 15:25 (UTC-0400):
Carlos E. R. wrote:
Just comment out the "baseurl" line and write your own. This is not detected as a repo change by yast/zypper.
Yes, I understand. That locks it in to one and only one specific repository. If that fails, if that performs badly, if there's network problems getting there, then you're screwed.
Only until you change it to something else that works as expected.
All this is requiring too much manual intervention, too much specialized investigation and preparation. It may be fine for the obsessive SA types, but I though the whole point of mirrorbrain was to avoid keeping stacks of repos and manually switching between the, and select, automatically, what is "good for you". The problem seems to lie in the fixed attitude of its algorithm which is insensitive to the performance of the server .. or something. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 07/26/2017 06:31 AM, Anton Aylward wrote:
The problem seems to lie in the fixed attitude of its algorithm which is insensitive to the performance of the server .. or something.
Exactly. It was supposed to spare us all getting a lame server. Said so in the writeups about it. It was also supposed to support Geo-location for picking close mirrors, which is a pointless exercise. We are internet-close to everywhere. We need mirror brain to use two criteria: Fast delivery and Up-to-date. Nothing else matters. Ping time doesn't matter. It might take me longer to ping a server in Germany, but if it can deliver faster than some crippled mirror at the nearest university, then that's the on I want. -- After all is said and done, more is said than done. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 27/07/17 12:54 PM, John Andersen wrote:
It might take me longer to ping a server in Germany, but if it can deliver faster than some crippled mirror at the nearest university, then that's the on I want.
+1 Internet geodesics and physical geodesics are beside the point. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
I knew of a way to alter that and use your own local redirector, using a local apache server - but I don't recall how, I used that two years ago, I think, for packman. I may be able to find that out another day if you are interested.
As I'm having quite some Leap machines here I rsync the repo on our main server and then use this via nfs-mounted directory, baseurl=dir:///extern/opensuse/update/leap/42.2/oss (there's also a way to use nfs://<server>/<dir> but I lack the correct syntax ATM) For the machine at home I rsync that repo to an external USB SSD, and use baseurl=hd:///update/leap/42.2/oss?device=/dev/disk/by-label/Leap-repos&filesystem=auto (only plug it in - zypper automounts it when needed...) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 07/26/2017 01:59 AM, Carlos E. R. wrote:
You can write a mirror in the config. The easiest manner (IMO) is editing the file:
/etc/zypp/repos.d/download.opensuse.org-oss.repo:
[download.opensuse.org-oss] name=Main Repository (OSS) enabled=1 autorefresh=1 baseurl=http://download.opensuse.org/distribution/leap/42.2/repo/oss/ path=/ type=yast2 keeppackages=0
Just comment out the "baseurl" line and write your own. This is not detected as a repo change by yast/zypper.
I knew of a way to alter that and use your own local redirector, using a local apache server - but I don't recall how, I used that two years ago, I think, for packman. I may be able to find that out another day if you are interested.
Finally have a chance to follow up on this thread. Right after I posted this thread, a friend was in a motorcycle accident and I had to take about a week off to take care of her at the hospital. Since that time, it seems that my original problem has been fixed with regard to the mirrors. I have another repo issue now with 42.3, but I think I will start a new thread for that one, since it is similar but a new issue. Thanks to everyone that responded. A lot of this is above my head, but it was helpful to gain some knowledge of the problem. -- George Box: 42.2 | KDE Plasma 5.8 | AMD Phenom IIX4 | 64 | 32GB Laptop #1: 42.2 | KDE Plasma 5.8 | AMD FX 7TH GEN | 64 | 32GB Laptop #2: 42.2 | KDE Plasma 5.8 | Core i5 | 64 | 8GB -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 25/07/17 23:03, Anton Aylward wrote:
On 25/07/17 03:49 AM, David C. Rankin wrote:
In the interim, pick one of the update mirrors that works for you and just create a new update_whatever.repo file in /etc/zypp/repos.d, then just call it by name (e.g. whatever you put between [....]) with zypper up -r .... (where .... is the name for the repo) and use that mirror directly.
Pick your mirror from one of:
http://download.opensuse.org/update/leap/42.2/oss/openSUSE:Leap:42.2:Update.... "I'm sorry, Dave ..."
I'm well aware of the other mirrors As another thread mentioned, I am capable of going to a repository and downloading by hand a specific RPM using wget or cURL, but that's not the way most of us work. and not a healthy way to update. It's tedious and a 'step and repeat process. Most ff us use Yast or Zypper.
Sadly, that means mirrorbrain makes the selection for us.
I'm unaware of any way I can load up /etc/zypp/<somenthing> to blacklist a specific repository or prioritize others.
I hope there is some mechanism available.
Whatever happened to Smart? Didn't that have the ability to do all sorts of marvellous things including, I think, what is talked about above? BC -- You are NOT entitled to your opinion. You are entitled to your INFORMED opinion. Nobody is entitled to be ignorant. Harlan Ellison -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (8)
-
Anton Aylward -
Basil Chupin -
Carlos E. R. -
David C. Rankin -
Felix Miata -
George from the tribe -
John Andersen -
Peter Suetterlin