[heroes] Weekends makes me wish I was using a different distro

Felix Miata

18 Jan 2020 18 Jan '20

01:20

I just waited 13.5 minutes for http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/ to completely load in SeaMonkey, 11.5 minutes before it even started to paint anything, 75,987,605 bytes of html. This is just after it took 45 minutes between submitting a new bug, and succeeding to add 3 additional attachments (in 4 parts: 3 15.2 initrds, one too large for acceptance which had to be split) to B.O.O.: http://bugzilla.opensuse.org/show_bug.cgi?id=1161238 http://ftp5.gwdg.de/pub/opensuse/distribution/leap/15.2/repo/oss/x86_64/ took 5 seconds to load html, 2,890,366 bytes. Something is very wrong that opensuse.org is setup to require such gargantuan HTML and 16000% longer load times. -- Evolution as taught in public schools is religion, not science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org

Show replies by date

wnereiz

18 Jan 18 Jan

04:32

Instead of complaining, more investigations can be done when we encounter these kind of issues. BTW, personally I don't think there is a reason to "using a different distro" just because these. So, calm down :) In your case, I found link [3] is super fast to load. [2] is opened with a certain lag, which is already known issue (at least for me). I think our bugzilla need to be updated. It is still in a very old version. But I know it is always decided by SUSE. The most serious problem is [1]. It is too slow to open which is not likely a DNS solving problem. It simply get no response from server. I tried to change the link from 15.2 to 15.1 [4] and it has no problem to load. I guess maybe there are something in [1] casued the low speed. [1] http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/ [2] http://bugzilla.opensuse.org/show_bug.cgi?id=1161238 [3] http://ftp5.gwdg.de/pub/opensuse/distribution/leap/15.2/repo/oss/x86_64/ [4] http://download.opensuse.org/distribution/leap/15.1/repo/oss/x86_64/ On Fri, Jan 17, 2020 at 08:20:57PM -0500, Felix Miata wrote:

...

I just waited 13.5 minutes for http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/ to completely load in SeaMonkey, 11.5 minutes before it even started to paint anything, 75,987,605 bytes of html. This is just after it took 45 minutes between submitting a new bug, and succeeding to add 3 additional attachments (in 4 parts: 3 15.2 initrds, one too large for acceptance which had to be split) to B.O.O.: http://bugzilla.opensuse.org/show_bug.cgi?id=1161238

http://ftp5.gwdg.de/pub/opensuse/distribution/leap/15.2/repo/oss/x86_64/ took 5 seconds to load html, 2,890,366 bytes.

Something is very wrong that opensuse.org is setup to require such gargantuan HTML and 16000% longer load times. -- Evolution as taught in public schools is religion, not science.

Team OS/2 ** Reg. Linux User #211409 ** a11y rocks!

Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org

-- wnereiz -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org

Bernhard M. Wiedemann

10:54

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 18/01/2020 01.20, Felix Miata wrote:

...

I just waited 13.5 minutes for http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/ to completely load in SeaMonkey, 11.5 minutes before it even started to paint anything, 75,987,605 bytes of html.

It was only 47370045 bytes here. I quickly checked with a smaller repo and it took wget 17 seconds to fetch 34kb of html. Then I checked on pontifex2 and it has a load of 100 and disk IO feels very slow with 71 rsyncd processes and repopush doing 100MB/s read + 24MB/s write in total. In addition to that, the leap 15.2/x86_64 dir has 111314 files instead of the expected 20k - eg it had 15 different versions of zypper-docker So cleanup is missing there. I removed some old zypper-docker rpms to see if they remain deleted - and they did. Just doing ls -l zypper-docker* in that dir took 30s. -----BEGIN PGP SIGNATURE----- iF0EARECAB0WIQRk4KvQEtfG32NHprVJNgs7HfuhZAUCXiLj+AAKCRBJNgs7Hfuh ZJ5uAKDhJ4ugyCJnKphVecnjqsJ5sepl5ACg+NgNPQzRFXAFLKYr5TJhBlTIwo0= =llBl -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org

Per Jessen

11:47

Bernhard M. Wiedemann wrote:

...

On 18/01/2020 01.20, Felix Miata wrote:

...
I just waited 13.5 minutes for http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/ to completely load in SeaMonkey, 11.5 minutes before it even started to paint anything, 75,987,605 bytes of html.

It was only 47370045 bytes here.

I quickly checked with a smaller repo and it took wget 17 seconds to fetch 34kb of html. Then I checked on pontifex2 and it has a load of 100 and disk IO feels very slow with 71 rsyncd processes and repopush doing 100MB/s read + 24MB/s write in total.

In addition to that, the leap 15.2/x86_64 dir has 111314 files instead of the expected 20k - eg it had 15 different versions of zypper-docker So cleanup is missing there.

The filesystem is also 91% full - yes, we need to get that cleaned up, every file has 3-4-5-6 copies. -- Per Jessen, Zürich (6.2°C) Member, openSUSE Heroes -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org

Per Jessen

11:29

Felix Miata wrote:

...

I just waited 13.5 minutes for http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/ to completely load in SeaMonkey, 11.5 minutes before it even started to paint anything, 75,987,605 bytes of html. This is just after it took 45 minutes between submitting a new bug, and succeeding to add 3 additional attachments (in 4 parts: 3 15.2 initrds, one too large for acceptance which had to be split) to B.O.O.: http://bugzilla.opensuse.org/show_bug.cgi?id=1161238

http://ftp5.gwdg.de/pub/opensuse/distribution/leap/15.2/repo/oss/x86_64/

...

took 5 seconds to load html, 2,890,366 bytes.

Something is very wrong that opensuse.org is setup to require such gargantuan HTML and 16000% longer load times.

Felix, that did not come from opensuse.org, but from ftp.gwdg.de. 5 seconds to download - you really think that is so bad? For myself just now, it took 1 second, but I'm a lot closer to Goettingen than you are :-) Also, that page lists 19623 files for download, on average 150 chars per file. There isn't much we can really do about it - unless you have some suggestions? -- Per Jessen, Zürich (6.6°C) Member, openSUSE Heroes -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org

Per Jessen

11:48

Per Jessen wrote:

...

Felix Miata wrote:

...
I just waited 13.5 minutes for http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/ to completely load in SeaMonkey, 11.5 minutes before it even started to paint anything, 75,987,605 bytes of html. This is just after it took 45 minutes between submitting a new bug, and succeeding to add 3 additional attachments (in 4 parts: 3 15.2 initrds, one too large for acceptance which had to be split) to B.O.O.: http://bugzilla.opensuse.org/show_bug.cgi?id=1161238

http://ftp5.gwdg.de/pub/opensuse/distribution/leap/15.2/repo/oss/x86_64/

...

...
took 5 seconds to load html, 2,890,366 bytes.

Something is very wrong that opensuse.org is setup to require such gargantuan HTML and 16000% longer load times.

Felix, that did not come from opensuse.org, but from ftp.gwdg.de. 5 seconds to download - you really think that is so bad? For myself just now, it took 1 second, but I'm a lot closer to Goettingen than you are :-)

Sorry, my mistake, I misread - as Bernhard wrote, download.o.o is way overloaded. -- Per Jessen, Zürich (6.3°C) Member, openSUSE Heroes -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org

Felix Miata

14:32

Per Jessen composed on 2020-01-18 12:29 (UTC+0100):

...

Felix Miata wrote:

...

...
I just waited 13.5 minutes for http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/ to completely load in SeaMonkey, 11.5 minutes before it even started to paint anything, 75,987,605 bytes of html. This is just after it took 45 minutes between submitting a new bug, and succeeding to add 3 additional attachments (in 4 parts: 3 15.2 initrds, one too large for acceptance which had to be split) to B.O.O.: http://bugzilla.opensuse.org/show_bug.cgi?id=1161238

...

http://ftp5.gwdg.de/pub/opensuse/distribution/leap/15.2/repo/oss/x86_64/

...
took 5 seconds to load html, 2,890,366 bytes.

Something is very wrong that opensuse.org is setup to require such gargantuan HTML and 16000% longer load times.

Felix, that did not come from opensuse.org, but from ftp.gwdg.de. 5 seconds to download - you really think that is so bad? For myself just now, it took 1 second, but I'm a lot closer to Goettingen than you are :-)

...

Also, that page lists 19623 files for download, on average 150 chars per file.

I don't know how you calculated this, but lines like this 800+ one shouldn't be necessary: <tr><td valign="top"><a href="http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/drbd-kmp-default-9.0.20%7E1+git.7dce3c8b_k5.3.12_lp152.1-lp152.1.10.x86_64.rpm"><img src="Index%20of%20_distribution_leap_15.2_repo_oss_x86_64-sm_files/rpm.png" alt="[ ]" width="16" height="16"></a></td><td><a href="http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/drbd-kmp-default-9.0.20%7E1+git.7dce3c8b_k5.3.12_lp152.1-lp152.1.10.x86_64.rpm">drbd-kmp-default-9.0.20~1+git.7dce3c8b_k5.3.12_lp152.1-lp152.1.10.x86_64.rpm</a></td><td align="right">07-Dec-2019 13:20 </td><td align="right">278K </td><td><a href="http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/drbd-kmp-default-9.0.20%7E1+git.7dce3c8b_k5.3.12_lp152.1-lp152.1.10.x86_64.rpm.mirrorlist">Details</a></td></tr> Here's the whole 75,987,605 byte file from download.opensuse.org: http://fm.no-ip.com/Tmp/SUSE/doo152repoOSSx8664-sm.html Loading in Chromium is taking forever too, 8 minutes so far and only into the m's. Just now https://mirrors.edge.kernel.org/opensuse/distribution/leap/15.2/repo/oss/x86... with wget was only 2,660,970 bytes, and loaded too quickly to measure: http://fm.no-ip.com/Tmp/SUSE/index-kernelorg.html Attempting same from http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/ (195.135.221.134) wget was still trying to connect after more than seven minutes. I left the room to make breakfast. Eventually I got this 47,366,785 bytes: http://fm.no-ip.com/Tmp/SUSE/index-wget-doo.html

...

There isn't much we can really do about it - unless you have some suggestions?

Eradicate the tables and inline HTML markup (e.g. <td align="right">, <td valign="top">, and <img src="Index%20of%20_distribution_leap_15.2_repo_oss_x86_64-sm_files/rpm.png" alt="[ ]" width="16" height="16">). CSS was proven more efficient two decades ago. Even when normal cleanup is happening as it should, D.O.O. takes *many* times longer to load than GWDG, or kernel.org (latter of which only carries selected portions of openSUSE's repos). -- Evolution as taught in public schools is religion, not science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org

Knurpht-openSUSE

14:55

Op zaterdag 18 januari 2020 15:32:58 CET schreef Felix Miata:

...

Per Jessen composed on 2020-01-18 12:29 (UTC+0100):

...
Felix Miata wrote:

...
I just waited 13.5 minutes for http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/ to completely load in SeaMonkey, 11.5 minutes before it even started to paint anything, 75,987,605 bytes of html. This is just after it took 45 minutes between submitting a new bug, and succeeding to add 3 additional attachments (in 4 parts: 3 15.2 initrds, one too large for acceptance which had to be split) to B.O.O.: http://bugzilla.opensuse.org/show_bug.cgi?id=1161238

http://ftp5.gwdg.de/pub/opensuse/distribution/leap/15.2/repo/oss/x86_64/

...
took 5 seconds to load html, 2,890,366 bytes.

Something is very wrong that opensuse.org is setup to require such gargantuan HTML and 16000% longer load times.

Felix, that did not come from opensuse.org, but from ftp.gwdg.de. 5 seconds to download - you really think that is so bad? For myself just now, it took 1 second, but I'm a lot closer to Goettingen than you are :-)

Also, that page lists 19623 files for download, on average 150 chars per file.

I don't know how you calculated this, but lines like this 800+ one shouldn't be necessary: <tr><td valign="top"><a href="http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/d rbd-kmp-default-9.0.20%7E1+git.7dce3c8b_k5.3.12_lp152.1-lp152.1.10.x86_64.rp m"><img src="Index%20of%20_distribution_leap_15.2_repo_oss_x86_64-sm_files/rpm.png" alt="[ ]" width="16" height="16"></a></td><td><a href="http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/d rbd-kmp-default-9.0.20%7E1+git.7dce3c8b_k5.3.12_lp152.1-lp152.1.10.x86_64.rp m">drbd-kmp-default-9.0.20~1+git.7dce3c8b_k5.3.12_lp152.1-lp152.1.10.x86_64. rpm</a></td><td align="right">07-Dec-2019 13:20 </td><td align="right">278K </td><td><a href="http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/d rbd-kmp-default-9.0.20%7E1+git.7dce3c8b_k5.3.12_lp152.1-lp152.1.10.x86_64.rp m.mirrorlist">Details</a></td></tr>

Here's the whole 75,987,605 byte file from download.opensuse.org: http://fm.no-ip.com/Tmp/SUSE/doo152repoOSSx8664-sm.html

Loading in Chromium is taking forever too, 8 minutes so far and only into the m's.

Just now https://mirrors.edge.kernel.org/opensuse/distribution/leap/15.2/repo/oss/x8 6_64/ with wget was only 2,660,970 bytes, and loaded too quickly to measure: http://fm.no-ip.com/Tmp/SUSE/index-kernelorg.html

Attempting same from http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/ (195.135.221.134) wget was still trying to connect after more than seven minutes. I left the room to make breakfast. Eventually I got this 47,366,785 bytes: http://fm.no-ip.com/Tmp/SUSE/index-wget-doo.html

...
There isn't much we can really do about it - unless you have some suggestions?

Eradicate the tables and inline HTML markup (e.g. <td align="right">, <td valign="top">, and <img src="Index%20of%20_distribution_leap_15.2_repo_oss_x86_64-sm_files/rpm.png" alt="[ ]" width="16" height="16">). CSS was proven more efficient two decades ago. Even when normal cleanup is happening as it should, D.O.O. takes *many* times longer to load than GWDG, or kernel.org (latter of which only carries selected portions of openSUSE's repos). Not going to argue about html/css, but .... A user on discord now reports the tumleweed installer 'waiting for "Downloading installation system language" and on my laptop the wget commands last forever connecting .....

-- Gertjan Lettink a.k.a. Knurpht openSUSE Board Member openSUSE Forums Team -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org

Knurpht-openSUSE

14:59

Op zaterdag 18 januari 2020 15:55:44 CET schreef Knurpht-openSUSE:

...

Not going to argue about html/css, but .... A user on discord now reports the tumleweed installer 'waiting for "Downloading installation system language" and on my laptop the wget commands last forever connecting ..... And now it only takes a couple of secs ....

-- Gertjan Lettink a.k.a. Knurpht openSUSE Board Member openSUSE Forums Team -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org

Michael Ströder

15:15

On 1/18/20 3:59 PM, Knurpht-openSUSE wrote:

...

Op zaterdag 18 januari 2020 15:55:44 CET schreef Knurpht-openSUSE:

...
Not going to argue about html/css, but .... A user on discord now reports the tumleweed installer 'waiting for "Downloading installation system language" and on my laptop the wget commands last forever connecting ..... And now it only takes a couple of secs ....

My impression is that OBS is not able to publish (the home repos) anymore (or the queue is *very* long). Ciao, Michael. -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org

Per Jessen

15:15

Felix Miata wrote:

...

Per Jessen composed on 2020-01-18 12:29 (UTC+0100):

...
Felix Miata wrote:

...
...
I just waited 13.5 minutes for http://download.opensuse.org/distribution/leap/15.2/repo/oss/x86_64/ to completely load in SeaMonkey, 11.5 minutes before it even started to paint anything, 75,987,605 bytes of html. This is just after it took 45 minutes between submitting a new bug, and succeeding to add 3 additional attachments (in 4 parts: 3 15.2 initrds, one too large for acceptance which had to be split) to B.O.O.: http://bugzilla.opensuse.org/show_bug.cgi?id=1161238

...
http://ftp5.gwdg.de/pub/opensuse/distribution/leap/15.2/repo/oss/x86_64/

...
...
took 5 seconds to load html, 2,890,366 bytes.

Something is very wrong that opensuse.org is setup to require such gargantuan HTML and 16000% longer load times.

Felix, that did not come from opensuse.org, but from ftp.gwdg.de. 5 seconds to download - you really think that is so bad? For myself just now, it took 1 second, but I'm a lot closer to Goettingen than you are :-)

...
Also, that page lists 19623 files for download, on average 150 chars per file.

I don't know how you calculated this,

I just downloaded the index file, and grepped for lines containing 'href'. There are 19652 - file size is 2890366 - divided, by 19623 = 150.

...

but lines like this 800+ one shouldn't be necessary: <tr><td valign="top">

I don't have anything like that. No table, no nothing. Just a plain list of <a href="....."> .... maybe we are not talking about the same thing? http://ftp5.gwdg.de/pub/opensuse/distribution/leap/15.2/repo/oss/x86_64/ The format used will depend on which mirror you use, it is up the operator. I guess we use tablea to format it.

...

Here's the whole 75,987,605 byte file from download.opensuse.org: http://fm.no-ip.com/Tmp/SUSE/doo152repoOSSx8664-sm.html

Right, so our file is much bigger, I get it.

...

...
There isn't much we can really do about it - unless you have some suggestions?

Eradicate the tables and inline HTML markup (e.g. <td align="right">, <td valign="top">, and

75Mb does seem like a lot of noise, I agree. I might open a ticket on that. -- Per Jessen, Zürich (5.6°C) Member, openSUSE Heroes -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org

Felix Miata

18:27

Per Jessen composed on 2020-01-18 16:15 (UTC+0100):

...

Felix Miata wrote: ...

...
but lines like this 800+ one shouldn't be necessary: <tr><td valign="top">

...

I don't have anything like that. No table, no nothing. Just a plain list of <a href="....."> .... maybe we are not talking about the same thing?

Right. 800+ was from D.O.O., HTML tables.

...

http://ftp5.gwdg.de/pub/opensuse/distribution/leap/15.2/repo/oss/x86_64/

Gwdg gets the job done with a simple list of links, magnitudes more efficient, same as most mirrors.

...

The format used will depend on which mirror you use, it is up the operator. I guess we use tablea to format it.

Tables are the main problem. High latency makes it worse.

...

...
Here's the whole 75,987,605 byte file from download.opensuse.org: http://fm.no-ip.com/Tmp/SUSE/doo152repoOSSx8664-sm.html

...

Right, so our file is much bigger, I get it.

Web browser handling of large tables has been poor since last century, a well established fact. Huge tables of 100k+ lines are horrific.

...

...
...
There isn't much we can really do about it - unless you have some suggestions?

...

...
Eradicate the tables and inline HTML markup (e.g. <td align="right">, <td valign="top">, and

...

75Mb does seem like a lot of noise, I agree.

It's worse than just "a lot".

...

I might open a ticket on that.

Please do, unless you'd rather I do it. -- Evolution as taught in public schools is religion, not science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org

Per Jessen

18:32

Felix Miata wrote:

...

...
75Mb does seem like a lot of noise, I agree.

It's worse than just "a lot".

Yeah, it is. Like 25 times worse than the listing from GWDG.

...

...
I might open a ticket on that.

Please do, unless you'd rather I do it.

Go ahead - that way I'll have someone to remind me when I forget :-) This might even be low-hanging fruit, I'm not sure. It's essentially the apache auto-indexing setup, I think. -- Per Jessen, Zürich (5.1°C) Member, openSUSE Heroes -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org

Felix Miata

21:27

Per Jessen composed on 2020-01-18 19:32 (UTC+0100):

...

Felix Miata wrote:

...

...
...
75Mb does seem like a lot of noise, I agree.

...

...
It's worse than just "a lot".

...

Yeah, it is. Like 25 times worse than the listing from GWDG.

...

...
...
I might open a ticket on that.

...

...
Please do, unless you'd rather I do it.

...

Go ahead - that way I'll have someone to remind me when I forget :-)

...

This might even be low-hanging fruit, I'm not sure. It's essentially the apache auto-indexing setup, I think.

https://progress.opensuse.org/issues/62255 -- Evolution as taught in public schools is religion, not science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org

1621

Age (days ago)

1621

Last active (days ago)

List overview

Download

13 comments

6 participants

participants (6)

Bernhard M. Wiedemann
Felix Miata
Knurpht-openSUSE
Michael Ströder
Per Jessen
wnereiz