On Mon, May 28, 2012 at 7:13 AM, Nelson Marques <nmo.marques@gmail.com> wrote:
2012/5/28 Marguerite Su <i@marguerite.su>:
On Mon, May 28, 2012 at 6:18 AM, Nelson Marques <nmo.marques@gmail.com> wrote:
2012/5/27 Marguerite Su <i@marguerite.su>:
在 2012-5-28 上午3:25，"Nelson Marques" <nmo.marques@gmail.com>写道：
2012/5/27 Marguerite Su <i@marguerite.su>:
Hi, all,
users complained to me many times about yast2's slowness on forums.o.o, twitter or gtalk groups. (of course in a form of flame war like apt vs. yast, yum vs. yast...I don't want to say much about that.)
I didn't believe it, until today I broke my zypp cache, and had to retrieve them all...and I found many ugly things. let me be clear one by one.
1. yast-ncurses.
you have to use `LANG=en zypper [option]` if you set your locale to non-En environment.
or texts will be displayed as unreadable ??????.
see screenshot: http://paste.opensuse.org/40229867
any ideas on this one？
2. if you broke your zypp cache, yast-ncurses will always tell you that you have insufficient permissions so can only see part of its modules, even you logged in with root. (maybe not producible)
3. yast have to download every local package descrs, no matter whether you need it or not.
Yes, you and everyone else. It's that metadata that allows zypper to resolve dependencies and other things. If you don't have it available you defeat the whole purpose of a repository.
I think just download packages.en.gz and packages.your_locale.gz will be enough. (nowadays it downloads all of them...du, kr, zh_TW, pl, blabla...)
and it's the cause of its slowness..
it took me 40 minutes to refresh it all...even with local mirrors on.
You most likely have a transparent proxy somewhere in the middle that is introducing erratic behavior or limiting down; that sounds the most reasonable explanation.
hi，nelson，
do you mean some isp slow me down in the middle of the transition from server to me？
I am not network expert...I cant quite follow this part
Yes, there can be a problem somewhere in the network, and it might be bottlenecking somewhere, which could cause issues on your end.
I'll switch all my repo from d.o.o to twaren then try again.
if that comes faster...I'll change all the links on zh.o.o wiki to guide new users.
I have a 4MB download speed, but can only download at a speed of < 10KB/s.
I can top my home line to 1.2Mbytes/sec (12Mbit ADSL); And I can get far more higher speeds through my office.
yes.
so are you suggesting me upgrading my internet plan to use opensuse？like buy a new computer to install opensuse？
No, I'm trying to tell you that if it was a problem on dl.opensuse.org, we would suffer from it to. If I can get those speeds, you should as aswell; for a 4Mbit downstream window you should get around 400Kb/sec. If you are not getting them, that's because there's an issue somewhere, right ?
yes...actually I highly suspect the Eu-Asia continent fiber-wire lost its speed into China. (because I never see any Russian guys complain about slowness of yast.)
I'll do some traceroute and see.
Now, depending on the type of your connection there can be lots of factors, from transparent proxies, to network bottlenecks on your ISP side, crazed routing tables, etc.
For example, ADSL connections you have phisical factors, like the distance between your house to the central that serves you, etc.
if my computer is obsolete. i will. but actually 4mb is the largest internet speed in Chinese...I can't immigrate only to update my opensuse box. right？
so the question I want to ask is how to or is there any possible way to speed zypper refresh based on the infrastructure I have.
There's nothing you or anyone else can do on zypper if the problem is related with a network bottleneck for example or crazed routing tables.
I knew it's slow, but I don't know it's that slow...
the standard procedure to use yast is: a. launch b. min it and do something else c. after half an hour, operate, min it again d. wait until it closes itself.
I think we really need to improve it. like:
(1) try merge package descrs among repositories. most of same packages' descrs among repos are the same. we don't need to download it several times.
I would belive that this would introduce chaos and havoc in operational theaters that contemplate concorrent repositories. Why should my repo metadata be replaced by some other repo metadata? Simples example:
- My repo has a version of PostgreSQL without UUID support. If you merge metada and mine gets replaced with the one from a repo which has PostgreSQL built with UUID support, users will most likely get a broken PostgreSQL. You will blow up the majority of systems with concorrent repositories.
do you mean packages.en.gz is used to solve dependencies？I think they are just translations. I never mean to merge oss and k：d：l contents together. but packages descr. eg %description instead of gpg keys....or rpms themeselves.
If we are talking about the files in 'repodata' or similiar depending if it's rpm-db or yast2 (which I believe we are), those files are created so that the package manager tools (zypper, yum) can read the metadata from the RPMs available instead of querying them all.
So if you so for example a query on zypper and you dont have those files, there's an error. If you rmeove info from those files, it can lead to malformed dependencies.
Like I said in my last email, there's most likely a global community living from those repos, you might not have interest on descriptions, but I and other people have.
yes. but actually we were talking about different things...
I was talking about /var/cache/zypp/download.opensuse.org-oss/suse/setup/descr/package.locale.gz
It's most likely the same thing because you can have the description fields in the RPM with translations, for example in one spec file we can have multiple summaries for different languages.
I would suppose they will get splitted somewhere. So they are the same thing except the package.locale.gz has the metadata that is provided for your location, right ? :)
(I might be wrong though).
nope. Adrian said once on build-service ML. if you did that (multiple summaries/descriptions), you have to maintain all the openSUSE supported languages on your own... but that's also not exact...because nowadays yast can't parse those localed summaries/descriptions at all. if you write them in spec, it's just got ignored. coolo's 50-tools script only reads *.spec and withdraw English summaries and descriptions, then insert them into a big .pot file for translators. and those translated .po will be used to create package.locale.gz. actually there're no metadata...but just plain translated summaries and descriptions...it's yast that parse them and inset the translations into the right place.
and you were talking about /var/cache/zypp/download.opensuse.org-oss/suse/setup/descr/package.gz and packge.DU.gz
although they look like the same thing. but actually they're different...
maybe because not everything gets translated? and those which haven't local translations dont go into local.gz ?
nope. "not everything get translated" is right. there're 40000+ packages in such pot....almost the number of items of the whole openSUSE. no one can translate them all... but package.gz and package.DU.gz don't contain any summaries and descriptions. they contain libraries' and binaries' names (like /usr/bin/bash) and some numbers.( I don't know what they're for.) all summaries and descriptions are in package.en.gz.
and I didn't see anything is repodata controls dependencies.
When you do something like 'zypper search foobar' it searches those files with the metadata for a package named foobar. If you dont have those files zypper can't find anything. So if package 'foo' has a dependency on 'foobar' thats how zypper resolves the dependencies, by reading those files (if I'm wrong, someone correct me).
it's done by /var/cache/zypp/raw/*/repodata/*-primary.xml.gz normal repositories (not oss or non-oss) do not have a ./suse/setup/ directory. so it can't be ./suse/setup/package.gz to solve dependencies....
Same goes for files, versions etc... Why this is done this way? Simple... Imagine that you have a repo with 10.000 files... if everytime a user queries the repo for something it has to open all the 10.000 RPMs to read the metadata, imagine the overhead in I/O it would create.
Not saying it's perfect! But so far it works nice... You have the option also to save that information on SQLite databases instead of the traditional files, wich is useful if you have some sort of application doing some management on the repository itself.
dependencies seems to be solved by RPM program itself. :)
RPM can«t find dependencies from stuff that isn't installed. Zypper does it based on repository metadata. If I am wrong, then someone correct me ;)
eg：you may have two packages in different repos with the same %description. then why do we need to download such descriptions twice？
If dump a lot of RPMs into a directory and you run 'createrepo' on it. It extracts the metadata from all RPMs and creates those file lists or injects it into a sqlite database. This information (like I said before) is used by the package manager tools (zypper) to resolve dependencies and show information to the user.
Try to search a bit for what RPM NEVRA is.
packages.en.gz has nothing to do with dependencies. or specfiles
Then we are maybe speaking of different things. I was assuming you were mentioning the files on 'repodata'. If they are... we shouldnt mess with them.
they are copied out simple texts by coolo's script work in i18n 50-tools
(2) make a daemon to download them in background when system starts.
Uncool stuff and bloat. How would that feature help a system with 1000 days uptime? You can easilly setup a local repo with mrepo and run a local mirror to overcome all the dificulties you are presenting.
yes.
but I dont know how...
someone said descrs are downloaded from suse original server and packages are downloaded from mirrors.
so to me it means I can get a fast speed if I set up a server to fetch all rpms from a mirror and install them by rpm command. or if I use yast，I have to download descrs from original server. that will be slow.
if I have such server，why I still use yast....
(3) give user options to not download and display descrs/summaries, but to find and install only using package names.
You are away that by doing such, you are suggesting that we strip off zypper functionality that will affect dependency resolution? How will this help? For example:
- # zypper install /usr/bin/ssh
Now what does zypper do if he can't access the repo metadata? :)
do you mean package.en.gz is metadata controling dependencies？
Well maybe we need some yast expert to tell us....because I translated such packages.en.gz and didnt find any dependencies in it. I dont know if there are autoscripts adding such dependencies to my translations.
I'm pretty sure I'm not one of them, neither I want to be.
if there is. it will be design default. because rpm dont know chinese...how can it solve dependencies in packages.zh_cn.gz？
just some ideas in thin air....I don't how to implement it.
PS: I think we should remove the "OSS might be slow" string from translations and yast itself.
a warning is something you can avoid and don't do it. but repositories?
you have to enable it not matter how many warnings it warns, if you want to do things using it.
but are those things dangerous or risky? no. come on, it's our own OSS repo. it's just slow.
and can you avoid that slow? no...
so it sounds to warn users something they can't change. useless and feels forcibly.
it does us more harms than benefits if you think it deeper.
we give our users a bad first impression that it is slow, so it has to be fast like a flash to change their such minds.
Please consider that there is a world outside China ;)
yes...
without any part， it cant be called world...so we have to solve the problem in the worst place instead of the best place...
actually，off topic，opensuse has little users in China partially because it is hard to download（big iso）and update （slow yast）. most people's internet speed is even slower than mine....
What I meant is that openSUSE serves a global community, therefore, because I only use English, doesnt mean that German, Russian, Portuguese and other should be removed on behalf of my own needs because there are other people for which those are important.
yes.
it seems openSUSE has done it well. because zypp will download locale descriptions.(if translators contribute such translations.)
greetings，marguerite
from my Android.
Regards
Marguerite -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
-- Nelson Marques // I've stopped trying to understand sandwiches with a third piece of bread in the middle...
-- Nelson Marques // I've stopped trying to understand sandwiches with a third piece of bread in the middle...
Marguerite
-- Nelson Marques // I've stopped trying to understand sandwiches with a third piece of bread in the middle... -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org