[zypp-devel] mirrorlist support in zypp?
Repeat question from yesterday since I got no response: 1) How do I set up a repomd (yum) format repository that is set up using "mirrorlist=URL" in yum? 2) How can I do variable substitution in a URL? I want to set up a repomd repository (using mirrorlist function above) with a URL like: mirrorlist=http://some.domain.com/repo/path/mirror.pl?osver=$releasever&osname=sles Where "$releasever" is automatically replaced with the OS version. 3) How can I write a plugin to add (two) new variables in the substitution process? I want to set up repositories that are specific to dell system models and have a server-side cgi redirect the client to the correct repo based on the system model. Something like: mirrorlist=http://domain.com/repo/mirrors.pl?vendor=0x1028&model=0x0152&osname=sles&osver=$releasever Any pointers would be greatly appreciated. I would like to write something that works with opensuse and sles. -- Michael -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Wednesday 06 June 2007 07:57:36 Michael E Brown wrote:
Repeat question from yesterday since I got no response:
http://en.wikipedia.org/wiki/Time_zone
1) How do I set up a repomd (yum) format repository that is set up using "mirrorlist=URL" in yum?
This is not supported in released versions, altough this is exactly what is being developed.
2) How can I do variable substitution in a URL? I want to set up a repomd repository (using mirrorlist function above) with a URL like: mirrorlist=http://some.domain.com/repo/path/mirror.pl?osver=$releasever&osn ame=sles Where "$releasever" is automatically replaced with the OS version.
You can't. Mirror list is not supported. And also variable substitution. You can get the mirror list effect at the server level (configuring apache). Just like software.opensuse.org works.
3) How can I write a plugin to add (two) new variables in the substitution process? I want to set up repositories that are specific to dell system models and have a server-side cgi redirect the client to the correct repo based on the system model. Something like:
You can't. I think it is the first time we get feedback like this, that can actually be used to improve things. The problem implementing these standards is that finding documentation is really hard. So basically we need to standarize. And you can help with that. What variables are already allowed?
mirrorlist=http://domain.com/repo/mirrors.pl?vendor=0x1028&model=0x0152&osn ame=sles&osver=$releasever
What is model? What variables do you need appart of the ones YUM already provides (which I also don't know)
Any pointers would be greatly appreciated. I would like to write something that works with opensuse and sles.
Cheers Duncan -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Wed, Jun 06, 2007 at 10:05:43AM +0200, Duncan Mac-Vicar Prett wrote:
On Wednesday 06 June 2007 07:57:36 Michael E Brown wrote:
1) How do I set up a repomd (yum) format repository that is set up using "mirrorlist=URL" in yum?
This is not supported in released versions, altough this is exactly what is being developed.
2) How can I do variable substitution in a URL? I want to set up a repomd repository (using mirrorlist function above) with a URL like: mirrorlist=http://some.domain.com/repo/path/mirror.pl?osver=$releasever&osn ame=sles Where "$releasever" is automatically replaced with the OS version.
You can't. Mirror list is not supported. And also variable substitution.
You can get the mirror list effect at the server level (configuring apache). Just like software.opensuse.org works.
Mirrorlist functionality is a really useful capability. I dont see how you could you could support millions of users without it. It allows you to load balance between separate servers which may even have different paths to the repo. It also enables you to do really cool things, like have a cgi script that produces the mirror list. It can, for example, return geographically-close mirror based on the users client IP.
3) How can I write a plugin to add (two) new variables in the substitution process? I want to set up repositories that are specific to dell system models and have a server-side cgi redirect the client to the correct repo based on the system model. Something like:
You can't. I think it is the first time we get feedback like this, that can actually be used to improve things.
The problem implementing these standards is that finding documentation is really hard.
So basically we need to standarize. And you can help with that. What variables are already allowed?
Out of the box, yum has three variables it will replace: - $basearch: i386, x86_64 - $arch: i386, i486, i586, i686, etc, x86_64 - $releasever: OS version. ex: for opensuse 10.3, should == "10.3" For the most part, $arch isnt usually used.
mirrorlist=http://domain.com/repo/mirrors.pl?vendor=0x1028&model=0x0152&osn ame=sles&osver=$releasever
What is model? What variables do you need appart of the ones YUM already provides (which I also don't know)
I have written a small plugin to yum (iirc <50 LOC) to provide: - $sys_ven_id: For Dell == "0x1028" - $sys_dev_id: equal to the Dell model (uniquely assigned system id) Each Dell system has a two-byte "system id", which uniquely identifies that system model. The utility of this is that I can distribute a single config (below). Also, if a user moves hdd from system to system, the variables are automatically updated to the new machine model ids. mirrorlist=http://linux.dell.com/repo/hardware/mirrors.pl?sys_ven_id=$sys_ven_id&sys_dev_id=$sys_dev_id&osname=fc$releasever.$basearch&dellsysidpluginver=$dellsysidpluginver Breaking this down: - mirrorlist=http://linux.dell.com/repo/hardware/mirrors.pl This is the CGI script that is the brains of the whole operation This cgi script uses the variables that follow to provide the client with the list of appropriate repo urls. - sys_ven_id=$sys_ven_id pass in the vendor ID as a cgi variable. - sys_dev_id=$sys_dev_id pass in system model so cgi can select the correct repository - osname=fc$releasever.$basearch and the correct os - dellsysidpluginver=$dellsysidpluginver This is mostly for auditing purposes. -- Michael -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Wednesday 06 June 2007 16:44:34 Michael E Brown wrote:
Mirrorlist functionality is a really useful capability. I dont see how you could you could support millions of users without it. It allows you to load balance between separate servers which may even have different paths to the repo. It also enables you to do really cool things, like have a cgi script that produces the mirror list. It can, for example, return geographically-close mirror based on the users client IP.
Variable sustitution in urls looks really interesting. I will try to implement that. mirrorlist, why not handling it at the web server level? Duncan -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Wed, Jun 06, 2007 at 05:11:54PM +0200, Duncan Mac-Vicar Prett wrote:
On Wednesday 06 June 2007 16:44:34 Michael E Brown wrote:
Mirrorlist functionality is a really useful capability. I dont see how you could you could support millions of users without it. It allows you to load balance between separate servers which may even have different paths to the repo. It also enables you to do really cool things, like have a cgi script that produces the mirror list. It can, for example, return geographically-close mirror based on the users client IP.
Variable sustitution in urls looks really interesting. I will try to implement that.
mirrorlist, why not handling it at the web server level?
Several reasons. - In the current yum implementation, the client gets the mirror list and randomly chooses an entry from the list. - It also will fail over down the list if there is any connection problem with the first server. - The mirror list can list different servers that have the repo at their own path. - Some people dont have administrative access to the web server to change the web server configuration to redirect. The mirror list is simply a URL to a file containing a list of mirror paths. Because it is a URL, it can easily be configured to point at a cgi that outputs the list. -- Michael -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On 06/06/07, Michael E Brown
Several reasons. - In the current yum implementation, the client gets the mirror list and randomly chooses an entry from the list. - It also will fail over down the list if there is any connection problem with the first server. - The mirror list can list different servers that have the repo at their own path. - Some people dont have administrative access to the web server to change the web server configuration to redirect.
The mirror list is simply a URL to a file containing a list of mirror paths. Because it is a URL, it can easily be configured to point at a cgi that outputs the list.
The redirector used on software.opensuse.org and download.opensuse.org does solve most of the same problems. See http://en.opensuse.org/Build_Service/Redirector If the user adds the repository from software.opensuse.org or download.opensuse.org each request will be redirected to the appropriate location on a mirror. It should only redirect to mirrors which are actually working. The main problem with this approach is that the central redirector must always be up, especially if one is encouraging users to use the redirector rather than mirrors directly. Currently (d|o).opensuse.org is not very reliable, going down every few weeks. Currently every time it goes down a large chunk of users lose package management. Until it is at least co-hosted in Provo as well as Nuremberg I'm not sure it's a good idea to use for the automatically added repositories. Currently most of the mirrors the redirector redirects to (heanet, belnet, switch etc) are far more reliable than the redirector itself. _ Benjamin Weber -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
On Thu, Jun 07, 2007 at 07:56:22AM +0100, Benji Weber wrote:
On 06/06/07, Michael E Brown
wrote: Several reasons. - In the current yum implementation, the client gets the mirror list and randomly chooses an entry from the list. - It also will fail over down the list if there is any connection problem with the first server. - The mirror list can list different servers that have the repo at their own path. - Some people dont have administrative access to the web server to change the web server configuration to redirect.
The mirror list is simply a URL to a file containing a list of mirror paths. Because it is a URL, it can easily be configured to point at a cgi that outputs the list.
The redirector used on software.opensuse.org and download.opensuse.org does solve most of the same problems. See http://en.opensuse.org/Build_Service/Redirector
If the user adds the repository from software.opensuse.org or download.opensuse.org each request will be redirected to the appropriate location on a mirror. It should only redirect to mirrors which are actually working.
The main problem with this approach is that the central redirector must always be up, especially if one is encouraging users to use the redirector rather than mirrors directly. Currently (d|o).opensuse.org is not very reliable, going down every few weeks. Currently every time it goes down a large chunk of users lose package management. Until it is at least co-hosted in Provo as well as Nuremberg I'm not sure it's a good idea to use for the automatically added repositories.
Currently most of the mirrors the redirector redirects to (heanet, belnet, switch etc) are far more reliable than the redirector itself.
Thanks for the pointer. The yum mirrorlist functionality is useful because you can point to a file:// url if you want. For my purposes, I am not doing that, but others do. So I prototyped an extremely dumb example to make sure that my webservers support setting http status from within cgi scripts, and it appears to work. I am going to say I still prefer the mirrorlist functionality, though. It returns a larger list of addresses to the client, and the client can fail over to another server if there is a problem. This makes for much more reliable downloads. -- Michael -- To unsubscribe, e-mail: zypp-devel+unsubscribe@opensuse.org For additional commands, e-mail: zypp-devel+help@opensuse.org
participants (3)
-
Benji Weber
-
Duncan Mac-Vicar Prett
-
Michael E Brown