[opensuse] New drpmsync Version available
Hi folks, I finally found some time to work a bit on drpmsync. A new version is available from https://forgesvn1.novell.com/svn/drpmsync/trunk . New Features: - while applydeltarpm is running the next delta is fetched over the network. - new --get-filelist and --use-filelist options to save the filelist to a file or re-use a fetched one. - new --list and --list-recursive option to list a directory on the server. - trailing slashes are stripped from the source directory. The last point is actually quite important. I plan to make drpmsync more similar to rsync, this means that drpmsync server/dir . will *always* create a subdirectory in one of the next versions. So you'll need to add '/' or '/.' to your source specifications to stay compatible. Planned next steps: - source handling changes (see above). - support for mirror servers (i.e. what Robert suggested: Pull the filelist from the main server, get the deltas from a mirror server). - exclude and --exclude-arch. Enjoy, Michael. -- Michael Schroeder mls@suse.de main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
Hi, On Fri, 7 Jul 2006, Michael Schroeder wrote:
I finally found some time to work a bit on drpmsync. A new version is available from https://forgesvn1.novell.com/svn/drpmsync/trunk .
New Features: - while applydeltarpm is running the next delta is fetched over the network. - new --get-filelist and --use-filelist options to save the filelist to a file or re-use a fetched one. - new --list and --list-recursive option to list a directory on the server. - trailing slashes are stripped from the source directory.
The last point is actually quite important. I plan to make drpmsync more similar to rsync, this means that
drpmsync server/dir .
will *always* create a subdirectory in one of the next versions. So you'll need to add '/' or '/.' to your source specifications to stay compatible.
Planned next steps: - source handling changes (see above). - support for mirror servers (i.e. what Robert suggested: Pull the filelist from the main server, get the deltas from a mirror server).
This is a very welcomed extension. This way ftp-1.gwdg.de can contribute. I would never use the original drpmsync scripting thingies to run a drpmsync service on high-production machines, but I can help to "deliver" the files. A possible next step would be to use the already very much refined, very sophisticated redirection mechanism of download.opensuse.org.
- exclude and --exclude-arch.
Cheers -e -- Eberhard Moenkeberg (emoenke@gwdg.de, em@kki.org) --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
On 2006-07-07 21:56:52 +0200, Eberhard Moenkeberg wrote:
A possible next step would be to use the already very much refined, very sophisticated redirection mechanism of download.opensuse.org.
kick christoph ;) or should i do that for you?:) darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
Hi, On Fri, 7 Jul 2006, Marcus Rueckert wrote:
On 2006-07-07 21:56:52 +0200, Eberhard Moenkeberg wrote:
A possible next step would be to use the already very much refined, very sophisticated redirection mechanism of download.opensuse.org.
kick christoph ;) or should i do that for you?:)
Michael should do it himself. Between technicians, only direct contacts are prosperous. ;-)) Christoph is currently doing only port 80 redirects, and maybe a technical discussion between Michael and Christoph is the best next step. Cheers -e -- Eberhard Moenkeberg (emoenke@gwdg.de, em@kki.org) --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
On 2006-07-07 22:28:10 +0200, Eberhard Moenkeberg wrote:
Michael should do it himself. Between technicians, only direct contacts are prosperous. ;-)) Christoph is currently doing only port 80 redirects, and maybe a technical discussion between Michael and Christoph is the best next step.
+ ftp:// darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
Hi, On Fri, 7 Jul 2006, Marcus Rueckert wrote:
On 2006-07-07 22:28:10 +0200, Eberhard Moenkeberg wrote:
Michael should do it himself. Between technicians, only direct contacts are prosperous. ;-)) Christoph is currently doing only port 80 redirects, and maybe a technical discussion between Michael and Christoph is the best next step.
+ ftp://
Let them directly discuss what is possible and what is best.
From my view, I would prefer http connections to the mirrors (port80, not 8080). Less overhead against ftp.
Cheers -e -- Eberhard Moenkeberg (emoenke@gwdg.de, em@kki.org) --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
On Fri, 7 Jul 2006, Eberhard Moenkeberg wrote:
Michael should do it himself. Between technicians, only direct contacts are prosperous. ;-)) Christoph is currently doing only port 80 redirects, and maybe a technical discussion between Michael and Christoph is the best next step.
+ ftp://
Let them directly discuss what is possible and what is best.
From my view, I would prefer http connections to the mirrors (port80, not 8080). Less overhead against ftp.
Redirecting HTTP requests to FTP servers doesn't sound 'polite' to me, though CURL (and I guess most browsers) would be able to handle redirects like that... IMO we should stick with HTTP redirects. Regards Christoph --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
Hi, On Mon, 10 Jul 2006, Christoph Thiel wrote:
On Fri, 7 Jul 2006, Eberhard Moenkeberg wrote:
Michael should do it himself. Between technicians, only direct contacts are prosperous. ;-)) Christoph is currently doing only port 80 redirects, and maybe a technical discussion between Michael and Christoph is the best next step.
+ ftp://
Let them directly discuss what is possible and what is best.
From my view, I would prefer http connections to the mirrors (port80, not 8080). Less overhead against ftp.
Redirecting HTTP requests to FTP servers doesn't sound 'polite' to me, though CURL (and I guess most browsers) would be able to handle redirects like that...
IMO we should stick with HTTP redirects.
Yes. Cheers -e -- Eberhard Moenkeberg (emoenke@gwdg.de, em@kki.org) --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
On Fri, 7 Jul 2006, Marcus Rueckert wrote:
On 2006-07-07 22:28:10 +0200, Eberhard Moenkeberg wrote:
Michael should do it himself. Between technicians, only direct contacts are prosperous. ;-)) Christoph is currently doing only port 80 redirects, and maybe a technical discussion between Michael and Christoph is the best next step.
+ ftp://
?? Regards Christoph --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
On Fri, 7 Jul 2006, Eberhard Moenkeberg wrote:
A possible next step would be to use the already very much refined, very sophisticated redirection mechanism of download.opensuse.org.
I'm not sure if I understand correctly: Are you suggesting to use the download.openSUSE.org infrastructure to track the mirror status of drpmsync mirrors and use it's cache to redirect users to different drpmsync mirrors? (At the moment SL-OSS-factory/drpmsync/* is blacklisted in download.openSUSE.org, because it takes quite some time to cache the directory contents of that part of the ftp-tree...) Regards Christoph --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
Hi, On Mon, 10 Jul 2006, Christoph Thiel wrote:
On Fri, 7 Jul 2006, Eberhard Moenkeberg wrote:
A possible next step would be to use the already very much refined, very sophisticated redirection mechanism of download.opensuse.org.
I'm not sure if I understand correctly: Are you suggesting to use the download.openSUSE.org infrastructure to track the mirror status of drpmsync mirrors and use it's cache to redirect users to different drpmsync mirrors?
(At the moment SL-OSS-factory/drpmsync/* is blacklisted in download.openSUSE.org, because it takes quite some time to cache the directory contents of that part of the ftp-tree...)
OK, so forget it for now, or direct them all to ftp-1.gwdg.de. But before integrate /pub/opensuse/distribution/SL-OSS-factory/drpmsync/ into the rsync push service. Cheers -e -- Eberhard Moenkeberg (emoenke@gwdg.de, em@kki.org) --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
On Fri, Jul 07, 2006 at 09:46:28PM +0200, Michael Schroeder wrote:
- support for mirror servers (i.e. what Robert suggested: Pull the filelist from the main server, get the deltas from a mirror server).
Please apply what is described in https://bugzilla.novell.com/show_bug.cgi?id=183793 to the repository building process. After that we can run from a mirror in this mode without the need of the drpmsync server _at all_! And we don't suffer from mirrors that are slightly out of sync because the file will be of the same age as the rest of the repository. Well obviously the latter is not true during the sync of the mirror but during that time a consistent sync cannot be expected with any sync technology. My client at http://pi3.informatik.uni-mannheim.de/~schiele/mydrpmsync/ could do that already if just the repository provided that single file. Robert -- Robert Schiele Tel.: +49-621-181-2214 Dipl.-Wirtsch.informatiker mailto:rschiele@uni-mannheim.de "Quidquid latine dictum sit, altum sonatur."
On Fri, Jul 07, 2006 at 11:13:39PM +0200, Robert Schiele wrote:
On Fri, Jul 07, 2006 at 09:46:28PM +0200, Michael Schroeder wrote:
- support for mirror servers (i.e. what Robert suggested: Pull the filelist from the main server, get the deltas from a mirror server).
Please apply what is described in https://bugzilla.novell.com/show_bug.cgi?id=183793 to the repository building process.
I'm not sure that this is a good idea. Just ask the server, it should be much more responsive. Cheers, Michael. -- Michael Schroeder mls@suse.de main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
Hi, On Mon, 10 Jul 2006, Michael Schroeder wrote:
On Fri, Jul 07, 2006 at 11:13:39PM +0200, Robert Schiele wrote:
On Fri, Jul 07, 2006 at 09:46:28PM +0200, Michael Schroeder wrote:
- support for mirror servers (i.e. what Robert suggested: Pull the filelist from the main server, get the deltas from a mirror server).
Please apply what is described in https://bugzilla.novell.com/show_bug.cgi?id=183793 to the repository building process.
I'm not sure that this is a good idea. Just ask the server, it should be much more responsive.
It would even be OK if it only were a "half-good" idea. As Robert states, "Generate the list you would get from http://drpmsync.opensuse.org:8888/Factory/drpmsync/contents (maybe compressed) during the repository build process (like INDEX.gz). If this was done I could easily change my client to run as a drpmsync client _without_ the need of a drpmsync server at all." I really cannot see why this should not be done. If you are thinking "not necessary" try to see it as a "very much welcomed addition". Besides the "responsiveness" aspect there also is the "bandwidth" aspect. Cheers -e -- Eberhard Moenkeberg (emoenke@gwdg.de, em@kki.org) --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
On Mon, Jul 10, 2006 at 12:00:32PM +0200, Michael Schroeder wrote:
On Fri, Jul 07, 2006 at 11:13:39PM +0200, Robert Schiele wrote:
On Fri, Jul 07, 2006 at 09:46:28PM +0200, Michael Schroeder wrote:
- support for mirror servers (i.e. what Robert suggested: Pull the filelist from the main server, get the deltas from a mirror server).
Please apply what is described in https://bugzilla.novell.com/show_bug.cgi?id=183793 to the repository building process.
I'm not sure that this is a good idea. Just ask the server, it should be much more responsive.
Anyway, I added the --use-filelist and --get-filelist option exactly for that reason. M. --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
On Mon, Jul 10, 2006 at 12:00:32PM +0200, Michael Schroeder wrote:
On Fri, Jul 07, 2006 at 11:13:39PM +0200, Robert Schiele wrote:
Please apply what is described in https://bugzilla.novell.com/show_bug.cgi?id=183793 to the repository building process.
I'm not sure that this is a good idea. Just ask the server, it should be much more responsive.
I am sure this is a good idea because it eliminates the last single point of failure in the protocol. And that this is relevant is proven by the fact that this single point of failure made the service unusable for the last few weeks. It would be a very good idea if you argued why you think this is a bad idea. Just postulating this does not really convince me. Actually the contents file as shipped from the drpmsync server is an index over the repository. So what's actually wrong with shipping it _with_ the repository. I mean you are also shipping repository meta data although they could be provided by a proprietary protocol server as well. Robert -- Robert Schiele Tel.: +49-621-181-2214 Dipl.-Wirtsch.informatiker mailto:rschiele@uni-mannheim.de "Quidquid latine dictum sit, altum sonatur."
On Mon, Jul 10, 2006 at 12:58:47PM +0200, Robert Schiele wrote:
It would be a very good idea if you argued why you think this is a bad idea. Just postulating this does not really convince me.
Actually I don't need to convince you. You need to convince me.
Actually the contents file as shipped from the drpmsync server is an index over the repository. So what's actually wrong with shipping it _with_ the repository. I mean you are also shipping repository meta data although they could be provided by a proprietary protocol server as well.
What you're saying is like "the rsync server should put the contents on disk". I just changed the format of the on-disk filelist. It now includes an id and a md5sum. Please update to the current svn version. It now also supports on-disk filelist gzip compression. (As you can see I'm working in your direction, so that we can easily add a drpmsync.filelist file) Micha. -- Michael Schroeder mls@suse.de main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
On Mon, Jul 10, 2006 at 02:20:23PM +0200, Michael Schroeder wrote:
On Mon, Jul 10, 2006 at 12:58:47PM +0200, Robert Schiele wrote:
It would be a very good idea if you argued why you think this is a bad idea. Just postulating this does not really convince me.
Actually I don't need to convince you. You need to convince me.
No, I don't need. If Novell insists in keeping all of the vital infrastructure of the openSUSE project prorietary and you exploit that fact refusing to keep the discussion on a senseful level by not providing arguments for your opinions then I should most likely opt out at this point. In that case every further minute I invest here seems to be a complete waste of time and I should prefer investing that time in preparing my move to Vancouver. So you might want to think about how to communicate effectively. I confess that I might be wrong sometimes in my opinion but as long as you don't tell me _why_ you think I am wrong, I have to assume that I am not. Playing "You are wrong but I won't tell you why, guess again!" was something I did stop a short time after I left the kindergarden.
Actually the contents file as shipped from the drpmsync server is an index over the repository. So what's actually wrong with shipping it _with_ the repository. I mean you are also shipping repository meta data although they could be provided by a proprietary protocol server as well.
What you're saying is like "the rsync server should put the contents on disk".
Whoever puts it there. It should be visible through traditional protocols like rsync to support mirrors that do not run drpmsync _without_ the need to contact a single point of failure server before.
I just changed the format of the on-disk filelist. It now includes an id and a md5sum. Please update to the current svn version. It now also supports on-disk filelist gzip compression.
That's fine. Now we still need to _have_ this file somewhere on the disk to make use of that.
(As you can see I'm working in your direction, so that we can easily add a drpmsync.filelist file)
Very good. Now if you could provide arguments for your opinions on a regular basis this could make discussions much less annoying. Robert -- Robert Schiele Tel.: +49-621-181-2214 Dipl.-Wirtsch.informatiker mailto:rschiele@uni-mannheim.de "Quidquid latine dictum sit, altum sonatur."
On Mon, Jul 10, 2006 at 03:01:41PM +0200, Robert Schiele wrote:
No, I don't need. If Novell insists in keeping all of the vital infrastructure of the openSUSE project prorietary
drpmsync isn't proprietary. Stop spreading nonsense.
and you exploit that fact refusing to keep the discussion on a senseful level by not providing arguments for your opinions then I should most likely opt out at this point. In that case every further minute I invest here seems to be a complete waste of time and I should prefer investing that time in preparing my move to Vancouver.
Sure, if that suits you better, go ahead.
What you're saying is like "the rsync server should put the contents on disk".
Whoever puts it there. It should be visible through traditional protocols like rsync to support mirrors that do not run drpmsync _without_ the need to contact a single point of failure server before.
You didn't get my point: I said that arguing that drpmsync should put its filelist on disk is like saying that rsync should put its (i.e. rsync's) internal file list on the disk. The main problem with this is just that the data is outdated if there's an update just running, so you'll get a lot more errors about missing files than when the filelist is generated on the fly. Cheers, Michael. -- Michael Schroeder mls@suse.de main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
On Mon, Jul 10, 2006 at 03:19:48PM +0200, Michael Schroeder wrote:
On Mon, Jul 10, 2006 at 03:01:41PM +0200, Robert Schiele wrote:
No, I don't need. If Novell insists in keeping all of the vital infrastructure of the openSUSE project prorietary
drpmsync isn't proprietary. Stop spreading nonsense.
I said vital infrastructure, not drpmsync in particular. And this is not nonsense but a fact. The scripts for building the repository are proprietary and those are the ones I considered best to put this into.
and you exploit that fact refusing to keep the discussion on a senseful level by not providing arguments for your opinions then I should most likely opt out at this point. In that case every further minute I invest here seems to be a complete waste of time and I should prefer investing that time in preparing my move to Vancouver.
Sure, if that suits you better, go ahead.
Thanks for finally being open here. Now at least I have written consent that it does not make any sense.
Whoever puts it there. It should be visible through traditional protocols like rsync to support mirrors that do not run drpmsync _without_ the need to contact a single point of failure server before.
You didn't get my point: I said that arguing that drpmsync should put its filelist on disk is like saying that rsync should put its (i.e. rsync's) internal file list on the disk.
The main problem with this is just that the data is outdated if there's an update just running, so you'll get a lot more errors about missing files than when the filelist is generated on the fly.
No. For the drpmsync server itself it is perfectly ok to create the list on the fly. But there should be a static version available for non-drpmsync-server systems. The argument that this might get out of sync does not really match here because getting out of sync is ways better than having no server at all. Again: My proposal should not influence the way the drpmsync server operates at all. I intends to provide an _alternative_ way of doing things without doing any harm (but using about 3MB of disk space) to any other service. I would even provide a patch for the repository building scripts to do so but because they are prorietary (Yes, they are!) I cannot do this. And since Novell neither gives support to include this feature there nor does it provide any reason for not doing so, I will now follow your advice from above and cancel this project until Novell changes their general attitude to this project. Robert -- Robert Schiele Tel.: +49-621-181-2214 Dipl.-Wirtsch.informatiker mailto:rschiele@uni-mannheim.de "Quidquid latine dictum sit, altum sonatur."
On Mon, Jul 10, 2006 at 04:02:00PM +0200, Robert Schiele wrote:
You didn't get my point: I said that arguing that drpmsync should put its filelist on disk is like saying that rsync should put its (i.e. rsync's) internal file list on the disk.
The main problem with this is just that the data is outdated if there's an update just running, so you'll get a lot more errors about missing files than when the filelist is generated on the fly.
No. For the drpmsync server itself it is perfectly ok to create the list on the fly. But there should be a static version available for non-drpmsync-server systems. The argument that this might get out of sync does not really match here because getting out of sync is ways better than having no server at all.
But that was when the server was misconfigured. The cache file generation was broken due to a bad app armor configuration and the number of clients limited to two. Both things have been fixed last week.
Again: My proposal should not influence the way the drpmsync server operates at all. I intends to provide an _alternative_ way of doing things without doing any harm (but using about 3MB of disk space) to any other service.
Agreed, it wouldn't break things. I'm just a bit reluctant because of the outdated filelist problems, as already stated.
I would even provide a patch for the repository building scripts to do so but because they are prorietary (Yes, they are!) I cannot do this.
What do you mean? The repository is synced with drpmsync (and so the delta rpms are created). There is no secret script. The way to create the on disk filelist would just be another drpmsync call. Cheers, Michael. -- Michael Schroeder mls@suse.de main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
On Mon, Jul 10, 2006 at 04:15:58PM +0200, Michael Schroeder wrote:
On Mon, Jul 10, 2006 at 04:02:00PM +0200, Robert Schiele wrote:
No. For the drpmsync server itself it is perfectly ok to create the list on the fly. But there should be a static version available for non-drpmsync-server systems. The argument that this might get out of sync does not really match here because getting out of sync is ways better than having no server at all.
But that was when the server was misconfigured. The cache file generation was broken due to a bad app armor configuration and the number of clients limited to two. Both things have been fixed last week.
Sure this was mainly due to a bug. But what do you expect? That you never hit any bug in the future again? It is the whole idea of eliminating single points of failures to prevent suffering from _failures_ too hard.
Again: My proposal should not influence the way the drpmsync server operates at all. I intends to provide an _alternative_ way of doing things without doing any harm (but using about 3MB of disk space) to any other service.
Agreed, it wouldn't break things. I'm just a bit reluctant because of the outdated filelist problems, as already stated.
Well, if it finally proves that my idea was completely crap then we could easily remove this file again. But not even trying it just because there _might_ be a problem does not really make sense if the is no big risk involved.
I would even provide a patch for the repository building scripts to do so but because they are prorietary (Yes, they are!) I cannot do this.
What do you mean? The repository is synced with drpmsync (and so the delta rpms are created). There is no secret script. The way to create the on disk filelist would just be another drpmsync call.
Well, ok, maybe this is the source of some misunderstanding: My idea was to create this file immediately when creating the repository (on the same system that creates files like INDEX.gz or ARCHIVES.gz). In that case the file will be always in sync with the repository at any time a mirror sync is complete. Sure the file _could_ be also generated by the drpmsync server but I would recommend to decouple this completely (besides the fact that code might be shared and that the actual client makes use of the deltas generated by the real drpmsync server) because this file describes only states of the files contained in the repository. The logic how to use this information to apply patches is completely up to the client implementation. Robert -- Robert Schiele Tel.: +49-621-181-2214 Dipl.-Wirtsch.informatiker mailto:rschiele@uni-mannheim.de "Quidquid latine dictum sit, altum sonatur."
On Mon, Jul 10, 2006 at 05:19:12PM +0200, Robert Schiele wrote:
Again: My proposal should not influence the way the drpmsync server operates at all. I intends to provide an _alternative_ way of doing things without doing any harm (but using about 3MB of disk space) to any other service.
Agreed, it wouldn't break things. I'm just a bit reluctant because of the outdated filelist problems, as already stated.
Well, if it finally proves that my idea was completely crap then we could easily remove this file again. But not even trying it just because there _might_ be a problem does not really make sense if the is no big risk involved.
Ok, I give up. Lets give it a try.
I would even provide a patch for the repository building scripts to do so but because they are prorietary (Yes, they are!) I cannot do this.
What do you mean? The repository is synced with drpmsync (and so the delta rpms are created). There is no secret script. The way to create the on disk filelist would just be another drpmsync call.
Well, ok, maybe this is the source of some misunderstanding: My idea was to create this file immediately when creating the repository (on the same system that creates files like INDEX.gz or ARCHIVES.gz). In that case the file will be always in sync with the repository at any time a mirror sync is complete.
Sure the file _could_ be also generated by the drpmsync server but I would recommend to decouple this completely (besides the fact that code might be shared and that the actual client makes use of the deltas generated by the real drpmsync server) because this file describes only states of the files contained in the repository. The logic how to use this information to apply patches is completely up to the client implementation.
Maybe, but it IMHO makes no sense to put drpmsync code into the repository creation code. I'll just call drpmsync for the job. Cheers, Michael. -- Michael Schroeder mls@suse.de main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
On Mon, Jul 10, 2006 at 06:06:15PM +0200, Michael Schroeder wrote:
Ok, I give up. Lets give it a try.
Thanks!
Maybe, but it IMHO makes no sense to put drpmsync code into the repository creation code. I'll just call drpmsync for the job.
Ok, maybe the place where the deltas itself are created is the right place then. Robert -- Robert Schiele Tel.: +49-621-181-2214 Dipl.-Wirtsch.informatiker mailto:rschiele@uni-mannheim.de "Quidquid latine dictum sit, altum sonatur."
On Mon, Jul 10, 2006 at 06:46:05PM +0200, Robert Schiele wrote:
On Mon, Jul 10, 2006 at 06:06:15PM +0200, Michael Schroeder wrote:
Ok, I give up. Lets give it a try.
Thanks!
There's now a "/Factory/drpmsync/filelist.gz" file. Enjoy, Michael. -- Michael Schroeder mls@suse.de main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
Hi, On Mon, 10 Jul 2006, Michael Schroeder wrote:
On Mon, Jul 10, 2006 at 06:46:05PM +0200, Robert Schiele wrote:
On Mon, Jul 10, 2006 at 06:06:15PM +0200, Michael Schroeder wrote:
Ok, I give up. Lets give it a try.
Thanks!
There's now a "/Factory/drpmsync/filelist.gz" file.
Landed as ftp://ftp-1.gwdg.de/pub/opensuse/distribution/SL-OSS-factory/drpmsync/filelist.gz http://ftp-1.gwdg.de/pub/opensuse/distribution/SL-OSS-factory/drpmsync/filel... rsync://ftp-1.gwdg.de/pub/opensuse/distribution/SL-OSS-factory/drpmsync/filelist.gz Cheers -e -- Eberhard Moenkeberg (emoenke@gwdg.de, em@kki.org) --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
On Mon, Jul 10, 2006 at 10:58:40PM +0200, Eberhard Moenkeberg wrote:
Hi,
On Mon, 10 Jul 2006, Michael Schroeder wrote:
There's now a "/Factory/drpmsync/filelist.gz" file.
Landed as
ftp://ftp-1.gwdg.de/pub/opensuse/distribution/SL-OSS-factory/drpmsync/filelist.gz http://ftp-1.gwdg.de/pub/opensuse/distribution/SL-OSS-factory/drpmsync/filel... rsync://ftp-1.gwdg.de/pub/opensuse/distribution/SL-OSS-factory/drpmsync/filelist.gz
Ok, so using the URL rsync://ftp-1.gwdg.de/pub/opensuse/distribution/SL-OSS-factory as REPOS_RSYNC in my client now should allow syncing the repository to a system using the delta files (if old RPM files are already present). Robert -- Robert Schiele Tel.: +49-621-181-2214 Dipl.-Wirtsch.informatiker mailto:rschiele@uni-mannheim.de "Quidquid latine dictum sit, altum sonatur."
Hi, On Mon, 10 Jul 2006, Robert Schiele wrote:
On Mon, Jul 10, 2006 at 10:58:40PM +0200, Eberhard Moenkeberg wrote:
On Mon, 10 Jul 2006, Michael Schroeder wrote:
There's now a "/Factory/drpmsync/filelist.gz" file.
Landed as
ftp://ftp-1.gwdg.de/pub/opensuse/distribution/SL-OSS-factory/drpmsync/filelist.gz http://ftp-1.gwdg.de/pub/opensuse/distribution/SL-OSS-factory/drpmsync/filel... rsync://ftp-1.gwdg.de/pub/opensuse/distribution/SL-OSS-factory/drpmsync/filelist.gz
Ok, so using the URL rsync://ftp-1.gwdg.de/pub/opensuse/distribution/SL-OSS-factory as REPOS_RSYNC in my client now should allow syncing the repository to a system using the delta files (if old RPM files are already present).
Who had thought in the morning that we would make such a good step today. ;-)) Next should be to add the drpmsync directory into the OpenSUSE rsync push service, to deliver the new files just-in-time to the mirrors. ftp-1 currently is syncing every 4 hours, but "getting pushed" would ensure best actuality at every time. Cheers -e -- Eberhard Moenkeberg (emoenke@gwdg.de, em@kki.org) --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
On Mon, Jul 10, 2006 at 08:21:35PM +0200, Michael Schroeder wrote:
There's now a "/Factory/drpmsync/filelist.gz" file.
Good. On http://pi3.informatik.uni-mannheim.de/~schiele/mydrpmsync/ there is now a version of my client that can sync the repository from any rsync server that also has the delta files mirrored without contacting the drpmsync server. Whoever wants to try this is welcome. Note that you currently have to compile convert.c in the directory from which you want to run mydrpmsync. This is not clean but since this is a first quick hack it seems acceptable to me for a first shot. Robert -- Robert Schiele Tel.: +49-621-181-2214 Dipl.-Wirtsch.informatiker mailto:rschiele@uni-mannheim.de "Quidquid latine dictum sit, altum sonatur."
Hi, On Mon, 10 Jul 2006, Michael Schroeder wrote:
On Mon, Jul 10, 2006 at 03:01:41PM +0200, Robert Schiele wrote:
No, I don't need. If Novell insists in keeping all of the vital infrastructure of the openSUSE project prorietary
drpmsync isn't proprietary. Stop spreading nonsense.
"keep infrastructure proprietary" means just your damned file list that you seemed to refuse to put "on disk".
and you exploit that fact refusing to keep the discussion on a senseful level by not providing arguments for your opinions then I should most likely opt out at this point. In that case every further minute I invest here seems to be a complete waste of time and I should prefer investing that time in preparing my move to Vancouver.
Sure, if that suits you better, go ahead.
What you're saying is like "the rsync server should put the contents on disk".
Whoever puts it there. It should be visible through traditional protocols like rsync to support mirrors that do not run drpmsync _without_ the need to contact a single point of failure server before.
You didn't get my point: I said that arguing that drpmsync should put its filelist on disk is like saying that rsync should put its (i.e. rsync's) internal file list on the disk.
If rsync's internal file list could help other standard tools to fulfill a better service, this would be a good idea.
The main problem with this is just that the data is outdated if there's an update just running, so you'll get a lot more errors about missing files than when the filelist is generated on the fly.
So please talk to Christoph & friends about adding the whole drpmsync directory into the already existing rsync push service. This way, each single change would get propagated to the mirror servers instantly. Cheers -e -- Eberhard Moenkeberg (emoenke@gwdg.de, em@kki.org) --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
On Fri, Jul 07, 2006 at 09:46:28PM +0200, Michael Schroeder wrote:
I finally found some time to work a bit on drpmsync. A new version is available from https://forgesvn1.novell.com/svn/drpmsync/trunk .
I just uploaded an even newer version. It now supports rsync style filters: --include <pat> --exclude <pat> --include-arch <pat> --exclude-arch <pat> The arch filters don't work on an rsync source, as the rpm architecture information is not available in this case. Enjoy, Michael. -- Michael Schroeder mls@suse.de main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-unsubscribe@opensuse.org For additional commands, e-mail: opensuse-help@opensuse.org
participants (5)
-
Christoph Thiel
-
Eberhard Moenkeberg
-
Marcus Rueckert
-
Michael Schroeder
-
Robert Schiele