[opensuse-buildservice] Re: withbinaries copyproject without rebuild
On Thu, Jan 16, 2014 at 02:12:44AM +0300, Kanstantsin Shautsou wrote:
@mls i had all service files in disabled mode, probably that why i end with new md5 for the same sources. Checksum for the same files is random, right?
Every new service run gets a new id, as the result may be different. Dunno if the service code checks if they are all disabled or not. And my idea in getting reproducible hashes if files not changed. For example: file1 hash1 file2 hash2
On Jan 16, 2014, at 17:07 , Michael Schroeder <mls@suse.de> wrote: then final hash = hash(hash1, hash2). File name should be also part of hash i suppose.
I want rework it and do repository copy in one call and without copies to job dirs and without stopping scheduler. How better implement this? I think then "obs_admin --clone-repository" can switch to this functionality then.
Conceptually the scheduler "owns" the build tree, i.e. it is assumed that only the scheduler does any changes. That way we don't need any locking, as just one process does the modifications.
That's also why things like binary uploads go through putting the files into a job dir and letting the scheduler do the integration part.
So the repo copy would IMHO need to be similar: stage the copy into some tmp job dir and let the scheduler do the final renaming.
Thanks, and what files can i link()? Hard link is used for build->jobdir and then for jobdir->newprj_builddir. New project files will be hard linked with src project and this potentially can effect if some daemon do open() for files. Also BSUtil:cp do open for src and dst that will cause in content changes for all hard linked files -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Thanks, and what files can i link()? Hard link is used for build->jobdir and then for jobdir->newprj_builddir. New project files will be hard linked with src project and this potentially can effect if some daemon do open() for files. Also BSUtil:cp do open for src and dst that will cause in content changes for all hard linked files
There are only very few files that get appended to, the standard operation for file writing is to write the content in a tmp file and then rename to the destination. For the "build" area, the :jobhistory file is appended to, so don't hardlink that file. Maybe it's best to not link all files starting with ':'. The only other exception is the "history" file in the packages. It also is appended to. Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
So the repo copy would IMHO need to be similar: stage the copy into some tmp job dir and let the scheduler do the final renaming. But scheduler belong to some architecture and repo includes multiple arches...-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Tue, Jan 21, 2014 at 08:17:07PM +0300, Kanstantsin Shautsou wrote:
So the repo copy would IMHO need to be similar: stage the copy into some tmp job dir and let the scheduler do the final renaming. But scheduler belong to some architecture and repo includes multiple arches...
No, the repos used for building are architecture specific, i.e. the builds for "x86_64" will only fetch binaries from the "x86_64" area. The publisher mixes scheduler architecture when creating the published repositories. Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Jan 21, 2014, at 20:52 , Michael Schroeder <mls@suse.de> wrote:
On Tue, Jan 21, 2014 at 08:17:07PM +0300, Kanstantsin Shautsou wrote:
So the repo copy would IMHO need to be similar: stage the copy into some tmp job dir and let the scheduler do the final renaming. But scheduler belong to some architecture and repo includes multiple arches...
No, the repos used for building are architecture specific, i.e. the builds for "x86_64" will only fetch binaries from the "x86_64" area.
The publisher mixes scheduler architecture when creating the published repositories.
Cheers, Michael.
-- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} I don’t touch publisher, because i don’t preserve :repodone file.
Even if i split copy operation to $repo/$arch bases (instead of current $repo/$arch/$packid) i wouldn’t have ability to say all schedulers “wait”. I moved binary copying to srcserver but scheduler still starts builds https://github.com/KostyaSha/open-build-service/commit/5908e0b1dd297fc063e5f... Is there any way to lock $repo or $repo/$arch from scheduler? Seems like it detect new sources and schedule build before binaries are copied… Should sched be locked before sources copying?-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
participants (2)
-
Kanstantsin Shautsou -
Michael Schroeder