On Wed, 08 Sep 2010 16:36:21 +0200, "Dominique Leuenberger"
 wrote:
>>>> On 09/08/2010 at 4:26 PM,  wrote:
>> - I presume one of these is set up/removed  for each of the packages in
a
>> users project when time comes to compile the package.
>> - If that is done on-disk that would use alot of elapsed I/O time. That
>> would add up over time for the many thousands of packages built.
>> 
>> If that is the case, could a buildroot location be set in ram-memory
and
>> a
>> symbolic link setup be done to connect the users package and the memory
>> location.
>> The package built on faster ram instead of using slower disk if disk is
>> used.  Quicker to setup and cleanup the start and end of each process.
>> 
>> The reason I asked was would it help the through-put turnaround time of
>> jobs submitted.
>> Just my 20cents worth 
>> Cheers Glenn 
> 
> I do use that on my own host for local builds, mainly with a tmpfs
> pointing to /var/tmp/build-root.
> 
> Out of curiosity I thought I could time this here...
> 
> so: without tmpfs (physical IO):
> 
> time osc build
> real	0m52.067s
> user	0m31.046s
> sys	0m4.868s
> 
> 
> With tmpfs (4GB.. so if it ever requires more, bad luck):
> time osc build
> real	0m24.708s
> user	0m15.237s
> sys	0m2.448s
> 
> 
> Certainly not the most interesting package to win time, but yes,
generally
> it saved me 50% of time in this case.
> 
> Dominique

Scaled up , how many smallish < say 3GB packages could be done on a very
busy day by an obs host.
- I classify smallish/tiny as < 3GB because im used to dealing with multi
100-500 GB oracle databases in my day job.

Cheers Glenn :-)
-- 
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org
For additional commands, e-mail: opensuse-buildservice+help@opensuse.org

more important ... we got some packages which easily exceed those 8GB while building. darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org

On Wednesday 08 September 2010 17:43:36 doiggl@velocitynet.com.au wrote:

On Wed, 8 Sep 2010 17:13:58 +0200, Marcus Rueckert wrote:

...

more important ... we got some packages which easily exceed those 8GB while building.

darix

If you could exclude the bigger packages and get the smaller ones done. It would do alot of jobs. Perhaps do the bigger jobs on another host.

Yes, would be nice and IIRC we have also fate requests for this. One could even do dynamic repartition LVM volumes to match the build jobs. Now we just need someone who wants to work on this :) -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org

On Wed, Sep 8, 2010 at 10:36 AM, Dominique Leuenberger
 wrote:
>>>> On 09/08/2010 at 4:26 PM,  wrote:
>> - I presume one of these is set up/removed  for each of the packages in a
>> users project when time comes to compile the package.
>> - If that is done on-disk that would use alot of elapsed I/O time. That
>> would add up over time for the many thousands of packages built.
>>
>> If that is the case, could a buildroot location be set in ram-memory and a
>> symbolic link setup be done to connect the users package and the memory
>> location.
>> The package built on faster ram instead of using slower disk if disk is
>> used.  Quicker to setup and cleanup the start and end of each process.
>>
>> The reason I asked was would it help the through-put turnaround time of
>> jobs submitted.
>> Just my 20cents worth
>> Cheers Glenn
>
> I do use that on my own host for local builds, mainly with a tmpfs pointing to /var/tmp/build-root.
>
> Out of curiosity I thought I could time this here...
>
> so: without tmpfs (physical IO):
>
> time osc build
> real    0m52.067s
> user    0m31.046s
> sys     0m4.868s
>
>
> With tmpfs (4GB.. so if it ever requires more, bad luck):
> time osc build
> real    0m24.708s
> user    0m15.237s
> sys     0m2.448s
>
>
> Certainly not the most interesting package to win time, but yes, generally it saved me 50% of time in this case.
>
> Dominique

You'd probably get most of that speed by using a SSD for the tmpfs.

iirc.  Good SSD is about $3/GB, so you could put in 8 * 32GB for less
than $1K.  (I'm assuming 8 VMs with 32GB of tmpfs)

Not exactly cheap, but I assume a lot less $s than adding a full
additional server the next time the build farm needs to be added to.

Greg
-- 
To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org
For additional commands, e-mail: opensuse-buildservice+help@opensuse.org