[opensuse-buildservice] "the Open Build Service team having upgraded the build farm with SSDs"?
All, In http://news.opensuse.org/2012/09/05/opensuse-12-2-green-means-go/ it says "the Open Build Service team having upgraded the build farm with SSDs and using preinstall images to rapidly setup build virtual machines." News to me, can we have an update? How many? Is performance improved? etc? Thanks Greg -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Am Mittwoch, 5. September 2012, 17:59:36 schrieb Greg Freemyer:
All,
In http://news.opensuse.org/2012/09/05/opensuse-12-2-green-means-go/ it says
"the Open Build Service team having upgraded the build farm with SSDs and using preinstall images to rapidly setup build virtual machines."
News to me, can we have an update? How many? Is performance improved? etc?
No, it has not (build19 has them). But we need more time to find the reason.
Thanks Greg
-- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Wed, Sep 05, 2012 at 05:59:36PM -0400, Greg Freemyer wrote:
In http://news.opensuse.org/2012/09/05/opensuse-12-2-green-means-go/ it says
"the Open Build Service team having upgraded the build farm with SSDs and using preinstall images to rapidly setup build virtual machines."
News to me, can we have an update? How many? Is performance improved? etc?
Using preinstall images helped quite a bit to speed up setting up the build environment, my guess is that it took away 30 seconds on average. That's doesn't sound like much, but the average build time is about 7 minutes. Our SSD machine isn't that much faster then the other machines for some reason. We need to investigate this further. Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Thu, Sep 6, 2012 at 5:25 AM, Michael Schroeder <mls@suse.de> wrote:
On Wed, Sep 05, 2012 at 05:59:36PM -0400, Greg Freemyer wrote:
In http://news.opensuse.org/2012/09/05/opensuse-12-2-green-means-go/ it says
"the Open Build Service team having upgraded the build farm with SSDs and using preinstall images to rapidly setup build virtual machines."
News to me, can we have an update? How many? Is performance improved? etc?
Using preinstall images helped quite a bit to speed up setting up the build environment, my guess is that it took away 30 seconds on average. That's doesn't sound like much, but the average build time is about 7 minutes.
Our SSD machine isn't that much faster then the other machines for some reason. We need to investigate this further.
SATA interfaced SSDs are much faster than rotating disks for seeks, not linear reads. If you have the preinstall images and disk cache working well, that may be eliminating most of the advantage of a SSD. As a test, you could find a package with a large set of packages installed and maybe a kernel compile that takes a long time to run. If the SSD doesn't accelerate one of those 2 workloads, I would be surprised. If SATA interfaced SSDs don't help, you can look at native PCI express SSDs, but I don't know much about those including what the driver status is. Greg -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Thu, Sep 6, 2012 at 6:25 AM, Michael Schroeder <mls@suse.de> wrote:
Our SSD machine isn't that much faster then the other machines for some reason. We need to investigate this further.
Many builds may not be I/O bound. You should log into build19 and use iostat to monitor I/O activity, especially % utilization, to check that assumption. -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Thu, Sep 6, 2012 at 6:25 AM, Michael Schroeder <mls@suse.de> wrote:
"the Open Build Service team having upgraded the build farm with SSDs and using preinstall images to rapidly setup build virtual machines."
News to me, can we have an update? How many? Is performance improved? etc?
Using preinstall images helped quite a bit to speed up setting up the build environment, my guess is that it took away 30 seconds on average. That's doesn't sound like much, but the average build time is about 7 minutes.
You know what would speed up environment setup *and* decrease the amount of I/O? Hardlinking common files between build workers. If you check the monitor and tell it to show the repository being built, you'll see a lot of potential sharing. Sharing not only avoids the work of having to unpack the rpm, it also avoids the need to have two copies of the same file and thus lowers I/O requirements. Pulling it off, however, isn't easy, but the stork project did it for PlanetLab and it may be useful in the OBS. Maybe this is a good GSOC project ;-) -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Sat, Sep 08, 2012 at 04:36:52PM -0300, Claudio Freire wrote:
You know what would speed up environment setup *and* decrease the amount of I/O?
Hardlinking common files between build workers.
We already use a common storage for the rpms and the preinstall images. Sharing files from inside the vms isn't that easy for security/encapsulation reasons. Cheers, Michael. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Mon, Sep 10, 2012 at 6:53 AM, Michael Schroeder <mls@suse.de> wrote:
Hardlinking common files between build workers.
We already use a common storage for the rpms and the preinstall images. Sharing files from inside the vms isn't that easy for security/encapsulation reasons.
I know, but, like I said, stork{0] did do a lot of that. I'm not sure how secure it is, but it must be somewhat secure since it's intended for vservers. [0] http://www.cs.arizona.edu/stork/tutmain.html -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
participants (4)
-
Adrian Schröter -
Claudio Freire -
Greg Freemyer -
Michael Schroeder