Hello With the opensuse build service [obs] getting more usage every day I wonder if this would make a diffence :-
1. Use solid state disks [ssd] to improve build times of package rpms on, perhaps point the BuildRoot or _tmppath to the ssd's
Reason for asking. Improve the packaging times during busy times on obs.
Cheers Glenn.
On Sat, Apr 16, 2011 at 02:19, doiggl@velocitynet.com.au wrote:
Hello With the opensuse build service [obs] getting more usage every day I wonder if this would make a diffence :-
- Use solid state disks [ssd] to improve build times of package rpms
on, perhaps point the BuildRoot or _tmppath to the ssd's
Reason for asking. Improve the packaging times during busy times on obs.
Cheers Glenn.
Unfortunately there is a cost associated, but it would be cool to get a bunch of fusion IO cards and see how they perform.
Here is a youtube video of 1 fusion IO card handling 4500 movies streaming at once. If I understood it right.
http://www.youtube.com/watch?v=9J5xGwdmsuo
- Stephen
El 19/04/11 14:28, Stephen Shaw escribió:
On Sat, Apr 16, 2011 at 02:19, doiggl@velocitynet.com.au wrote:
Hello With the opensuse build service [obs] getting more usage every day I wonder if this would make a diffence :-
- Use solid state disks [ssd] to improve build times of package rpms
on, perhaps point the BuildRoot or _tmppath to the ssd's
Reason for asking. Improve the packaging times during busy times on obs.
Cheers Glenn.
Unfortunately there is a cost associated, but it would be cool to get a bunch of fusion IO cards and see how they perform.
But those are really expensive... 160GB storage for a few _thousands_ of dollars.. :)
On Tue, 19 Apr 2011, Stephen Shaw wrote:
On Sat, Apr 16, 2011 at 02:19, doiggl@velocitynet.com.au wrote:
Hello With the opensuse build service [obs] getting more usage every day I wonder if this would make a diffence :-
- Use solid state disks [ssd] to improve build times of package rpms
on, perhaps point the BuildRoot or _tmppath to the ssd's
Reason for asking. Improve the packaging times during busy times on obs.
Cheers Glenn.
Unfortunately there is a cost associated, but it would be cool to get a bunch of fusion IO cards and see how they perform.
The laptop I do all my builds from before submitting to OBS has an SSD in it, and I must say, it's delightful :)
% osc build --ccache
And I'm off to the races!
That said, this is a simple Corsair X64, I've priced higher quality SSDs for servers before and it's a good jump in price from standard laptop SSDs still :(
Cheers - R. Tyler Croy -------------------------------------- Code: http://github.com/rtyler Chatter: http://identi.ca/agentdero http://twitter.com/agentdero
On Tue, Apr 19, 2011 at 1:28 PM, Stephen Shaw sshaw@decriptor.com wrote:
On Sat, Apr 16, 2011 at 02:19, doiggl@velocitynet.com.au wrote:
Hello With the opensuse build service [obs] getting more usage every day I wonder if this would make a diffence :-
- Use solid state disks [ssd] to improve build times of package rpms
on, perhaps point the BuildRoot or _tmppath to the ssd's
Reason for asking. Improve the packaging times during busy times on obs.
Cheers Glenn.
Unfortunately there is a cost associated, but it would be cool to get a bunch of fusion IO cards and see how they perform.
Here is a youtube video of 1 fusion IO card handling 4500 movies streaming at once. If I understood it right.
Clearly buying SSD costs money.
But buying a $1000 250 GB SSD for one of the build servers would be cheaper than buying a whole new server.
So the next time an expansion to the build farm is budgeted, I hope getting SSDs for the buildroot is considered as an option.
If the thread from 6 months ago is still accurate, having a SSD might double the compile speed of a build farm node.
Greg
El 19/04/11 14:45, R. Tyler Croy escribió:
The laptop I do all my builds from before submitting to OBS has an SSD in it, and I must say, it's delightful :)
% osc build --ccacheAnd I'm off to the races!
Yeah, it is pretty cool, I use an Intel SSD X25-M and is indeed a pleasure to build on it.
Unfortunately such deployment in the OBS can get quite expensive.
Am Dienstag, 19. April 2011, 19:42:50 schrieb Cristian Rodríguez:
El 19/04/11 14:28, Stephen Shaw escribió:
On Sat, Apr 16, 2011 at 02:19, doiggl@velocitynet.com.au wrote:
Hello With the opensuse build service [obs] getting more usage every day I wonder if this would make a diffence :-
- Use solid state disks [ssd] to improve build times of package rpms
on, perhaps point the BuildRoot or _tmppath to the ssd's
Reason for asking. Improve the packaging times during busy times on obs.
Cheers Glenn.
Unfortunately there is a cost associated, but it would be cool to get a bunch of fusion IO cards and see how they perform.
But those are really expensive... 160GB storage for a few _thousands_ of dollars.. :)
Its more effective to place workers in tmpfs . Enough RAM needed ;) .
Best, Jan-Simon
On Tue, Apr 19, 2011 at 12:00, Jan-Simon Möller dl9pf@gmx.de wrote:
Am Dienstag, 19. April 2011, 19:42:50 schrieb Cristian Rodríguez:
El 19/04/11 14:28, Stephen Shaw escribió:
On Sat, Apr 16, 2011 at 02:19, doiggl@velocitynet.com.au wrote:
Hello With the opensuse build service [obs] getting more usage every day I wonder if this would make a diffence :-
- Use solid state disks [ssd] to improve build times of package rpms
on, perhaps point the BuildRoot or _tmppath to the ssd's
Reason for asking. Improve the packaging times during busy times on obs.
Cheers Glenn.
Unfortunately there is a cost associated, but it would be cool to get a bunch of fusion IO cards and see how they perform.
But those are really expensive... 160GB storage for a few _thousands_ of dollars.. :)
Its more effective to place workers in tmpfs . Enough RAM needed ;) .
Best, Jan-Simon
It would be cool if Intel donated a bunch of SSDs so that they could get their logo up on the OBS site.
Cheers, Stephen
On 19/04/11 18:49, Greg Freemyer wrote:
On Tue, Apr 19, 2011 at 1:28 PM, Stephen Shawsshaw@decriptor.com wrote:
On Sat, Apr 16, 2011 at 02:19,doiggl@velocitynet.com.au wrote:
Hello With the opensuse build service [obs] getting more usage every day I wonder if this would make a diffence :-
- Use solid state disks [ssd] to improve build times of package rpms
on, perhaps point the BuildRoot or _tmppath to the ssd's
Reason for asking. Improve the packaging times during busy times on obs.
Cheers Glenn.
Unfortunately there is a cost associated, but it would be cool to get a bunch of fusion IO cards and see how they perform.
Here is a youtube video of 1 fusion IO card handling 4500 movies streaming at once. If I understood it right.
Clearly buying SSD costs money.
But buying a $1000 250 GB SSD for one of the build servers would be cheaper than buying a whole new server.
So the next time an expansion to the build farm is budgeted, I hope getting SSDs for the buildroot is considered as an option.
If the thread from 6 months ago is still accurate, having a SSD might double the compile speed of a build farm node.
Don't forget that workers are very, very write-heavy. This is not the best I/O pattern for SSD; and of course the vast majority of data written during a build-preparation is never re-read.
I've not had time to dig into it but I understand that the MeeGo OBS used SSDs (not, AFAIK, Intel ones) and they failed so much they were replaced.
I'm not sure who has more info about this.
So the message is : try it for a while before buying a whole bunch of them. (it is still on my todo list).
David
On Tue, Apr 26, 2011 at 11:46 AM, David Greaves david@dgreaves.com wrote:
On 19/04/11 18:49, Greg Freemyer wrote:
On Tue, Apr 19, 2011 at 1:28 PM, Stephen Shawsshaw@decriptor.com wrote:
On Sat, Apr 16, 2011 at 02:19,doiggl@velocitynet.com.au wrote:
Hello With the opensuse build service [obs] getting more usage every day I wonder if this would make a diffence :-
- Use solid state disks [ssd] to improve build times of package rpms
on, perhaps point the BuildRoot or _tmppath to the ssd's
Reason for asking. Improve the packaging times during busy times on obs.
Cheers Glenn.
Unfortunately there is a cost associated, but it would be cool to get a bunch of fusion IO cards and see how they perform.
Here is a youtube video of 1 fusion IO card handling 4500 movies streaming at once. If I understood it right.
Clearly buying SSD costs money.
But buying a $1000 250 GB SSD for one of the build servers would be cheaper than buying a whole new server.
So the next time an expansion to the build farm is budgeted, I hope getting SSDs for the buildroot is considered as an option.
If the thread from 6 months ago is still accurate, having a SSD might double the compile speed of a build farm node.
Don't forget that workers are very, very write-heavy. This is not the best I/O pattern for SSD; and of course the vast majority of data written during a build-preparation is never re-read.
Why is a write heavy load not good for a SSD? Performance or Lifetime?
For performance, the big issue is random i/o vs sequential.
A rotating disk and a SSD should have similar sequential speeds.
But for random i/o the SSD should be much, much faster in that it should barely slow down due to a random i/o write pattern.
Also, the mkfs version in 11.4 tells the SSD that all of the sectors in the partition are unused (trim'ed) prior to starting the format. That should ensure consistent highspeed operation out of the SSD. If the build farm is up to 11.4 yet, just the mkfs.ext4 package could be used I suspect.
For lifetime, I don't know. But a typical build seems to take several minutes, so the writes to the SSD don't seem like they would be that outrageously heavy.
Greg
On Tue, 26 Apr 2011 17:06:04 -0400, Greg Freemyer greg.freemyer@gmail.com wrote:
On Tue, Apr 26, 2011 at 11:46 AM, David Greaves david@dgreaves.com
wrote:
On 19/04/11 18:49, Greg Freemyer wrote:
On Tue, Apr 19, 2011 at 1:28 PM, Stephen Shawsshaw@decriptor.com wrote:
On Sat, Apr 16, 2011 at 02:19,doiggl@velocitynet.com.au wrote:
Hello With the opensuse build service [obs] getting more usage every day I wonder if this would make a diffence :-
- Use solid state disks [ssd] to improve build times of package
rpms on, perhaps point the BuildRoot or _tmppath to the ssd's
Reason for asking. Improve the packaging times during busy times on obs.
Cheers Glenn.
Unfortunately there is a cost associated, but it would be cool to get a bunch of fusion IO cards and see how they perform.
Here is a youtube video of 1 fusion IO card handling 4500 movies streaming at once. If I understood it right.
Clearly buying SSD costs money.
But buying a $1000 250 GB SSD for one of the build servers would be cheaper than buying a whole new server.
So the next time an expansion to the build farm is budgeted, I hope getting SSDs for the buildroot is considered as an option.
If the thread from 6 months ago is still accurate, having a SSD might double the compile speed of a build farm node.
Don't forget that workers are very, very write-heavy. This is not the best I/O pattern for SSD; and of course the vast majority of data written during a build-preparation is never re-read.
Why is a write heavy load not good for a SSD? Performance or Lifetime?
For performance, the big issue is random i/o vs sequential.
A rotating disk and a SSD should have similar sequential speeds.
But for random i/o the SSD should be much, much faster in that it should barely slow down due to a random i/o write pattern.
Also, the mkfs version in 11.4 tells the SSD that all of the sectors in the partition are unused (trim'ed) prior to starting the format. That should ensure consistent highspeed operation out of the SSD. If the build farm is up to 11.4 yet, just the mkfs.ext4 package could be used I suspect.
For lifetime, I don't know. But a typical build seems to take several minutes, so the writes to the SSD don't seem like they would be that outrageously heavy.
Greg
More info from the company which did above demo -Glenn http://www.iomax.com.au/products.htm
On Fri, 29 Apr 2011 13:39:29 +1000, doiggl@velocitynet.com.au wrote:
On Tue, 26 Apr 2011 17:06:04 -0400, Greg Freemyer greg.freemyer@gmail.com wrote:
On Tue, Apr 26, 2011 at 11:46 AM, David Greaves david@dgreaves.com
wrote:
On 19/04/11 18:49, Greg Freemyer wrote:
On Tue, Apr 19, 2011 at 1:28 PM, Stephen Shawsshaw@decriptor.com wrote:
On Sat, Apr 16, 2011 at 02:19,doiggl@velocitynet.com.au wrote:
Hello With the opensuse build service [obs] getting more usage every day I wonder if this would make a diffence :-
- Use solid state disks [ssd] to improve build times of
package
rpms on, perhaps point the BuildRoot or _tmppath to the ssd's
Reason for asking. Improve the packaging times during busy times on obs.
Cheers Glenn.
Unfortunately there is a cost associated, but it would be cool to
get
a bunch of fusion IO cards and see how they perform.
Here is a youtube video of 1 fusion IO card handling 4500 movies streaming at once. If I understood it right.
Clearly buying SSD costs money.
But buying a $1000 250 GB SSD for one of the build servers would be cheaper than buying a whole new server.
So the next time an expansion to the build farm is budgeted, I hope getting SSDs for the buildroot is considered as an option.
If the thread from 6 months ago is still accurate, having a SSD might double the compile speed of a build farm node.
Don't forget that workers are very, very write-heavy. This is not the best I/O pattern for SSD; and of course the vast majority of data written during a build-preparation is never re-read.
Why is a write heavy load not good for a SSD? Performance or Lifetime?
For performance, the big issue is random i/o vs sequential.
A rotating disk and a SSD should have similar sequential speeds.
But for random i/o the SSD should be much, much faster in that it should barely slow down due to a random i/o write pattern.
Also, the mkfs version in 11.4 tells the SSD that all of the sectors in the partition are unused (trim'ed) prior to starting the format. That should ensure consistent highspeed operation out of the SSD. If the build farm is up to 11.4 yet, just the mkfs.ext4 package could be used I suspect.
For lifetime, I don't know. But a typical build seems to take several minutes, so the writes to the SSD don't seem like they would be that outrageously heavy.
Greg
More info from the company which did above demo http://www.iomax.com.au/products.htm
See the links to the areas covered which covered - 4,500 DVDs playing from a single ioDrive Octal. - Running 512 Virtual machines running on one VMWare View host. - Over 1,000 DVDs playing from a single ioDrive. Also http://iomax.com.au/performance.htm Interesting indeed. I wonder if one could be tried out ?
-Glenn
On Fri, 29 Apr 2011 13:39:29 +1000, doiggl@velocitynet.com.au wrote:
On Tue, 26 Apr 2011 17:06:04 -0400, Greg Freemyer greg.freemyer@gmail.com wrote:
On Tue, Apr 26, 2011 at 11:46 AM, David Greaves david@dgreaves.com
wrote:
On 19/04/11 18:49, Greg Freemyer wrote:
On Tue, Apr 19, 2011 at 1:28 PM, Stephen Shawsshaw@decriptor.com wrote:
On Sat, Apr 16, 2011 at 02:19,doiggl@velocitynet.com.au wrote:
Hello With the opensuse build service [obs] getting more usage every day I wonder if this would make a diffence :-
- Use solid state disks [ssd] to improve build times of
package
rpms on, perhaps point the BuildRoot or _tmppath to the ssd's
Reason for asking. Improve the packaging times during busy times on obs.
Cheers Glenn.
Unfortunately there is a cost associated, but it would be cool to
get
a bunch of fusion IO cards and see how they perform.
Here is a youtube video of 1 fusion IO card handling 4500 movies streaming at once. If I understood it right.
Clearly buying SSD costs money.
But buying a $1000 250 GB SSD for one of the build servers would be cheaper than buying a whole new server.
So the next time an expansion to the build farm is budgeted, I hope getting SSDs for the buildroot is considered as an option.
If the thread from 6 months ago is still accurate, having a SSD might double the compile speed of a build farm node.
Don't forget that workers are very, very write-heavy. This is not the best I/O pattern for SSD; and of course the vast majority of data written during a build-preparation is never re-read.
Why is a write heavy load not good for a SSD? Performance or Lifetime?
For performance, the big issue is random i/o vs sequential.
A rotating disk and a SSD should have similar sequential speeds.
But for random i/o the SSD should be much, much faster in that it should barely slow down due to a random i/o write pattern.
Also, the mkfs version in 11.4 tells the SSD that all of the sectors in the partition are unused (trim'ed) prior to starting the format. That should ensure consistent highspeed operation out of the SSD. If the build farm is up to 11.4 yet, just the mkfs.ext4 package could be used I suspect.
For lifetime, I don't know. But a typical build seems to take several minutes, so the writes to the SSD don't seem like they would be that outrageously heavy.
Greg
More info from the company which did above demo -Glenn http://www.iomax.com.au/products.htm
Hello Was there any progress in improving obs build times/throughput or using ssd's, considering obs hardware was replaced in the last week. Thanks Glenn
Hello With the Open Build Service (OBS) getting busier now days. Is there any chance of trying out a card to improve build times and throughput.
This continues the discussion from here - http://lists.opensuse.org/opensuse-buildservice/2011-04/msg00318.html
I saw some pricing links here: http://www.fusionio.com/press-releases/new-storage-class-memory-beats-fusion...
Cheers Glenn
On 04.06.2012 14:18, doiggl@velocitynet.com.au wrote:
Hello With the Open Build Service (OBS) getting busier now days. Is there any chance of trying out a card to improve build times and throughput.
This continues the discussion from here
If you send some, I'm sure we can try it. Building from memory sounds like the cheaper solution still.
Greetings, Stephan
On Mon, 04 Jun 2012 14:27:05 +0200, Stephan Kulow coolo@suse.de wrote:
On 04.06.2012 14:18, doiggl@velocitynet.com.au wrote:
Hello With the Open Build Service (OBS) getting busier now days. Is there any chance of trying out a card to improve build times and throughput.
This continues the discussion from here
If you send some, I'm sure we can try it. Building from memory sounds like the cheaper solution still.
Greetings, Stephan
Hello Stephan I received a response about the card and their team asked the following details:
From the Customer Support for Fusion-io[1]
Thank you for your interest in Fusion-io. Please respond with your location (City, State, Country) and a contact number that you can be reached at and we'll get the appropriate sales person in contact with you to discuss your opportunity.
Thank You, Customer Support
[1] The logged Fusion-IO request is here: https://sup.fusionio.com/cases/browse/CASE-17162
so reply with details: Please respond with your location (City, State, Country)
I'll forward it on to Fusion-IO Support and see if their card will speed up the obs system.
Cheers Glenn
On Tue, 05 Jun 2012 23:40:44 +1000, doiggl@velocitynet.com.au wrote:
On Mon, 04 Jun 2012 14:27:05 +0200, Stephan Kulow coolo@suse.de wrote:
On 04.06.2012 14:18, doiggl@velocitynet.com.au wrote:
Hello With the Open Build Service (OBS) getting busier now days. Is there
any
chance of trying out a card to improve build times and throughput.
This continues the discussion from here
http://lists.opensuse.org/opensuse-buildservice/2011-04/msg00318.html
If you send some, I'm sure we can try it. Building from memory sounds like the cheaper solution still.
Greetings, Stephan
Hello Stephan I received a response about the card and their team asked the following details:
From the Customer Support for Fusion-io[1] Thank you for your interest in Fusion-io. Please respond with your location (City, State, Country) and a contact number that you can be reached at and we'll get the appropriate sales person in contact with
you
to discuss your opportunity.
Thank You, Customer Support
[1] The logged Fusion-IO request is here: https://sup.fusionio.com/cases/browse/CASE-17162
so reply with details: Please respond with your location (City, State, Country)
I'll forward it on to Fusion-IO Support and see if their card will speed up the obs system.
Cheers Glenn
Hello, Just following up on this: Im in the process on responding to Fusion-IO Support on this case [1]
[1] - Fusion-IO Support are asking for the following: Thank you for your interested in Fusion-io. Please respond with your location (City, State, Country) and a contact number that you can be reached at and we'll get the appropriate sales person in contact with you to discuss your opportunity.
The logged Fusion-IO request is here, but need to log on to see it. https://sup.fusionio.com/cases/browse/CASE-17162
Question. So who in the opensuse-buildservice team would be able to test Fusion-io card. See [2] for details of the card. Provide the above details to me, and I will notify Fusion-IO with the contact details. With some luck someone from Fusion-io will contact you and send you a card to test with.
[2] Info about the cards. http://www.fusionio.com/platforms/iodrive/ http://www.fusionio.com/platforms/
Cheers Glenn
On Sat, 09 Jun 2012 01:30:38 +1000, doiggl@velocitynet.com.au wrote:
On Tue, 05 Jun 2012 23:40:44 +1000, doiggl@velocitynet.com.au wrote:
On Mon, 04 Jun 2012 14:27:05 +0200, Stephan Kulow coolo@suse.de
wrote:
On 04.06.2012 14:18, doiggl@velocitynet.com.au wrote:
Hello With the Open Build Service (OBS) getting busier now days. Is there
any
chance of trying out a card to improve build times and throughput.
This continues the discussion from here
http://lists.opensuse.org/opensuse-buildservice/2011-04/msg00318.html
If you send some, I'm sure we can try it. Building from memory sounds like the cheaper solution still.
Greetings, Stephan
Hello Stephan I received a response about the card and their team asked the following details:
From the Customer Support for Fusion-io[1] Thank you for your interest in Fusion-io. Please respond with your location (City, State, Country) and a contact number that you can be reached at and we'll get the appropriate sales person in contact with
you
to discuss your opportunity.
Thank You, Customer Support
[1] The logged Fusion-IO request is here: https://sup.fusionio.com/cases/browse/CASE-17162
so reply with details: Please respond with your location (City, State, Country)
I'll forward it on to Fusion-IO Support and see if their card will
speed
up the obs system.
Cheers Glenn
Hello, Just following up on this: Im in the process on responding to Fusion-IO Support on this case [1]
[1]
- Fusion-IO Support are asking for the following:
Thank you for your interested in Fusion-io. Please respond with your location (City, State, Country) and a contact number that you can be reached at and we'll get the appropriate sales person in contact with
you
to discuss your opportunity.
The logged Fusion-IO request is here, but need to log on to see it. https://sup.fusionio.com/cases/browse/CASE-17162
Question. So who in the opensuse-buildservice team would be able to test Fusion-io card. See [2] for details of the card. Provide the above details to me, and I will notify Fusion-IO with the contact details. With some luck someone from Fusion-io will contact you and send you a
card
to test with.
[2] Info about the cards. http://www.fusionio.com/platforms/iodrive/ http://www.fusionio.com/platforms/
Cheers Glenn
Hello, Some sales of ioDrive , details: Cheers Glenn
iodrive - Auctions only http://www.ebay.com/sch/Computers-Tablets-Networking-/58058/i.html?_sop=3&am...
iodrive - Buy it now http://www.ebay.com/sch/Computers-Tablets-Networking-/58058/i.html?_sop=3&am...
buildservice@lists.opensuse.org
-
Cristian Rodríguez -
David Greaves -
doiggl＠velocitynet.com.au -
Greg Freemyer -
Jan-Simon Möller -
R. Tyler Croy -
Stephan Kulow -
Stephen Shaw