[yast-devel] YCP vs Ruby: RAM consumption in installation
Hi all, I'd like to extend the David's message about the working installer, I have some interesting data for you. I did some tests regarding the memory consumption in the YaST installer after switching to Ruby because there were some concerns about increased memory requirements... YCP vs Ruby: RAM consumption ============================ Testing setup: HW: VirtualBox virtual machine with 4 CPUs and 1GB RAM SW: openSUSE-12.3, 64bit installation, English language, graphical (Qt) installation. The Ruby installation ISO was from http://download.opensuse.org/repositories/YaST:/Head:/ruby/images/iso/ For YCP installation I used the final 12.3 DVD ISO image. Originally I wanted to put the results directly here, but a spreadsheet document looks better, it's available here: https://docs.google.com/spreadsheet/ccc?key=0ArPH-PKhmo3kdFRFUy05WG50ckpCVHJpVXFJRGYwd2c&usp=sharing Summary ------- The overall increase in RAM usage is about 20-30MB (depending on how you measure it). This is not bad if you consider that just the new inst-sys image needs about 5MB more space (because of the added ruby packages). But the converted YaST code currently does _not_ include comments, adding them will slightly increase all YaST files (but should not be much as inst-sys uses compressed squashfs images) Conclusion ---------- The RAM size needed is not much bigger for running YaST in Ruby and the currently needed extra RAM space can be even decreased after some inst-sys tweaking (see the next paragraph) to like 15-20MB more than the old YCP installer. This is really good news! Possible improvements: ----------------------- Currently we include _complete_ ruby in inst-sys, including parts which are not needed for the installer (like webrick web server, test unit, rdoc, rake, xmlrpc, ....). It's possible to remove these parts and save some space in inst-sys which is stored in RAM. It's possible to include just a part of an RPM package to inst-sys instead of complete package. Also the added fast_gettext gem could be filtered to save few kilobytes... Later we could also do some cleanup and use native Ruby library instead of our implementation (e.g. use native net/ftp.rb instead of our FTP.ycp (converted to FTP.rb), the same applies to String.ycp, URL.ycp...) -- Ladislav Slezák Appliance department / YaST Developer Lihovarská 1060/12 190 00 Prague 9 / Czech Republic tel: +420 284 028 960 lslezak@suse.com SUSE -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
El 27/05/13 05:43, Ladislav Slezak escribió:
Hi all,
I'd like to extend the David's message about the working installer, I have some interesting data for you.
I did some tests regarding the memory consumption in the YaST installer after switching to Ruby because there were some concerns about increased memory requirements...
Summary -------
The overall increase in RAM usage is about 20-30MB (depending on how you measure it).
Conclusion ----------
The RAM size needed is not much bigger for running YaST in Ruby and the currently.
Well, even if increased RAM usage was 100 MB will still be a non-issue and trying to do anything about it at this stage will be premature optimization. IMHO resource usage is the last thing to look at, first the converter must work and the generated code run correctly, then stuff gets included into the distribution as soon as possible, only then it might make sense to try reducing footprint. -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Hi, I tried to do some tests to find out with how much of RAM the installer still works and where its limits are. I tested with VirtualBox environment. 512 MB: Installer can run with source from HTTP in graphical mode 384 MB: Installer can only run from local (CD) source, but in graphical mode as well. Running from HTTP failed even in text-mode 256 MB: Could not install the system any more In cases where the installer succeeded, I did not notice any slow down. I also could not find any regression compared to YCP version, however, I did not really try to and only went the standard installation path. I did not preform the same tests with YCP-based installer to see whether there is any difference, though. Anyway, these results look good to me, great work! Jiri On 05/27/2013 11:43 AM, Ladislav Slezak wrote:
Hi all,
I'd like to extend the David's message about the working installer, I have some interesting data for you.
I did some tests regarding the memory consumption in the YaST installer after switching to Ruby because there were some concerns about increased memory requirements...
YCP vs Ruby: RAM consumption ============================
Testing setup: HW: VirtualBox virtual machine with 4 CPUs and 1GB RAM SW: openSUSE-12.3, 64bit installation, English language, graphical (Qt) installation.
The Ruby installation ISO was from http://download.opensuse.org/repositories/YaST:/Head:/ruby/images/iso/ For YCP installation I used the final 12.3 DVD ISO image.
Originally I wanted to put the results directly here, but a spreadsheet document looks better, it's available here:
https://docs.google.com/spreadsheet/ccc?key=0ArPH-PKhmo3kdFRFUy05WG50ckpCVHJpVXFJRGYwd2c&usp=sharing
Summary -------
The overall increase in RAM usage is about 20-30MB (depending on how you measure it).
This is not bad if you consider that just the new inst-sys image needs about 5MB more space (because of the added ruby packages).
But the converted YaST code currently does _not_ include comments, adding them will slightly increase all YaST files (but should not be much as inst-sys uses compressed squashfs images)
Conclusion ----------
The RAM size needed is not much bigger for running YaST in Ruby and the currently needed extra RAM space can be even decreased after some inst-sys tweaking (see the next paragraph) to like 15-20MB more than the old YCP installer.
This is really good news!
Possible improvements: -----------------------
Currently we include _complete_ ruby in inst-sys, including parts which are not needed for the installer (like webrick web server, test unit, rdoc, rake, xmlrpc, ....). It's possible to remove these parts and save some space in inst-sys which is stored in RAM. It's possible to include just a part of an RPM package to inst-sys instead of complete package.
Also the added fast_gettext gem could be filtered to save few kilobytes...
Later we could also do some cleanup and use native Ruby library instead of our implementation (e.g. use native net/ftp.rb instead of our FTP.ycp (converted to FTP.rb), the same applies to String.ycp, URL.ycp...)
--
Ladislav Slezák Appliance department / YaST Developer Lihovarská 1060/12 190 00 Prague 9 / Czech Republic tel: +420 284 028 960 lslezak@suse.com SUSE
-- Regards, Jiri Srain Project Manager --------------------------------------------------------------------- SUSE LINUX, s.r.o. e-mail: jsrain@suse.com Lihovarska 1060/12 tel: +420 284 084 659 190 00 Praha 9 fax: +420 284 084 001 Czech Republic http://www.suse.com -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
El 28/05/13 03:25, Jiri Srain escribió:
Hi,
I tried to do some tests to find out with how much of RAM the installer still works and where its limits are.
I tested with VirtualBox environment.
512 MB: Installer can run with source from HTTP in graphical mode 384 MB: Installer can only run from local (CD) source, but in graphical mode as well. Running from HTTP failed even in text-mode 256 MB: Could not install the system any more
I tried with the isos mentioned in this post plus booting with "zcache" 256 MB : Installer crashed (probably needs to handle OOM better IMHO) 320 MB: Installer runs in graphical mode, albeit sluggish, however it dies on the partitioning and formatting of disks (the installer does not die but mkfs tools do) -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 05/29/2013 12:36 AM, Cristian Rodríguez wrote:
El 28/05/13 03:25, Jiri Srain escribió:
Hi,
I tried to do some tests to find out with how much of RAM the installer still works and where its limits are.
I tested with VirtualBox environment.
512 MB: Installer can run with source from HTTP in graphical mode 384 MB: Installer can only run from local (CD) source, but in graphical mode as well. Running from HTTP failed even in text-mode 256 MB: Could not install the system any more
I tried with the isos mentioned in this post plus booting with "zcache"
256 MB : Installer crashed (probably needs to handle OOM better IMHO)
320 MB: Installer runs in graphical mode, albeit sluggish, however it dies on the partitioning and formatting of disks (the installer does not die but mkfs tools do)
Failed in my reports meant that (in most cases) it started correctly, but was incredibly slow and crashed at some point of time - before reaching the installation proposal. Jiri -- Regards, Jiri Srain Project Manager --------------------------------------------------------------------- SUSE LINUX, s.r.o. e-mail: jsrain@suse.com Lihovarska 1060/12 tel: +420 284 084 659 190 00 Praha 9 fax: +420 284 084 001 Czech Republic http://www.suse.com -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 05/27/2013 11:43 AM, Ladislav Slezak wrote:
Originally I wanted to put the results directly here, but a spreadsheet document looks better, it's available here:
https://docs.google.com/spreadsheet/ccc?key=0ArPH-PKhmo3kdFRFUy05WG50ckpCVHJpVXFJRGYwd2c&usp=sharing
Very nice and complete data. Thanks! I've cloned the document and added some graphs just to give the data a graphical representation: https://docs.google.com/spreadsheet/ccc?key=0AhWDup9rfv8cdE83WmlrelpwVzhGX0p...
Possible improvements: -----------------------
Currently we include _complete_ ruby in inst-sys, including parts which are not needed for the installer (like webrick web server, test unit, rdoc, rake, xmlrpc, ....). It's possible to remove these parts and save some space in inst-sys which is stored in RAM. It's possible to include just a part of an RPM package to inst-sys instead of complete package.
Also the added fast_gettext gem could be filtered to save few kilobytes...
It's great to have a plan. I believe this can wait till the code is good enough to be switched to Ruby in master. Then we can do some optimizations.
Later we could also do some cleanup and use native Ruby library instead of our implementation (e.g. use native net/ftp.rb instead of our FTP.ycp (converted to FTP.rb), the same applies to String.ycp, URL.ycp...)
Additionally, it seems that the YCP code currently caches some data that are not needed anymore. And the Ruby implementation does the same: it copies the data instead of sharing them - which is expected as it's been planned to behave the same way as YCP after the conversion. I see a big potential for minimizing the memory usage after some data-handling related refactoring in converted YaST. Thanks and Bye Lukas -- Lukas Ocilka, Cloud & Systems Management Department SUSE LINUX s.r.o., Praha -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 27/05/13 11:43, Ladislav Slezak wrote:
Hi all,
I'd like to extend the David's message about the working installer, I have some interesting data for you.
I did some tests regarding the memory consumption in the YaST installer after switching to Ruby because there were some concerns about increased memory requirements...
Also, we have to have in mind that YaST in Ruby still uses the YCP engine. All native access is still done via YCP and it will not be a complete comparison until all those modules are loaded in a more "native" way. Even if that does not guarantee less memory usage. -- Duncan Mac-Vicar P. - http://www.suse.com/ SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
participants (5)
-
Cristian Rodríguez -
Duncan Mac-Vicar P. -
Jiri Srain -
Ladislav Slezak -
Lukas Ocilka