[opensuse-project] GSOC Project "Better Cloud images with KIWI" Aspirant
Hi, Let me quickly grap the opportunity to introduce myself. I am a Master of Science(FOSS) student from India . I have completed my project Hepunion filesystem with CERN last summer. You can find more details about me and my project here https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2013/disd... I would like to tell a little bit about this project so as to let the community know about my skill sets- Hepunion filesystem is a stackable filesystem I was developing for CERN( the European Organization for Nuclear Research). CERN needs a Union filesystem for LHCb(Large Hadron Collider Beauty) to provide fast diskless booting for LHCb's nodes. For such an implementation, they need a file system with two branches a Read-Write and a Read Only so they decided to write a completely new union file system called Hepunion. The filesystem is stable till 2.6.34 kernel as of now and is being planned to upgraded to newer kernel versions. Now this year also I want to participate in a linux based project. As per my discussion with Saurabh Sood, I came across the project "Better Cloud images with KIWI" . I would like to contribute to this project so as to learn new things and contribute to the OpenSuse Community. I would like to admit that I am a novice in Perl since major amount of my work has been limited to kernel space so I am basically a C programmer though I have written scripts in Perls for automation and testing. I have worked on Embedded Systems(both ARM and X86) and have worked on projects like Linux From Scratch so have some better understanding of concepts involved in Linux Boot Process. Kindly let me know the prerequisites needed to contribute to this project so that I can apply for the same as a GSOC Project for 2014. Regards, Saket Sinha -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org To contact the owner, email: opensuse-project+owner@opensuse.org
Hi Saket,
You can find more details about me and my project here https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2013/disd...
Thanks for your interest. I have read the article about HEPunion and I'm sure the knowledge in that area is very helpful to get you started fast in what we do with kiwi. In kiwi we have experimented with many union approaches using e.g clicfs, unionfs, deltafs, aufs but they are all running as fuse userspace processes which lead to other implementations like device-mapper-snapshot and btrfs+seed. As of today our preferred union solution is overlayfs because it has found its way into the kernel and works great Thanks again for your interest. Regards, Marcus -- Public Key available gpg --keyserver pgp.mit.edu --recv-keys 0xCCE3C6A2 ------------------------------------------------------- Marcus Schäfer (Res. & Dev.) SUSE LINUX Products GmbH Tel: 0911-740 53 0 Maxfeldstrasse 5 FAX: 0911-740 53 479 D-90409 Nürnberg GF: Jeff Hawn,Jennifer Guild, Felix Imendörffer HRB: 21284 (AG Nürnberg) Germany http://www.suse.de ------------------------------------------------------- -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org To contact the owner, email: opensuse-project+owner@opensuse.org
Hi Marcus, I have some questions regarding Kiwi internal working from a developer's point of view. I have gone through the Kiwi System image cookbook but that mainly targets end-users rather than the developers. 1. kiwi takes the initrd, kernel details and the user defined config.xml as input and creates the desired image. I would like to know the details as to how it does that. 2. As you have said-
In kiwi we have experimented with many union approaches using e.g clicfs, unionfs, deltafs, aufs but they are all running as fuse userspace processes which lead to other implementations like device-mapper-snapshot and btrfs+seed.
Why and where do you implement this in Kiwi 3. Now regarding the GSOC project. Kiwi supports Amazon Elastic Compute Cloud (Amazon EC2) which, AFAIK, is a web service that provides re-sizable compute capacity in the cloud. According to Kiwi Cloud Cookbook- For a Simple Storage Service backed Amazon Machine Image, a bundle with a manifest XML file is required and KIWI uses the Amazon tools to create this bundle for you. The steps involved are- 1. Register an Amazon EC2 account. 2. Modify the config.xml with the EC2 account information 3. kiwi --prepare 4. kiwi --create Now the image gets created. The generated image needs to be transfered over to Amazon which is done by the ec2-upload-bundle tool. Now my question is this, what does this GSOC project aim to change in these above steps. What would be the final deliverable of this project? Regards, Saket Sinha -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org To contact the owner, email: opensuse-project+owner@opensuse.org
On 01/26/2014 06:02 AM, Saket Sinha wrote:
Hi Marcus,
I have some questions regarding Kiwi internal working from a developer's point of view. I have gone through the Kiwi System image cookbook but that mainly targets end-users rather than the developers.
1. kiwi takes the initrd, kernel details and the user defined config.xml as input and creates the desired image. I would like to know the details as to how it does that.
Here is the high level overview: - create a directory (location provided by the user) - use the build system package manager and install the packages that are listed as "bootstrap" - chroot into the directory - now use the package manager inside the chroot and install all the other packages Repeat the above process for the initrd. The initrd is created based on a description provided by kiwi, that can also be a user defined description. There is also some clean up code that prunes the initrd image we build. Create a disk image file install the bootloader, put the initrd in place and all the stuff from the first list I provided.
2. As you have said-
In kiwi we have experimented with many union approaches using e.g clicfs, unionfs, deltafs, aufs but they are all running as fuse userspace processes which lead to other implementations like device-mapper-snapshot and btrfs+seed.
Why and where do you implement this in Kiwi
3. Now regarding the GSOC project.
Kiwi supports Amazon Elastic Compute Cloud (Amazon EC2) which, AFAIK, is a web service that provides re-sizable compute capacity in the cloud. According to Kiwi Cloud Cookbook-
We just completely changed the way we build the EC2 images and the doc also changed with it.
For a Simple Storage Service backed Amazon Machine Image, a bundle with a manifest XML file is required and KIWI uses the Amazon tools to create this bundle for you.
We decided to no longer let kiwi do this but shift the bundle and manifest creation to the user, the doc has been updated accordingly. For EC2 images we now create a disk image rather than a flat filesystem image as we did before.
The steps involved are- 1. Register an Amazon EC2 account. 2. Modify the config.xml with the EC2 account information 3. kiwi --prepare 4. kiwi --create Now the image gets created. The generated image needs to be transfered over to Amazon which is done by the ec2-upload-bundle tool.
Now my question is this, what does this GSOC project aim to change in these above steps. What would be the final deliverable of this project?
EC2 is pretty much done now. But there are other cloud formats that need attention. One thing that needs to go away is the initrd regeneration on boot for cloud images. There is a fair amount of code restructuring to do to make the kiwi work flow more structured and get the code base into better shape for unit testing. From a user perspective almost nothing would change. HTH, Robert -- Robert Schweikert MAY THE SOURCE BE WITH YOU SUSE-IBM Software Integration Center LINUX Tech Lead Public Cloud Architect rjschwei@suse.com rschweik@ca.ibm.com 781-464-8147 -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org To contact the owner, email: opensuse-project+owner@opensuse.org
First of All, I request you to guide me little bit through the code. Does a Kiwi Developer Manual exist or something similar that can help a developer get started to contribute to KIWI?
One thing that needs to go away is the initrd regeneration on boot for cloud images.
Can you guide me exactly where in the code we are doing it. I, being a novice, am having problem in going through the code.
There is a fair amount of code restructuring to do to make the kiwi work flow more structured and get the code base into better shape for >unit testing
What exactly needs to be changed? Kindly bear with me, considering the fact I am just a beginner and hope to catch up soon, who is wanting to contribute to this community. Regards, Saket Sinha -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org To contact the owner, email: opensuse-project+owner@opensuse.org
participants (3)
-
Marcus Schäfer -
Robert Schweikert -
Saket Sinha