[opensuse-buildservice] Specification for OBS light project concept
Hello, those who have discussed with me during the MeeGo Conference in San Francisco, know that I have started a small project to create a Light version of OBS. The goal of the project is to ease the access to OBS for embedded developers and initial investigation team which have to select an embedded OS, by creating a tool which follows their traditional development process (working locally in chroot) but keeps the compatibility with the OBS. Some of the module that we are planning could potentially be of interest for the real OBS (called Full OBS in my spec). In particular the the automatic creation of patch files from a modified chroot and the UI for MIC2 could become generic features. All created new code will be GPL2. Your feedback is welcome. All discussion will take place on the MeeGo distribution-tools mailing list. http://lists.meego.com/listinfo/meego-distribution-tools The file is on the MeeGo Wiki. http://wiki.meego.com/images/SpecOBSLight-1_V-0-6.pdf -- Dominig ar Foll MeeGo TV Open Source Technology Centre Intel SSG -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Tuesday, 14. June 2011, 10:53:47 schrieb Dominig ar Foll:
Hello,
those who have discussed with me during the MeeGo Conference in San Francisco, know that I have started a small project to create a Light version of OBS.
The goal of the project is to ease the access to OBS for embedded developers and initial investigation team which have to select an embedded OS, by creating a tool which follows their traditional development process (working locally in chroot) but keeps the compatibility with the OBS.
Some of the module that we are planning could potentially be of interest for the real OBS (called Full OBS in my spec). In particular the the automatic creation of patch files from a modified chroot and the UI for MIC2 could become generic features. All created new code will be GPL2.
Your feedback is welcome. All discussion will take place on the MeeGo distribution-tools mailing list.
http://lists.meego.com/listinfo/meego-distribution-tools
The file is on the MeeGo Wiki.
I had a brief look and I did not understand if you plan to use just the "build" script directly or if you are about to invent something else. The build script can be used complete standalone at least or embeeded in osc local builds or obs service side builds. So is it a problem that an OBS server exists at all for these developers ? Or would it be find to do a local build via "osc build" in chroot using the resources from a remote OBS server ? -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
I had a brief look and I did not understand if you plan to use just the "build" script directly or if you are about to invent something else. For the build part we plan to reuse the build script. No need to reinvent the wheel. The build script can be used complete standalone at least or embeeded in osc local builds or obs service side builds. Yes osc can do that. What osc does not do easily is to re-extract the change done in the chroot to create a patch to apply to the original
package. In embedded we are targeting a developer population which works in the chroot directly. One of their first need to is to run makeconfig to change the kernel configuration and create a config patch. What is not that simple with the existing tool set.
So is it a problem that an OBS server exists at all for these developers ? Yes. The selection of an OS is generally done by a team of a few guys generally under the CTO office. These guy are normally given a few weeks to check a few alternative OS and as new projects are often "top secret" using an OBS requires to install a private instance.
Once you add the complexity of the task (intermediate I would say) to the embedded culture of these teams, you really sees that OBS can be a major obstacle to MeeGo selection while it is a fantastic tool that would be very useful for the future of any embedded project. OBS Light target to feel that gap.
Or would it be find to do a local build via "osc build" in chroot using the resources from a remote OBS server ? If you look at the UI in the Spec you will notice that we are planing to allow the dual mode (local or remote OBS repo as reference). Our goal is to help embedded project to move to OBS.
Some part OBS light will be created as plug-ins to osc (e.g. the patch creation tool) and that could be of a general value for all OBS users. -- Dominig -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Tue, Jun 14, 2011 at 02:44:27PM +0200, Dominig ar Foll (Intel OTC) wrote:
I had a brief look and I did not understand if you plan to use just the "build" script directly or if you are about to invent something else. For the build part we plan to reuse the build script. No need to reinvent the wheel. The build script can be used complete standalone at least or embeeded in osc local builds or obs service side builds. Yes osc can do that. What osc does not do easily is to re-extract the change done in the chroot to create a patch to apply to the original package. In embedded we are targeting a developer population which works in the chroot directly.
One of their first need to is to run makeconfig to change the kernel configuration and create a config patch. What is not that simple with the existing tool set.
So is it a problem that an OBS server exists at all for these developers ? Yes. The selection of an OS is generally done by a team of a few guys generally under the CTO office. These guy are normally given a few weeks to check a few alternative OS and as new projects are often "top secret" using an OBS requires to install a private instance.
Once you add the complexity of the task (intermediate I would say) to the embedded culture of these teams, you really sees that OBS can be a major obstacle to MeeGo selection while it is a fantastic tool that would be very useful for the future of any embedded project.
Why not just use Yocto, as that is exactly what that project is for? greg k-h -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Why not just use Yocto, as that is exactly what that project is for?
Yes Yocto target that area but Yocto provides far less packages than MeeGo and does not provides the OBS once that the team will grow. In TV or IVI team of a few 100's of developers is not uncommon and when the team is that large a full OBS is a great tool. Our goal is to bridge the gap while staying on MeeGo and an rpm based distro, because that is what our users needs on the long run. -- Dominig -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Tue, Jun 14, 2011 at 10:40:06PM +0200, Dominig ar Foll (Intel OTC) wrote:
Why not just use Yocto, as that is exactly what that project is for?
Yes Yocto target that area but Yocto provides far less packages than MeeGo and does not provides the OBS once that the team will grow. In TV or IVI team of a few 100's of developers is not uncommon and when the team is that large a full OBS is a great tool.
Our goal is to bridge the gap while staying on MeeGo and an rpm based distro, because that is what our users needs on the long run.
Then trying to strip down OBS seems "odd" in that in the long run, you will need the whole thing. Anyway, good luck with this. greg k-h -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On 14/06/11 13:44, Dominig ar Foll (Intel OTC) wrote:
I had a brief look and I did not understand if you plan to use just the "build" script directly or if you are about to invent something else. For the build part we plan to reuse the build script. No need to reinvent the wheel.
I'm also looking at making OBS fit more easily with the developer workflow in a sizeable MeeGo/OBS project in Teleca/LGE. (And by developer I don't mean just the elite code hackers who inhabit lkml etc; I need to cover junior devs too). My initial thoughts had been to prepare something like: * osc chroot : to prepare a clean build area, probably with gdb etc too * mount -bind /path/to/vcs-managed-src /path/inside/chroot/src * developer hacks and saves in their 'desktop' environment * osc 'rebuild' : runs rpmbuild with hacks but without cleaning chroot. (since I did something like this a couple of years ago) Issues include: * support multiple packages in one mega-chroot? * updating chroot efficiently? * not rm -rf'ing a build chroot with a mount -bind .git tree (!) * create a project or team specific meta-package which depends on various useful rpms to allow osc build -x my_dev_env to be easy. I really haven't had time to properly analyse my requirements or the possible solutions so this is very much a hand-waving and a "me too" email. I will say that I'm in favour of a solution using a real OBS - an appliance VM would be fine.
The build script can be used complete standalone at least or embeeded in osc local builds or obs service side builds. Yes osc can do that. What osc does not do easily is to re-extract the change done in the chroot to create a patch to apply to the original package.
This seems too hard to support in any meaningful general case. How do I know if the Makefile that just appeared is because I made it or because I ran autoconf? OK, you could just support modifications but then you risk losing real work in new files. I was going to allow access to git inside the chroot and probably looking at some git/quilt workflow for those who like the patch approach. There is a very nice policy in a supporting tool in debian's git-buildpackage which I'm in the process of 'porting' to rpm over the next week (I hope). see : https://honk.sigxcpu.org/piki/development/debian_packages_in_git/ Some plugin like "osc trialbuild" would use that policy to extract patches from the git tree and update the spec; send them to the OBS and build there.
In embedded we are targeting a developer population which works in the chroot directly.
Agreed. One of my first patches to OBS was to make the UID of the chroot user match the real user so you could use the 'desktop' editor against the chroot path.
Or would it be find to do a local build via "osc build" in chroot using the resources from a remote OBS server ? If you look at the UI in the Spec you will notice that we are planing to allow the dual mode (local or remote OBS repo as reference). Our goal is to help embedded project to move to OBS.
Whilst this isn't precisely my goal, I know that a reasonably large subset of devs (especially in the hardware adaptation layers) will have come from that area so I'm also interested in making their environment seamless as they move into a production use of the OBS in their integration workflow.
Some part OBS light will be created as plug-ins to osc (e.g. the patch creation tool) and that could be of a general value for all OBS users.
I wonder if we can start by providing some useful wrappers, workflows and the like around osc and the current system? NB. If there's not much interest on the obs list then I'm happy to continue discussions on the meego-distribution-tools list. David -- "Don't worry, you'll be fine; I saw it work in a cartoon once..." -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
I apologize for top- and in-between posting, and perhaps somewhat naive thinking, but I just couldn't stop for my brains for making stupid question(s): Since most developer hardware are capable of running virtual appliances (using Vmware's player, KVM, or other such software) would it be out-of-reach of obs/osc developers to implement osc as VM client node, which would register itself to OBS service as a build node. If a local OBS would be used, then the full build environment could be part of the VM appliance and the build process would be executed in local workstation. All required configurations, repository settings and such would be included to the project by local system admin. Perhaps all developer tools [editors etc], which are required in that project, should be included in appliance which would allow really "standard desktop implmentation"; if such is needed. After development cycle, the cleanup of development desktop would require just the deletion of local VM appliance. All other VM benefits would also be included, such as distributed build process, platform independent development, load balancing etc. 06/19/2011 09:07 PM, David Greaves kirjoitti:
On 14/06/11 13:44, Dominig ar Foll (Intel OTC) wrote:
I had a brief look and I did not understand if you plan to use just the "build" script directly or if you are about to invent something else. For the build part we plan to reuse the build script. No need to reinvent the wheel.
I'm also looking at making OBS fit more easily with the developer workflow in a sizeable MeeGo/OBS project in Teleca/LGE. (And by developer I don't mean just the elite code hackers who inhabit lkml etc; I need to cover junior devs too).
Could the developers in this project use "standard" environment, which would actually exist as a virtual desktop, or virtual appliance? Would it be plausible in resource wise?
My initial thoughts had been to prepare something like: * osc chroot : to prepare a clean build area, probably with gdb etc too * mount -bind /path/to/vcs-managed-src /path/inside/chroot/src * developer hacks and saves in their 'desktop' environment * osc 'rebuild' : runs rpmbuild with hacks but without cleaning chroot. (since I did something like this a couple of years ago)
Issues include: * support multiple packages in one mega-chroot? * updating chroot efficiently? * not rm -rf'ing a build chroot with a mount -bind .git tree (!) * create a project or team specific meta-package which depends on various useful rpms to allow osc build -x my_dev_env to be easy.
How much of this could be achieved with implementing more modular installation setup of the packages or just in coding process of developers? [this was not a real question, just thinkin aloud :]
I really haven't had time to properly analyse my requirements or the possible solutions so this is very much a hand-waving and a "me too" email.
I will say that I'm in favour of a solution using a real OBS - an appliance VM would be fine.
The build script can be used complete standalone at least or embeeded in osc local builds or obs service side builds. Yes osc can do that. What osc does not do easily is to re-extract the change done in the chroot to create a patch to apply to the original package.
This seems too hard to support in any meaningful general case. How do I know if the Makefile that just appeared is because I made it or because I ran autoconf? OK, you could just support modifications but then you risk losing real work in new files.
I was going to allow access to git inside the chroot and probably looking at some git/quilt workflow for those who like the patch approach. There is a very nice policy in a supporting tool in debian's git-buildpackage which I'm in the process of 'porting' to rpm over the next week (I hope). see : https://honk.sigxcpu.org/piki/development/debian_packages_in_git/
Some plugin like "osc trialbuild" would use that policy to extract patches from the git tree and update the spec; send them to the OBS and build there.
In embedded we are targeting a developer population which works in the chroot directly.
Agreed. One of my first patches to OBS was to make the UID of the chroot user match the real user so you could use the 'desktop' editor against the chroot path.
Or would it be find to do a local build via "osc build" in chroot using the resources from a remote OBS server ? If you look at the UI in the Spec you will notice that we are planing to allow the dual mode (local or remote OBS repo as reference). Our goal is to help embedded project to move to OBS.
Whilst this isn't precisely my goal, I know that a reasonably large subset of devs (especially in the hardware adaptation layers) will have come from that area so I'm also interested in making their environment seamless as they move into a production use of the OBS in their integration workflow.
Some part OBS light will be created as plug-ins to osc (e.g. the patch creation tool) and that could be of a general value for all OBS users.
I wonder if we can start by providing some useful wrappers, workflows and the like around osc and the current system?
NB. If there's not much interest on the obs list then I'm happy to continue discussions on the meego-distribution-tools list.
David
If this sounds even slightly interesting, I can explain more deeper. If this has been said earlier allready, I apoplogize, but I haven't noticed that. yours, Tommi -- WasaLab Oy, P.O.Box 365, FIN-65101 Vaasa t2r@wasalab.com, tel +358-44-767 7770 helpdesk@wasalab.com, tel +358-45-130 4000 -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Le 20/06/11 07:36, Tommi Rintala a écrit :
I apologize for top- and in-between posting, and perhaps somewhat naive thinking, but I just couldn't stop for my brains for making stupid question(s):
The only stupids people are the one who do not ask questions :-)
Since most developer hardware are capable of running virtual appliances (using Vmware's player, KVM, or other such software) would it be out-of-reach of obs/osc developers to implement osc as VM client node, which would register itself to OBS service as a build node.
Issue is that when you run an investigation project in a company to select a new OS, your project is not public. It's actually in general highly secret getting a local system is perceived as a high value in that period. Going an a public OBS even with a VM would requires a religion change. Experience (I have tried) prove to be out of reach. Secondly, method required to work with OBS make a few things fairly complex for embedded developers. In particular, when you port MeeGo on a new special HW, you first need to create a new Kernel config and that with OCS is not simple. Our goal is to make that type of things easy but enabling them to use the method that they know (work in the chroot).
If a local OBS would be used, then the full build environment could be part of the VM appliance and the build process would be executed in local workstation. All required configurations, repository settings and such would be included to the project by local system admin. Perhaps all developer tools [editors etc], which are required in that project, should be included in appliance which would allow really "standard desktop implmentation"; if such is needed. After development cycle, the cleanup of development desktop would require just the deletion of local VM appliance. All other VM benefits would also be included, such as distributed build process, platform independent development, load balancing etc.
To be honest, as the first implementation of light OBS will be done on OpenSUSE only (simpler). We might use a VM to deliver the version 1.0. But still we need a tool to help embedded people to create patches in RPM. But I agree that your idea to develop it in a first step with a full OBS running as a local appliance in a VM in lieu of the GostOBS module proposed is a good short cut for an early phase1. I am a bit afraid by the complexity of delivering the VM as installation is difficult to fully automate but the idea is interesting and might simplify the project maintenance. I will push Ronan (project maintainer) to get a look at that idea. Thanks. -- Dominig
06/19/2011 09:07 PM, David Greaves kirjoitti:
On 14/06/11 13:44, Dominig ar Foll (Intel OTC) wrote:
I had a brief look and I did not understand if you plan to use just the "build" script directly or if you are about to invent something else. For the build part we plan to reuse the build script. No need to reinvent the wheel.
I'm also looking at making OBS fit more easily with the developer workflow in a sizeable MeeGo/OBS project in Teleca/LGE. (And by developer I don't mean just the elite code hackers who inhabit lkml etc; I need to cover junior devs too).
Could the developers in this project use "standard" environment, which would actually exist as a virtual desktop, or virtual appliance? Would it be plausible in resource wise?
My initial thoughts had been to prepare something like: * osc chroot : to prepare a clean build area, probably with gdb etc too * mount -bind /path/to/vcs-managed-src /path/inside/chroot/src * developer hacks and saves in their 'desktop' environment * osc 'rebuild' : runs rpmbuild with hacks but without cleaning chroot. (since I did something like this a couple of years ago)
Issues include: * support multiple packages in one mega-chroot? * updating chroot efficiently? * not rm -rf'ing a build chroot with a mount -bind .git tree (!) * create a project or team specific meta-package which depends on various useful rpms to allow osc build -x my_dev_env to be easy.
How much of this could be achieved with implementing more modular installation setup of the packages or just in coding process of developers? [this was not a real question, just thinkin aloud :]
I really haven't had time to properly analyse my requirements or the possible solutions so this is very much a hand-waving and a "me too" email.
I will say that I'm in favour of a solution using a real OBS - an appliance VM would be fine.
The build script can be used complete standalone at least or embeeded in osc local builds or obs service side builds. Yes osc can do that. What osc does not do easily is to re-extract the change done in the chroot to create a patch to apply to the original package.
This seems too hard to support in any meaningful general case. How do I know if the Makefile that just appeared is because I made it or because I ran autoconf? OK, you could just support modifications but then you risk losing real work in new files.
I was going to allow access to git inside the chroot and probably looking at some git/quilt workflow for those who like the patch approach. There is a very nice policy in a supporting tool in debian's git-buildpackage which I'm in the process of 'porting' to rpm over the next week (I hope). see : https://honk.sigxcpu.org/piki/development/debian_packages_in_git/
Some plugin like "osc trialbuild" would use that policy to extract patches from the git tree and update the spec; send them to the OBS and build there.
In embedded we are targeting a developer population which works in the chroot directly.
Agreed. One of my first patches to OBS was to make the UID of the chroot user match the real user so you could use the 'desktop' editor against the chroot path.
Or would it be find to do a local build via "osc build" in chroot using the resources from a remote OBS server ? If you look at the UI in the Spec you will notice that we are planing to allow the dual mode (local or remote OBS repo as reference). Our goal is to help embedded project to move to OBS.
Whilst this isn't precisely my goal, I know that a reasonably large subset of devs (especially in the hardware adaptation layers) will have come from that area so I'm also interested in making their environment seamless as they move into a production use of the OBS in their integration workflow.
Some part OBS light will be created as plug-ins to osc (e.g. the patch creation tool) and that could be of a general value for all OBS users.
I wonder if we can start by providing some useful wrappers, workflows and the like around osc and the current system?
NB. If there's not much interest on the obs list then I'm happy to continue discussions on the meego-distribution-tools list.
David
If this sounds even slightly interesting, I can explain more deeper. If this has been said earlier allready, I apoplogize, but I haven't noticed that.
yours,
Tommi
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
participants (6)
-
Adrian Schröter
-
David Greaves
-
Dominig ar Foll
-
Dominig ar Foll (Intel OTC)
-
Greg KH
-
Tommi Rintala