[opensuse-kde] Cross-desktop KDE packages building - macros
Hello, a couple of IRC meetings back, we talked about advertising the buildservice as a tool for upstream KDE developers to package their applications. As a part of this, I have packaged a simple KDE app in home:llunak:kde (taskbarswitch) that with the same .spec file builds also for Fedora and Mandriva (where same .spec file means there is no conditional stuff per distribution). So it looks like it shouldn't be that difficult to make it simple for most kde-apps kind of apps, where their developers would just create one .spec and get it built for all RPM distros, instead of adjusting it per-distro (which, of course, would be an option too, but that's more complicated). The trickery can be done in macros and in project config (right now 'osc meta prjconf home:llunak:kde'). Does this make sense, or do you think it would be better to have something that would create the .spec with per-distro code? In order to continue, I'd like to do some changes and I have some questions. - The first question is: What is %kde4_runtime_requires actually good for? I would like to know the exact purpose, since it doesn't quite make sense to me. As far as I understand it, it is a minimal KDE version requirement, but this way it is specified by the repository, instead by the application itself. Which doesn't make sense to me for most apps, if I build an app with 4.2, it should work with 4.3 too. - In order to avoid people copy&pasting the KDE macros all around, I would like to split them out of kde4-filesystem into kde4-macros. Then people wanting to use the unified way could just link this package to their KDE repository. This would provide an easy way to adjust the macros as necessary. I don't see any big problem here, except for the mentioned %kde4_runtime_requires needing the KDE version of the repository. - And then, of course, there is Debian/Kubuntu with a completely different packaging system :(. I'm still investigating, but it looks like packaging something for both rpm and deb is a big mess. Is here somebody who has any experience with packaging debs, preferably in the buildservice? -- Lubos Lunak KDE developer -------------------------------------------------------------- SUSE LINUX, s.r.o. e-mail: l.lunak@suse.cz , l.lunak@kde.org Lihovarska 1060/12 tel: +420 284 084 672 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz -- To unsubscribe, e-mail: opensuse-kde+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kde+help@opensuse.org
On Tuesday 23 June 2009, Lubos Lunak wrote:
- The first question is: What is %kde4_runtime_requires actually good for? I would like to know the exact purpose, since it doesn't quite make sense to me. As far as I understand it, it is a minimal KDE version requirement, but this way it is specified by the repository, instead by the application itself.
it is supposed to pull in the KDE runtime dependencies (kdebase4-runtime and friends) for this particular application to function correctly, and ensure that at least a particular version of it is installed, namely the one that it was build against and likely requires it due to C++ ABI.
Which doesn't make sense to me for most apps, if I build an app with 4.2, it should work with 4.3 too.
So you're wondering about the "< 4.2.60" part in the requires? This is not strictly needed, we added it because we had several people who had KDE:KDE4:UNSTABLE:Desktop and KDE:KDE4:Factory:Desktop enabled as a repository and then the solver decided to take decisions that do not work. I had filed a bugreport to implement to make repositories conflict each other, but this was rejected, as all dependencies should be expressed in the package itself. therefore we added this "hack". we can remove the "<" part if that is the part that worries you.
- In order to avoid people copy&pasting the KDE macros all around, I would like to split them out of kde4-filesystem into kde4-macros. Then people wanting to use the unified way could just link this package to their KDE repository. This would provide an easy way to adjust the macros as necessary. I don't see any big problem here, except for the mentioned %kde4_runtime_requires needing the KDE version of the repository.
why do you need kde4-filesystem to be seperate? it is just a list of empty directories plus this macro file. just link this package should be enough (for building it does not hurt, you will only get the macros, the additional directories have no effect). This was considered but it just does not make any difference to split the packages, and having just one package that provides all the magic is better than, well. two. you can remove the directories from kde4-filesystem for non-suse distros if that is the other issue, but fedora and mandriva will also want to have those directories owned (mandriva for sure, not clear about fedora). Greetings, Dirk -- To unsubscribe, e-mail: opensuse-kde+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kde+help@opensuse.org
On Wednesday 24 of June 2009, Dirk Müller wrote:
On Tuesday 23 June 2009, Lubos Lunak wrote:
Which doesn't make sense to me for most apps, if I build an app with 4.2, it should work with 4.3 too.
So you're wondering about the "< 4.2.60" part in the requires? This is not strictly needed, we added it because we had several people who had KDE:KDE4:UNSTABLE:Desktop and KDE:KDE4:Factory:Desktop enabled as a repository and then the solver decided to take decisions that do not work.
I had filed a bugreport to implement to make repositories conflict each other, but this was rejected, as all dependencies should be expressed in the package itself. therefore we added this "hack". we can remove the "<" part if that is the part that worries you.
I originally wanted to have just one set of macros for all build repositories, where these versions would be a problem, but I've already figured that wouldn't work, so this can be handled per build repository with the right version numbers. Still, I think the < shouldn't be there. Old KDE apps should work with newer KDE base libraries.
- In order to avoid people copy&pasting the KDE macros all around, I would like to split them out of kde4-filesystem into kde4-macros. Then people wanting to use the unified way could just link this package to their KDE repository. This would provide an easy way to adjust the macros as necessary. I don't see any big problem here, except for the mentioned %kde4_runtime_requires needing the KDE version of the repository.
why do you need kde4-filesystem to be seperate? it is just a list of empty directories plus this macro file. just link this package should be enough (for building it does not hurt, you will only get the macros, the additional directories have no effect).
I wanted to have a separate package with just the macros, to make it simple, I didn't know the directories didn't matter. But right now I still don't know how to solve the problem of using the same macros in all build repositories. Something for opensuse-buildservice@ I guess. -- Lubos Lunak KDE developer -------------------------------------------------------------- SUSE LINUX, s.r.o. e-mail: l.lunak@suse.cz , l.lunak@kde.org Lihovarska 1060/12 tel: +420 284 028 972 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz -- To unsubscribe, e-mail: opensuse-kde+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kde+help@opensuse.org
On Thursday 25 June 2009, Lubos Lunak wrote:
Still, I think the < shouldn't be there. Old KDE apps should work with newer KDE base libraries.
Feel free to remove it from Factory:Desktop. as I said, it is a workaround for a different problem, and so far it was a good-enough hack.
I wanted to have a separate package with just the macros, to make it simple, I didn't know the directories didn't matter. But right now I still don't know how to solve the problem of using the same macros in all build repositories. Something for opensuse-buildservice@ I guess.
by using the same package (sourcelink, _aggregate etc) or by setting a prj config. Alternative you can also add specific .spec files for each individual repository, depending on what you're actually trying to achieve. Greetings, Dirk -- To unsubscribe, e-mail: opensuse-kde+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kde+help@opensuse.org
On Friday 26 of June 2009, Dirk Müller wrote:
On Thursday 25 June 2009, Lubos Lunak wrote:
Still, I think the < shouldn't be there. Old KDE apps should work with newer KDE base libraries.
Feel free to remove it from Factory:Desktop. as I said, it is a workaround for a different problem, and so far it was a good-enough hack.
Ok, but I will keep to my testing repository until I get this all ready.
I wanted to have a separate package with just the macros, to make it simple, I didn't know the directories didn't matter. But right now I still don't know how to solve the problem of using the same macros in all build repositories. Something for opensuse-buildservice@ I guess.
by using the same package (sourcelink, _aggregate etc) or by setting a prj config. Alternative you can also add specific .spec files for each individual repository, depending on what you're actually trying to achieve.
I think I've already handled this, I was suggested on opensuse-buildservice@ to create my own build repositories. I currently have 4 simple packages in home:llunak:kde that build and (mostly :) ) work on openSUSE, Fedora and Mandriva (with the same simple .spec file) and additionally also on Kubuntu (which is a bit of a pain for me). I now need to sort out all remaining important differences in paths and package names, figure out how to do Kubuntu in a less insane way, write howtos and it's ready to go. To continue on the topic, I'd like to clean up our kde4 rpm macros a bit. The plan is of course that the .spec files are written in a way that they are openSUSE .spec files and other distros are handled by mapping, but it would be nice to first sort them out. First of all, we don't have enough filepath macros. Meaning %_kde_share_dir and so on. For example, we put appdata in /usr/share/kde4/apps, but Mandriva has it in /usr/share/apps, while still having /usr/share/kde4/services, and we don't have a macro for that. I would like to extend our set to have one macro per location, so that we would write just something like %_kde_service_dir/kded/whatever instead of manually specifying the rest of the path after %_kde_share_dir. Also, our own macros are inconsistent. Some macros start with _kde_, some with _kde4_. Standard macros are like %_libdir (not %_lib_dir), but ours are %_kde_share_dir. I would like to make them consistent (and keep the old names for backwards compatibility of course), so that it's easier to use for potential new packagers (which later could also consider helping with maintaining KDE for openSUSE and we don't want to scare those away, do we ;) ? ). So, I checked also Fedora and Mandriva, and I would like the list to be: %_kde4_prefix %{_prefix} %_kde4_bindir %{_bindir} %_kde4_libdir %{_libdir} %_kde4_modules %{_libdir}/kde4 %_kde4_libexecdir %{_libexecdir}/kde4 %_kde4_sysconfdir %{_sysconfdir} %_kde4_sharedir %{_prefix}/share/kde4 %_kde4_appsdir %{_kde4_sharedir}/apps %_kde4_configdir %{_kde4_sharedir}/config %_kde4_applicationsdir %{_kde4_prefix}/share/applications/kde4 %_kde4_servicesdir %{_kde4_sharedir}/services %_kde4_servicetypes %{_kde4_sharedir}/servicetypes %_kde4_iconsdir %{_kde4_prefix}/share/icons %_kde4_docdir %{_kde4_prefix}/share/doc %_kde4_sbindir %{_kde4_prefix}/sbin %_kde4_includedir %{_kde4_prefix}/include Not that I insist on any specific form, but having this in some way would make things simpler. Another inconsistency is %cmake_kde4. Both on Mandriva and Fedora they imply an out-of-source build, but we have to explicitly state '-d build'. Are there actually any packages where this matters? I would like to change %cmake_kde4 to work like on Fedora and Mandriva, and ignore any passed -d for backwards compatibility. I will fix any possible package that would need adjusting because of this. In relation to this, although not specific to cross-distro builds, I'd like to change %make_jobs to something like this: %make_jobs \ if test -d doc; then \ %(__make} %{_smp_mflags} -C doc \ fi \ %{__make} %{?jobs:-j %{jobs}} VERBOSE=1 \ %{nil} I.e. first build documentation and only then run really in paralel. Most of our packages are not suitable for paralel building, paralel build in doc/ causes many meinproc instances that quickly eat all memory and I often run into this issue when testing a package locally (OOM-murderer kicks in and you know how much I hate that bloody broken thing). Comments, objections? -- Lubos Lunak KDE developer -------------------------------------------------------------- SUSE LINUX, s.r.o. e-mail: l.lunak@suse.cz , l.lunak@kde.org Lihovarska 1060/12 tel: +420 284 028 972 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz -- To unsubscribe, e-mail: opensuse-kde+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kde+help@opensuse.org
On Saturday 27 June 2009, Lubos Lunak wrote:
I think I've already handled this, I was suggested on opensuse-buildservice@ to create my own build repositories. I currently have 4 simple packages in home:llunak:kde that build and (mostly :) ) work on openSUSE, Fedora and Mandriva (with the same simple .spec file) and additionally also on Kubuntu (which is a bit of a pain for me).
I don't see the difference to a kde4-filesystem (or name it kde4-macros) package that writes a slightly different macros.kde4 file depending on the distro it was built against. I think it is also possible to set the prjconf depending on the distro you're building against (I know there is some %if conditional, several prjconfs useit).
So, I checked also Fedora and Mandriva, and I would like the list to be: %_kde4_prefix %{_prefix} %_kde4_bindir %{_bindir} %_kde4_libdir %{_libdir} %_kde4_modules %{_libdir}/kde4 %_kde4_libexecdir %{_libexecdir}/kde4 %_kde4_sysconfdir %{_sysconfdir} %_kde4_sharedir %{_prefix}/share/kde4 %_kde4_appsdir %{_kde4_sharedir}/apps %_kde4_configdir %{_kde4_sharedir}/config %_kde4_applicationsdir %{_kde4_prefix}/share/applications/kde4 %_kde4_servicesdir %{_kde4_sharedir}/services %_kde4_servicetypes %{_kde4_sharedir}/servicetypes %_kde4_iconsdir %{_kde4_prefix}/share/icons %_kde4_docdir %{_kde4_prefix}/share/doc %_kde4_sbindir %{_kde4_prefix}/sbin %_kde4_includedir %{_kde4_prefix}/include
I think that this is an excellent idea, and I like that cleanup very much.
Another inconsistency is %cmake_kde4. Both on Mandriva and Fedora they imply an out-of-source build, but we have to explicitly state '-d build'.
I think we can remove that, given that out-of-source build works with every package that I tried.
In relation to this, although not specific to cross-distro builds, I'd like to change %make_jobs to something like this: %make_jobs \ if test -d doc; then \ %(__make} %{_smp_mflags} -C doc \ fi \ %{__make} %{?jobs:-j %{jobs}} VERBOSE=1 \ %{nil}
Makes sense for me, and can't hurt. I don't know what %{_smp_mflags} is. wouldn't it be good enough to build it serially?
run into this issue when testing a package locally (OOM-murderer kicks in and you know how much I hate that bloody broken thing).
Sure. note that I tried to implement this already via icecream, assuming that you build via icecream. If not, you should </shameless plug>.
Comments, objections?
Go ahead. Let me know when I can help with the conversion. Greetings, Dirk -- To unsubscribe, e-mail: opensuse-kde+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kde+help@opensuse.org
On Sunday 28 of June 2009, Dirk Müller wrote:
On Saturday 27 June 2009, Lubos Lunak wrote:
I think I've already handled this, I was suggested on opensuse-buildservice@ to create my own build repositories. I currently have 4 simple packages in home:llunak:kde that build and (mostly :) ) work on openSUSE, Fedora and Mandriva (with the same simple .spec file) and additionally also on Kubuntu (which is a bit of a pain for me).
I don't see the difference to a kde4-filesystem (or name it kde4-macros)
There is a huge difference : After I change a macro, it doesn't trigger a rebuild of everything out there :).
package that writes a slightly different macros.kde4 file depending on the distro it was built against. I think it is also possible to set the prjconf depending on the distro you're building against (I know there is some %if conditional, several prjconfs useit).
Maybe if I knew buildservice better when I started, I would have done this using conditionals (although they're not exactly simple to use together with macros), but I actually consider my current setup quite elegant. Instead of standard build repositories I have a setup of special build repositories created using home:llunak:distro, they provide everything needed for the cross-distro magic and it's up to the packager to use them or the standard build repos.
So, I checked also Fedora and Mandriva, and I would like the list to be: %_kde4_prefix %{_prefix} ... %_kde4_includedir %{_kde4_prefix}/include
I think that this is an excellent idea, and I like that cleanup very much.
Ok, I'll try adding them to UNSTABLE first, just in case. If it doesn't become more unstable after that, I'll move it to K:Factory.
In relation to this, although not specific to cross-distro builds, I'd like to change %make_jobs to something like this: %make_jobs \ if test -d doc; then \ %(__make} %{_smp_mflags} -C doc \ fi \ %{__make} %{?jobs:-j %{jobs}} VERBOSE=1 \ %{nil}
Makes sense for me, and can't hurt. I don't know what %{_smp_mflags} is. wouldn't it be good enough to build it serially?
It is (e.g.) -j2. People with more cpus/cores presumably have also more memory. And I don't want to wait unnecessarily while building locally.
run into this issue when testing a package locally (OOM-murderer kicks in and you know how much I hate that bloody broken thing).
Sure. note that I tried to implement this already via icecream, assuming that you build via icecream. If not, you should </shameless plug>.
I don't see how icecream can help me here. Generating documentation is not compilation, the limiting here could be only done by make (boy, do I miss unsermake :-/ ). Oh, you mean meinproc via icecream? Hmm ... interesting idea.
Comments, objections?
Go ahead. Let me know when I can help with the conversion.
Ok. I'll yell if there is a problem. -- Lubos Lunak KDE developer -------------------------------------------------------------- SUSE LINUX, s.r.o. e-mail: l.lunak@suse.cz , l.lunak@kde.org Lihovarska 1060/12 tel: +420 284 028 972 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz -- To unsubscribe, e-mail: opensuse-kde+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kde+help@opensuse.org
On Monday 29 June 2009, Lubos Lunak wrote:
I don't see the difference to a kde4-filesystem (or name it kde4-macros) There is a huge difference : After I change a macro, it doesn't trigger a rebuild of everything out there :).
Which is good for debugging, but bad for fixing issues. if you fix something, you want the fix to propagate :)
Maybe if I knew buildservice better when I started, I would have done this using conditionals (although they're not exactly simple to use together with macros), but I actually consider my current setup quite elegant. Instead of standard build repositories I have a setup of special build repositories created using home:llunak:distro, they provide everything needed for the cross-distro magic and it's up to the packager to use them or the standard build repos.
I'm not sure, I consider it hackish, but thats my feeling.
Ok, I'll try adding them to UNSTABLE first, just in case. If it doesn't become more unstable after that, I'll move it to K:Factory.
I have a script for cleaning up the file lists anyway according to macro names, so you don't have to do that, I can just run it.
Makes sense for me, and can't hurt. I don't know what %{_smp_mflags} is. wouldn't it be good enough to build it serially? It is (e.g.) -j2. People with more cpus/cores presumably have also more memory. And I don't want to wait unnecessarily while building locally.
yep,but a machine with 64 cores will still die on meinproc.
Sure. note that I tried to implement this already via icecream, assuming that you build via icecream. If not, you should </shameless plug>. I don't see how icecream can help me here. Generating documentation is not compilation, the limiting here could be only done by make (boy, do I miss unsermake :-/ ). Oh, you mean meinproc via icecream? Hmm ... interesting idea.
no, icecream has the feature to serialize anything that is not a compile job. e.g, if you do ln -s /opt/icecream/bin/icecc /opt/icecream/bin/meinproc4 and use meinproc4 (adjusting $PATH), then it will make sure that meinproc4 is not run in parallel at all, while other things still parallelize as needed. Greetings, Dirk -- To unsubscribe, e-mail: opensuse-kde+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kde+help@opensuse.org
participants (2)
-
Dirk Müller -
Lubos Lunak