[opensuse-packaging] 11-check-pkgconfig-deps issues
Hi ;-) 11-check-pkgconfig-deps has some issues, that causes increasing package dependency bloat. let say we have packages X, Y, Z where Z is being checked by this script. Z : Requires Y that provides Y.pc Y: Requires X that provides X.pc when Z is checked, it complains that there is a missing Require on X which is not really correct,and will led packagers to add X as a dependency of Z and on.. and on.. and on... over the whole dependency tree..thing that will of course end in a unfixable mess as time passes, as dependencies are always added but never removed in this case,even when some intermediate package doesn't use a particular library or component anymore. The second issue, is that this script does not account Requires.private that should be taken in consideration _only_ when the package contains static libraries, the really bad news is that pkgconfig is utterly broken handling Requires.private and doesnt really do the right thing either :-( Fixing this issues will make this check a bit more useful, but will still let a lot of stuff to be fixed, including most pkgconfig scripts that are frequently broken as well :-| -- "If this is the best God can do, I am not impressed" -George Carlin (1937-2008) Cristian Rodríguez R. Software Developer Platform/OpenSUSE - Core Services SUSE LINUX Products GmbH Research & Development http://www.opensuse.org/
Cristian Rodríguez wrote in Sun 03/01 2009 at 15:28 -0300:
Hi ;-)
11-check-pkgconfig-deps has some issues, that causes increasing package dependency bloat. let say we have packages X, Y, Z where Z is being checked by this script.
Z : Requires Y that provides Y.pc Y: Requires X that provides X.pc
when Z is checked, it complains that there is a missing Require on X which is not really correct,and will led packagers to add X as a dependency of Z and on.. and on.. and on... over the whole dependency tree..thing that will of course end in a unfixable mess as time passes, as dependencies are always added but never removed in this case,even when some intermediate package doesn't use a particular library or component anymore.
Are you sure? check-pkgconfig-deps is not recursive. Note that another tool - check-libtool-deps enters into a recursion for required packages that don't specify their .la requirements. It has a historic background - in the time the check was written, only about a half of .la files was packaged in -devel package.
The second issue, is that this script does not account Requires.private that should be taken in consideration _only_ when the package contains static libraries,
I guess that it should not be complicated to implement it.
the really bad news is that pkgconfig is utterly broken handling Requires.private and doesnt really do the right thing either :-(
http://bugs.freedesktop.org/show_bug.cgi?id=9917
Fixing this issues will make this check a bit more useful, but will still let a lot of stuff to be fixed, including most pkgconfig scripts that are frequently broken as well :-|
In case of enforcing of strict devel file split policy, we may simply backport find_requires/provides from rpm 4.6 and do all these things automatically, and no check will be needed any more. -- Best Regards / S pozdravem, Stanislav Brabec software developer --------------------------------------------------------------------- SUSE LINUX, s. r. o. e-mail: sbrabec@suse.cz Lihovarská 1060/12 tel: +420 284 028 966, +49 911 740538747 190 00 Praha 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
Stanislav Brabec escribió:
Cristian Rodríguez wrote in Sun 03/01 2009 at 15:28 -0300:
Hi ;-)
11-check-pkgconfig-deps has some issues, that causes increasing package dependency bloat. let say we have packages X, Y, Z where Z is being checked by this script.
Z : Requires Y that provides Y.pc Y: Requires X that provides X.pc
when Z is checked, it complains that there is a missing Require on X which is not really correct,and will led packagers to add X as a dependency of Z and on.. and on.. and on... over the whole dependency tree..thing that will of course end in a unfixable mess as time passes, as dependencies are always added but never removed in this case,even when some intermediate package doesn't use a particular library or component anymore.
Are you sure? check-pkgconfig-deps is not recursive.
That's exactly the problem, the lack of at least some recursive checks ;) looks like I failed to express the problem in words, so let's do a practical example.. osc co GNOME:Factory vte see the -devel package requires they say: Requires: %{name} = %{version} fontconfig-devel glib2-devel gtk2-devel ncurses-devel pango-devel xorg-x11-devel They should say Requires: %{name} = %{version} gtk2-devel now it you rebuild the package you will see: .. testing devel dependencies required by pkgconfig .pc files Error: Missing "Requires: glib2-devel" in vte-devel (/usr/lib64/pkgconfig/vte.pc requires glib-2.0.pc). Error: Missing "Requires: glib2-devel" in vte-devel (/usr/lib64/pkgconfig/vte.pc requires gobject-2.0.pc). Error: Missing "Requires: pango-devel" in vte-devel (/usr/lib64/pkgconfig/vte.pc requires pango.pc). headers installed by vte-devel only require gtk2-devel (and the dependencies of it , in this particular case pango-devel that will pull glib2-devel into the system) all the other Requires will be either pulled by other -devel packages or are simple not needed (like ncurses-devel) Is it now more clear ? -- "If this is the best God can do, I am not impressed" -George Carlin (1937-2008) Cristian Rodríguez R. Software Developer Platform/OpenSUSE - Core Services SUSE LINUX Products GmbH Research & Development http://www.opensuse.org/
Cristian Rodríguez wrote in Wed 03/04 2009 at 17:47 -0300:
Stanislav Brabec escribió:
Are you sure? check-pkgconfig-deps is not recursive.
That's exactly the problem, the lack of at least some recursive checks ;) looks like I failed to express the problem in words, so let's do a practical example..
osc co GNOME:Factory vte
see the -devel package requires they say:
Requires: %{name} = %{version} fontconfig-devel glib2-devel gtk2-devel ncurses-devel pango-devel xorg-x11-devel
They should say
Requires: %{name} = %{version} gtk2-devel
now it you rebuild the package you will see:
.. testing devel dependencies required by pkgconfig .pc files Error: Missing "Requires: glib2-devel" in vte-devel (/usr/lib64/pkgconfig/vte.pc requires glib-2.0.pc). Error: Missing "Requires: glib2-devel" in vte-devel (/usr/lib64/pkgconfig/vte.pc requires gobject-2.0.pc). Error: Missing "Requires: pango-devel" in vte-devel (/usr/lib64/pkgconfig/vte.pc requires pango.pc).
headers installed by vte-devel only require gtk2-devel (and the dependencies of it , in this particular case pango-devel that will pull glib2-devel into the system) all the other Requires will be either pulled by other -devel packages or are simple not needed (like ncurses-devel)
Is it now more clear ?
Yes, I understand and don't agree with you. vte.pc cleanly says, that glib2-devel is required. Even if it is just now cleanly superfluous, it's wise to add it explicitly. In past we migrated from neededforbuild (list of all packages was required) to BuildRequires (only first level dependencies are required), such simplification was very welcome. Somebody created a tool to optimize requirements and everything worked pretty well. A bit later, several low-level packages changed its dependencies. High level packages did not get devel packages they expected and failed, or even worse, we got crippled packages. As an intelligent being, you can recognize, that gtk2-devel will require glib2-devel "forever", but say libsvg-devel was just a "random" dependency in the chain of gtk2-devel. It's not case of a stupid script. If we implement it, we may lose these dependencies in time, and the recommendation may even depend on distribution. As it is only a warning, it can product bad surprises. Note: Automatic generation of BuildRequires and devel Requires from pkg-config instead of ugly check would be a good idea. But due to strict limitations of rpmbuild design, it would not be straight-forward. I experimented with this idea in past. pkg-buildrequires works pretty well (I already tried it on several hundreds packages). A bit more work needs to be done before the "hard use" (analyze AC_CHECK_LIB results, respect .la files, if they are packages). http://software.opensuse.org/search?baseproject=openSUSE%3A11.1&p=1&q=pkg-buildrequires -- Best Regards / S pozdravem, Stanislav Brabec software developer --------------------------------------------------------------------- SUSE LINUX, s. r. o. e-mail: sbrabec@suse.cz Lihovarská 1060/12 tel: +420 284 028 966 190 00 Praha 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
Stanislav Brabec escribió:
A bit later, several low-level packages changed its dependencies. High level packages did not get devel packages they expected and failed,
So it had the expected behaviuor.. if something critical changes, we want to know about it right ? or we will carry a huge amount of unneded dependencies till the end of times ?
As an intelligent being, you can recognize, that gtk2-devel will require glib2-devel "forever", but say libsvg-devel was just a "random" dependency in the chain of gtk2-devel.
That's exactly my worry, those "random" dependencies are never, ever removed from packages, and there is a _huge_ mess..
Note: Automatic generation of BuildRequires and devel Requires from pkg-config
Really ? .pc files are usually loated or just wrong, while doing this is certainly interesting, those have to be fixed first IMHO. -- "If this is the best God can do, I am not impressed" -George Carlin (1937-2008) Cristian Rodríguez R. Software Developer Platform/OpenSUSE - Core Services SUSE LINUX Products GmbH Research & Development http://www.opensuse.org/
Cristian Rodríguez wrote:
Stanislav Brabec escribió:
A bit later, several low-level packages changed its dependencies. High level packages did not get devel packages they expected and failed,
So it had the expected behaviuor.. if something critical changes, we want to know about it right ? or we will carry a huge amount of unneded dependencies till the end of times ?
It is not expected behavior. Suppose you are building your package against the two different versions of openSUSE (you even may install exactly the same package to more versions of openSUSE). In both of them the package has exactly the same dependencies. In one distro, the devel file works because the needed package is required implicitly by one of low level packages. In the second one, its use will fail. Solution: Never expect that low level packages will import some package for you and you are always on the safe side.
As an intelligent being, you can recognize, that gtk2-devel will require glib2-devel "forever", but say libsvg-devel was just a "random" dependency in the chain of gtk2-devel.
That's exactly my worry, those "random" dependencies are never, ever removed from packages, and there is a _huge_ mess..
Yes, it's another story and we have to invent a better way than manual copy and paste. rpm-4.6 may be answer for Requires. For BuildRequires, more sophisticated tool is required. pkg-buildrequires can do it, but it is a ugly tool by principle.
Note: Automatic generation of BuildRequires and devel Requires from pkg-config
Really ? .pc files are usually loated or just wrong, while doing this is certainly interesting, those have to be fixed first IMHO.
Starting Note that rpm-4.6 automatically collects dependencies from .pc files. If they are wrong, Fedora will fix have to them. -- Best Regards / S pozdravem, Stanislav Brabec software developer --------------------------------------------------------------------- SUSE LINUX, s. r. o. e-mail: sbrabec@suse.cz Lihovarská 1060/12 tel: +420 284 028 966, +49 911 740538747 190 00 Praha 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
On Thu, 5 Mar 2009, Stanislav Brabec wrote:
Cristian Rodríguez wrote:
Stanislav Brabec escribió:
A bit later, several low-level packages changed its dependencies. High level packages did not get devel packages they expected and failed,
So it had the expected behaviuor.. if something critical changes, we want to know about it right ? or we will carry a huge amount of unneded dependencies till the end of times ?
It is not expected behavior. Suppose you are building your package against the two different versions of openSUSE (you even may install exactly the same package to more versions of openSUSE). In both of them the package has exactly the same dependencies. In one distro, the devel file works because the needed package is required implicitly by one of low level packages. In the second one, its use will fail.
Solution: Never expect that low level packages will import some package for you and you are always on the safe side.
No, the solution is of course for -devel packages to Require what they require for building an application against themselves. Recursively expanding them to BuildRequires of an application package is just wrong IMHO. So our Requires should already be as complete as what .pc files specify. Richard.
Le jeudi 05 mars 2009, à 10:42 -0300, Cristian Rodríguez a écrit :
Stanislav Brabec escribió:
Note: Automatic generation of BuildRequires and devel Requires from pkg-config
Really ? .pc files are usually loated or just wrong, while doing this is certainly interesting, those have to be fixed first IMHO.
And fixing them is a good thing, so we should just do it ;-) Vincent -- Les gens heureux ne sont pas pressés. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
Richard Guenther wrote in Wed 03/05 2009 in 16:58 +0100:
On Thu, 5 Mar 2009, Stanislav Brabec wrote:
Solution: Never expect that low level packages will import some package for you and you are always on the safe side.
No, the solution is of course for -devel packages to Require what they require for building an application against themselves. Recursively expanding them to BuildRequires of an application package is just wrong IMHO.
If I understood correctly, Cristian wanted just opposite: - vte requires gtk2, gtk2 requires glib2 - vte _itself_ also directly refers to glib2 and requires it Cristian wanted to remove glib2-devel from Requires to make Requires simpler and considered: - vte-devel works perfectly - check-pkgconfig-deps complains I opposed: It's correct that it complains: vte.pc refers to glib-2.0.pc.
So our Requires should already be as complete as what .pc files specify.
Yes, I agree. check-pkgconfig-deps just exactly maps required pc files to packages and the wants these packages. -- Best Regards / S pozdravem, Stanislav Brabec software developer --------------------------------------------------------------------- SUSE LINUX, s. r. o. e-mail: sbrabec@suse.cz Lihovarská 1060/12 tel: +420 284 028 966, +49 911 740538747 190 00 Praha 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
Vincent Untz escribió:
And fixing them is a good thing, so we should just do it ;-)
Yes, but we have to fix pkgconfig first on its handling of Libs.private, Requires.private, then the next step is to Figure if the "Requires*", and "Libs*" are correct, send those fixes to upstream and then profit ;) -- "If this is the best God can do, I am not impressed" -George Carlin (1937-2008) Cristian Rodríguez R. Software Developer Platform/OpenSUSE - Core Services SUSE LINUX Products GmbH Research & Development http://www.opensuse.org/
Cristian Rodríguez wrote:
Vincent Untz escribió:
And fixing them is a good thing, so we should just do it ;-)
Yes, but we have to fix pkgconfig first on its handling of Libs.private, Requires.private, then the next step is to Figure if the "Requires*", and "Libs*" are correct, send those fixes to upstream and then profit ;)
Its not a sufficient fix to be able to build semi-static libraries: http://bugs.freedesktop.org/show_bug.cgi?id=9917 -- Best Regards / S pozdravem, Stanislav Brabec software developer --------------------------------------------------------------------- SUSE LINUX, s. r. o. e-mail: sbrabec@suse.cz Lihovarská 1060/12 tel: +420 284 028 966 190 00 Praha 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
participants (4)
-
Cristian Rodríguez
-
Richard Guenther
-
Stanislav Brabec
-
Vincent Untz