[opensuse-packaging] Communication problem within packagers
Hello, At first I'm a maintainer of PokerTH for packaging on openSUSE systems. I want to discuss with you a simple problem. Unfortunately I see a problem with a bad consultation between you and me as packager with dependet package in OBS. In my case the package PokerTH needs boost package. Any time the boost package was updated from 1.38 to 1.39. The Upstream from PokerTH know about a established problem in boost 1.39, so the new library let crash the PokerTH-Server. For my case I have to consult the boost packager for patching the boost package. The patching is a least problem. The biggest problem is the nonconsultation with all of packager of dependet packages before any update of the dependet packages. The maintainer of package (e.g. boost) is completely blind to me as packager (PokerTH), but backwards I know which package depend on other sub-packages. This communication order isn't a problem e.g.: Package PokerTH -> Package boost -> Package glibc Package Wesnoth -> Package boost -> Package glibc In this case all packages are completely blind afterwards e.g.: Package glibc -> Package boost -> Package PokerTH Package glibc -> Package boost -> Package Wesnoth This example isn't only for boost, but also other dependet packages. I think, all other packagers have the same problem. On this note we need a simple communication platform for packager like Wiki-pages or something like that. On this platform every packager see which package depends with other package and can easily consulte the other packager or inform of a upcoming update of sub-package. In this case we have a chance to discuss about potential bugs, incompatiblity and so on. I wish more communication with packager from dependet packages. What do you think about this situation? How we can change this situation? Regards, Sebastian -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
On Mon, 17 Aug 2009, Sebastian Siebert wrote:
Hello,
At first I'm a maintainer of PokerTH for packaging on openSUSE systems. I want to discuss with you a simple problem.
Unfortunately I see a problem with a bad consultation between you and me as packager with dependet package in OBS. In my case the package PokerTH needs boost package. Any time the boost package was updated from 1.38 to 1.39. The Upstream from PokerTH know about a established problem in boost 1.39, so the new library let crash the PokerTH-Server. For my case I have to consult the boost packager for patching the boost package. The patching is a least problem. The biggest problem is the nonconsultation with all of packager of dependet packages before any update of the dependet packages. The maintainer of package (e.g. boost) is completely blind to me as packager (PokerTH), but backwards I know which package depend on other sub-packages.
This communication order isn't a problem e.g.: Package PokerTH -> Package boost -> Package glibc Package Wesnoth -> Package boost -> Package glibc
In this case all packages are completely blind afterwards e.g.: Package glibc -> Package boost -> Package PokerTH Package glibc -> Package boost -> Package Wesnoth
This example isn't only for boost, but also other dependet packages. I think, all other packagers have the same problem.
On this note we need a simple communication platform for packager like Wiki-pages or something like that. On this platform every packager see which package depends with other package and can easily consulte the other packager or inform of a upcoming update of sub-package. In this case we have a chance to discuss about potential bugs, incompatiblity and so on. I wish more communication with packager from dependet packages.
What do you think about this situation? How we can change this situation?
If you are aware about a problem in boost 1.39 and you depend on boost
you should notify the packager of boost of this problem, for example
by filing a bugreport. If you do this in advance then this situation
may be avoided.
I see no other way the situation can be improved.
Richard.
--
Richard Guenther
Hello Richard, On Mon, 17 Aug 2009, Richard Guenther wrote:
On Mon, 17 Aug 2009, Sebastian Siebert wrote: [..]
On this note we need a simple communication platform for packager like Wiki-pages or something like that. On this platform every packager see which package depends with other package and can easily consulte the other packager or inform of a upcoming update of sub-package. [..] If you are aware about a problem in boost 1.39 and you depend on boost you should notify the packager of boost of this problem, for example by filing a bugreport. If you do this in advance then this situation may be avoided.
Am I prescient? The boost-packager probably get's the new version first (that's his job, so to speak).
I see no other way the situation can be improved.
It's more about being able to look up "who might get hit by an update". Something like being able to notify dependent packagers... The deps are all there. Being able to send a message like "Hey all you guys that use this package, I'm gonna update to version x.y.z. Be prepared. This update may be tricky. Digestive-end-products might hit the air-moving-device. Set your OBS-builds to disabled, run a testbuild with only one of them, or preferrably a local build, once the update's done. File a bug as usual to ... if the testbuild fails." to all packagers of dependent packages (and only those), would be quite nice to have. E.g. packagers could set their build to disabled until the build of the dependance is through, grab that package locally and do a local testbuild (or one OBS build) -- before hitting the OBS with (possibly) a "wave" of doomed-to-fail builds. Sure, dependant packagers can file bugs after the SHF. But possibly, by then tons of packages are "Failed", having uselessly blocked the OBS. It'd just be nice to be able to notify and/or to communicate _before_ the fact. A Wiki or whatever would be just one way to document the dependencies. I myself'd love to have a "packages depending on this package" in osc/the WebUI. And a "notify dependant packagers". Really nice would be an option, that packagers could set: [X] disable this build if a dependend on package is set to "may break" (or whatever one might call that state), or if the packager of the depended-on package notifies dependant package(r)s [about a pending update], and notify me that I/we should watch it / look into it. (that could be split into more options, it's just about the concept now) For packagers of glibc, gcc, kernel etc., like you, AFAIK, this feature'd be rather irrelevant (what package does _not_ require one of those?), but for the bulk of packages, and esp. the more exotic libraries, that have much fewer "users" / dependant packagers, it'd be very-nice-to-have. Oh, and consider the scenario of someone getting tired of building something. How's he to know who uses his packages? And, BTW: I don't know what you package, but there are a lot of "broken" upstream-packages out there, where we lowly packagers do have to fight hard to blend / bend / break them into a sane openSUSE system. And not all of us are as well versed as you with the OBS, with .spec, with the various build-tools (autotools is actually the least painful, in my experience), with good old "plain" GNU make, ... Sascha and I sure could use help on bending 'freemedforms' into shape, you are herewith invited to join the effort (warning: it uses qmake and far too many build-path- and binary-path derived paths, hardcoded into the binary or somesuch... Fun! Not! -dnh, and that's me talking, building RPMs for his own system and no profit, for about 8-9 years, blending / bending / breaking them to fit his - as of the 16th - now officially 10 year old installation[0]. dh@slarty[5]: ~ (0)$ uname -r 2.4.37.5 dh@slarty[5]: ~ (0)$ perl -v | grep built This is perl, v5.10.0 built for i686-linux-thread-multi-64int-ld dh@slarty[5]: ~/OTR (0)$ rpm -qa --last | headntail -1 qv-0.9.1-1 Thu 30 Jul 2009 04:02:15 bc-1.04-74 Mon 16 Aug 1999 07:19:26 dh@slarty[5]: ~ (0)$ cat /etc/SuSE-release SuSE Linux 6.2 (i386) VERSION = 6.2 Go figure. Yes, most of the system belongs to some .rpm or other. I think I can claim the belt of the 42nd-dan of rpm or something like that. I don't just sit in the ivory tower of the OBS, only using the current openSUSE or Factory. (and if that is or seems derogatory of you, feel free to write (via PM or here) or flame me (via PM). I'd love to know more about what you do at SuSE). IIRC we already had some discussion about gcc stuff (moderated by/via pth). [0] I actually did install it a bit before, but had to reinstall quite a lot, and none of the older packages survived that. Files in /etc/ imply July 23rd 1999 as installation date, e.g. ltrace.conf, at.deny, arenarc~, aliases.susenew, auto.misc, lmhosts, sshd.config~, conf.modules.orig, login.defs.orig.o ... /etc/permissions~ and 2 others even imply July 20th ;) -- "All mushrooms are edible. However, some of them only once" -- Ino!~ -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
On Tuesday 18 of August 2009, David Haller wrote:
It's more about being able to look up "who might get hit by an update".
Something like being able to notify dependent packagers... The deps are all there. Being able to send a message like
"Hey all you guys that use this package, I'm gonna update to version x.y.z. Be prepared. This update may be tricky. Digestive-end-products might hit the air-moving-device. Set your OBS-builds to disabled, run a testbuild with only one of them, or preferrably a local build, once the update's done. File a bug as usual to ... if the testbuild fails."
to all packagers of dependent packages (and only those), would be quite nice to have.
I agree that being able to find this out would be nice, but it wouldn't help in this case. If this would happen everytime, everybody would learn to ignore these mails by the 50th one at the latest, which for some packages could be within a month. Do people really care that much that libattr has been upgraded from 2.4.43 to 2.4.44 and as a result most probably nothing interesting happens at all? That a minor version upgrade of a library broke some application is unfortunate, but, assuming the developers of Boost are not a bunch of crazy retards who do it just for the fun of it, it's simply a bug and that can happen. -- Lubos Lunak KDE developer -------------------------------------------------------------- SUSE LINUX, s.r.o. e-mail: l.lunak@suse.cz , l.lunak@kde.org Lihovarska 1060/12 tel: +420 284 084 672 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
We hit a similar issue on our internal OBS instance, so I made some mods to the server so that a developer could grab a dependency graph and determine all packages that depend on their OBS package (it actually goes down to the Subversion code project itself, but that's not required.) So the developer can be in kernel-source and do svndownstream deps ls and get a listing all OBS packages that depend on kernel-source. What happens is svndownstream gets a list of all the buildinfos (which a mod to the server merges into one xml file at /build/_builddependencytree) and then uses the bdep entries and some python code in svndownstream (which is built on top of osc) to generate a dependency graph. The problem with it is my implementation. I couldn't follow all the code in bs_sched, so we had to switch to generating the full depgraph only daily (since it takes 8-9 minutes). I'm sure someone that actually wrote sub buildinfo() or bs_sched could improve on that time since in theory bs_sched has a depgraph already and there's potential for caching if you know you're going to ask for everyone's buildinfo. It's just bs_sched is a big loop and I couldn't figure out how to extract it any of the information it had in it. Sebastian Siebert wrote:
Hello,
At first I'm a maintainer of PokerTH for packaging on openSUSE systems. I want to discuss with you a simple problem.
Unfortunately I see a problem with a bad consultation between you and me as packager with dependet package in OBS. In my case the package PokerTH needs boost package. Any time the boost package was updated from 1.38 to 1.39. The Upstream from PokerTH know about a established problem in boost 1.39, so the new library let crash the PokerTH-Server. For my case I have to consult the boost packager for patching the boost package. The patching is a least problem. The biggest problem is the nonconsultation with all of packager of dependet packages before any update of the dependet packages. The maintainer of package (e.g. boost) is completely blind to me as packager (PokerTH), but backwards I know which package depend on other sub-packages.
This communication order isn't a problem e.g.: Package PokerTH -> Package boost -> Package glibc Package Wesnoth -> Package boost -> Package glibc
In this case all packages are completely blind afterwards e.g.: Package glibc -> Package boost -> Package PokerTH Package glibc -> Package boost -> Package Wesnoth
This example isn't only for boost, but also other dependet packages. I think, all other packagers have the same problem.
On this note we need a simple communication platform for packager like Wiki-pages or something like that. On this platform every packager see which package depends with other package and can easily consulte the other packager or inform of a upcoming update of sub-package. In this case we have a chance to discuss about potential bugs, incompatiblity and so on. I wish more communication with packager from dependet packages.
What do you think about this situation? How we can change this situation?
Regards,
Sebastian
-- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
Am Montag, 17. August 2009 17:23:16 schrieb Luke Imhoff:
We hit a similar issue on our internal OBS instance, so I made some mods to the server so that a developer could grab a dependency graph and determine all packages that depend on their OBS package (it actually goes down to the Subversion code project itself, but that's not required.) So the developer can be in kernel-source and do svndownstream deps ls and get a listing all OBS packages that depend on kernel-source.
What happens is svndownstream gets a list of all the buildinfos (which a mod to the server merges into one xml file at /build/_builddependencytree) and then uses the bdep entries and some python code in svndownstream (which is built on top of osc) to generate a dependency graph.
The problem with it is my implementation. I couldn't follow all the code in bs_sched, so we had to switch to generating the full depgraph only daily (since it takes 8-9 minutes). I'm sure someone that actually wrote sub buildinfo() or bs_sched could improve on that time since in theory bs_sched has a depgraph already and there's potential for caching if you know you're going to ask for everyone's buildinfo. It's just bs_sched is a big loop and I couldn't figure out how to extract it any of the information it had in it.
We have actually the "whatdependson" call as (the last?) missing feature for our 1.7 release. We have already the :depend data on the server, so it should be not too hard to get this implemented. However, I would be very interessted in your code creating the graphes. It would be nice if we could integrate this into our web interface (maybe even interactive via clicking on an item and jump to that package ;) bye adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-packaging+help@opensuse.org
Adrian Schröter wrote:
Am Montag, 17. August 2009 17:23:16 schrieb Luke Imhoff:
We hit a similar issue on our internal OBS instance, so I made some mods to the server so that a developer could grab a dependency graph and determine all packages that depend on their OBS package (it actually goes down to the Subversion code project itself, but that's not required.) So the developer can be in kernel-source and do svndownstream deps ls and get a listing all OBS packages that depend on kernel-source.
What happens is svndownstream gets a list of all the buildinfos (which a mod to the server merges into one xml file at /build/_builddependencytree) and then uses the bdep entries and some python code in svndownstream (which is built on top of osc) to generate a dependency graph.
The problem with it is my implementation. I couldn't follow all the code in bs_sched, so we had to switch to generating the full depgraph only daily (since it takes 8-9 minutes). I'm sure someone that actually wrote sub buildinfo() or bs_sched could improve on that time since in theory bs_sched has a depgraph already and there's potential for caching if you know you're going to ask for everyone's buildinfo. It's just bs_sched is a big loop and I couldn't figure out how to extract it any of the information it had in it.
We have actually the "whatdependson" call as (the last?) missing feature for our 1.7 release.
We have already the :depend data on the server, so it should be not too hard to get this implemented.
However, I would be very interessted in your code creating the graphes. It would be nice if we could integrate this into our web interface (maybe even interactive via clicking on an item and jump to that package ;)
bye adrian
The main class doing all the calculations based on the /build/_binarytree and /build/_builddependencytree is DependencyGraph in build.py. The binarytree and builddependencytree are just calls to the server that merge together all the _binary or bdeps entries for the _buildinfo calls for all packages. You can infer the layout from DependencyGraph.fetch(). Creating a tree is down in the DependencyGraph.tree(). It gives output like emerge --tree: PROJECT PACKAGE REPOSITORY ARCH DEPENDENT_PROJECT DEPENDENT_PACKAGE DEPENDENT_REPOSITORY DEPENDENT_ARCH The tree is just based on a topological sort (http://en.wikipedia.org/wiki/Topological_sort) generated by DependencyGraph.topologicalSort() that includes the items depth in the tree so the tree can be printed with indentation for the levels. DependencyGraph.dot() will generate a dot file that can be passed to graphviz to make pretty (or scary depending on your point of view and how many things are interdependent) pictures of the dependency graph.
participants (6)
-
Adrian Schröter -
David Haller -
Lubos Lunak -
Luke Imhoff -
Richard Guenther -
Sebastian Siebert