[Bug 221276] New: FATE #301145: ease providing new pci.ids
https://bugzilla.novell.com/show_bug.cgi?id=221276 Summary: FATE #301145: ease providing new pci.ids Product: openSUSE 10.2 Version: Beta 2 plus Platform: Other OS/Version: Other Status: NEW Severity: Normal Priority: P5 - None Component: Basesystem AssignedTo: mjancar@novell.com ReportedBy: mjancar@novell.com QAContact: qa@suse.de When a KMP is supporting some new hardware unknown to lspci, hwinfo and similar tools, the corresponding vendors are getting support calls as the system is listing hardware components as 'unknown' which in the users eyes translates into "doesn't recognize my component!" possible solutions - update pci.ids frequently via maintenance - make /usr/share/pci.ids a symlink, define location for 'all of them' and allow packages to bring their own, curren tcopy. may come with pci.ids-lint tool to make sure file is always extended or corrected and know ids are not dropped. - modularize pci.ids file this is really annoying partners so I'd prefer to have a technical solution but it's not blocking operation. providing frequent maintenance updates of the pci.ids file until we have a sound solution in SP1 could make our partners happy. References: ~garloff/scripts/Python/merge-pciids contains my prototype for creating /usr/share/pci.ids The distributed (and updated via update-pciids) pci.ids file would be moved to /usr/share/pci.ids.d/pci.ids.dist. In that directory, all other pci.ids would be dropped, with a naming convention (RPMname) that should avoid conflicts. The used /usr/share/pci.ids file would be a %ghost in the filelist and built in %post of pciutils. Then we either play %trigger tricks or just require all KMPs that needs new pci.ids to call merge-pciids in their %post script. The prototype is operational and tested well -- only drawback might be that we require python on the platform. (But if some feels this is a problem, he can rewrite the script in a low level language.) Partner: Novell NDA: no NDA Discussion: #1: gp@suse.de (2006-08-28 16:18:26) [reply] Kurt, I believe you expressed interest to drive this as architect? We'll probably want to release this with a maintenance update rather sooner than later, not just SP1, if possible without undue risk/effor. #2: garloff@suse.de (2006-08-28 16:27:06) [reply] It is a cosmetic shortcoming. Nevertheless, we should not give any partner any excuse to not use the partner driver update process, so I'm all for working on a solution. I'll drive it. My preferred solution would be to modularize pci.ids, so make pcitutils read the IDs that a package (typically KMP) may put in files dropped in /usr/share/pci.ids.d/ in addition to the IDs in the classical /usr/share/pci.ids. This obviously needs some policy for how those files may be named and what contents might be allowed. We need to investigate whether this is feasible: If there are many other consumers of pci.ids except pciutils, then we may need to change the file to a symlink somewhere below /var/ that's constructed out of the pci.ids.orig file and all the additional files somewhere. The reconstruction would need to be done by a litte script invoked via rpm %post or %trigger sections. Comments? #3: froh@suse.de (2006-08-29 10:02:18) [reply] a very simple solution could be to move the pci.ids file into a separate subpackage and to move ownership to the ones in charge of the driver process then for KMPs that are within the process (silver level partner) we simply provide an update of that pci-ids package whenever needed. the helpfull contribution from engineering could be a pci-ids-lint that checks the pci-ids file is valid and that flags red yellow green which PCI ids have been removed, modified or added respectively. That makes it trivial enough to review the updated pci-ids file that is submitted by the partner #6: garloff@suse.de (2006-09-22 10:20:07) [reply] Susanne, your suggestion implies that every driver update that requires s new PCI ID, we'd issue a new pciids.rpm. I'm not sure this is a good idea. It also implies that all driver updates are done via the ones in charge of the driver process -- a restriction that we did not want to build into the process. #8: froh@suse.de (2006-09-22 15:47:13) [reply] Indeed some partners will provide their KMPs on their own, didn't think of that. Then that leaves us with modularizing the pci.ids file. Would you prefer constructing a pci.ids file from pci.ids.d/* and leaving utilities allone? that would also stay compatible with partner tools that we are not awae of. Or would you prefer the utilities to use some libpciids to merge them on the fly? #10: garloff@suse.de (2006-09-25 18:14:28) [reply] Just merging them is easiest and least risky. I have a prototype ready for this (python script). #7: garloff@suse.de (2006-09-22 10:45:47) [reply] Does not look too bad: g32:/ # for name in /sbin/* /bin/* /lib64/* /usr/lib64/* \ /usr/bin/* /usr/sbin/* /usr/X11R6/bin/* /usr/X11R6/lib64* \ /opt/gnome/lib64/* /opt/gnome/bin/* /opt/kde3/lib64/* \ /opt/kde3/bin/*; do echo -en "\r$name: "; \ strings $name | grep pci.ids; done /sbin/lspci: /usr/share/pci.ids /sbin/setpci: /usr/share/pci.ids /sbin/update-pciids: SRC="http://pciids.sourceforge.net/v2.2/pci.ids" DEST=/usr/share/pci.ids /usr/bin/systool: /usr/local/share/pci.ids /usr/sbin/hald: /usr/share/pci.ids pci_ids_load /usr/sbin/s2ram: /usr/share/pci.ids: /usr/sbin/sysp: /usr/local/share/pci.ids So on first sight it would seem that only pciutils and hald would need to be adapted. Question remains whether we could do such a thing for old products -- or whether we better dynamically construct the classical pci.ids here. #4: kukuk@suse.de (2006-08-30 13:32:06) [reply] I hope everybody is aware of /sbin/update-pciids, which for example could be called by the post install script of KMP packages? #5: garloff@suse.de (2006-09-22 10:18:09) [reply] /sbin/update-pciids will pull the latest pci.ids from sourceforge. So it would only serve its purpose if the new IDs have been merged upstream when the driver is installed. Not sure whether that limitation is a problem. A more serious drawback is that it would result in the file /usr/share/pci.ids not being clean any more w.r.t. the RPM database. #9: froh@suse.de (2006-09-22 15:48:51) [reply] That would require access to sourceforge at the time the KMP is installed. I doubt this is alwasy the case. I'd rather prefer to bring the updated pci.ids along. #11: kukuk@suse.de (2006-10-09 15:04:27) [reply] I don't like the approach of merging pci.ids files from other ones. We know that this very problematic and in the end we will get yet another SUSEconfig script to do the job. Please enhance the pciutils to support a /usr/share/pci.ids.d/ directory. There third party vendors can drop in their files and this problem of having an inconsistent pci.ids file is solved for all time. #12: agruen@suse.de (2006-10-09 16:11:39) [reply] I agree with the /usr/share/pci.ids.d/ approach. #13: garloff@suse.de (2006-10-09 23:02:44) [reply] Well, with the merging approach, you would still just drop files into /usr/share/pci.ids.d/ directory (with some naming convention to avoid conflicts). The only difference is that you'd build the classic pci.ids file from pci.ids.dist and the (other) files in /usr/share/pci.ids.d/ so noone that happens to assume the existence of this file will fail. The downside is that you need to call the merge-pciids program in the %post section of pciutils and of every KMP that drops a file there. (Well, you might be able to play %trigger tricks alternatively ...) Not requiring the merge-pciids is certainly more elegant -- but that looks to risky an option for OLD distributions. I'm happy to be convinced otherwise. #14: kukuk@suse.de (2006-10-11 11:18:54) [reply] We know that %trigger tricks will not work and that, at the end, we will need yet another SuSEconfig script, about which people will complain and which will not be called in all cases, means the file is out of sync. #15: garloff@suse.de (2006-10-22 00:30:11) [reply] I fail to see a need for a SUSEconfig script. Just call it from %post in pciutils and every KMP that drops a file into /usr/share/pci.ids.d/. (And from the update-pci-ids script.) As it's in the own interest of the KMP providers to have the ability to drop pci ids there and that's not possible today, we just need to document that except for dropping a file (with a to be defined naming convention) they also have to put the call in the %post section of the RPM, otherwise it won't work for them. No chance for hidden failure. #16: garloff@suse.de (2006-10-22 00:39:44) [reply] Thorsten, we need to come to an agreement how to best proceed, otherwise we won't make any progress. The reasoning is known: Just having pciutils read from multiple files is the more elegant.solution, as you don't need to keep derived files in your filesystem that way that could become stale. That would be a code change to pciutils that we should get upstream. Question is whether we'd want tot touch released products (SLES10 and possibly also SLES9). The advantage of a merge script (which I have a working implentation for in python) is that it does not touch pciutils code. Only update-pci-ids would be changed to overwrite pci.ids.dist and the spec files would get added calls in %post. Advantage is that you preserve the existence of a complete pci.ids file, so anyone who uses it (outside of pciutils?) has the full information as well. Disadvantage is that you'd end up with having data twice on the system. And possibly issues with keeping things up-to-date. As Thorsten and I don't seem to be able to agree, I'd like to find a way forward to come to a decision. Should the dist meeting decide. The TPM meeting? Or simply Gerald and Thorsten? And I'd certainly like to hear from the package maintainer about the effort and the chances to get a pciutils code change upstream. #17: mjancar@suse.cz (2006-10-23 14:22:59) [reply] The way to go is IMHO /usr/share/pci.idsd.d/ for (vendor) packages, user defined entries and updates from the sf.net, and /usr/share/pci.ids as the default that nothing (except pciutils-ids package update) should touch, with the former taking precedence when they conflict about an entry. As I know the pciutils author personaly I don't expect much problems getting this upstream. #18: garloff@suse.de (2006-10-27 01:37:16) [reply] OK, what you describe still allows the merge with a script/tool or a merge inside pciutils. I assume you prefer the pciutils merge -- it certainly is the architetually nicer approach. Are you able to implement it? #19: kukuk@suse.de (2006-10-27 12:24:51) [reply] Marian, can you please look at this and speak with the upstream maintainer? I have a short proof of concept patch for this, but it does not work yet with compressed file support enabled (trivial to fix) and id_insert() always reports the entry was already added. I don't understand why, here somebody with more time need to look at it. #20: kukuk@suse.de (2006-10-27 13:56:59) [reply] ~/Export/pciutils-pci.ids.d.diff contains my current diff, the id_insert() duplicate problem is fixed, file compression is still disabled. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #1 from mjancar@novell.com 2006-11-15 08:12 MST ------- bug created to join Martin to the discussion -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #2 from mj@ucw.cz 2006-11-15 10:44 MST ------- I don't think it's a good idea. Your proposal is introducing a lot of extra complexity (not only scanning of multiple files, but also having to define precedence in case of record collisions), while solving only a part of a more general problem: Recently, the available hardware and consequently also the pci.ids database was evolving with a much faster pace than releases of most Linux distributions. The users are regularly seeing hardware which is unidentified with the pci.ids from the distribution and they are equally disapponted if it's a hardware specially listed in some driver package or not. Aside from that, some drivers cannot provide a list of PCI ID's they support, because they are class drivers, so even for the supported devices the list is going to be incomplete. This shows that splitting the ID database isn't likely to help much and that you need to have a mechanism for updating pci.ids between releases anyway (it can be anything from the original update-pciids to issuing new rpm's with just the pci.ids or making your own mirror) and the vendor-supplied ID's can be handled in the same way. Also, please remember that there are many other users of the ID database, so submitting the entries normally is certainly better. On the other hand, I understand your wish to have all supported devices identified (at least in the simple cases of non-class drivers), so I am offering to give your maintainer of pciutils a right to approve new database submissions. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #3 from froh@novell.com 2006-11-16 09:41 MST ------- I see your concern of possibly not getting the PCI-IDs into the database, right? Indeed a certain laziness on the vendor side would be suported by embedding the merge code right into the pciutils. hmmm... We are trying to mitigate between two interests: We definitively want the pciids database to be up to date and accurate, on http://pci-ids.ucw.cz/ as well as on the customer system. Now to update the customer system with the current flat pciids database, we need to update a package on that system. That involves our maitnenance process where the code is going through several hands and is taking time that we'd rather spend on fixing real bugs in the code. For that reason we are looking for a modular plug-in system for the pci-ids. I'm just wondering if Kurt's stand-allone merge-pciids script could help us better achieve both objectives? If we add a --diff option to merge-pciids (that runs a diff -u on the newly generated pciids, compared ot the original one), then the vendor can trivially submit an update to pci-ids@ucw.cz If we submit on behalf of the vendor, with the permissipons you offered we can then in the same vein approve these Novell submitted pci-ids to the database so the pciids maintainers dn't need ot review those coming directly from Novell. And at the same time we don't need to reiterate our lspci package through the maintenance process but can provide a pci-id plugin dir to the component vendors. We'd then add a hook to the KMPs to run the script locally. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #4 from mj@ucw.cz 2006-11-16 14:48 MST ------- In this case, my biggest concern is not that the updates could be missing from the central database (I trust your maintainers to propagate changes to upstream properly), but that you are missing the big picture. The ID's submitted by your vendors are only a tiny fraction of all the updates in pci.ids and your users are going to miss all of them, not only the vendor-submitted ones. Even if the vendor has provided a package, the users would probably like to identify the device _before_ they find the vendor's package (in many cases it's the crucial step in discovering that some vendor's package exists). The proposed ID file merging solves close to nothing. I think that what you need is a general mechanism for updating pci.ids with both the vendor-supplied and community-supplied ID's. I agree that updating the whole pciutils package is too troublesome, but there are many other ways, e.g.: (1) you can split off a separate package with the pci.ids file, which will be released more frequently (and which probably won't need so much scrutiny as code changes). This is similar in spirit to the debian-volatile repository (although Debian has not put pci.ids there yet). (2) you can use something like the update-pciids script to download the ID's from time to time (preferably from your own mirror to avoid too much load on the SF servers). I agree that both possibilities could be problematic on machines not connected to the network, but so is installing any vendor's packages there :-) Also, with (1) the vendor's package can depend on your ID package of at least a given revision, making sure that the ID's will be present. With (2), it can trigger the network update. After pondering on all this, I'm convinced that the right solution is to improve your maintenance process instead of adding hacks to the pciutils. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #5 from froh@novell.com 2006-11-17 02:21 MST ------- Hmmm... I think there is diverging asumptions on 'users' and when they use lspci and what for. If they use lspci to identify which driver they need, then yes, I can very well follow your train of thoughts. If the relevant use that we have to support however is the support case, where users identify their hardware with lspci *after* installing backported drivers, then we haev a different situation. So your question for the big picture is very valid and we'll investigate what exactly we need the updated pciids for and decide from that which path to follow. thanks for your input, Martin! -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #7 from mj@ucw.cz 2006-11-17 03:42 MST ------- Well, I dare say that I have a bit more complete data on what the users use lspci for ;-) Most of the endless stream of complaints about unidentified devices and submits of new ID's for unindentified devices ends up in my mailbox, not in distributors' bug reports (if bugs entered to Debian BTS and database submits by distribution maintainers are a representative sample). The support case you see is just the tip of the proverbial iceberg. Sure, it's probably what you hear about the most, but that the big picture is different is a matter of fact, not a question :) Anyway, regardless of what the relevant use is, my proposal works always. So I suggest to stop pondering on relevant uses (which is doomed to be inaccurate) and implement what's obviously correct. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #12 from mj@ucw.cz 2006-11-21 13:12 MST ------- As nobody replied, I'll reply to myself :) I think that it's obvious that your merging patch (or utility) is much less useful to the users of the pciutils than timely distribution of _all_ new ID's and, frankly, it's the users whom I care about the most, not Novell's partners. (Although I would of course prefer an approach which is useful for everybody.) So unless somebody proves the above reasoning wrong, I don't see a reason for accepting any such patches to upstream. Anyway, if you'd like to cooperate in distributing the ID's, I am open to suggestions and willing to help. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #13 from mjancar@novell.com 2006-11-22 03:20 MST ------- Well, we don't have the ID's, and there is not much point in passing the new ID's through Novell, that would be just an unnecessary additional step without any real benefit for anyone (including your database, it is IMHO better to collect the ID's on just one place). When a customer purchases a very new branded machine and gets a software disk for it, everything should just work after he installs it. The problem is that the path vendor -> pci.ids database -> distribution -> customer that the ID's have to take introduces delays, while if the vendor had a way to put the few ID's necessary for the particular hardware on his disk, everything would work out of the box. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 froh@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |NEEDINFO Info Provider| |adrian@novell.com ------- Comment #14 from froh@novell.com 2006-11-22 04:15 MST ------- Hmmm... Just thinking openSUSE build server... Not sure if you are familiar with it Martin? it's a build server building for many distros, including fedora and debian. Could be a nice vehicle to provide the latest and greatest pciids to all distros automatically and integrated with the respective package management. Adrian, do we have the attachment of source repos in place yet? Or alternatively: how about helping Martin with a script to update a pciids project (with repos for sles, openSUSE, fedora and debian) so he can trivially update the repo from the command line? hmmm... that might then even do as a cvs/svn trigger. Martin, would that work for you? Adrian, what would be the reccomended approach? I think having these repos available should do for our partners. Need to check, but I think this can work, at least for sles10. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #15 from mj@ucw.cz 2006-11-22 04:42 MST ------- Marian, the new ID's needn't pass through Novell, except maybe for some automated generation of a package. The vendor only has to submit the entries to the database, wait for them to be approved (this is where Novell can help to get ID's of their partners approved faster) and for a package to be automatically generated. This can be done in a day. I like the Susanne's idea of using the build servers for that. It seems that I could easily extend the scripts around the PCI ID database to build packages whenever new entries are approved (together with the pci.ids snapshots we already generate). Then SUSE can propagate this package to the distribution (either only to releases or for example as monthly updates) and the vendors can submit their ID's and then upgrade the package in the distribution to the most recent snapshot. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 mjancar@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- AssignedTo|mjancar@novell.com |anicka@novell.com Status|NEEDINFO |NEW Info Provider|adrian@novell.com | -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 anicka@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #22 from garloff@novell.com 2007-01-19 19:38 MST ------- Update: I have submitted packages to SP1 with the merge script. I am aware of one shortcoming: The script does not currently support .gz files; it's easy to add (import gzip; gzip.open()), but it's too late for me in the night to do concentrated testing. The script by-the-way is more error tolerant meanwhile; it would discard the unparseable entries. Thanks for pointing out a syntax change. Can you quickly describe the new rule? Feel free to rely on me for maintaining the script for now. I do agree that it's good to coordinate with maintainers; in fact I'm often the one pushing engineers to do so. But I thought Thorsten's patch was rejected there? :-( -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #24 from mj@ucw.cz 2007-01-20 02:52 MST -------
The script by-the-way is more error tolerant meanwhile; it would discard the unparseable entries.
This is some sort of a joke? ;-) Regardless what the script really does (you haven't shown me yet), silently ignoring errors is unacceptable. The ID file format is context-sensitive and ignoring any line can change the context, leading to complete misinterpretation of the following entries. What could be considered safe is skipping until the first entry starting in column 0, but still I believe that the only sane response is to report an error and fail.
I do agree that it's good to coordinate with maintainers; in fact I'm often the one pushing engineers to do so. But I thought Thorsten's patch was rejected there? :-(
It was rejected for a reason. In the discussion above, I pointed out that both the patch and the merging script fail to solve the main problem, which is users having out-of-date ID files. Your issue with Novell's partners is just a minor special case of the whole problem and I believe we should solve the whole, not introduce random work-arounds for special cases. Also, I mentioned that the ID file is regularly used to find drivers for devices, so making the ID appear only after you install the driver package doesn't work. I think I have proposed a clean solution to the general problem and so far nobody from Novell has demonstrated any flaw in it, or shown that it wouldn't be adequate for their partners. Hence I am not willing to accept a patch which is obviously inferior. Last, but not least: I think it's not wise to have lots of utilities with hard-coded format of the ID file. The format is evolving and it has changed several times in the history, so the less places we have to maintain, the better. Currently it's hard-wired only in the PCI library and in the PCI ID database maintainers' scripts and one should think carefully before adding more. (By the way, the ID snapshot package in the build service are almost ready, I will add regular updates today.) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #25 from garloff@novell.com 2007-01-20 03:12 MST ------- Martin,
The script by-the-way is more error tolerant meanwhile; it would discard the unparseable entries.
This is some sort of a joke? ;-) Regardless what the script really does (you haven't shown me yet), silently ignoring errors is unacceptable. The ID file format is context-sensitive and ignoring any line can change the context,
I'm fully aware of this; that's why the parser must understand what it does and we can't simply concatenate files. The script is everything but quiet about it. Just put nonsense files in there and watch it complain. But I decided against aborting, so we don't end up having an empty pci.ids. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #26 from mj@ucw.cz 2007-01-20 03:21 MST ------- I cannot watch as you haven't shown me the script yet :-) Also, if merging fails, you can just use the main ID file unmerged, it's much better than trying to recover from the error somehow (which is, as I already shown, unlikely to work). -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #27 from anicka@novell.com 2007-01-20 03:35 MST ------- Kurt: BTW, if we would like to use pciids package provided by upstream in buildservice, we will have to split a merge utility from pciids package. As I am reading this discussion, I guess that upstream is not willing to take care of packaging a merge script with fresh pciids and maintaining it for us. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #28 from anicka@novell.com 2007-01-20 03:45 MST ------- Martin: the merge script was just commited for Factory. I will add it here as attachment for your convenience. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #29 from anicka@novell.com 2007-01-20 03:48 MST ------- Created an attachment (id=114023) --> (https://bugzilla.novell.com/attachment.cgi?id=114023&action=view) merge script -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #30 from mj@ucw.cz 2007-01-20 04:20 MST ------- Kurt: I've read the script and I am a bit puzzled: (1) I think that it's not true that the script complains about entries it doesn't understand. At line 246 (the end of the parseline method) I can clearly see that if a line is not understood, it's treated as a comment for the following entry, without even a glimpse of message. Collisions are reported, but syntax errors really aren't. (2) Have you ever tried to run it with `--help'? Instead of writing a usage message (which it tries to), it ends up with an exception. (3) The whole thing smells of overengineering. Spending 352 lines of complex Python code to give a wrong solution of a trivial problem cannot be justified. Moreover, your script has quadratic time complexity in the worst case, because you are using sorted lists where a simple Python dictionary would suffice. By the way, my maintenance script for doing a similar thing has something like 20 lines :) [Well, it doesn't have proper error reporting either, but at least it proves that the problem is trivial.] So no, I am really not going to accept anything even remotely close to this. Also, I still miss any explanation of why such a thing is needed at all. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #33 from mj@ucw.cz 2007-01-22 06:34 MST ------- It seems that you are really going to release this crap, without even trying to fix the fatal errors which I've pointed at, or at least reply to the criticism in public. Has Novell become a company where stupid management decisions override solid technical arguments? If it is so, please consider my offer to give you approval rights for the database, or to automatically build packages with snapshots, withdrawn. If you are not willing to play a fair game, I don't need to play any. I offered cooperation and help to your users, you decided not to care. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #35 from ories@novell.com 2007-01-22 08:18 MST ------- (In reply to comment #33)
It seems that you are really going to release this crap, without even trying to fix the fatal errors which I've pointed at, or at least reply to the criticism in public. Has Novell become a company where stupid management decisions override solid technical arguments?
Martin, I am part of the technical project management team for Novell's SLE products. I'd like to let you know that Kurt will probably not be able to respond back the next 2 days. Please allow us some more time to process the feedback you have provided so far. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #36 from anicka@novell.com 2007-01-22 08:32 MST ------- Martin kindly sent me an upstream merge script he uses for merging pci.ids. I wrote a little shell wrapper for running the script in our distribution. I am going to add both as an attachment. We can easily switch from Kurt's script to this one now and thus fix the bugs mentioned above. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #37 from anicka@novell.com 2007-01-22 08:38 MST ------- Created an attachment (id=114177) --> (https://bugzilla.novell.com/attachment.cgi?id=114177&action=view) merge script from upstream -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #38 from mj@ucw.cz 2007-01-22 08:56 MST ------- Oliver, thanks for the response. As far as I understand, SP1 is going to be released soon and I would really like to avoid releasing it with a completely wrong merging script. I have offered my own script I use for maintaining the ID's to Anna. It's not perfect (as the whole idea of merging multiple databases is a little dubious), but it definitely works much better than the Kurt's script. However, my rant was not only due to bugs in the script, it was a response to the whole situation. I have given several sound reasons why the whole idea of merging ID files is wrong and I also offered a solution which I believe to be much better. But Kurt has ignored these arguments and pushed his merging script anyway, even though your own package maintainer disagreed. I respect your right to do anything you wish (and the license permits) with my package, but I still believe that we should try to cooperate with each other if possible. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #42 from garloff@novell.com 2007-01-22 11:10 MST ------- I guess you are refering to the format VID\tDID\tSVID\tSDID Name format. Yes, that was indeed an extension that makes sens if you only want to add a subsys vendor device ID combination ... We could continue the discussion on that, but it's pointless ... I trust you have carefully implemented your own merge tool. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #43 from mj@ucw.cz 2007-01-22 13:24 MST ------- Hello!
(1) The script does not complain when parsing, but later when it spits out the database and finds entries that don't have a valid format.
Are we really speaking about the same script? I'm trying the one attached to this bug and it definitely doesn't complain at all when fed with a bogus entry. (I've just tried that once more to be sure.)
(2) The script does not actually implement --long-options, only short ones, currently. Would be a one-liner patch to change, if we determine that it's important.
This is also not true. The script explicitly compares for "--verbose", "--help" and so on, only the comparison can never succeed, because an exception is thrown by getopt beforehand.
Sidenote: Runtime of this script is not an issue. You simply don't care at all if installation of the package takes a second longer or not. It's not a frequent operation.
Quadratic behavior on lists with tens of thousands of entries can easily take minutes, not seconds, especially in interpreted languages.
I'd love that -- but unfortunately the scenario also needs to work with ZenWorks, which tends to always install the latest package it finds anywhere.
So you want to say that this whole business is done because of bug (or deficiency to say it politely) in ZenWorks, which you don't want to fix in this release, right? Why not just add an exception to ZenWorks that it shouldn't try to update this exact package automatically? It's a kludge, but a kludge for a single version, while you will have to maintain the merge script kludge for years and I will have to handle bug reports on random pci.ids additions by your partners for years. Also, I completely failed to notice any previous discussion on this ;)
(Also note that partners prefer to have self-contained KMPs without relying on us updating pciutils-ids).
Please also note that users obviously prefer the contrary as they want to be able to identify the device _before_ they install a driver.
ad comment #33: Let me give you the advice to chose your words more carefully when you comment on other engineers' work. People might react as unpolitely as you were and chose to ignore you. That's not the style in which we should interact.
I have started politely and gently, explaining, asking for your reasons and offering solutions which are useful for everybody. You have chosen to ignore my arguments and questions and just pushed your solution without any word of explanation and without even studying the file format. You have also chosen to ignore the opinion of your maintainer of pciutils. You assert things which are obviously not true (see above and several times before). And then you try to preach about manners. Funny, eh? I believe that in such situations good engineers should care about facts and not much about other people's feelings.
ad comment #38 and #39: I appreciate you finding a solution that's more in line with the way things are done upstream.
No, that's not the way things are done upstream -- upstream doesn't handle somebody-and-his-dog's local databases. However, it's at least a way which is not obviously broken and not likely to collide with upstream. To conclude with something positive: If you would like to accept my offer of building the ID snapshot packages and include them ocassionally in your updates, I will be glad to cooperate. If not, well, then you are on your own and I will happily redirect all bug reports for your pciutils package to you. Best regards. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #44 from garloff@novell.com 2007-01-23 10:08 MST ------- Martin, if you like we can continue discussing on the python script that you hate so much. I could point out that some of things you claim are untrue. It's probably not worth recording it in bugzilla, so let me control myself. Package management is a non-trivial task and a discipline where it's not easy to always do the right thing; the solution to dependency solving is not necessarily unique; and once you have conflicts it gets only harder. I won't discuss why in ZenWorks, some cases are handled differently from YaST. If you want to shoot further attacks on our tools and the way we are doing things, feel free. And if it helps you, include as many "totally broken", "crap", "fatal error" attributes as you like. I'm just saying it certainly does not help the recipient to accept your message. But again, let's move it to email then. Having automatically build pciutils-ids packages would be a nice thing and help us to not require our partners to include extra IDs so often. Thanks for the offer! -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #45 from mj@ucw.cz 2007-01-23 14:29 MST -------
if you like we can continue discussing on the python script that you hate so much. I could point out that some of things you claim are untrue.
Then please show at least one in private mail. I still stand behind all the assertions I made and I have enough facts to back them by. Even my assertion that the script is broken (i.e., wrong and hard to fix without rewriting it substantially) and unnecessarily complicated, included a proof. I am not attacking your tools (except for your merge script, but as I have said, these are facts, not attacks), but I definitely do attack your way of "cooperation", which is based on ignorance of other people and of their arguments, even including your own maintainers. For example, since November I was politely explaining why I think my approach is better and asked for counterexamples, but until your yesterday's comment, you didn't consider it necessary to explain anything or even answer in public. And now, when confronted with facts, you only attempt to save yourself by speaking about manners and about package management being a non-trivial task instead, but nothing concrete. That doesn't work with me, I am a hacker and not a manager and I'm glad it's so. Anyway, back to technical stuff, so that we can close this bug. If I prepare the automatic builds of pciids packages, are you willing to include them in distribution updates occassionally? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #46 from kukuk@novell.com 2007-04-16 05:11 MST ------- Can we close this now? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #47 from ories@novell.com 2007-04-16 08:20 MST ------- still in testing, ETA for the results is hopefully tomorrow -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ories@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |RESOLVED Resolution| |FIXED ------- Comment #48 from ories@novell.com 2007-04-17 08:27 MST ------- tested/verified suceesfully according to https://bugzilla.novell.com/tr_show_case.cgi?case_id=346856 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #49 from froh@novell.com 2007-04-20 01:14 MST ------- cool. THX! to all who contributed :) !! remains one Q: I guess the pciids should be in the KMP and the merge script should be called from the %post trigger, right? This needs to be added to the KMP Manual for code10: https://forgesvn1.novell.com/svn/driver-process/trunk/doc/KernelModulePackag... also, if we ever touch the pciids package or update it with a service pack, will we ensure we have the current pci.ids file? if not we should run the merge script in a %triggerin -- pciutils-ids instead of a %post. After whe have determined what the technical reccomendation is, the PLDP team can work that into the document. Oli, not sure where/how the documentation step should be tracked, what's the normal porcess for that? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ------- Comment #50 from ories@novell.com 2007-04-20 03:58 MST ------- (In reply to comment #49)
cool. THX! to all who contributed :) !!
remains one Q:
I guess the pciids should be in the KMP and the merge script should be called from the %post trigger, right?
This needs to be added to the KMP Manual for code10:
https://forgesvn1.novell.com/svn/driver-process/trunk/doc/KernelModulePackag...
the release notes will contain following on that subject: Updating the pci.ids database The pci.ids database can now be merged with more recent updates. Kernel Module Packages (kmp) which are adding drivers for new components to the system benefit from that feature. In order to accomplish this a kmp has to provide the updated data in a separate file located at /usr/share/pci.ids.d/ following the standard pci.ids file syntax. The %post section of the kmp package specification needs to include following code: %post if [ -x /usr/bin/merge-pciids -a -x /usr/bin/perl ]; then /usr/bin/merge-pciids else echo "ERROR: merge-pciids or perl not found" fi
also, if we ever touch the pciids package or update it with a service pack, will we ensure we have the current pci.ids file? if not we should run the merge script in a %triggerin -- pciutils-ids instead of a %post.
I assume that is not addressed yet for SP1, we need to implement it in the first maintenance update of pciutils, will open a bug for it
After whe have determined what the technical reccomendation is, the PLDP team can work that into the document.
Oli, not sure where/how the documentation step should be tracked, what's the normal porcess for that?
see above, will forward that snippet to Ann -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ories@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- OtherBugsDependingO| |266526 nThis| | -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 ories@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- OtherBugsDependingO|266526 | nThis| | ------- Comment #51 from ories@novell.com 2007-04-20 04:06 MST ------- created https://bugzilla.novell.com/show_bug.cgi?id=266526 to track it for the next maintenance update of pciutils -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=221276 froh@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #114177|merge script from upstream |merge script from upstream, compressed twice, to description| |be sure ;) Attachment #114177|merge.tar.gz |merge.tgz.gz filename| | -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
participants (1)
-
bugzilla_noreply@novell.com