[opensuse-packaging] Extensions for specification of application binary/programming interfaces?
XY=$(zypper --terse search --match-words --provides "$1") && echo "$XY" | tail --lines +$(($(echo "$XY" | grep --line-number --regexp=--+ | cut --delimiter : --fields 1) + 1)) | cut --delimiter \| --fields 2 } elfring@Sonne:~> show_names abi abi-compliance-checker gimp llvm-clang nodejs
Hello, I have tried the following queries out on package information which is available on my openSUSE Tumbleweed system at the moment. elfring@Sonne:~> show_names() { php5 python-base python3-base ruby2.1 ruby2.2 vdr elfring@Sonne:~> show_names api | wc -l 87 Does such a display indicate further possibilities for the specification of details around the usage of application binary interfaces and application programming interfaces? How do you think to improve any product descriptions which are provided (by RPM files for example) in different degree? Would it make sense to clarify a kind of certification mark duty for more software components? Regards, Markus -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Wednesday 2016-01-20 18:23, SF Markus Elfring wrote:
elfring@Sonne:~> show_names() {
XY=$(zypper --terse search --match-words --provides "$1") && echo "$XY" | tail --lines +$(($(echo "$XY" | grep --line-number --regexp=--+ | cut --delimiter : --fields 1) + 1)) | cut --delimiter \| --fields 2 }
elfring@Sonne:~> show_names api | wc -l 87
Does such a display indicate further possibilities for the specification of details around the usage of application binary interfaces and application programming interfaces?
It does not. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
Does such a display indicate further possibilities for the specification of details around the usage of application binary interfaces and application programming interfaces?
It does not.
Would you like to explain your view a bit more? Regards, Markus -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Wednesday 2016-01-20 18:58, SF Markus Elfring wrote:
XY=$(zypper --terse search --match-words --provides "$1") && echo "$XY" | tail --lines +$(($(echo "$XY" | grep --line-number --regexp=--+ | cut --delimiter : --fields 1) + 1)) | cut --delimiter \| --fields 2 }
Does such a display indicate further possibilities for the specification of details around the usage of application binary interfaces and application programming interfaces?
It does not.
Would you like to explain your view a bit more?
You took a motor engine part list, noticed the words "circuit board" appears in it, and concluded that the car's sales flyer needs to enumerate the pin assignments. You got to be kidding. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
You took a motor engine part list, noticed the words "circuit board" appears in it, and concluded that the car's sales flyer needs to enumerate the pin assignments.
There are also some ordinary certification mark duties involved, aren't there?
You got to be kidding.
Will it become useful to specify the properties "application binary interfaces" and "application programming interfaces" for more software components in a consistent way? How do you think to improve the software packaging situation so that dependency resolution will be a bit safer? Regards, Markus -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Wednesday 2016-01-20 19:35, SF Markus Elfring wrote:
How do you think to improve the software packaging situation so that dependency resolution will be a bit safer?
With the use of checksum-based identifiers, like it is already done with the "srcmd5" attribute in kernel modules, or the debug-id in ELF files. That however has nothing to do with grepping a list of all RPM provs for the substring /api/. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
How do you think to improve the software packaging situation so that dependency resolution will be a bit safer?
With the use of checksum-based identifiers, like it is already done with the "srcmd5" attribute in kernel modules, or the debug-id in ELF files.
This is another interesting implementation detail.
That however has nothing to do with grepping a list of all RPM provs for the substring /api/.
Can certification mark duties evolve also around various software libraries and their users? Would anybody (besides me) like to reduce the difficulties for safe dependency resolution a bit more by making the distinction between the properties "application binary interface" and "application programming interfaces consistent in specifications for some (or only popular) software packages? Regards, Markus -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Wednesday 2016-01-20 20:05, SF Markus Elfring wrote:
That however has nothing to do with grepping a list of all RPM provs for the substring /api/.
Can certification mark duties evolve also around various software libraries and their users? Would anybody (besides me) like to reduce the difficulties for safe dependency resolution a bit more by making the distinction between the properties "application binary interface" and "application programming interfaces consistent in specifications for some (or only popular) software packages?
I suggest you cut the salesbabble, and come up with a *concrete* case where the existing RPM mechanisms are insufficient, instead of making up hypothetical blurry scenarios involving some unspecified packages and interfaces. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
I suggest you cut the salesbabble, and come up with a *concrete* case where the existing RPM mechanisms are insufficient, instead of making up hypothetical blurry scenarios involving some unspecified packages and interfaces.
How do you think about to make the combination of the properties "application binary interface" and "application programming interface" safer for software libraries like the following? * libmodman1 / libproxy1 https://bugzilla.opensuse.org/show_bug.cgi?id=937157#c5 * yast2-core / libyui6 https://bugzilla.suse.com/show_bug.cgi?id=937665#c2 * Difficulties with activation of another current Nvidia graphic driver https://forums.opensuse.org/showthread.php/512847-Difficulties-with-activati... Can specific certification marks help to make the reuse of some software packages a bit safer? Would you like to improve dependency resolution a bit more? Regards, Markus -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Friday 2016-01-22 20:46, SF Markus Elfring wrote:
I suggest you cut the salesbabble, and come up with a *concrete* case where the existing RPM mechanisms are insufficient, instead of making up hypothetical blurry scenarios involving some unspecified packages and interfaces.
How do you think about to make the combination of the properties "application binary interface" and "application programming interface" safer for software libraries like the following?
* libmodman1 / libproxy1 https://bugzilla.opensuse.org/show_bug.cgi?id=937157#c5 * yast2-core / libyui6 https://bugzilla.suse.com/show_bug.cgi?id=937665#c2
They both are a duplicate of https://bugzilla.opensuse.org/show_bug.cgi?id=903974 where I already bemoaned the lack of symbol versions.
Can specific certification marks help to make the reuse of some software packages a bit safer?
Definitely.
Would you like to improve dependency resolution a bit more?
The dependency *resolution* is fine. The problem is with lazy upstream developers who do not update these "marks" *at all* when they made a change. They must do one of the following: - change the SONAME (the mark changes from "libmodman.so.1" -> "libmodman.so.2") - add symbol versions (the mark changes from "libmodman.so.1(V_1)" to "libmodman.so.1(V_2)")
* Difficulties with activation of another current Nvidia graphic driver https://forums.opensuse.org/showthread.php/512847-Difficulties-with-activati...
TL;DR. The ".run" installer from nvidia is a homebrew solution and tramples on files managed by RPM - which subsequently get replaced by RPM at some point again. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
They both are a duplicate of https://bugzilla.opensuse.org/show_bug.cgi?id=903974[https://bugzilla.opensuse.org/show_bug.cgi?id=903974] where I already bemoaned the lack of symbol versions.
Would you like to improve such situations anyhow? Have you got any further ideas to fix similar issues?
Can specific certification marks help to make the reuse of some software packages a bit safer?
Definitely.
Thanks for your acknowledgement.
The dependency *resolution* is fine.
Interesting view …
The problem is with lazy upstream developers who do not update these "marks" *at all* when they made a change.
Do they need any additional and enhanced tools which will make the desired tag (or SONAME) maintenance more convenient and safe?
* Difficulties with activation of another current Nvidia graphic driver https://forums.opensuse.org/showthread.php/512847-Difficulties-with-activati...
The ".run" installer from nvidia is a homebrew solution and tramples on files managed by RPM - which subsequently get replaced by RPM at some point again.
How do you think about the general idea to express the properties "application binary interface" and "application programming interfaces" more often? Would any consistent usage of RPM capabilities help in such use cases? https://docs.fedoraproject.org/ro/Fedora_Draft_Documentation/0.1/html/RPM_Gu... Can they be automatically determined? Which specification style do you prefer at corresponding places? Examples: * gimp(abi) = 4 gimp(api) = 2.0 * php(api) = 20131106 php(zend-abi) = 20131226 php-api = 20131106 php-zend-abi = 20131226 * python(abi) = 2.7 Regards, Markus -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Saturday 2016-01-23 11:20, SF Markus Elfring wrote:
The problem is with lazy upstream developers who do not update these "marks" *at all* when they made a change.
Do they need any additional and enhanced tools which will make the desired tag (or SONAME) maintenance more convenient and safe?
We have tools. The "libabigail" package has utilities to find incompatibilities. They are especially useful for languages with complex symbol names, for example as they occur with C++. The problem is that developers just don't do anything about it even if they knew about the existence of these tools.
Would any consistent usage of RPM capabilities help in such use cases? https://docs.fedoraproject.org/ro/Fedora_Draft_Documentation/0.1/html/RPM_Gu...
No. Those are just freeform strings that serve no purposes with regard to ABI compatibility. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
We have tools. The "libabigail" package has utilities to find incompatibilities. They are especially useful for languages with complex symbol names, for example as they occur with C++.
Interesting …
The problem is that developers just don't do anything about it even if they knew about the existence of these tools.
Are there any more possibilites to reduce unwanted consequences from such situations?
https://docs.fedoraproject.org/ro/Fedora_Draft_Documentation/0.1/html/RPM_Gu...
Those are just freeform strings that serve no purposes with regard to ABI compatibility.
Should RPM capabilities help to manage specific software requirements? How should the available and really usable versions be determined for application programming interfaces which will fit to a selected application binary interface of a needed component? Regards, Markus -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Saturday 2016-01-23 12:55, SF Markus Elfring wrote:
The problem is that developers just don't do anything about it even if they knew about the existence of these tools.
Are there any more possibilites to reduce unwanted consequences from such situations?
Automatic generation of tags based upon the ABI -- basically the srcmd5 fingerprint approach I suggested.
https://docs.fedoraproject.org/ro/Fedora_Draft_Documentation/0.1/html/RPM_Gu...
Those are just freeform strings that serve no purposes with regard to ABI compatibility.
Should RPM capabilities help to manage specific software requirements?
They already do on a daily basis.
How should the available and really usable versions be determined for application programming interfaces which will fit to a selected application binary interface of a needed component?
That is a different category, which does not visibly happen in practice, and which is unrelated to the three bugreports mentioned earlier. -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
Are there any more possibilites to reduce unwanted consequences from such situations?
Automatic generation of tags based upon the ABI -- basically the srcmd5 fingerprint approach I suggested.
Can this information be queried as a RPM capability?
Should RPM capabilities help to manage specific software requirements?
They already do on a daily basis.
How do you think about to describe data around the requirements of ABIs and APIs for various software libraries and their users a bit more explicitly (instead of encoding such information often in a main version number)?
How should the available and really usable versions be determined for application programming interfaces which will fit to a selected application binary interface of a needed component?
That is a different category, which does not visibly happen in practice, and which is unrelated to the three bugreports mentioned earlier.
How is the software packaging situation different here? How are the chances to improve the corresponding data processing? Regards, Markus -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Saturday 2016-01-23 16:06, SF Markus Elfring wrote:
Are there any more possibilites to reduce unwanted consequences from such situations?
Automatic generation of tags based upon the ABI -- basically the srcmd5 fingerprint approach I suggested.
Can this information be queried as a RPM capability?
That makes no sense. You need to work on your grammar.
Should RPM capabilities help to manage specific software requirements?
They already do on a daily basis.
How do you think about to describe data around the requirements of ABIs and APIs for various software libraries and their users a bit more explicitly (instead of encoding such information often in a main version number)?
This is already being practised in case you have not noticed.
How should the available and really usable versions be determined for application programming interfaces which will fit to a selected application binary interface of a needed component?
That is a different category, which does not visibly happen in practice, and which is unrelated to the three bugreports mentioned earlier.
How is the software packaging situation different here?
It is not.
How are the chances to improve the corresponding data processing?
What? -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
Automatic generation of tags based upon the ABI -- basically the srcmd5 fingerprint approach I suggested.
Can this information be queried as a RPM capability?
That makes no sense.
Why?
You need to work on your grammar.
Would you like to explain this feedback a bit more?
How do you think about to describe data around the requirements of ABIs and APIs for various software libraries and their users a bit more explicitly (instead of encoding such information often in a main version number)?
This is already being practised in case you have not noticed.
To which practice do you refer here? Have you got another aspect for further clarification in mind eventually?
How should the available and really usable versions be determined for application programming interfaces which will fit to a selected application binary interface of a needed component?
That is a different category, which does not visibly happen in practice, and which is unrelated to the three bugreports mentioned earlier.
How is the software packaging situation different here?
It is not.
Which difference did you see in the category then?
How are the chances to improve the corresponding data processing?
What?
How will the evolution of "software fingerprint" measurements affect the desired dependency resolution? Regards, Markus -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
How do you think about to describe data around the requirements of ABIs and APIs for various software libraries and their users a bit more explicitly (instead of encoding such information often in a main version number)?
This is already being practised in case you have not noticed.
Do you know any more software developments (besides the packages "libabigail-tools" and "abi-compliance-checker") which can improve the handling for the combination of the properties "application binary interface" and "application programming interface" to some degree? Will any advanced queries on the package database help to clarify special cases for some software dependencies? Are you also looking for further ideas to achieve safer and more convenient solutions? Regards, Markus -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
On Wednesday 2016-01-27 15:28, SF Markus Elfring wrote:
How do you think about to describe data around the requirements of ABIs and APIs for various software libraries and their users a bit more explicitly (instead of encoding such information often in a main version number)?
This is already being practised in case you have not noticed.
Do you know any more software developments (besides the packages "libabigail-tools" and "abi-compliance-checker") which can improve the handling for the combination of the properties "application binary interface" and "application programming interface" to some degree?
http://upstream.rosalinux.ru/ (note the notice that the service at that URL is discontinued, with the software having been released on github) -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
Jan Engelhardt wrote:
On Saturday 2016-01-23 11:20, SF Markus Elfring wrote:
The problem is with lazy upstream developers who do not update these "marks" *at all* when they made a change.
Do they need any additional and enhanced tools which will make the desired tag (or SONAME) maintenance more convenient and safe?
We have tools. The "libabigail" package has utilities to find incompatibilities. They are especially useful for languages with complex symbol names, for example as they occur with C++.
The problem is that developers just don't do anything about it even if they knew about the existence of these tools.
I wrote a review bot that compares abi changes in submit/maintenance requests and complains if there are incompatibilities: https://github.com/openSUSE/osc-plugin-factory/tree/master/abichecker It uses the abi-compliance-checker tool which maybe was not necessarily the best choice, let's say it that way :-) It's not as reliable as I had wished for so I so far hesiated to activate it for Factory as is. Maybe it's worth investigating the use of libabigail for it at some point. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
Le mardi 26 janvier 2016 à 12:03 +0100, Ludwig Nussel a écrit :
Jan Engelhardt wrote:
On Saturday 2016-01-23 11:20, SF Markus Elfring wrote:
The problem is with lazy upstream developers who do not update these "marks" *at all* when they made a change.
Do they need any additional and enhanced tools which will make the desired tag (or SONAME) maintenance more convenient and safe?
We have tools. The "libabigail" package has utilities to find incompatibilities. They are especially useful for languages with complex symbol names, for example as they occur with C++.
The problem is that developers just don't do anything about it even if they knew about the existence of these tools.
I wrote a review bot that compares abi changes in submit/maintenance requests and complains if there are incompatibilities: https://github.com/openSUSE/osc-plugin-factory/tree/master/abichecker
It uses the abi-compliance-checker tool which maybe was not necessarily the best choice, let's say it that way :-) It's not as reliable as I had wished for so I so far hesiated to activate it for Factory as is. Maybe it's worth investigating the use of libabigail for it at some point.
And as a side comment regarding libabigail (I already told Ludwig, but let's share the word), if you find missing features in it, contact the upstream developer, he is very interested in integrating new features ! -- Frederic Crozat Enterprise Desktop Release Manager SUSE -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
I wrote a review bot that compares abi changes in submit/maintenance requests and complains if there are incompatibilities: https://github.com/openSUSE/osc-plugin-factory/tree/master/abichecker
Does this approach depend on data from "DWARF debug information"?
It uses the abi-compliance-checker tool which maybe was not necessarily the best choice, let's say it that way :-) It's not as reliable as I had wished for so I so far hesiated to activate it for Factory as is.
Would you like to share any more experiences?
Maybe it's worth investigating the use of libabigail for it at some point.
It seems that both approaches deal with XML files. How similar are the involved data structures (or XML data type definitions/schemas)? Regards, Markus -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
SF Markus Elfring wrote:
I wrote a review bot that compares abi changes in submit/maintenance requests and complains if there are incompatibilities: https://github.com/openSUSE/osc-plugin-factory/tree/master/abichecker
Does this approach depend on data from "DWARF debug information"?
Yes.
It uses the abi-compliance-checker tool which maybe was not necessarily the best choice, let's say it that way :-) It's not as reliable as I had wished for so I so far hesiated to activate it for Factory as is.
Would you like to share any more experiences?
What do you want to know? :-) From the top of my head I remember that we had problems with the results in one product as the debug information created by gcc was sometimes not quite correct so we got false positives. The implementation of abi-compliance-checker is fragile as it relies on parsing output from readelf etc. Also, it loads the full data into RAM so huge packages like libreoffice can make it run OOM. A fundamental problem with only looking at the debug information is that one can't know whether a symbol or structure is part of a public or private API. So some libraries produce false positives. The abi-compliance-checker author also offers a method to evaluate the public headers in order to filter out private symbols. That requires recompiling the software though which is not so straight forward to do in a review bot. The reports abi-compliance-checker creates are quite nice though.
Maybe it's worth investigating the use of libabigail for it at some point.
It seems that both approaches deal with XML files. How similar are the involved data structures (or XML data type definitions/schemas)?
I didn't look into libabigail so I can't tell. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
From the top of my head I remember that we had problems with the results in one product as the debug information created by gcc was sometimes not quite correct so we got false positives.
Would you like to reduce or even avoid the dependency on DWARF debug information then?
The implementation of abi-compliance-checker is fragile as it relies on parsing output from readelf etc. Also, it loads the full data into RAM so huge packages like libreoffice can make it run OOM.
Are there any more software development challenges to consider?
A fundamental problem with only looking at the debug information is that one can't know whether a symbol or structure is part of a public or private API. So some libraries produce false positives. The abi-compliance-checker author also offers a method to evaluate the public headers in order to filter out private symbols. That requires recompiling the software though which is not so straight forward to do in a review bot.
Does such information point any details out which are interesting for further consideration?
The reports abi-compliance-checker creates are quite nice though.
Are there any plans to integrate such tools into the Open Build Service in a more convenient way?
How similar are the involved data structures (or XML data type definitions/schemas)?
I didn't look into libabigail so I can't tell.
How do you think about to improve the desired clarification of software dependencies by a discussion of a topic like fine-tuning for data structure designs? Regards, Markus -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org
participants (4)
-
Frederic Crozat -
Jan Engelhardt -
Ludwig Nussel -
SF Markus Elfring