[opensuse-factory] New repo-checker to be deployed on August 14th
packagers, A complete rewrite of the tool that runs as factory-repo-checker on OBS is scheduled for deployment on August 14th as the new user repo-checker. This new tool fixes a number of major shortcomings and bugs with the previous tool that result in errors detected that were previously not detected. As such simple version updates may be rejected until these issues are resolved. Both to provide a grace period and tooling improvement going forward the tool will now regularly post comments on packages with issues in their corresponding devel project. The 112 comments went out Wednesday on a variety of packages. Issues with any ring package will block an entire letter stagings just as a failing builds have previously. Due to the issues with the previous tool a number of problems currently exist with ring packages. In order to roll out the tool I started to fix the problems, but ended up finding too many so a whitelist will be used and relaxed as issues are fixed. See the Factory config [1] repo_checker-package-whitelist{,-x86_64,-i586} lines to see which packages are whitelisted and work to resolve those problems. The complete list of problems can be found in the repo_checker file [2]. If you help fix ring problems it is appreciated if you link to the requests in the tracking issue [3] so the whitelist can be relaxed as they are accepted. Overall, the new tool is significantly simpler in terms of lines of code, code complexity, and workflow complexity. Also faster! See the initial rewrite pull request for more details [4]. Summary of changes/features: - all architectures are checked (currently i586, x86_64) instead of just x86_64 - requests are checked in staging groups instead of devel projects - no more review spam during intermediary stages - properly handle ghc-* requests - cycle check based on entire group which properly catches problems - no longer a requires the latest version from devel - expect reviews not to be accepted until the entire group is built and passes - summary comments on staging projects to notify submitters of problems - new user repo-checker on requests - comments on packages in devel projects based on current Factory:standard The new repo-checker is currently soft-enabled for Factory in that it will be checking all requests, but only commenting when it sees a problem and having no effect on reviews. On August 14th all new requests will require approval from the new repo-checker user on OBS. An example of problems detected during the soft-enablement that the old repo checker passed over can be seen currently on Staging:B [5]. For a summary of planned improvements to the tool see the issues on github [5]. Report issues there using the title prefix 'repo_checker: '. Enjoy! [1] https://build.opensuse.org/package/view_file/openSUSE:Factory:Staging/dashbo... [2] https://build.opensuse.org/package/view_file/openSUSE:Factory:Staging/dashbo... [3] https://github.com/openSUSE/osc-plugin-factory/issues/1012 [4] https://github.com/openSUSE/osc-plugin-factory/pull/964 [5] https://build.opensuse.org/project/show/openSUSE:Factory:Staging:B [6] https://github.com/openSUSE/osc-plugin-factory/issues?&q=is%3Aissue%20is%3Aopen%20repo_checker -- Jimmy -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 08/05/2017 12:34 AM, Jimmy Berry wrote:
Both to provide a grace period and tooling improvement going forward the tool will now regularly post comments on packages with issues in their corresponding devel project. The 112 comments went out Wednesday on a variety of packages.
Thanks. This is good way of getting attention. Sometimes the comments are in the wrong package. For example, I've noticed boost is on there, but the problem is with OpenMPI being broken. I'm not sure if this is by design or in error, but it risks spamming lots of packages just because innermost dependency is uninstallable. It's still better to know earlier rather than later. Currently these problems were only discovered by end-users.
The new repo-checker is currently soft-enabled for Factory in that it will be checking all requests, but only commenting when it sees a problem and having no effect on reviews. On August 14th all new requests will require approval from the new repo-checker user on OBS.
In this case will this block something like boost because OpenMPI is uninstallable? - Adam -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Saturday, August 5, 2017 5:41:09 AM CDT Adam Majer wrote:
On 08/05/2017 12:34 AM, Jimmy Berry wrote:
Both to provide a grace period and tooling improvement going forward the tool will now regularly post comments on packages with issues in their corresponding devel project. The 112 comments went out Wednesday on a variety of packages. Thanks. This is good way of getting attention.
Sometimes the comments are in the wrong package. For example, I've noticed boost is on there, but the problem is with OpenMPI being broken. I'm not sure if this is by design or in error, but it risks spamming lots of packages just because innermost dependency is uninstallable.
I did consider this, but decided to go with the simplest approach first. Note that the relevant error is also posted on the openmpi package. It does not seem completely wrong since the error in fact effects a boost sub-package as well. It seems like maintainers might want to know that their package's dependencies are broken as that effects the usage of their package. It would be possible to build a tree and detect the root of the problem and only post there, but that would require parsing the details of these errors. Currently, the install check and file conflict tools do not provide structured output so parsing must be done in addition to mapping the binaries back to the packages from which they originate. Long story short, the new tool should prevent the majority of such issues from ever making it into Factory which should make these devel project comments rare. If it turns out to be otherwise or worth the extra effort it can be done. My hope is that maintainers simply fix these issues prior to wanting to update their packages since they do effect users. In the future such problems should only make it into Factory due to updates in separate stagings being accepted in one round and introducing a problem or due to more complex packaging relationships which are not seen during the staging process.
It's still better to know earlier rather than later. Currently these problems were only discovered by end-users.
Indeed. The semi-regular emails to Factory regarding such issues will hopefully decrease.
The new repo-checker is currently soft-enabled for Factory in that it will be checking all requests, but only commenting when it sees a problem and having no effect on reviews. On August 14th all new requests will require approval from the new repo-checker user on OBS.
In this case will this block something like boost because OpenMPI is uninstallable?
Yes. Obviously, the release team may choose to manually override, but the tool will not accept the review. Short of the recent introduction of parsing the output it was impossible for the tool to even know the package to which the errors related much less the source of a tree of problems (see above). Since these are in fact real problems I would like to see them simply resolved rather than avoiding them. Most of these problems are relatively simple to fix, but are not always noticed. The idea behind this being that if build time requirements were not capable of being met (even if another package's fault) those would block an update from entering Factory. Enforcing install time requirements has been the goal of the previous tool and this is simply bringing it closer to reality.
- Adam
For some examples see the issue referenced in the original mail for a list of SRs I created to fix a number of file conflicts. For details on the parsing and mapping of binary to package see the PR that added the devel package comments [1]. [1] https://github.com/openSUSE/osc-plugin-factory/pull/1042 -- Jimmy -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
participants (2)
-
Adam Majer -
Jimmy Berry