On Thursday 20 April 2017, Michael Matz wrote:
On Thu, 20 Apr 2017, Ruediger Meier wrote:
No, you don't see *the* results. You can see *some* results of different binaries which were built using a different build script (spec file), different BuildRequires and probably on a different build host.
While your arguments are valid and true in the fullest generic sense, in the particular case we're talking about here (and in fact for most such cases) it's moot. The binaries _are_ the same, so the tests are valid.
And while it is also true that it would be even nicer if more packages would support installed testing, that's something for upstream to solve usually. (Sometimes you can get away with running make check with the appropriate setting of various makefile variables). But it would also be nicer if I had a pink zombie pony; alas it is not so.
So, for the time being, and with the realities we're working in, and considering that some packages are in fact in the critical path and hence create a _real_ problem when they build for a long time, the %-testsuite extra .spec files are the sensible work around. The other realistic work-around would be to not run the testsuite at all, and that would be worse.
I agree that splitting off *testsuite is not the most critical thing.
However my concerns are not only about the *testsuite packages but mainly about our general way how to optimize the build cycles. Look at our util-linux package. It's almost impossible to understand these spec files and AFAIK no other distro is building ul in a similar manner, completely different than supposed by upstream and autotools inventors.
To fix that we could add a subpackage providing the whole original build directory and install this later (at QA time) to run the missing check only.
True, that would work as well. It would also create many new extremely large binary packages nobody except automatic testers would use. If it actually solves any problem we in reality have is doubtful.
Regarding reality problems. Here you see an accepted and forwarded submit request where all the duplicated spec files are not in sync, applying different patch-sets: https://build.opensuse.org/request/show/487529