On Dienstag, 2. April 2019 17:12:39 CEST Jan Engelhardt wrote:
On Tuesday 2019-04-02 16:02, Vinzenz Vietzke wrote:
Am Dienstag, 2. April 2019, 15:34:09 CEST schrieb Stefan Seyfried:
The April's fools joke is obviously talking about "PTS being sufficient and creating comparable results".
I know PTS but what I meant to ask Ludwig what he (and the Leap release team) would prefer for test results.
Phoronix generally seems to focus on time/work. (Time-to-completion for a fixed task; or number of operations for a fixed time quantum)
So if you can make whatever software you are analyzing trade CPU for memory (use less CPU, use more memory), you will score "higher" in most tallies. At least that is my impression of PTS results as published on Phoronix.
The problem with Phoronix is not the PTS, but the "evaluation" done in the articles. The PTS outputs raw metrics. These metrics include both "speed" and "efficiency" (e.g. frames/second/watt, i.e frames/joule) indicators. Of course the latter depends on ways to e.g. measure power draw. For desktops, this requires external power loggers with realtime data via USB/serial, while for Laptops the internal measurement suffices. So you can get both metrics, time to completion and energy per workload. *Both* are of interest here. If you use the same hardware, the same distribution at a given version and just exchange the kernel itself, you are quite safe to argue any statistical relevant changes are due to the change of kernel. When you manage to pick the same kernel version with just different PREEMPT configs, you can attribute differences to PREEMPT. The last point is where Phoronix regularly fails, it changes to many variables, so you can assess there *is* a difference, but it can not identify a reason for it. The PTS is just a tool. Use it correctly, and you get valid results. Kind regards, Stefan-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org