Knurpht-openSUSE schrieb:

Op vrijdag 25 januari 2019 08:50:03 CET schreef Axel Braun:

...

Morning all,

we had recently a mail chain about perception of openSUSE. Today I stumbled over https://www.phoronix.com/scan.php?page=article&item=jan2019-win-server&num=1 in which the performance of TW does not really look good, compared to other distros.

Personally I dont share this view, but I'm only using it on a laptop, and quite happy with it. Can one comment on the server side?

I run TW on a VPS, 2 Xeon cores, 4 GB RAM and have absolutely no complaints re. performance. But .... through some forums posts where people complained about TW booting slower I found that openSUSE starts quite some services others don't start until the admin tells them to start. We have the btrfs-maintenance service (others don't use btrfs), on some postfix is not used and so on. One user managed to get his TW booting ~as fast as Manjaro by disabling some services and removing stuff from initrd.

Well, that is something that can be worked on. The btrfs-maintenance service already isn't meant to get enabled by default anymore. postfix likely is quite superflous too. Cron probably used to pull it in but a default install should actually only use systemd timers nowadays. Mostly a matter of someone taking the topic to clean up a bit. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Am Freitag, 25. Januar 2019, 10:10:43 CET schrieb Michal Kubecek:

For instance, just few days ago he published an "out-of-the-box 10GbE network benchmark" which is extremely crappy even for Phoronix standards. He completely ignored that some distributions enable connection tracking and set a complex set of netfilter rules while others do not setup any kind of firewall and compares network latency numbers between them. In one "bandwidth test" he reported that the fastest result among all tested distribution was "847 Mbit/s" (through 10Gb/s ethernet) but didn't stop for a minute to say "Hey, something is obviously wrong here." and didn't do so even after two other programs reported numbers more than 10 times higher.

To second that somehow: I don't have the source at my hands anymore but recently came across a blog post where someone ran Phoronix' "test suite" benchmarks multiple times with the same settings and on the same system. He compared the results and got *vastly* different figures. I mean, there were deviations up to 25%! That is something to keep in mind even for benchmarks from other sources if they use that test suite... Regards, vinz. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Citeren Knurpht-openSUSE :

Op vrijdag 25 januari 2019 11:30:11 CET schreef Vinzenz Vietzke:

...

Am Freitag, 25. Januar 2019, 10:10:43 CET schrieb Michal Kubecek:

...

For instance, just few days ago he published an "out-of-the-box 10GbE network benchmark" which is extremely crappy even for Phoronix standards. He completely ignored that some distributions enable connection tracking and set a complex set of netfilter rules while others do not setup any kind of firewall and compares network latency numbers between them. In one "bandwidth test" he reported that the fastest result among all tested distribution was "847 Mbit/s" (through 10Gb/s ethernet) but didn't stop for a minute to say "Hey, something is obviously wrong here." and didn't do so even after two other programs reported numbers more than 10 times higher.

To second that somehow:

I don't have the source at my hands anymore but recently came across a blog post where someone ran Phoronix' "test suite" benchmarks multiple times with the same settings and on the same system. He compared the results and got *vastly* different figures. I mean, there were deviations up to 25%!

That is something to keep in mind even for benchmarks from other sources if they use that test suite...

Regards, vinz. So, let's conclude that Phoronix is a click-bait site. A hardware geek in my town had experiences likewise. Question arises: who's paying the guy?

I'll bite: from https://www.phoronix.com/scan.php?page=phoronix_premium "At Phoronix our primary revenue stream is through the advertisements that you see on our web-site. We depend upon this revenue in order to maintain our operations and continue serving the Linux and free software communities. When readers use such services as AdBlock, our revenue suffers. With the growing usage of these services and the request of readers who would like to better support our operations, we have established a Phoronix Premium subscription service. Phoronix was started in 2004 by Michael Larabel who continues to serve as the principal author for the hundreds of articles published on Phoronix each month. A common misconception is that Phoronix has a whole team of writers and significant resources like other publications, but that's not the case due to the greater percentage of those blocking ads, not many companies advertising specifically for the enthusiast/gaming Linux niche, etc. Thus most of the work falls onto the shoulders of one man who frequently invests 100+ hour weeks working on Phoronix." So basically, as long as people keep visiting this site (or even become paid subscribers), he will stay in business. The sheer volume of articles speaks volumes about how thorough the research can be. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 25/01/2019 10.10, Michal Kubecek wrote:

On Friday, 25 January 2019 8:50 Axel Braun wrote:

...

we had recently a mail chain about perception of openSUSE. Today I stumbled over

...

Honestly, the most wrong thing about Phoronix is that there are people who take its "benchmarks" and their results seriously. Please don't be one of them.

But that's it, that people read those benchmarks and don't listen to their critics. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)

On 25/01/2019 13.12, Vinzenz Vietzke wrote:

Am Freitag, 25. Januar 2019, 12:09:50 CET schrieb Carlos E. R.:

...

...

Honestly, the most wrong thing about Phoronix is that there are people who take its "benchmarks" and their results seriously. Please don't be one of them.

But that's it, that people read those benchmarks and don't listen to their critics.

Which leads to two possible ways of handling it: 1) Try to jump on every discussion where Leap, Tumbleweed or whatever openSUSE "product" is benchmarked wrong. 2) Try to tell everyone what's cool, handy and impressive about the openSUSE stuff.

Both ways are legit imho. But I know which one I'd choose - for the sake of my mind and mental sanity. ;)

There is another way, but you will not like it. It is what the car industry in Europe does. There is an standardized test that measures mileage (NEDC, now WLTP). Well, the cars are optimized for that test. The "real" mileage might be different, usually worse, but the figure that is printed and which buyers use, is the one from the standardized test. We could have the installed system "next next next" faster, for instance. I do not know if this is good, probably not. I'm not sure if this phoronix test is what critics use to say "openSUSE is slow". If it is, it does not matter if the test is rubbish. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)

Op vrijdag 25 januari 2019 14:01:42 CET schreef Liam Proven:

On 1/25/19 1:27 PM, Carlos E.R. wrote:

...

There is another way, but you will not like it.

It is what the car industry in Europe does. There is an standardized test that measures mileage (NEDC, now WLTP). Well, the cars are optimized for that test. The "real" mileage might be different, usually worse, but the figure that is printed and which buyers use, is the one from the standardized test.

We could have the installed system "next next next" faster, for instance.

I do not know if this is good, probably not.

I'm not sure if this phoronix test is what critics use to say "openSUSE is slow". If it is, it does not matter if the test is rubbish.

This.

It would drive further difference between SLE* and openSUSE. Whether that is a good or a bad thing is not my call.

I do not like to see this Phoronix-bashing. It is a useful and valuable site, as the only one in the *large* field of PC performance comparison sites and communities which is Linux-based. The Linux world _needs_ this. If we Linux users wish to see Linux continue to make headway against Windows, then sites like this are _necessary_.

If someone spends their own money on computers, then they want to know which are fast and which are slow, which are better or worse value for money. There *are* differences.

(For instance, I personally avoid all "Celeron", "Pentium Dual Core" and other cut-down CPUs. I'd rather have an older, faster model. Others opinions differ; for instance, if someone routinely overclocks their hardware.)

As well as differences between makes and models, there are also differences between performance under Windows and under Linux -- obviously.

Phoronix is the _only_ site doing such comparisons.

That is a valuable service.

If, perhaps, aspects of the design of openSUSE make it seem slower in benchmarks, either on first boot or in general, well, that is going to make it seem less appealing to people who want to get more performance out of their hardware by running Linux instead of Windows.

That being the case, there's only one answer: eliminate those performance differentials. Make it faster.

The answer is *not* to deride the people doing the measurements, because people do care about those measurements. That's life. Deal with it. Liam, Critisizing Phoronix != bashing. But let's be fair, his benchmarks lack info. Which services are started at boot, what is in each distro's initrd, which filesystems are being used etc. etc. And this has been the same for ages. I even tried to talk about this with Laravel, no go. It's not for openSUSE's sake that I want this a proper comparison, it just needs to be proper. Last but not least: without 'results' the site would not exist. So ...... And, I remember his 'mir' posts...... And mir ...... -- Gertjan Lettink a.k.a. Knurpht openSUSE Board Member openSUSE Forums Team

-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Op vrijdag 25 januari 2019 14:26:46 CET schreef Liam Proven:

On 1/25/19 2:14 PM, Knurpht-openSUSE wrote:

...

Critisizing Phoronix != bashing.

Isn't it?

You don't think that "Phoronix is a click-bait site" is bashing? Those are your words. They look like bashing more than criti

...

But let's be fair, his benchmarks lack info. Which services are started at boot, what is in each distro's initrd, which filesystems are being used etc. etc.

As far as I can see, isn't the thing that he compares default installs without specific optimizations?

That seems pretty fair to me. It is how I would test.

I used to do this sort of performance-testing for a living. I ran PC Pro Magazine's in-house testing labs for some years. I did the 32-bit port of PC Pro's Windows benchmark suite. I have a little knowledge in this area.

If I was comparing the Windows installs of 2 vendors, for instance, then the _point_ of the exercise was to compare them as shipped, _not_ to optimize them first and _then_ compare them. What we were trying to find out for our readers was how quick they were out of the box.

It would be _less fair_ if I optimized them first. For instance, what if I only knew of optimizations for one particular hardware config and so did not apply them to a different vendor's equipment?

Then some machines would be optimized better than others. That is _increasing_ the unfairness.

If special steps are needed on openSUSE that are not needed on rival distros, then it is _fair_ *not* to include those steps.

No? No. Which does not mean we cannot improve things.

-- Gertjan Lettink a.k.a. Knurpht openSUSE Board Member openSUSE Forums Team -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 1/25/19 2:42 PM, Michal Kubecek wrote:

No. It's not about "optimizations". It's about choices like which services we want to start by default. Which modules to load on boot. If we want to setup firewall by default. Maybe also how long timeout we want for the boot menu. (I sincerely hope he does not count _that_ into the "boot time" but I wouldn't be really surprised.)

As an extreme, we could drop everything except what is necessary to get a login prompt by default and we would get super fast "boot time in default install". On the other hand, our users would get quite angry that they have to enable all kind of services (including those they had no idea they need) to make their system usable. Is it what you want?

So what you are saying is, you would not want to see the default configuration, as found in a clean install, to be benchmarked. You want specific changes to that configuration for better performance. Is that right? Am I reading you correctly? If so, *that is optimizing the system*. "Optimizing", in the context of benchmarking software performance, means "making specific changes to improve performance". Turning stuff off to make it faster than the default config _is optimizing it_.

That's just sad.

Then I am afraid that you are not going to like the entire world of performance testing.

Even more sad that you still believe what you said above after having some experience.

How else would you do it? Seriously. I have worked for some 15 different computer magazines and websites in Britain, Germany, America and other countries. I used to be the editor of Heise UK, for example. This is how Heise works. This is how benchmarking works. What would you change? -- Liam Proven - Technical Writer, SUSE Linux s.r.o. Corso II, Křižíkova 148/34, 186-00 Praha 8 - Karlín, Czechia Email: lproven@suse.com - Office telephone: +420 284 241 084 -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 25/01/2019 14.54, Patrick Shanahan wrote:

* Liam Proven [01-25-19 08:49]:

...

On 1/25/19 2:42 PM, Michal Kubecek wrote:

...

No. It's not about "optimizations". It's about choices like which services we want to start by default. Which modules to load on boot. If we want to setup firewall by default. Maybe also how long timeout we want for the boot menu. (I sincerely hope he does not count _that_ into the "boot time" but I wouldn't be really surprised.)

As an extreme, we could drop everything except what is necessary to get a login prompt by default and we would get super fast "boot time in default install". On the other hand, our users would get quite angry that they have to enable all kind of services (including those they had no idea they need) to make their system usable. Is it what you want?

So what you are saying is, you would not want to see the default configuration, as found in a clean install, to be benchmarked. You want specific changes to that configuration for better performance.

Is that right? Am I reading you correctly?

If so, *that is optimizing the system*.

"Optimizing", in the context of benchmarking software performance, means "making specific changes to improve performance".

Turning stuff off to make it faster than the default config _is optimizing it_.

...

That's just sad.

Then I am afraid that you are not going to like the entire world of performance testing.

...

Even more sad that you still believe what you said above after having some experience.

How else would you do it? Seriously. I have worked for some 15 different computer magazines and websites in Britain, Germany, America and other countries. I used to be the editor of Heise UK, for example. This is how Heise works.

This is how benchmarking works. What would you change?

probably move the discussion of opensuse-offtopic where it fits.

No, it fits on one of the main lists. It is not chit chat, it affects our future. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)

On 25/01/2019 15.06, Michal Kubecek wrote:

On Friday, 25 January 2019 14:47 Liam Proven wrote:

...

So what you are saying is, you would not want to see the default configuration, as found in a clean install, to be benchmarked. You want specific changes to that configuration for better performance.

Is that right? Am I reading you correctly?

If so, *that is optimizing the system*.

It's not. Let's use a car analogy (as usual). Say we want to compare two car models from different manufacturers. We get car A from one vendor and car B from another. Let's say one line in the table is supposed to be weight of the car. Car A comes with extra set of winter wheels, roof bike holder and full tank as a bonus while car B comes without any of that and even without a reserve wheel and a wrench.

Mr. Larabel - and apparently also you - would just weight both cars as they came and call that "fair". I, on the other hand, would try to make the conditions - as equal as possible by taking the bonus equipment out before weighting both cars (or leaving the same amount of equipment in both).

What you advocate for would be equivalent to punishing the vendor who provides their customer a rooftop box by leaving it on when measuring maximum speed and fuel consumption. Is it "fair"? Not in my dictionary.

Well, in car NEDC testing, the car is tested as is (meaning as the car model is officially described, out of the factory). What to remove is specified in the test docs. Otherwise, no optimizations.

...

How else would you do it? Seriously. I have worked for some 15 different computer magazines and websites in Britain, Germany, America and other countries. I used to be the editor of Heise UK, for example. This is how Heise works.

Even more sad.

You may not agree, but that is how it is done. Just a fact. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)

On Friday, January 25, 2019 8:17:03 AM CST Carlos E. R. wrote:

Well, in car NEDC testing, the car is tested as is (meaning as the car model is officially described, out of the factory). What to remove is specified in the test docs. Otherwise, no optimizations.

This is the point people seem to miss with "out-of-box" and why people care. There are plenty of users that are not interested in tuning every part of the system to get the best performance. They just want to install it and use it. As such out-of-box is very relevant. In fact if you are comparing distros that's a major part the default configuration. Clearly if I try hard enough I can make distros very similar to each-other as they are running the same upstream software and as such I would get more similar results. At that point you are benchmarking the upstream software and not the distro. Crappy defaults are a serious problem, no one is expert in everything nor wants to be. -- Jimmy -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Friday, 25 January 2019 18:02 Liam Proven wrote:

Yes. In fact, to be more specific, if someone is interested in comparing different Linux distros, then that means that they are not committed to one distro. If they were, they wouldn't be interested in such test results.

And yet Phoronix has been running for 14 years, so clearly they *are*.

People who are even more clueless than the author are interested. Let's try to educate them, explain why the Phoronix style "benchmarks" have no value. That will do way more good than making our distribution less usable just to rank better in pointless Phoronix tables. Michal Kubecek -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Fri, Jan 25, 2019 at 02:43:51PM -0600, Jimmy Berry wrote:

On Friday, January 25, 2019 11:26:41 AM CST Michal Kubecek wrote:

...

People who are even more clueless than the author are interested. Let's try to educate them, explain why the Phoronix style "benchmarks" have no value. That will do way more good than making our distribution less usable just to rank better in pointless Phoronix tables.

If the benchmarks are useless than so are distros.

I'm sorry but this implication is too strong to just set it as a postulate. Please explain the logic that led you to it.

Why bother shipping different version combinations of software and configs if that is useless?

...and now you are building even more questionable conclusions on that unfounded implication.

No one is suggesting making the distribution less usable...at best that is hyperbole. One merely needs to step back and understand their are multiple audiences for both the distro and benchmarks. If you are using openSUSE as the basis for large-scale NAS deployment you will want to optimze it for that or any distro. If you are someone doing this for your home server in a couple hours of the weekend what you get out of the box is far more important to you.

How about this? Either I want a firewall with connection tracking on that particular box or I don't. In the former case, I'll have to configure and enable it on the distributions which do not provide it out of the box; in the latter, I'll have to disable it on those which do. The important point is that if I'm interested in performance of my box, I'm interested either in performance with the firewall or without. The performance I'm interested in is the performance of the system I'm going to actually use. Whether it's with a firewall or without is my choice independent of what distribution I'm going to use and its defaults. Just imagine it. You go to Phoronix, pick a distribution which ranks best in their "benchmarks", install it the "next-next-next" way, do not even choose what packages you want to install, which services you want to run and start using it with the default config? What if you want CIFS and the "fastest" distribution doesn't install Samba by default? Do you just shrug and say "Pity... but what can I do?" Maybe there really are users thinking like this - but I strictly disagree with the idea of tailoring our design decision to them. If I remember correctly, few years ago there was some discussion about what should be openSUSE's driving idea and target audience. The result was that we want to be the best distribution for software development. Does that sound as if our target audience were people who leave the decision if they want a firewall or not to distribution defaults? I don't think so. Michal Kubecek -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 1/25/19 6:26 PM, Michal Kubecek wrote:

Let's try to educate them, explain why the Phoronix style "benchmarks" have no value.

https://en.wikipedia.org/wiki/King_Canute_and_the_tide -- Liam Proven - Technical Writer, SUSE Linux s.r.o. Corso II, Křižíkova 148/34, 186-00 Praha 8 - Karlín, Czechia Email: lproven@suse.com - Office telephone: +420 284 241 084 -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Friday, January 25, 2019 11:08:51 AM CST Brüns, Stefan wrote:

If one system comes without an active firewall and the other does, then the whole comparison is absurd.

A sports car is faster than a truck, but they address different needs.

Exactly, but no one would deny that benchmarking the 0-60mph time of a sports car vs a truck would be relevant for someone looking to find the vehicle with the fastest acceleration time. So then the same is true of distros. Sure I could replace the engine in the truck, remove most of the weight, etc and get the truck to 0-60mph in the same time, but that is something for someone familiar with the mechanics involved and wanting to spend the time. Similarly if a distro's out-of-the-box configuration is better suited for some usecases this is equally important for a non-mechanic or someone who just wants the thing to work well. Nothing absurd about that in the slightest. -- Jimmy -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Friday, January 25, 2019 3:47:25 PM CST Stefan Brüns wrote:

No, the problem is the benchmarks are selling the truck for a sports car. There is no mentioning what the differences are. According to the article, the capabilities are all the same.

The topic of selling distros as the same is not something I see phoronix do. In fact the opposite is rather the point of the benchmarks.

Reducing the comparison to mere speed does not show the whole picture, it is misinformation. A casual reader will just see, oh, a bunch of Linux distributions, they are all the same, but some are faster - what a crappy job done by the others ...

Depending on a user's needs some distros do things worse than others. Clearly, I can transport a bunch of stuff in a car or a truck. Definitely larger objects and more easily in a truck, but I could buy a trailer for the car. This of course requires me to look into transport methods as they pertain to cars. The car vs truck analogy ironically demonstrates my point perfectly. Vehicles are essentially distros. They ship a method of generating power (engine), climate control, entertainment systems, seats, safety features, etc., but provide them in very different configurations. As a casual vehicle buyer I generally only care about a couple high-level requirements and not so much about the details of how they are accomplished. The problem with distros is they are not quite as transparent and easily compared. If you don't like the benchmarks ignore them, but they accomplish what they set out to. It would be like saying the various vehicle review magazines and sites do not tell the whole story...indeed...that is not the point. -- Jimmy -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Friday, January 25, 2019 11:11:10 AM CST Michal Kubecek wrote:

Again: this is *not* (only) about *tuning*. "Tuning" would be e.g. (for the ethernet performance tests) setting net.core.[rw]mem_max sysctl, NIC offloading flags etc. I'm not talking about that. Comparing packet latencies between distribution A with connection tracking and complex set of rules against distribution B with no firewall (and conntrack) means that you are running a completely different test for each, not that you "did not tune".

The same holds for the selection of services to run by default and measuring the "boot time". As I said before, we could disable everything not strictly necessary for login prompt (or sddm login screen) and we would get perfect "boot time" rank in Phoronix benchmarks. Of course, such default installation would be essentially unusable. Do we make a mistake by not going this way? I don't think so, there are more important things than rank in a pointless benchmark article.

But that is entirely depended on what your benchmarks are attempting to compare. If was was comparing different upstream applications for doing the same thing then yes having the same settings would be relevant. Clearly I should be able to take any rolling Linux distro and configure them to achieve virtually the same results. Absolutely nothing interesting about that. If you are not interested in out-of-the-box information that is entirely fine, but to come to some conclusion that they are pointless or absurd is exactly that absurd. You have assumed the goal of these benchmarks to be one other than the actual goal.

...

Clearly if I try hard enough I can make distros very similar to each-other as they are running the same upstream software and as such I would get more similar results. At that point you are benchmarking the upstream software and not the distro.

Even that is not true. You would get differences from e.g. compiler version and flags, kernel configuration etc.

But according to your argument then I should make those the same as well since that is not fair. Why do you draw the line arbitrarily at run-time configuration? That's the whole point, distros are not the same. People want to know how they compare for similar tasks. Not be told that they can spend a week learning the inner workings of everything and re-configure on distro to achieve the same thing they get automatically from another.

Do you call the choice to prepare firewall config and enable the firewall or to enable other services (that other distribution might not) a "crappy default"?

For someone who's usecase does not require them yes. For our target audience perhaps not. It is somewhat beside the point. My focus of crappy defaults is anything that is optimized for the usual case and not the majority of our target. If someone is setting up a gitlab instance and needs really good IO performance than a btrfs root volume with /srv sub-volume is probably not the best option and would be a bad default for them. Clearly they chould choose a different file system, but just as easily a benchmark against a distro with a different default file system of gitlab performance would make that clear. -- Jimmy -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Friday, January 25, 2019 3:28:13 PM CST Michal Kubecek wrote:

But having conntrack and complex set of rules or nothing at all, that's not "tuning", that's completely different test.

Sure, but I would bet the majority of Linux users do not even know what those are let alone how or why they would configure them. Hence all they see are the same results the benchmarks show, but without the need to find out themselves. Still boils down to assuming a goal that is simply not the goal. -- Jimmy -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 1/25/19 2:01 PM, Liam Proven wrote:

I do not like to see this Phoronix-bashing. It is a useful and valuable site, as the only one in the *large* field of PC performance comparison sites and communities which is Linux-based. The Linux world _needs_ this.

I don't think the site is valuable as long as the benchmark methods remain unscientific and questionable.

If we Linux users wish to see Linux continue to make headway against Windows, then sites like this are _necessary_.

Most Linux users don't care about competing with Windows. Linux has already surpassed Windows in all markets that are relevant. The only market left is the desktop market but there is not money to be made which is why the market has been on a decline for several years now.

If someone spends their own money on computers, then they want to know which are fast and which are slow, which are better or worse value for money. There *are* differences.

Sure. But what do you gain if the benchmarks produce incorrect or misleading results?

As well as differences between makes and models, there are also differences between performance under Windows and under Linux -- obviously.

Those performance differences are negligible for 95% of the userbase as the CPU is idling most of the time anyway. For users which seriously care about performance (e.g. HPC users), there are better and more scientific ways of running benchmarks.

Phoronix is the _only_ site doing such comparisons.

That is a valuable service.

There are tons of sites and computer magazines that do benchmarks.

If, perhaps, aspects of the design of openSUSE make it seem slower in benchmarks, either on first boot or in general, well, that is going to make it seem less appealing to people who want to get more performance out of their hardware by running Linux instead of Windows.

No user is going to run Linux over Windows for performance reasons. In fact, I would argue that the majority of users choose their operating system over the applications they are running. What do I gain by installing Linux when I need to work with the Adobe Creative Suite?

That being the case, there's only one answer: eliminate those performance differentials. Make it faster.

There is no point in trying to make it faster when the benchmark methods to determine the performance were flawed in the first place.

The answer is *not* to deride the people doing the measurements, because people do care about those measurements. That's life. Deal with it.

If the methods are completely flawed, it absolutely is. If someone tried to publish scientific paper based on the work methods that Phoronix is using, it would get torn into pieces by the scientific community in no time. Adrian -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Op vrijdag 25 januari 2019 14:33:12 CET schreef Carlos E. R.:

On 25/01/2019 14.19, John Paul Adrian Glaubitz wrote:

...

On 1/25/19 2:01 PM, Liam Proven wrote:

...

I do not like to see this Phoronix-bashing. It is a useful and valuable site, as the only one in the *large* field of PC performance comparison sites and communities which is Linux-based. The Linux world _needs_ this.

I don't think the site is valuable as long as the benchmark methods remain unscientific and questionable.

Irrelevant, because that is the test site most people consult. What you or me think of the site is irrelevant. That would make it some fake news site. Lots of people consult X.com, so what?

-- Gertjan Lettink a.k.a. Knurpht openSUSE Board Member openSUSE Forums Team -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 1/25/19 2:54 PM, Carlos E. R. wrote:

So what, if the voters read them?

If you want more installs, more users, you have to consider those sites.

Do you have a source that backs your claim that Phoronix has a measurable impact on the users' decisions to install a certain Linux distribution? Adrian -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 1/25/19 3:30 PM, Carlos E. R. wrote:

Just what users tells me in conversations. But I'll find some for you.

Those are just anecdotes. If you want to convince a scientist, you should present hard facts in the form of statistics. Adrian -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 1/25/19 3:38 PM, John Paul Adrian Glaubitz wrote:

Those are just anecdotes. If you want to convince a scientist, you should present hard facts in the form of statistics.

http://blog.danwin.com/don-t-forget-the-plural-of-anecdote-is-data/ The correct original quote, from Raymond Wolfinger, is: "The plural of anecdote is data." https://quoteinvestigator.com/2017/12/25/data/ It's amusing that most people get this precisely wrong. -- Liam Proven - Technical Writer, SUSE Linux s.r.o. Corso II, Křižíkova 148/34, 186-00 Praha 8 - Karlín, Czechia Email: lproven@suse.com - Office telephone: +420 284 241 084 -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 1/25/19 2:33 PM, Carlos E. R. wrote:

...

I don't think the site is valuable as long as the benchmark methods remain unscientific and questionable.

Irrelevant, because that is the test site most people consult. What you or me think of the site is irrelevant.

I think you are seriously overestimating the relevance that Phoronix has. SUSE's primary customer base are enterprise users and I can guarantee you that an enterprise customer is not choosing their distribution over Phoronix testsuite results. Adrian -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 1/25/19 2:40 PM, John Paul Adrian Glaubitz wrote:

I think you are seriously overestimating the relevance that Phoronix has.

SUSE's primary customer base are enterprise users and I can guarantee you that an enterprise customer is not choosing their distribution over Phoronix testsuite results.

This is the same error in thinking that cost Novell the NOS industry. Big businesses choose things based on the opinions of technical experts. People become technical experts because of years of skill and experience. They do that learning because they are interested. They engaged their interest because they played around with the tools and technology. When someone plays around for the purposes of learning, they do not use expensive enterprise tools. They use free tools or cheap educational tools. So in this business, they get started with some old junk PCs and a free copy of whatever software is involved. Possibly, as in my case, by temporarily installing some expensive commercial stuff that is, in future, going to go onto a real customer's machine. But you do some test runs first, in the lab, so you know how it works and what it will do. *Nobody* does the first ever deployment on a real live (or future live) customer system. And nobody worth employing *only* has knowledge they got from a training course. If they haven't practiced, they're not worth having. So what you must do, in order to eventually win that lucrative enterprise market, is make sure that future enterprise tech staff can get free versions of your products, or cheap educational versions, to play around with so that they can get to know it. Every big company started out as a small company. Every professional started out as a beginner. Netware 2 and 3 were great small-company products, which did not scale well to bigger deployments. But they ended up in big deployments, because those big companies started off as 1 or 2 people in a shared office, and when they got a secretary or a receptionist, they needed to network their 3 or 4 computers. Netware 4 and NDS were great for big companies, because they were designed to scale using a network directory. But they were a pain to install on a single machine in a small office, because all that NDS stuff just got in the way. Unfortunately, at the same time, Windows NT 4 Server came along. It looked like Win95, it was nearly as easy to deploy as Win95, and you could just work it out. And when you did, it did everything Netware 4 did, and it was also a good application server -- for instance it could run a local Web proxy, or a dial-on-demand firewall. It was terrible for large companies, because it had no network directory, but it took over the low end. So 4 or 5 years later, it took over the high end as well, and Netware died. Source: I was a trained, certified Netware and NT Server engineer at the time, doing this in real life, alongside writing about it. About a decade later, there were 2 types of Linux distro: * ones you paid money for and came with manuals and support * free ones, which were much harder to install and set up Professional distros like SUSE Pro dominated the commercial world. At home, people played around with Red Hat Linux because it was free, but it was kinda clunky. Mark Shuttleworth made US $600 million selling Thawte to Verisign. After going to the International Space Station ($20M) he started a Linux company to give something back to the community. He took the leading freeware distro, Debian, famous for being hard to install, configure and use, with no fancy paid version, and he made an easy-to-install, easy-to-use, freeware distro. It really caught on. Any kid learning Linux used Ubuntu because it was free and it was easier than Red Hat Linux or Slackware or Gentoo or whatever. A decade later, many of those kids are professional Linux engineers. And what do they recommend to their boss? The distro they know best. The free one they played around with at night after school. Ubuntu. Is it less mature? Yes. Does it have a special supported enterprise version? No. So are you getting anything less or inferior to a paid-for "full version"? No. It's the same. Result -- slightly to Canonical's surprise at first, I think -- Ubuntu becomes a widely-used server distro. Just as NT 4 did. Because you could find your way, it was easy, and it was the same as the cheapo home version. Summary No, enterprises do *not* choose distros based on benchmarks on free ad-supported websites. Absolutely. You're right. They base their choices on experts' opinions. And those experts started out learning... from free ad-supported websites. So neglect the free home stuff for amateurs, hackers and know-nothing beginners, and you're doomed. -- Liam Proven - Technical Writer, SUSE Linux s.r.o. Corso II, Křižíkova 148/34, 186-00 Praha 8 - Karlín, Czechia Email: lproven@suse.com - Office telephone: +420 284 241 084 -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 25/01/2019 14.19, John Paul Adrian Glaubitz wrote:

On 1/25/19 2:01 PM, Liam Proven wrote:

...

If, perhaps, aspects of the design of openSUSE make it seem slower in benchmarks, either on first boot or in general, well, that is going to make it seem less appealing to people who want to get more performance out of their hardware by running Linux instead of Windows.

No user is going to run Linux over Windows for performance reasons.

On the contrary, I have found quite some users that claim their computers are faster running Linux than Windows.

In fact, I would argue that the majority of users choose their operating system over the applications they are running. What do I gain by installing Linux when I need to work with the Adobe Creative Suite?

Well, of course. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)

On 1/25/19 11:12 AM, Liam Proven wrote:

On 1/25/19 3:37 PM, Carlos E. R. wrote:

...

On the contrary, I have found quite some users that claim their computers are faster running Linux than Windows.

Exactly. Most, in fact, I'd say.

In fact I did say:

https://www.quora.com/Why-is-Ubuntu-faster-than-Windows/answer/Liam-Proven

As a comparison, I just recently bought myself a Sony Vaio P as a toy. It's a very unusual take on the netbook form factor -- a very wide but tiny PC.

It is maxed out with 2GB of RAM.

It boots Windows 10 in about 5 minutes, and launching even simple apps provokes a frenzy of swapping.

It runs Xubuntu LTS quite comfortably. Chromium and Firefox are unusable, but as a portable writing tool, it's quite nice.

My partner's fairly high-end quad-core Core i5 PC with nVidia graphics, 8GB of RAM and a big 1TB hard disk takes several minutes to get to the login screen in Win10.

In Linux Mint it boots to the desktop in about a quarter of the time.

In general use as a desktop OS, Linux is indeed *noticeably* faster than Windows.

This is one thing that attracts many people to it in the first place: it's a way to revive old PCs, without the expense (and difficulty) of getting a current Windows licence, reinstalling the OS, finding all the drivers, replacing all the apps, etc.

As such, there's considerable competition for who can be the fastest full-function Linux, with a full desktop, selection of apps, and so on.

My Toshiba laptop with a 4th generation i7 CPU, 12 GB RAM, and an SSD takes longer for the BIOS to start than for Linux to boot. Of course it is tuned, which emphasizes that out-of-the-box tests without specs mean nothing. Slightly off topic, but an opportunity to teach/preach: I volunteer one day a week at a non-profit organization in Kansas City that refurbishes donated computers and resells them at low cost to low-income people. A Core 2 Duo desktop system with Windows 10 license, 4 GB RAM, and an 80 GB hard drive sells for $25. Those systems are dogs as far as performance, but the interesting part for this discussion is what happens to the donations that have even less performance. If the machine runs 64-bit code, then we install openSUSE 15.0 with an LXDE desktop, a single ext4 partition on the hard drive and a full complement of applications. These machines boot faster than Windows systems with i7 CPUs! Most importantly to our users, these systems are FREE unless they need wifi, which we sell for $10. For systems that can only run 32-bit code, we install Q4OS with a configuration as close as possible to the 64-bit versions. We have distributed more than 100 Linux systems into the community. To my knowledge, only one has been returned "because it was not Windows"! Larry -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 1/25/19 2:33 PM, Michal Kubecek wrote:

It _would_ be a valuable service if the numbers presented had some meaning. They don't. Unfortunately, Michael Larabel, after all those years he is running the site still didn't learn a bit and didn't get any further than any beginner who just runs some program and writes the numbers that fell out of it into a nice table. And that's just sad. That's not "deriding", that's how it is.

Doing benchmarsks properly is hard, it requires a lot of knowledge and lot of experience. You need knowledge and experience to know how to set up the test systems,

You are still missing the point here. This isn't about an enterprise storage farm with 40 GigE SAN connections. It's "which distro is fastest?" You are comparing it to the wrong thing.

So we are supposed to stop providing preconfigured firewall just to make us look better in some tables of pseudorandom numbers some moron decided to put on his site (which other morons interpret as "openSUSE is slow"?

Ubuntu comes with a firewall. Fedora comes with a firewall. Just about every mainstream distro comes with a firewall. If openSUSE's is slower for some reason, find out why the competitors' _aren't_. Don't blame the place doing the testing that found the difference. -- Liam Proven - Technical Writer, SUSE Linux s.r.o. Corso II, Křižíkova 148/34, 186-00 Praha 8 - Karlín, Czechia Email: lproven@suse.com - Office telephone: +420 284 241 084 -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Fri, 25 Jan 2019, Michal Kubecek wrote:

On Friday, 25 January 2019 15:09 Liam Proven wrote:

...

...

So we are supposed to stop providing preconfigured firewall just to make us look better in some tables of pseudorandom numbers some moron decided to put on his site (which other morons interpret as "openSUSE is slow"? Ubuntu comes with a firewall.

Fedora comes with a firewall.

And both ranked similar to Leap in the "10Gb/s ethernet test". Does ClearLinux (which was "much faster")? Did the Debian and FreeBSD installations that were also "much faster"? Actually... we don't even know because Michael Larabel somehow forgot to tell us (but he didn't forget to inform us about what GPU and monitor did the test system have because that's apparently way more important in ethernet performance test). He didn't even bother to mention actual parameters iperf and netperf were run with so that we cannot even check his numbers.

The answer to this is usually the benchmark suite is open-source, so you can have a look yourself. Something we do from time-to-time when looking at odd numbers from benchmark comparisons of compilers... Oh, and when pointing out specific issues with tests he's usually open to fixing those. But yes, it happens too often that the pretty graphs are results of GIGO ... Richard.

Michal Kubecek

...

Just about every mainstream distro comes with a firewall.

If openSUSE's is slower for some reason, find out why the competitors' _aren't_.

-- Richard Biener SUSE LINUX GmbH, GF: Felix Imendoerffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nuernberg) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Am Freitag, 25. Januar 2019, 14:01:42 CET schrieb Liam Proven:

I do not like to see this Phoronix-bashing.

Just to make this clear: My intention wasn't to bash but to point out problems. Problems that are not in the to-be-tested distribution's hands which makes it even harder to tackle on a technical basis.

Phoronix is the _only_ site doing such comparisons.

That is a valuable service.

Yes, as long as you take it with a grain of salt. And that's missing quite frequent in the public reception. Or in other words: being the only one doing such benchmarks doesn't inhibit objective criticism.

The answer is *not* to deride the people doing the measurements, because people do care about those measurements. That's life. Deal with it.

Not seeing where you're pulling derision from within this thread. Regards, vinz. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Op vrijdag 25 januari 2019 14:23:44 CET schreef Joachim Wagner:

...

We could have the installed system "next next next" faster, for instance.

To stay with your analogy to car mileage testing, the "next next next" system should be optimised for performance on the phoronix test and nothing else. No worries if the system is slow or unusable in general. I think I read somewhere that they remove the car seats and side mirrors for the test. You cannot drive your car on a public road like that either.

To make it clear to users, we could add an option "optimised for phoronix test" to the installer page that asks to choose between kde, gnome and server.

Any volunteers to implement and maintain this?

Joachim Thanks, had the best laugh in ages.

-- Gertjan Lettink a.k.a. Knurpht openSUSE Board Member openSUSE Forums Team -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Am Freitag, 25. Januar 2019, 12:09:50 CET schrieb Carlos E. R.:

...

...

Honestly, the most wrong thing about Phoronix is that there are people who take its "benchmarks" and their results seriously. Please don't be one of them.

But that's it, that people read those benchmarks and don't listen to their critics.

Which leads to two possible ways of handling it: 1) Try to jump on every discussion where Leap, Tumbleweed or whatever openSUSE "product" is benchmarked wrong. 2) Try to tell everyone what's cool, handy and impressive about the openSUSE stuff.

Both ways are legit imho. But I know which one I'd choose - for the sake of my mind and mental sanity. ;)

Regards, vinz. Vinz, I did both for years, ended up with 2) as the thing that does work once in a while, where 1) is mostly a waste of energy and time.. Hence my reluctance in

Op vrijdag 25 januari 2019 13:12:39 CET schreef Vinzenz Vietzke: the 'Tumbleweed defence' thread. It indeed is about apples vs. elephants. IMNSHO. -- Gertjan Lettink a.k.a. Knurpht openSUSE Board Member openSUSE Forums Team -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Am Freitag, 25. Januar 2019, 17:56:09 CET schrieb Axel Braun:

What are our options? - first of all, we may ask Phoronix to disclose methodology and default setup for all distributions. Not even the question was ansered if the distros run a DE or just text mode - we could try to verify ourself - we could optimize default setup

Next to technical, we could question in public the correctness of the results.

I don't think approaching Michael Larabel might lead to any success. Which is okay for me but leaves us with the problematic public reception of oS. Therefore it would be good to communicate technical decisions and reasons in a more accessible way. What comes up to my mind is some series of blog posts, explaining things and interviewing developers briefly. All in all, openSUSE is just keeping it's good work way too much under the hood when it comes to "Jack User", as you say. So, to not just demand for progress but also to step forward: I'm totally willing to work on that issues, to create some kind of plan. Anyone willing to join? Regards, vinz. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Am 25. Januar 2019 18:08:16 MEZ schrieb Vinzenz Vietzke :

Am Freitag, 25. Januar 2019, 17:56:09 CET schrieb Axel Braun:

...

What are our options? - first of all, we may ask Phoronix to disclose methodology and default setup for all distributions. Not even the question was ansered if the distros run a DE or just text mode - we could try to verify ourself - we could optimize default setup

Next to technical, we could question in public the correctness of the results.

I don't think approaching Michael Larabel might lead to any success. Which is okay for me but leaves us with the problematic public reception of oS.

Therefore it would be good to communicate technical decisions and reasons in a more accessible way. What comes up to my mind is some series of blog posts, explaining things and interviewing developers briefly.

All in all, openSUSE is just keeping it's good work way too much under the hood when it comes to "Jack User", as you say.

So, to not just demand for progress but also to step forward: I'm totally willing to work on that issues, to create some kind of plan. Anyone willing to join?

Yes Cheers Axel -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org