Hello! On 7/28/22 13:44, Richard Biener wrote:

there's been rumors and confusion about ALP raising the x86_64 architecture level to v3 which requires CPUs with AVX2 support and how this will affect openSUSE Factory. Sofar openSUSE Factory was supposed to be not affected but I think it makes sense to re-consider the current -v1 usage given what other distros are doing.

I assume the other distros are most likely Fedora, Ubuntu and Arch?

Thus I would propose raising the x86_64 architecture level to x86-64-v2 for openSUSE Factory. v2 requires CMPXCHG16B, [SL]AHF, POPCNT and SSE4_2 (and the predecessor SSE extensions) but not AVX, BMI or FMA.

I think it would be good to know which Intel and AMD CPUs were the first to introduce x86_64-v2, so that users know which CPUs will stop working with Tumbleweed. Adrian

On Thu, 28 Jul 2022, John Paul Adrian Glaubitz wrote:

Hello!

On 7/28/22 13:44, Richard Biener wrote:

...

there's been rumors and confusion about ALP raising the x86_64 architecture level to v3 which requires CPUs with AVX2 support and how this will affect openSUSE Factory. Sofar openSUSE Factory was supposed to be not affected but I think it makes sense to re-consider the current -v1 usage given what other distros are doing.

I assume the other distros are most likely Fedora, Ubuntu and Arch?

...

Thus I would propose raising the x86_64 architecture level to x86-64-v2 for openSUSE Factory. v2 requires CMPXCHG16B, [SL]AHF, POPCNT and SSE4_2 (and the predecessor SSE extensions) but not AVX, BMI or FMA.

I think it would be good to know which Intel and AMD CPUs were the first to introduce x86_64-v2, so that users know which CPUs will stop working with Tumbleweed.

If you run Tumbleweed you can do

/lib64/ld-linux-x86-64.so.2 --help [..] Subdirectories of glibc-hwcaps directories, in priority order: x86-64-v4 x86-64-v3 (supported, searched) x86-64-v2 (supported, searched)

and see which architecture level is supported by your CPU. Mine above supports v3 but not v4. Richard.

On Thursday 2022-07-28 13:50, John Paul Adrian Glaubitz wrote:

I think it would be good to know which Intel and AMD CPUs were the first to introduce x86_64-v2, so that users know which CPUs will stop working with Tumbleweed.

I'm still hoping for /bin/lscpu to show the architecture level someday.

John Paul Adrian Glaubitz composed on 2022-07-28 13:50 (UTC+0200):

Richard Biener wrote:

...

Thus I would propose raising the x86_64 architecture level to x86-64-v2 for openSUSE Factory. v2 requires CMPXCHG16B, [SL]AHF, POPCNT and SSE4_2 (and the predecessor SSE extensions) but not AVX, BMI or FMA.

What is it these functions add? What proportion of users with v2 support would actually notice any difference? Is this level in line with whatever Win11 requires?

I think it would be good to know which Intel and AMD CPUs were the first to introduce x86_64-v2, so that users know which CPUs will stop working with Tumbleweed.

+1 Does this mean dropping 32bit TW too? Which 64bit CPUs will be forced to switch to 32bit TW, and which to other distros? Support for older hardware has been a Linux tradition. Would it be feasible to have two 64bit TWs? If yes, one could go beyond v2, the other stay right where TW is now. I just finished zypper dup on a Pentium III with i810 graphics that Debian 11 fully supports, while Debian 12 roughly matches TW[2] in failing iGPU support from the Intel DDX[1]. :( [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1016151 [2] (EE) open /dev/dri/card0: No such file or directory (EE) open /dev/fb0: No such file or directory (EE) intel(0): AGP GART support is not available. -- Evolution as taught in public schools is, like religion, based on faith, not based on science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata

On Thursday 2022-07-28 14:08, Fridrich Strba wrote:

On 28/07/2022 14:06, Jan Engelhardt wrote:

...

Shouldn't we drop i586 before dropping any x86_64-v0 class hw? Feels a bit strange to support most recent and most ancient components, with an uncanny hole in the middle.

I am pretty happy that openSUSE runs on my Centrino M from 15 years ago though :)

Yes, but the question was a serious one. Like, for this v0/v1 device: # lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 32 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 2 On-line CPU(s) list: 0,1 Vendor ID: GenuineIntel Model name: Intel(R) Atom(TM) CPU N450 @ 1.66GHz CPU family: 6 Model: 28 Thread(s) per core: 2 Core(s) per socket: 1 Socket(s): 1 Stepping: 10 CPU max MHz: 1667.0000 CPU min MHz: 1000.0000 BogoMIPS: 3324.94 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts nopl cpuid aperfmperf pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm movbe lahf_lm dtherm L1d cache: 24 KiB (1 instance) L1i cache: 32 KiB (1 instance) L2 cache: 512 KiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0,1 Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec store bypass: Not affected Vulnerability Spectre v1: Not affected Vulnerability Spectre v2: Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected # ld.so --help ... Shared library search path: (libraries located via /etc/ld.so.cache) /lib64 (system search path) /usr/lib64 (system search path) Subdirectories of glibc-hwcaps directories, in priority order: x86-64-v4 x86-64-v3 x86-64-v2 Legacy HWCAP subdirectories under library search path directories: x86_64 (AT_PLATFORM; supported, searched) tls (supported, searched) avx512_1 x86_64 (supported, searched)

On Thu, Jul 28, 2022 at 2:06 PM Jan Engelhardt wrote:

...

Thus I would propose raising the x86_64 architecture level to x86-64-v2 for openSUSE Factory. Shouldn't we drop i586 before dropping any x86_64-v0 class hw?

is there even x86_64 Variant0 and Variant1 as well? can anyone please provide a robust and working means to determine the current level for us current leap 15.4 (and/or even 15.3) users? I found some script via https://unix.stackexchange.com/posts/631226/revisions by sephen kitt there. My real meta hardware results with this script only give me -v2 (very few) but mostly virtually all -v1 results. But then again when looking into /proc/cpuinfo on my hardwares I find stuff I can not interpret or understand (e.g. finding SSSE3 (three S) but no SSE3 and still SSE4_x ) I am mostly an avid AMD fan here. I also ask for sane decision for the future of Leap, I make a lot of use of aged hardware and I always liked it that the underdog AMD invented the x86_64 world and succeeded with it and not the giant monopolist named intel, but only as a side remarkt. I wonder why cut x86_64 (V0 or V1 or whatever early stages of it? in general but not going there incrementally? I dont wana create e-waste of my hardware and I suppose others are happy with their x64 world as well compared to really old x86/32bit/i686/i585 stuff when I understand all these sub ABIs at all. Please dont abandon our hardware when it is already in the x64 world (x86_64). thanks. ty

On Thu, Jul 28, 2022 at 7:44 AM Richard Biener wrote:

though)

Comments welcome.

This is long overdue, the amount of CPUs not supporting this features is pretty small and not worth keeping.

On Thu, 28 Jul 2022 20:24, cagsm wrote:

On Thu, Jul 28, 2022 at 6:12 PM Dirk Müller wrote:

...

...

can anyone please provide a robust and working means to determine the current level for us current leap 15.4 (and/or even 15.3) users?

First of all, there is no change whatsoever planned for Leap users, so Leap users don't have to worry about anything.

okay this means there is no new CPU feature requirements for Leap 15.5 and successor?

...

But to answer your question, simply run this: podman run registry.opensuse.org/opensuse/tumbleweed:latest /lib64/ld-linux-x86-64.so.2 --help

is this a single line command? I first also needed to zypper in podman

then all your above two(?) lines in a single line gave me a huge output. Does the current leap 15.3 and 15.4 /lib64/ld-linux-x86-64.so.2 not have these parameters and or outputs? its call only creates an error message as result.

never dealt with podman and containers so far. where does the bits of this download or packages get actually stored locally? thanks lots. ty

otherwise a short awk program will also give answers: [code] #!/usr/bin/awk -f BEGIN { while (!/flags/) if (getline < "/proc/cpuinfo" != 1) exit 1 if (/lm/&&/cmov/&&/cx8/&&/fpu/&&/fxsr/&&/mmx/&&/syscall/&&/sse2/) level = 1 if (level == 1 && /cx16/&&/lahf/&&/popcnt/&&/sse4_1/&&/sse4_2/&&/ssse3/) level = 2 if (level == 2 && /avx/&&/avx2/&&/bmi1/&&/bmi2/&&/f16c/&&/fma/&&/abm/&&/movbe/&&/xsave/) level = 3 if (level == 3 && /avx512f/&&/avx512bw/&&/avx512cd/&&/avx512dq/&&/avx512vl/) level = 4 if (level > 0) { print "CPU supports x86-64-v" level; exit level + 1 } exit 1 } [/code] hint: in case of linefolding: each line starting with "if (level..." ends with "...level = [number]" this was taken from a stackexchange post about 1,5 years ago, url was: https://unix.stackexchange.com/questions/631217/how-do-i-check-if-my-cpu-sup... - Yamaban

On Thursday 2022-07-28 20:51, Yamaban wrote:

otherwise a short awk program will also give answers: [code] #!/usr/bin/awk -f

BEGIN { while (!/flags/) if (getline < "/proc/cpuinfo" != 1) exit 1 if (/lm/&&/cmov/&&/cx8/&&/fpu/&&/fxsr/&&/mmx/&&/syscall/&&/sse2/) level = 1 if (level == 1 && /cx16/&&/lahf/&&/popcnt/&&/sse4_1/&&/sse4_2/&&/ssse3/) level = 2 if (level == 2 && /avx/&&/avx2/&&/bmi1/&&/bmi2/&&/f16c/&&/fma/&&/abm/&&/movbe/&&/xsave/)

This is slightly incorrect. A hypothetical machine which has AVX2 but lacks AVX1 would still print level 2.

On 7/28/22 23:40, Marius Kittler wrote:

Am Donnerstag, 28. Juli 2022, 15:30:15 CEST schrieb Neal Gompa:

...

To the best of my knowledge, no community distribution has elected to move to a higher level yet. It has been brought up a few times in Fedora and Arch, but as far as I know, the change hasn't happened yet in a way that eliminates x86_64-v1 in these distributions.

When I've been following-up Arch Linux correctly, they've decided to provide x86_64-v3 packages *in addition* to their existing x86_64 packages (which will remain unaffected for now). The option to make x86_64-v2 the new baseline (replacing existing x86_64 packages) was also on the table but they decided against it because x86_64-v2 would bring almost no benefit, at least not enough to justify the hardware requirement. (Note that providing x86_64-v3 packages has not been carried out yet. It looks like they want to focus on their build automation first. Not a concern for openSUSE, though.)

I suppose this decision makes most sense. No users on old hardware will be negatively affected and users with newer hardware can switch to x86_64-v3 packages.

The simplest way of achieving this, that would on the other hand use the most build resources would be to add a new arch "x86_64-v3" which would leave people with a choice. Whether we have the spare build power for this would be the main question. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

On Thu, 28 Jul 2022 11:44:03 +0000 (UTC) Richard Biener wrote:

Thus I would propose raising the x86_64 architecture level to x86-64-v2 for openSUSE Factory. v2 requires CMPXCHG16B, [SL]AHF, POPCNT and SSE4_2 (and the predecessor SSE extensions) but not AVX, BMI or FMA.

...

Comments welcome.

Hi, Are there some data available about the actual benefit of this change? On the one hand I am very for making the best use of available CPU features. On the other hand I dislike to obsolete hardware which is still perfectly working and usable. I am aware that for specific types of applications, algorithms or calculations modern CPU features can bring a big benefit. But I assume that most users are not running this type of workload. I also experienced in the past that a self compiled kernel optimized for i586 on a Pentium had noticeable better performance then the original distribution kernel built for i386 for specific tasks. But I think with x86_64 there is already a huge improvement of common available CPU features, and that the additional added features since then have very limited benefit for standard software (common libraries, kernel, ...). Therefore I think it would really be important to have some data about the expected benefits for standard use cases on -v2 or -v3 CPUs. And yes, I have a machine which would be obsoleted by this change. Kind regards, Dieter

Hi, On Thu, 04 Aug 2022 18:40:23 +0200 Martin Jambor wrote:

...

Are there some data available about the actual benefit of this change?

Over the weekend, I have quickly gathered some SPEC CPU benchmarks results comparing the different x86_64 versions:

https://jamborm.github.io/spec-2022-07-29-levels/index.html

thanks, very interesting results. I admit this gain is even less than I expected. Kind regards, Dieter

dieter wrote:

...

Thus I would propose raising the x86_64 architecture level to x86-64-v2 for openSUSE Factory. v2 requires CMPXCHG16B, [SL]AHF, POPCNT and SSE4_2 (and the predecessor SSE extensions) but not AVX, BMI or FMA. ... Comments welcome. Hi, Are there some data available about the actual benefit of this change? Therefore I think it would really be important to have some data about

On Thu, 28 Jul 2022 11:44:03 +0000 (UTC) Richard Biener wrote: the expected benefits for standard use cases on -v2 or -v3 CPUs.

https://openbenchmarking.org/result/2103142-HA-UARCHLEVE55 Only x86-64-v3 has good gains (sometimes).

Please make gradual upgrade: from {i586+SSE1 | x86-64} to {x86-64 | x86-64-v2}. Otherwise system with Phenom II X6 and 16 GiB of RAM will have to use 32-bit Tumbleweed and will be limited to 4 GiB of usable RAM (2 GiB system + 2 GiB user). IMHO supporting x86-64 feature levels v1 + v2 + v3 can be cheaper than today's 32-bit i586+SSE1 and x86-64.

On 8/5/22 18:55, Richard Biener wrote:

On Fri, 5 Aug 2022, Nikolai Nikolaevskii wrote:

...

dieter wrote:

...

...

Thus I would propose raising the x86_64 architecture level to x86-64-v2 for openSUSE Factory. v2 requires CMPXCHG16B, [SL]AHF, POPCNT and SSE4_2 (and the predecessor SSE extensions) but not AVX, BMI or FMA. ... Comments welcome. Hi, Are there some data available about the actual benefit of this change? Therefore I think it would really be important to have some data about

On Thu, 28 Jul 2022 11:44:03 +0000 (UTC) Richard Biener wrote: the expected benefits for standard use cases on -v2 or -v3 CPUs.

https://openbenchmarking.org/result/2103142-HA-UARCHLEVE55 Only x86-64-v3 has good gains (sometimes).

Only AVX with x86-64-v3 has the chance to double throughput of vectorized code, plus -v3 has BMI which can make differences in crypto and hashing.

But as it is always with bechmarks - if you get to choose it you can choose the outcome of the experiment. There's definitely special cases benefiting of SSE 4.2 but the gain with -v2 is expected to be limited compared to -v3, but -v3 is understood to be a no-go for openSUSE.

I'll note that while some specialized libraries come with separate code paths for ISAs they benefit from and appropriately dispatch via CPUid most developers are simply too lazy to even think about that. You may say that performance may not matter in most cases but performance often also translates to less energy use which for mobile uses might be even more important than performance (but that's even more difficult to measure, of course).

For me as a general user I think the sort of real life benchmarks that may make an influence on whether this is worthwhile on a wider scale is stuff like if I left my laptop on battery playing youtube for 3 hrs would using V3 instructions result in significantly better battery life. If someone could present an argument saying that this change will provide much better battery efficiency for a large number of our users that becomes a much more compelling argument for saying that we are chosing to no longer support hardware that a small number of our user use. (or it justifies using the processing power to build both). Where as if all we really know is it will make some certain specific workloads a bit faster for some then its much harder to make such an argument. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

* Larry Len Rainey [08-05-22 09:16]:

Hello,

On Fri, Aug 05 2022, Dan Čermák wrote:

dieter writes:

Hi,

On Thu, 04 Aug 2022 18:40:23 +0200 Martin Jambor wrote:

Are there some data available about the actual benefit of this change?

Over the weekend, I have quickly gathered some SPEC CPU benchmarks results comparing the different x86_64 versions:

https://jamborm.github.io/spec-2022-07-29-levels/index.html

thanks, very interesting results. I admit this gain is even less than I expected.

Yes, the gain is small, but a few percent gain will result in e.g. HPC users to pick another distribution. But yeah, the gain in these benchmarks is really surprisingly small.

Why not just do the -v3 only on Tumbleweed x86_64 and leave Leap as it is. Those that want speed probably are on Tumbleweed and those on "antique hardware" can still run Leap.

To me that seems like a great compromise.

and all outlooks are different. I still run tumbleweed on dual core Intel Core2 Duo T6600 [MCP] arch: Penryn Yorkfield -- (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri http://en.opensuse.org openSUSE Community Member facebook/ptilopteri Photos: http://wahoo.no-ip.org/piwigo paka @ IRCnet oftc

Hi, On Fri, 05 Aug 2022 07:59:18 +0200 Dan Čermák wrote:

dieter writes:

...

Hi,

On Thu, 04 Aug 2022 18:40:23 +0200 Martin Jambor wrote:

...

...

Are there some data available about the actual benefit of this change?

Over the weekend, I have quickly gathered some SPEC CPU benchmarks results comparing the different x86_64 versions:

https://jamborm.github.io/spec-2022-07-29-levels/index.html

thanks, very interesting results. I admit this gain is even less than I expected.

Yes, the gain is small, but a few percent gain will result in e.g. HPC users to pick another distribution. But yeah, the gain in these benchmarks is really surprisingly small.

Actually these gains are so small that I personally would conclude that it makes no sense to switch to -v2 or even -v3. There are even benchmarks in which the -v3 code is "significantly" slower - in the scope of measured differences. For HPC use cases I assume these users would compile the performance relevant libraries and programs really optimized for their very hardware. For many users the CPU is idle most of the time. For them a speed improvement of 2% or even 20% is hardly noticeable. Even for developers - I think if the build time of a project drops from 100 minutes to 98 minutes most of the time does not matter. Of course this is my personal opinion and others may have different priorities and every saved second my be essential to them. About running Tumbleweed on PCs older than 10 years: compared to PCs from 20 years ago these machines can be horrendously fast. And they can still perfectly fulfill many tasks. Tumbleweed is a good distribution and provides current software and security updates, so why not. I do not know what hardware most of the openSUSE user base is using. But from my own case switching to -v2 or higher would mean I can no longer use Tumbleweed on a machine, or at least would not get security updates or bugfixes anymore if I just stay at a snapshot which still works. Kind regards, Dieter

Am 05.08.22 um 11:25 schrieb Richard Biener:

I'll note that while some specialized libraries come with separate code paths for ISAs they benefit from and appropriately dispatch via CPUid most developers are simply too lazy to even think about that. You may say that performance may not matter in most cases but performance often also translates to less energy use which for mobile uses might be even more important than performance (but that's even more difficult to measure, of course).

Totally with you on the energy consumption, but I have to slightly disagree about how usual such dispatching is. Lots of software that benefits most from vectorization already does something like that, most notably anything in the "multimedia" arena, like audio/video codecs, filters, and so on. Same goes for cryptography. It's hard to see exactly how widespread it is, but even with lots of libraries missing .symtab I find a few on my system: $ for lib in /usr/lib64/*.so.{[0-9],[0-9][0-9],[0-9][0-9][0-9]}; do readelf -a --wide $lib | grep _sse2 >/dev/null && echo $lib; done /usr/lib64/libc.so.6 /usr/lib64/libfftw3f.so.3 /usr/lib64/libfftw3.so.3 /usr/lib64/libmpeg2.so.0 /usr/lib64/libm.so.6 /usr/lib64/libmvec.so.1 /usr/lib64/libnettle.so.8 /usr/lib64/libsodium.so.23 /usr/lib64/libSvtAv1Enc.so.1 /usr/lib64/libvidstab.so.1.1 /usr/lib64/libvisual-0.4.so.0 /usr/lib64/libwebrtc_audio_processing.so.1 $ for lib in /usr/lib64/*.so.{[0-9],[0-9][0-9],[0-9][0-9][0-9]}; do readelf -a --wide $lib | grep _sse3 >/dev/null && echo $lib; done /usr/lib64/libsodium.so.23 /usr/lib64/libx265.so.199 $ for lib in /usr/lib64/*.so.{[0-9],[0-9][0-9],[0-9][0-9][0-9]}; do readelf -a --wide $lib | grep _ssse3 >/dev/null && echo $lib; done /usr/lib64/libc.so.6 /usr/lib64/libQt5Gui.so.5 /usr/lib64/libsodium.so.23 /usr/lib64/libSvtAv1Enc.so.1 /usr/lib64/libx265.so.199 $ for lib in /usr/lib64/*.so.{[0-9],[0-9][0-9],[0-9][0-9][0-9]}; do readelf -a --wide $lib | grep _sse4 >/dev/null && echo $lib; done /usr/lib64/libc.so.6 /usr/lib64/libde265.so.0 /usr/lib64/libjavascriptcoregtk-4.0.so.18 /usr/lib64/libm.so.6 /usr/lib64/libmvec.so.1 /usr/lib64/libsodium.so.23 /usr/lib64/libx265.so.199 $ for lib in /usr/lib64/*.so.{[0-9],[0-9][0-9],[0-9][0-9][0-9]}; do readelf -a --wide $lib | grep _avx >/dev/null && echo $lib; done /usr/lib64/ld-linux-x86-64.so.2 /usr/lib64/ld-lsb-x86-64.so.3 /usr/lib64/libcblas.so.3 /usr/lib64/libc.so.6 /usr/lib64/libfftw3.so.3 /usr/lib64/libjavascriptcoregtk-4.0.so.18 /usr/lib64/liblapacke.so.3 /usr/lib64/libm.so.6 /usr/lib64/libmvec.so.1 /usr/lib64/libopenblas_pthreads.so.0 /usr/lib64/libopenblas.so.0 /usr/lib64/libsodium.so.23 /usr/lib64/libvmaf.so.1 $ for lib in /usr/lib64/*.so.{[0-9],[0-9][0-9],[0-9][0-9][0-9]}; do readelf -a --wide $lib | grep _popcnt >/dev/null && echo $lib; done /usr/lib64/libjavascriptcoregtk-4.0.so.18 /usr/lib64/libzvbi.so.0 Granted, scientific software tends to be an issue, perhaps because the implicit assumption is that people will build their own with -march=native. But if that's common enough I'd rather have those packages built in an additional version (which could even be x86_64-v3) than up the requirements across the board. But even there it's not unheard of, I mean look at the madness that is GMP: they have versions of their functions for at least a handful of different CPUs. That's not even just taking the instruction sets into account but often hand-tuned for a specific chip. Aaron

On Thu, Aug 4, 2022 at 12:40 PM Martin Jambor wrote:

Over the weekend, I have quickly gathered some SPEC CPU benchmarks results comparing the different x86_64 versions:

https://jamborm.github.io/spec-2022-07-29-levels/index.html

Huh, Martin ..thanks for taking the time to do this..it was disappointing to the extreme. So is this a case of a) most code been written "unaware" of vectorization b) the compiler is not aggressive enough c) we do not get warnings of this kind of code or all of them..

Am 07.08.22 um 00:30 schrieb Aaron Puchert:

Am 05.08.22 um 11:25 schrieb Richard Biener:

...

I'll note that while some specialized libraries come with separate code paths for ISAs they benefit from and appropriately dispatch via CPUid most developers are simply too lazy to even think about that.  You may say that performance may not matter in most cases but performance often also translates to less energy use which for mobile uses might be even more important than performance (but that's even more difficult to measure, of course).

Totally with you on the energy consumption, but I have to slightly disagree about how usual such dispatching is. Lots of software that benefits most from vectorization already does something like that, most notably anything in the "multimedia" arena, like audio/video codecs, filters, and so on. Same goes for cryptography.

It's hard to see exactly how widespread it is, but even with lots of libraries missing .symtab I find a few on my system:

$ for lib in /usr/lib64/*.so.{[0-9],[0-9][0-9],[0-9][0-9][0-9]}; do readelf -a --wide $lib | grep _sse2 >/dev/null && echo $lib; done /usr/lib64/libc.so.6 /usr/lib64/libfftw3f.so.3 /usr/lib64/libfftw3.so.3 /usr/lib64/libmpeg2.so.0 /usr/lib64/libm.so.6 /usr/lib64/libmvec.so.1 /usr/lib64/libnettle.so.8 /usr/lib64/libsodium.so.23 /usr/lib64/libSvtAv1Enc.so.1 /usr/lib64/libvidstab.so.1.1 /usr/lib64/libvisual-0.4.so.0 /usr/lib64/libwebrtc_audio_processing.so.1 $ for lib in /usr/lib64/*.so.{[0-9],[0-9][0-9],[0-9][0-9][0-9]}; do readelf -a --wide $lib | grep _sse3 >/dev/null && echo $lib; done /usr/lib64/libsodium.so.23 /usr/lib64/libx265.so.199 $ for lib in /usr/lib64/*.so.{[0-9],[0-9][0-9],[0-9][0-9][0-9]}; do readelf -a --wide $lib | grep _ssse3 >/dev/null && echo $lib; done /usr/lib64/libc.so.6 /usr/lib64/libQt5Gui.so.5 /usr/lib64/libsodium.so.23 /usr/lib64/libSvtAv1Enc.so.1 /usr/lib64/libx265.so.199 $ for lib in /usr/lib64/*.so.{[0-9],[0-9][0-9],[0-9][0-9][0-9]}; do readelf -a --wide $lib | grep _sse4 >/dev/null && echo $lib; done /usr/lib64/libc.so.6 /usr/lib64/libde265.so.0 /usr/lib64/libjavascriptcoregtk-4.0.so.18 /usr/lib64/libm.so.6 /usr/lib64/libmvec.so.1 /usr/lib64/libsodium.so.23 /usr/lib64/libx265.so.199 $ for lib in /usr/lib64/*.so.{[0-9],[0-9][0-9],[0-9][0-9][0-9]}; do readelf -a --wide $lib | grep _avx >/dev/null && echo $lib; done /usr/lib64/ld-linux-x86-64.so.2 /usr/lib64/ld-lsb-x86-64.so.3 /usr/lib64/libcblas.so.3 /usr/lib64/libc.so.6 /usr/lib64/libfftw3.so.3 /usr/lib64/libjavascriptcoregtk-4.0.so.18 /usr/lib64/liblapacke.so.3 /usr/lib64/libm.so.6 /usr/lib64/libmvec.so.1 /usr/lib64/libopenblas_pthreads.so.0 /usr/lib64/libopenblas.so.0 /usr/lib64/libsodium.so.23 /usr/lib64/libvmaf.so.1 $ for lib in /usr/lib64/*.so.{[0-9],[0-9][0-9],[0-9][0-9][0-9]}; do readelf -a --wide $lib | grep _popcnt >/dev/null && echo $lib; done /usr/lib64/libjavascriptcoregtk-4.0.so.18 /usr/lib64/libzvbi.so.0

Of course to be fair these are specialized libraries in some sense, but as I've argued in another thread, maybe the gap between SSE2 and SSE4.2 only helps in special circumstances. Certainly these code paths are small relative to the overall size of /usr/lib64, but they likely make up a considerable portion of a typical user's run time. (What the grepping didn't find are libraries like libavcodec.so or libx264.so with AVX instructions, and I presume most video codecs use some kind of acceleration.) When I'm watching a video on Firefox (with the default media.ffmpeg.vaapi.enabled = false) and run "perf record" on the RDD process, I see lots of AVX instructions. Which is to say: yes, newer instructions aren't widely used, but they're widely used where it matters. So at least I don't have the feeling that my new stuff (if you can count AVX as new) is not being used properly. Aaron

On 8/7/22 19:55, Nikolai Nikolaevskii wrote:

Larry Len Rainey [05 aug 22 13:14] wrote:

...

Why not just do the -v3 only on Tumbleweed x86_64 and leave Leap as it is. Those that want speed probably are on Tumbleweed and those on "antique hardware" can still run Leap. To me that seems like a great compromise.

SUSE ALP (~ SLE 16, June 2024) supposedly will be x86-64-v3 only. Derived openSUSE Leap (16.x?) possibly will be x86-64-v2 only. Tumbleweed can be used for "antique" hardware.

To clarify if SUSE ALP ends up being x86-64-v3 then whatever openSUSE derives will likely also follow that as the default setup as its likely we would probably still use the SLE binaries directly. Where however we have a bit more flexibility is if there was the demand we could rebuild all the packages as an x86-64-v1 "port" in the same way we currently rebuild armv7l for Leap 15. It should also be said SUSE hasn't made a final decision on which version of X86_64 they'll use for ALP, I believe this is still being evaluated. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

Cristian Rodríguez wrote:

On Thu, Aug 4, 2022 at 12:40 PM Martin Jambor mjambor@suse.cz wrote:

...

Over the weekend, I have quickly gathered some SPEC CPU benchmarks results comparing the different x86_64 versions: https://jamborm.github.io/spec-2022-07-29-levels/index.html Huh, Martin ..thanks for taking the time to do this..it was disappointing to the extreme. So is this a case of a) most code been written "unaware" of vectorization b) the compiler is not aggressive enough c) we do not get warnings of this kind of code or all of them..

d) (4th option): code is already using new instructions via system components https://www.phoronix.com/news/Glibc-Dropping-SSSE3-Paths For clean comparison disable some CPU instructions via BIOS settings or other way.

Martin Jambor writes:

Hello,

On Fri, Aug 05 2022, Dan Čermák wrote:

...

dieter writes:

...

Hi,

On Thu, 04 Aug 2022 18:40:23 +0200 Martin Jambor wrote:

...

...

Are there some data available about the actual benefit of this change?

Over the weekend, I have quickly gathered some SPEC CPU benchmarks results comparing the different x86_64 versions:

https://jamborm.github.io/spec-2022-07-29-levels/index.html

thanks, very interesting results. I admit this gain is even less than I expected.

Yes, the gain is small, but a few percent gain will result in e.g. HPC users to pick another distribution. But yeah, the gain in these benchmarks is really surprisingly small.

please note that I measured SPEC only because I already have the setup to run it quickly and then compare the results. The results show some benefits, for example the improvements in 531.deepsjeng_r is most likely because of POPCNT and the software you use may contain more such opportunities to utilize these "new" instructions. And I do think that at some point old HW just should not be considered a blocker to progress - and more than a decade seems like old to me.

But while I am myself somewhat undecided about switching to -v2, I would like the openSUSE community to look at the -v3 improvements too, because the floating-point ones are IMHO big. ImageMagick alone runs 20% faster and that means that compiling for older HW wastes a lot of compute power (and thus a lot of energy too) on new machines.

Can we estimate when we will want to switch to -v3? (I assume we don't want to do it now.) In two years? Four years? Six years? Never?

I think that arriving at some consensus would benefit not just planners at both openSUSE and SUSE but also users still relying on machines that do not support -v3 (I regularly use one too and also would like to keep it for a few more years :-).

I will object to switching to x86_64 v3 as the only option in the foreseeable future (at least for the next decade), because this baseline just got introduced and people will still be using perfectly working machines whom we would cutoff. I consider this a disservice to our users given that this is a *technical* problem that should be fixed. Cheers, Dan -- Dan Čermák Software Engineer Development tools SUSE Software Solutions Germany GmbH Frankenstrasse 146 90461 Nürnberg Germany (HRB 36809, AG Nürnberg) Managing Director/Geschäftsführer: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman

Hello, On Wed, Aug 10 2022, Dan Čermák wrote:

Martin Jambor writes:

...

Hello,

On Fri, Aug 05 2022, Dan Čermák wrote:

...

dieter writes:

...

Hi,

On Thu, 04 Aug 2022 18:40:23 +0200 Martin Jambor wrote:

...

...

Are there some data available about the actual benefit of this change?

Over the weekend, I have quickly gathered some SPEC CPU benchmarks results comparing the different x86_64 versions:

https://jamborm.github.io/spec-2022-07-29-levels/index.html

thanks, very interesting results. I admit this gain is even less than I expected.

Yes, the gain is small, but a few percent gain will result in e.g. HPC users to pick another distribution. But yeah, the gain in these benchmarks is really surprisingly small.

please note that I measured SPEC only because I already have the setup to run it quickly and then compare the results. The results show some benefits, for example the improvements in 531.deepsjeng_r is most likely because of POPCNT and the software you use may contain more such opportunities to utilize these "new" instructions. And I do think that at some point old HW just should not be considered a blocker to progress - and more than a decade seems like old to me.

But while I am myself somewhat undecided about switching to -v2, I would like the openSUSE community to look at the -v3 improvements too, because the floating-point ones are IMHO big. ImageMagick alone runs 20% faster and that means that compiling for older HW wastes a lot of compute power (and thus a lot of energy too) on new machines.

Can we estimate when we will want to switch to -v3? (I assume we don't want to do it now.) In two years? Four years? Six years? Never?

I think that arriving at some consensus would benefit not just planners at both openSUSE and SUSE but also users still relying on machines that do not support -v3 (I regularly use one too and also would like to keep it for a few more years :-).

I will object to switching to x86_64 v3 as the only option in the foreseeable future (at least for the next decade), because this baseline just got introduced and people will still be using perfectly working machines whom we would cutoff. I consider this a disservice to our users given that this is a *technical* problem that should be fixed.

I'll be happy if I am proven wrong but I disagree that this is a "problem" that can be "fixed" into non-existence (except if we provide both -v1 and -v3). Rather than a problem - technical or otherwise - we are facing a trade-off that is inherent in the nature of the situation. Either we do a "disservice" to users of old machines because their n years (and ticking) old machines will stop working or we do a "disservice" to users of new machines who have workloads which run slower. The answer may well be that even in 2030 it will be more important that machines from 2018 work but we should acknowledge that we have decided to impose the price on a different set of our users. Alternatively openSUSE can somehow cough up the resources to provide an additional distribution flavor - where it will bear the hardware and electricity cost of the trade-off. Or we can decide that having multiple x86_64 flavors is more important than having a 32bit version and stop building i686, which would be a disservice to all the happy users of that. Another trade-off. The trade-off can be made smaller and indeed is by many projects dynamically dispatching into routines specialized for a given ISA version or even specific CPU. For many users this can cover all of their interesting workload, for example if their only use for AVX2 is video codecs. But that is not the case for all. Real ImageMagick is probably a good example but I have not checked thoroughly. Implementing dynamic dispatching in projects which don't have it basically requires that upstream authors of the projects do it and it may often be a lot of work and quite some complexity to be dealt with forever. I hope the above makes sense, regardless of anyone's opinion on what we should or shouldn't do or when. Martin

On Wed, Aug 10, 2022 at 06:33:29PM +0200, Martin Jambor wrote:

Hello,

On Wed, Aug 10 2022, Dan Čermák wrote:

...

Martin Jambor writes:

...

Hello,

On Fri, Aug 05 2022, Dan Čermák wrote:

...

dieter writes:

...

Hi,

On Thu, 04 Aug 2022 18:40:23 +0200 Martin Jambor wrote:

...

> Are there some data available about the actual benefit of this > change?

Over the weekend, I have quickly gathered some SPEC CPU benchmarks results comparing the different x86_64 versions:

https://jamborm.github.io/spec-2022-07-29-levels/index.html

thanks, very interesting results. I admit this gain is even less than I expected.

Yes, the gain is small, but a few percent gain will result in e.g. HPC users to pick another distribution. But yeah, the gain in these benchmarks is really surprisingly small.

please note that I measured SPEC only because I already have the setup to run it quickly and then compare the results. The results show some benefits, for example the improvements in 531.deepsjeng_r is most likely because of POPCNT and the software you use may contain more such opportunities to utilize these "new" instructions. And I do think that at some point old HW just should not be considered a blocker to progress - and more than a decade seems like old to me.

But while I am myself somewhat undecided about switching to -v2, I would like the openSUSE community to look at the -v3 improvements too, because the floating-point ones are IMHO big. ImageMagick alone runs 20% faster and that means that compiling for older HW wastes a lot of compute power (and thus a lot of energy too) on new machines.

Can we estimate when we will want to switch to -v3? (I assume we don't want to do it now.) In two years? Four years? Six years? Never?

I think that arriving at some consensus would benefit not just planners at both openSUSE and SUSE but also users still relying on machines that do not support -v3 (I regularly use one too and also would like to keep it for a few more years :-).

I will object to switching to x86_64 v3 as the only option in the foreseeable future (at least for the next decade), because this baseline just got introduced and people will still be using perfectly working machines whom we would cutoff. I consider this a disservice to our users given that this is a *technical* problem that should be fixed.

I'll be happy if I am proven wrong but I disagree that this is a "problem" that can be "fixed" into non-existence (except if we provide both -v1 and -v3).

Rather than a problem - technical or otherwise - we are facing a trade-off that is inherent in the nature of the situation. Either we do a "disservice" to users of old machines because their n years (and ticking) old machines will stop working or we do a "disservice" to users of new machines who have workloads which run slower.

I would like to see this 'disservice' to users of newer hardware somehow clarified. From the benchmarks AFAICT v2 is not even overall gain, and while v3 is very slightly better I doubt it would be noticable for, say, a desktop. HPC is not real reason either because 1) it's moving towards using GPUs anyway 2) if you still do calculations on CPU you are likely going to use specialized libraries that have dynamic CPU detection, anyway Finally if you want to squeeze last fraction of a percent of performance out of your CPU you will have to taylor the compiler options to your particular CPU, and do your own bemchmarking - the prebuilt binaries are not really heplfule for that. And there is OBS for packaging your own. I have not heard about a specific case where not having some more recent instructions brings in complex code patching, extra maintenance, or any such problem. In any case if such use case does exist I would like to hear which specific CPU feature enables the optimization, not some random bag of unrelated CPU features called x86_64-vn.

The answer may well be that even in 2030 it will be more important that machines from 2018 work but we should acknowledge that we have decided to impose the price on a different set of our users.

If we cannot point out any noticeable price the users of newer CPUs are actually paying for the old CPU support then we shoud keep the support even in 2030 I suppose. Although I have my doubts about that - CPU obsolescence generally happens in generations by some developments that render the CPU generation practically insufficient in some way, and while the Core(2) CPUs are usable today they are nearing their practical usability EOL.

Alternatively openSUSE can somehow cough up the resources to provide an additional distribution flavor - where it will bear the hardware and electricity cost of the trade-off. Or we can decide that having multiple x86_64 flavors is more important than having a 32bit version and stop building i686, which would be a disservice to all the happy users of that. Another trade-off.

The trade-off can be made smaller and indeed is by many projects dynamically dispatching into routines specialized for a given ISA version or even specific CPU. For many users this can cover all of their interesting workload, for example if their only use for AVX2 is video codecs.

But that is not the case for all. Real ImageMagick is probably a good example but I have not checked thoroughly. Implementing dynamic dispatching in projects which don't have it basically requires that upstream authors of the projects do it and it may often be a lot of work and quite some complexity to be dealt with forever.

I appreciate that ImageMagick execution speed is a nice compiler benchmark but I have to ask how many people run ImageMagick all day. Sure, there are some web services that might use it for image processing but I have no idea how many are out there. I vaguely remember some specialized C++ image processing libraries for image recognition but I also remember that I had to build them myself, anyway. Thanks Michal

On Wed, Aug 10, 2022 at 7:40 PM Michal Suchánek wrote:

Yes, Gentoo is notorious, and at least a few others exist.

okay i didnt come around too many linux distros i guess.

It's impractical because of compilation time, especially on old machines that would be left out by requiring newer CPU features. Many would not be even able to compile many packages at all due to insufficient memory.

well doesnt these whole arguments contradict the reasoning that it would be exactly the new fancy hardware owners of incapable to compile their stuff and optimize it to their needs and new(er) components? i think it would make sense to provide basic x86_64 binaries or a minimal os booting up stuff and the compile (not that often) the rest of stuff. or provide everything in the basic platform binaries and compile all updates patches etc only the files and packages that come as updates and renew themselves or as the user chooses so. are varios distros not as well trying to attempt to go for reproducible builds? this would need to implement the counter-checking on the users hardware anyways, or am i misunderstanding the reproducible-builds stuff? i think even (open)suse has some reproducible builds project if i am not mistaken. maybe i should take a look at gentoo then or at lfs. ty.

On Wed, Aug 10, 2022 at 03:27:33PM -0400, Felix Miata wrote:

Michal Suchánek composed on 2022-08-10 19:32 (UTC+0200):

...

while the Core(2) CPUs are usable today they are nearing their practical usability EOL.

I don't see it as CPU per se. I have multiple Core2Duos, an i5-11400 and various in between. Newer CPU generations mainly mean more threads per price point. CPU

Each new iteration tends to be about 10-20% fater for some definition of faster which accumulates over time. If raw CPU power was the problem we would not get new Celerons and APUs, though.

speed range hasn't been evolving. The big differences I see are all I/O-related, DDR2 vs. DDR3 vs. DDR4, NVME vs SATA, maximum supported RAM 4G vs. various multiples of 4G, and availability of USB3 and its ostensible competitors. Basic

Sure, and at some point some of the IO capabilities become essential rendering the systems that don't have them impractical to use. Different video interfaces and screen size limits exist, too.

internet, writing and spreadsheet functionality still exists in 2G max RAM PCs,

Which is very close to the 4G limit mentioned above and potential bottleneck in the near future.

just not with flagship desktop environments or lots of browser tabs, PCs for people with minimal budgets, depending on what they can get for little to no cost.

Some Core2 systems go up to 16G but getting the memory is very expenive so you generally have to live with whatever is installed there or get a new system. Note that on many old systems you would require memory size that was atypical at the time the systems were mainstream to get the maximum possible memory so it's unlikely to be available second hand. Thanks Michal

On Wed 2022-08-10, cagsm wrote:

are varios distros not as well trying to attempt to go for reproducible builds? this would need to implement the counter-checking on the users hardware anyways, or am i misunderstanding the reproducible-builds stuff?

The point of reproducible builds is that they are reproducible which (ideally) means identical. Building on local systems with "random" compiler flags goes against that and opens a different can of worms. I am not saying it doesn't have benefits, and some are really interested in that. It's just something different than what openSUSE distros aim for - which includes ease of consumption. Gerald

Hi Martin, Martin Jambor writes:

Hello,

On Wed, Aug 10 2022, Dan Čermák wrote:

...

Martin Jambor writes:

...

Hello,

On Fri, Aug 05 2022, Dan Čermák wrote:

...

dieter writes:

...

Hi,

On Thu, 04 Aug 2022 18:40:23 +0200 Martin Jambor wrote:

...

> Are there some data available about the actual benefit of this > change?

Over the weekend, I have quickly gathered some SPEC CPU benchmarks results comparing the different x86_64 versions:

https://jamborm.github.io/spec-2022-07-29-levels/index.html

thanks, very interesting results. I admit this gain is even less than I expected.

Yes, the gain is small, but a few percent gain will result in e.g. HPC users to pick another distribution. But yeah, the gain in these benchmarks is really surprisingly small.

please note that I measured SPEC only because I already have the setup to run it quickly and then compare the results. The results show some benefits, for example the improvements in 531.deepsjeng_r is most likely because of POPCNT and the software you use may contain more such opportunities to utilize these "new" instructions. And I do think that at some point old HW just should not be considered a blocker to progress - and more than a decade seems like old to me.

But while I am myself somewhat undecided about switching to -v2, I would like the openSUSE community to look at the -v3 improvements too, because the floating-point ones are IMHO big. ImageMagick alone runs 20% faster and that means that compiling for older HW wastes a lot of compute power (and thus a lot of energy too) on new machines.

Can we estimate when we will want to switch to -v3? (I assume we don't want to do it now.) In two years? Four years? Six years? Never?

I think that arriving at some consensus would benefit not just planners at both openSUSE and SUSE but also users still relying on machines that do not support -v3 (I regularly use one too and also would like to keep it for a few more years :-).

I will object to switching to x86_64 v3 as the only option in the foreseeable future (at least for the next decade), because this baseline just got introduced and people will still be using perfectly working machines whom we would cutoff. I consider this a disservice to our users given that this is a *technical* problem that should be fixed.

I'll be happy if I am proven wrong but I disagree that this is a "problem" that can be "fixed" into non-existence (except if we provide both -v1 and -v3).

No, it cannot be fixed into non-existance by some technical solution ;-) But I think it could be addressed by adding support for different x86_64 baselines in rpm, zypper & dnf, podman, docker, etc., so that we can get away with rebuilding just a small subset of packages, where a newer baseline actually *matters*. That will not result in a terrible strain on our workers, while also allowing users of old hardware to still use openSUSE. Cheers, Dan -- Dan Čermák Software Engineer Development tools SUSE Software Solutions Germany GmbH Frankenstrasse 146 90461 Nürnberg Germany (HRB 36809, AG Nürnberg) Managing Director/Geschäftsführer: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman

On Thu, Aug 11, 2022 at 1:44 AM Gerald Pfeifer wrote:

On Wed 2022-08-10, cagsm wrote:

...

are varios distros not as well trying to attempt to go for reproducible builds? this would need to implement the counter-checking on the users hardware anyways, or am i misunderstanding the reproducible-builds stuff?

The point of reproducible builds is that they are reproducible which (ideally) means identical.

Building on local systems with "random" compiler flags goes against that and opens a different can of worms. I am not saying it doesn't have benefits, and some are really interested in that. It's just something different than what openSUSE distros aim for - which includes ease of consumption.

Reproducible builds (as Debian defines them) have a weaker value proposition when the distribution doesn't accept binary builds from random computers into the repositories that could be built any which way. If you have total control of the inputs used to do the build, and have the ability to replay build environments (which not all distros have the ability to do), then specifically spending *more* time on this is not as important. That's why Debian and openSUSE spend time on this, while you don't see Fedora doing it so much, for example. That's not to say they don't *care* about reproducible builds, but that the motivation to do more is significantly lessened due to what they already have. And like Gerald said, screwing around with build flags violates reproducible builds too. :) -- 真実はいつも一つ！/ Always, there's only one truth!

On Thu, 11 Aug 2022 09:17:40 +0200 Dan Čermák wrote:

...

I could see a ImageMagic-V3 as an option for those with hardware that can take advantage of it and just plain ImageMagic for those that cannot.

Yes, that's what I'd like to see as well, but currently rpm does not support this natively (it could be worked around via different multiple repositories though, but I'd like a proper solution to be honest).

I do not know if it possible from the build steps to produce all the -vX binaries/DSOs in one go and package them all into one "fat" rpm. In principle it would mean to do the compile, link and install steps several times with different compiler settings and installation targets. For the rpm installation I could imagine that by default the -v1 version is installed (a link in the filesystem pointing to the included -v1 version) and a rpm-post scriptlet detects the architecture level supported by the current CPU and changes this link to the highest possible variant - for packages where this really matters. Kind regards, Dieter

On Thu, 11 Aug 2022, Jan Engelhardt wrote:

On Thursday 2022-08-11 09:15, Dan Čermák wrote:

...

But I think it could be addressed by adding support for different x86_64 baselines in rpm, zypper & dnf, podman, docker, etc., so that we can get away with rebuilding just a small subset of packages, where a newer baseline actually *matters*. That will not result in a terrible strain on our workers, while also allowing users of old hardware to still use openSUSE.

How hard could it be? glibc has been built as glibc and glibc.i686 for a long time (and now it's _multibuilded, but still doing the same).

But glibc.i686 is just used as -32bit for x86_64 systems. I don't think you get glibc.i686 automagically installed on i586 systems or that there's anything preventing you from installing it when your system isn't capable of running it (which would then be quite fatal I guess ;)) Richard.

On Thu, 11 Aug 2022 08:22:14 +0000 (UTC) Richard Biener wrote:

But glibc.i686 is just used as -32bit for x86_64 systems. I don't think you get glibc.i686 automagically installed on i586 systems or that there's anything preventing you from installing it when your system isn't capable of running it (which would then be quite fatal I guess ;))

On an Athlon-XP I got automatically glibc.i686 long time ago, so it seems at this time there was a mechanism to select the best fitting version. I think I did not get kernel-pae automatically. About unsupported instructions: when software does runtime CPU dispatching properly there is no problem with including instructions not supported by the installed CPU, and therefore rpm can hardly check this. Kind regards, Dieter

On Thursday 2022-08-11 10:33, Dan Čermák wrote:

...

How hard could it be? glibc has been built as glibc and glibc.i686 for a long time (and now it's _multibuilded, but still doing the same).

The problem is that i686 and x86_64 are distinct architectures for rpm. x86_64v2 and x86_64v3 are not

rpm should have configuration directives for what's distinct and what isn't. /usr/lib/rpm/rpmrc:arch_compat: x86_64: amd64 em64t athlon noarch rpmrc:arch_compat: athlon: i686 After all, if glibc.i586 and glibc.i686 are distinct, then glibc.x86_64 and glibc.x64v3 can be distinct.

Larry Len Rainey writes:

Why not do what python and ffmpeg do for multiple versions.

We have multiple python versions and ffmpeg3 ffmpeg4 and ffmpeg5. Why not ImageMagic and ImageMagicv3?

Create a obs and obsv3 repos (non-obs and non-obsv3 too?) put the obsv3 repository in for those that want it.

Users would do this to get the v3 version:

zypper rm ImageMagic

zypper in ImageMagicv3

The executable names stay the same but the rpm gets fetched from the obsv3 repo.

Unfortunately then users can very easily shoot themselves in the foot by installing the v3 version on an old system. Our tooling simply does not understand subarchitectures/baselines. Cheers, Dan -- Dan Čermák Software Engineer Development tools SUSE Software Solutions Germany GmbH Frankenstrasse 146 90461 Nürnberg Germany (HRB 36809, AG Nürnberg) Managing Director/Geschäftsführer: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman

dieter writes:

On Thu, 11 Aug 2022 09:17:40 +0200 Dan Čermák wrote:

...

...

I could see a ImageMagic-V3 as an option for those with hardware that can take advantage of it and just plain ImageMagic for those that cannot.

Yes, that's what I'd like to see as well, but currently rpm does not support this natively (it could be worked around via different multiple repositories though, but I'd like a proper solution to be honest).

I do not know if it possible from the build steps to produce all the -vX binaries/DSOs in one go and package them all into one "fat" rpm. In principle it would mean to do the compile, link and install steps several times with different compiler settings and installation targets. For the rpm installation I could imagine that by default the -v1 version is installed (a link in the filesystem pointing to the included -v1 version) and a rpm-post scriptlet detects the architecture level supported by the current CPU and changes this link to the highest possible variant - for packages where this really matters.

I don't think this is a great idea, as the rpm size will just blow up needlessly causing more disk usage and larger downloads for *everyone*. Cheers, Dan -- Dan Čermák Software Engineer Development tools SUSE Software Solutions Germany GmbH Frankenstrasse 146 90461 Nürnberg Germany (HRB 36809, AG Nürnberg) Managing Director/Geschäftsführer: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman

On Thursday 2022-08-11 14:18, Larry Len Rainey wrote:

Why not do what python and ffmpeg do for multiple versions. We have multiple python versions and ffmpeg3 ffmpeg4 and ffmpeg5. Why not ImageMagic and ImageMagicv3?

python/ffmpeg is about different APIs, not about what instructions the code utilizes (but which is the topic of x86_64-v2).

On Thursday 2022-08-11 16:24, Michal Suchánek wrote:

...

But then rpm itself doesn't do any magic and will happily install even arm architecture packages to your x86_64 system IIRC.

Considering you can do `rpm --chroot`... and with things like qemu-via-binfmt, could actually *use* the ARM package, it make sense that rpm did not want to impose restrictions. It's a glorified cpio unpacker that keeps a record of all unpacked files in a database. zypper applies the architecture logic (like proposing glibc.i686 on 686-capable x86); that all looks like a good layering design.

Unfortunately there is no support for having the required support libraries for multiple architectures.

I see two blockers. First, arch-specific libraries would have to live in arch-specific directories, like is the case with /usr/lib/x86_64-linux-gnu on Debian. Secondly, the autodependencies should be rewritten, from $ rpm -q --provides glibc glibc = 2.35-6.1 glibc(x86-64) = 2.35-6.1 ld-linux-x86-64.so.2()(64bit) i586$ rpm -q --provides glibc ld-linux-x86-64.so.2 to glibc = 2.35-6.1 glibc(x86-64) = 2.35-6.1 ld-linux-x86-64.so.2()(x86_64) i586$ rpm -q --provides glibc ld-linux-x86-64.so.2()(i586)

On Thu, 11 Aug 2022, Jan Engelhardt wrote:

On Thursday 2022-08-11 16:24, Michal Suchánek wrote:

...

...

But then rpm itself doesn't do any magic and will happily install even arm architecture packages to your x86_64 system IIRC.

Considering you can do `rpm --chroot`... and with things like qemu-via-binfmt, could actually *use* the ARM package, it make sense that rpm did not want to impose restrictions. It's a glorified cpio unpacker that keeps a record of all unpacked files in a database. zypper applies the architecture logic (like proposing glibc.i686 on 686-capable x86); that all looks like a good layering design.

...

Unfortunately there is no support for having the required support libraries for multiple architectures.

I see two blockers.

First, arch-specific libraries would have to live in arch-specific directories, like is the case with /usr/lib/x86_64-linux-gnu on Debian.

Workaround: have the different variants conflict ...

Secondly, the autodependencies should be rewritten, from

$ rpm -q --provides glibc glibc = 2.35-6.1 glibc(x86-64) = 2.35-6.1 ld-linux-x86-64.so.2()(64bit) i586$ rpm -q --provides glibc ld-linux-x86-64.so.2

to

glibc = 2.35-6.1 glibc(x86-64) = 2.35-6.1 ld-linux-x86-64.so.2()(x86_64) i586$ rpm -q --provides glibc ld-linux-x86-64.so.2()(i586)

why? That would make it difficult to use a library as dependency the user would be able to choose from x86-64-v[123]? Richard. k

On Mon, Aug 15, 2022 at 07:21:56AM +0000, Richard Biener wrote:

On Thu, 11 Aug 2022, Dan Čermák wrote:

...

Richard Biener writes:

[..]

...

...

The suggestion of multiple repositories is just one possible technical implementation of such annotation, using system provides and package requires would be another (then with the technical issue of having multiple same named but different filenamed packages in the same repository?).

[OT] Why requires though? I don't think it's a good idea to explicitly require a library with a certain instruction set. Provides on the other hand would be a workaround, but I really think that architectures are the proper way how this should be handled.

To elaborate, the "system" would have a Provides: hw-x86_64-version = 3 for example and a package built for x86_64-v2 would then have a Requires: hw-x86_64-version >= 2. The Provides would be magically

But that does not make any sense. The x86_64 'versions' are just random bags of features that CPU vedors randomly pick to (not) include in the CPUs. If anything these dependencie hould be for specific CPU features. Thanks Michal

On Mon, Aug 15, 2022 at 09:54:51AM +0200, Jan Engelhardt wrote:

On Monday 2022-08-15 09:24, Richard Biener wrote:

...

...

First, arch-specific libraries would have to live in arch-specific directories, like is the case with /usr/lib/x86_64-linux-gnu on Debian.

Workaround: have the different variants conflict ...

That would probably sort of work, except rpm/zypper are really bad at resolving such complex dependencies. Already seen a lot of ugliness with kernel variants.

glibc.ppc64le.rpm and glibc.x86_64.rpm already conflict in practice without any additions just because they share files in the same location (/lib64/libc.so.6 and /lib64/libc.so.6).

Which is broken. There is no reason to not have both glibc.ppc64le.rpm and glibc.x86_64.rpm, they provide completely different feature. It also randomly does not conflict with glibc.armv7l or glibc.ppc for no real reason.

...

...

Secondly, the autodependencies should be rewritten, from

$ rpm -q --provides glibc ld-linux-x86-64.so.2()(64bit) i586$ rpm -q --provides glibc ld-linux-x86-64.so.2

to

ld-linux-x86-64.so.2()(x86_64) i586$ rpm -q --provides glibc ld-linux-x86-64.so.2()(i586)

why? That would make it difficult to use a library as dependency the user would be able to choose from x86-64-v[123]?

Yes, you are right. But my thought here was that

perhaps we should make sure you can't accidentally mix *totally incompatible* RPM deps (such as installing libfoo1.ppc64le on a glibc.x86_64 system)

Why shouldn't you? Just include glibc.ppc64le as well. Yes, it should not be enabled by default, and if you do enable it you should make sure you do not install any vital system tool as non-native binary because you won't be able to boot otherwise but other than that non-native tools are perfectly usable.

before even tackling the one-direction-is-compatible characteristic of ({somearch, somearchv2}).

Architecture 'version' is nonense. It's just random set of unrelated features that CPU vendors sometimes pick to include, sometimes not. x86 CPUs manufactured last year are not even all x86_64-v3. Most may have most of the CPU features that were arbitrarily picked as 'v3' but not all of them have all. There is similar problem with ppc64le definition - at its inception IBM arbitrarily decided that ppc64le platform compliant systems should have HTM, and now Power10 does not have it although it has a number of other features not available at the time so it's technically not a compilant ppc64le platform. Thanks Michal

I know I'm late to the party, but I'm still running Leap on a HP microserver G7 bought in 2011, which has this: Vendor ID: AuthenticAMD CPU family: 16 Model: 6 Model name: AMD Athlon(tm) II Neo N36L Dual-Core Processor

/tmp/test.awk CPU supports x86-64-v1

I believe there are still a lot these around and will be in 2025 (or whenever Leap 15 is EOL). I'm sure the project has some statement on sustainability somewhere; this decision would be a good opportunity to show that we are serious about it. Best Martin

On Thu, Aug 11, 2022 at 10:59 AM Jan Engelhardt wrote:

On Thursday 2022-08-11 16:24, Michal Suchánek wrote:

...

...

But then rpm itself doesn't do any magic and will happily install even arm architecture packages to your x86_64 system IIRC.

Considering you can do `rpm --chroot`... and with things like qemu-via-binfmt, could actually *use* the ARM package, it make sense that rpm did not want to impose restrictions. It's a glorified cpio unpacker that keeps a record of all unpacked files in a database. zypper applies the architecture logic (like proposing glibc.i686 on 686-capable x86); that all looks like a good layering design.

...

Unfortunately there is no support for having the required support libraries for multiple architectures.

I see two blockers.

First, arch-specific libraries would have to live in arch-specific directories, like is the case with /usr/lib/x86_64-linux-gnu on Debian.

Secondly, the autodependencies should be rewritten, from

$ rpm -q --provides glibc glibc = 2.35-6.1 glibc(x86-64) = 2.35-6.1 ld-linux-x86-64.so.2()(64bit) i586$ rpm -q --provides glibc ld-linux-x86-64.so.2

to

glibc = 2.35-6.1 glibc(x86-64) = 2.35-6.1 ld-linux-x86-64.so.2()(x86_64) i586$ rpm -q --provides glibc ld-linux-x86-64.so.2()(i586)

Indeed. In fact, I had proposed such enhancements to the elf autodeps code in RPM quite some time ago[1]. However, it doesn't solve subarch stuff, mostly because it wasn't a thing I had to think about when I wrote it... [1]: https://github.com/rpm-software-management/rpm/pull/1038 -- 真実はいつも一つ！/ Always, there's only one truth!

Hello openSUSE! Talking about benchmarks, I spoke with Product Management earlier today, and we want to have data. PM seems to be expecting a larger gain than I see mentioned here. So the plan is to run a benchmark with some test suite (perhaps https://www.phoronix-test-suite.com/ ?). As of now, SUSE:ALP does not seem to be built with any -v3 flags. Once this is the case, we can use such an image and compare it to, e.g., v1-based TW minimal install. I expect that this will be later in September. Another outcome: We have to have a community -v3 ALP image, to support the non-paid to paid "seamless" migration use case for ALP. As mentioned before, PM doesn't block us from Rebuilding ALP as v1 or v2 in an openSUSE:Step- like fashion (this was also discussed on Rel-eng call two weeks ago or similar). Doug also had an idea to ask via survey@ who has -v3 hw available etc. So perhaps that would help. Investing into -v2 ALP rebuild will take some effort, and I want to make sure that such investment is backed with data and also understand the gain, etc. Lubos On Fri, 2022-08-05 at 07:59 +0200, Dan Čermák wrote:

dieter writes:

...

Hi,

On Thu, 04 Aug 2022 18:40:23 +0200 Martin Jambor wrote:

...

...

Are there some data available about the actual benefit of this change?

Over the weekend, I have quickly gathered some SPEC CPU benchmarks results comparing the different x86_64 versions:

https://jamborm.github.io/spec-2022-07-29-levels/index.html

thanks, very interesting results. I admit this gain is even less than I expected.

Yes, the gain is small, but a few percent gain will result in e.g. HPC users to pick another distribution. But yeah, the gain in these benchmarks is really surprisingly small.

Cheers,

Dan

On Fri, 2022-08-05 at 07:59 +0200, Dan Čermák wrote:

dieter writes:

...

Hi,

On Thu, 04 Aug 2022 18:40:23 +0200 Martin Jambor wrote:

...

...

Are there some data available about the actual benefit of this change?

Over the weekend, I have quickly gathered some SPEC CPU benchmarks results comparing the different x86_64 versions:

https://jamborm.github.io/spec-2022-07-29-levels/index.html

thanks, very interesting results. I admit this gain is even less than I expected.

Yes, the gain is small, but a few percent gain will result in e.g. HPC users to pick another distribution. But yeah, the gain in these benchmarks is really surprisingly small.

Cheers,

Dan

Richard Biener composed on 2022-07-28 11:44 (UTC):

there's been rumors and confusion about ALP raising the x86_64 architecture level to v3 which requires CPUs with AVX2 support and how this will affect openSUSE Factory. Sofar openSUSE Factory was supposed to be not affected but I think it makes sense to re-consider the current -v1 usage given what other distros are doing.

Thus I would propose raising the x86_64 architecture level to x86-64-v2 for openSUSE Factory. v2 requires CMPXCHG16B, [SL]AHF, POPCNT and SSE4_2 (and the predecessor SSE extensions) but not AVX, BMI or FMA.

Intel has apparently released v2 as recently as 2017-Q1 at least (Kaby Lake): https://ark.intel.com/content/www/us/en/ark/products/97453/intel-pentium-pro... I purchased mine new in box December 2017. # lscpu | egrep 'S Model|lags' BIOS Model name: Intel(R) Pentium(R) CPU G4600 @ 3.60GHz... Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl est tm2 ssse3 sdbg cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave rdrand lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp fsgsbase tsc_adjust smep erms invpcid mpx rdseed smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d # lscpu | grep lags | grep avx # No avx or avx2. That means this PC won't be 10 years old yet when 15.5 support ends. If ALP requires v3, that means my (currently) 3rd newest PC will no longer be supported without a switch to TW, or someone else's stable distro release; or this CPU supports avx/avx2 but doesn't advertise it. My other Kaby Lake, same age, doesn't have this limitation: # inxi -Cxx CPU: Info: dual core model: Intel Core i3-7100T bits: 64 type: MT MCP arch: Kaby Lake level: v3 rev: 9 cache: L1: 128 KiB L2: 512 KiB L3: 3 MiB Speed (MHz): avg: 800 min/max: 800/3400 cores: 1: 800 2: 800 3: 800 4: 800 bogomips: 27199 Flags: avx avx2.... -- Evolution as taught in public schools is, like religion, based on faith, not based on science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata

On Sun, 11 Sep 2022, Felix Miata wrote:

Richard Biener composed on 2022-07-28 11:44 (UTC):

...

there's been rumors and confusion about ALP raising the x86_64 architecture level to v3 which requires CPUs with AVX2 support and how this will affect openSUSE Factory. Sofar openSUSE Factory was supposed to be not affected but I think it makes sense to re-consider the current -v1 usage given what other distros are doing.

...

Thus I would propose raising the x86_64 architecture level to x86-64-v2 for openSUSE Factory. v2 requires CMPXCHG16B, [SL]AHF, POPCNT and SSE4_2 (and the predecessor SSE extensions) but not AVX, BMI or FMA.

Intel has apparently released v2 as recently as 2017-Q1 at least (Kaby Lake):

Yes, Intel is known for too much product segmentation by fusing off perfectly usable features from some SKUs. Also for Comet Lake btw (Celeron G5900T). I've just not seen it being done for the Ice Lake family (Tiger Lake, Alder Lake), but you'll never know with Intel. So at least The Celeron G6900T (Alder Lake) now has AVX2 _not_ fused off (ok, they fused off AVX512 instead). Richard.

added to my previous post ... if its true that this only affects code at compile time based on instructions sets available then the use of _xvx package sufixes should mean in real terms that some apps/libs may not have any advantages in a v2 or v4 compile as they do not take advantage of v2 or v4 instructions to any advantage meaning they may only ever need to be v0 or v1 versions of them provided and held on the servers keeping it so the most relivent higest versions are only ever installed while keeping support for lower cpu models thoughts ? david (achiestdragon)

Ben Holmes wrote:

...

I wonder how many packages *really* benefit (that is, enough to warrant keeping them) from v3/v4 instructions?

well the issue thats got my attention is on one hand there saying a move to v2 but at the same time saying the performance benifits are with v3 , at the same time though removing v0 /v1 hardware in favour of v2 is not gong to provide the performance increase to v3/v4 users making the whole issue rather pointless and putting a vast number of users in a position they need new hardware , it maybe fine for redhat to go this route since they market for new machines and business users (dell for example provides redhat as a o/s option on new machines ) but tumbleweed is mostly home/hobby use and most installs i suspect are on ex lease kit or self builds , its not fair on users to be left out because the cpu gets removed from the current versions , so although i see the need to provide more optimised code for newer cpus there dose need to be provision to suport the other cpus , the first most users will know is when they do there weekly zapper -dup , and reboot to find there machine is no longer supported , not that it will tell them why its not running , not that some will know what x86_64 vX means, but this also needs to be avoided , at the same time implimenting the v0,1,2,3,4,n version system now would allow not just v2 optimised code , but allow for v3 and 4 optimised code to be introduced then and only then (by looking at actual server download logs) will you see for shure if removing v0, or v1 is going to affect the user base practicaly its should not be hard to impliment eather a xvx sufix or i86_64_x(n) sufix and provide zypper support for that just like the whole x86_32 to x86_64 issue this has been brewing for some time , but untill now its just been left at the same time its posible to impliment it so it choses the best version to suit the chipset and update say a v0 to v2 even to v3 or v4 as the binaries are added to the servers the issue i think is ether server space , given the extra binarys needed , or workload of the compile farm for producing those binaries , (could we get an answer on this ?) by the sounds of it unless there is an advantage performance wise there should be no need to produce a v2 version of code over a v0 - v1 version , but there is more a need to provide a v3 version in some cases , and maybe in others a v4 also while still providing lower version support if this is not resolved now then i suspect it wont be too long before v2 gets dropped in favor of v3, for the same reasons users will not forget and swich distros to ones that do still support there hardware , if you dont know how many users then i would be cautious sorry to ramble on about it , but i know of a number of users that will end up with no machine to use after the switch to v2 minimum when it trickles down to tumbleweed , with no way to afford newer machines and no idea what they would need to get (second hand) if they could afford to , let alone be in a position to know that this is comming untill it does , probabaly account for about 5 to 10 machines made into e waste on that update, just on a small number of users i know of, and about 4 machines of my own (that i may of been able to hand down to them to help , other than there also going to be unsuported ) rather it only highlights the lack of actual performance increases there have been in the last 20 years, and "bricking " v0-v1 cpus for only marganal performance advantages is not going to be looked on kindly by the community that is mostly using legacy hardware of that type, and at the same time not providing the v3 and v4 optimisations is not going to be taken lightly by those that do have such cpus and are not seeing any gains

thinking more , diferent file lists sould work, then it also should be posible just to have the dirs containg rpms compiled for spesific version where available , without having to simlink other versions to that , each file list xml could be autoscripted (long term) to produce the list spesific to each sub arch without the need to simlink , at least it make managment of it easyer also keeping sub version spesific files in there own dir and having the file lists to determin the apropriate ones to install for that version , it will not only allow for v2 optimisations to be introduced but we could push ahead with v3 and 4 optimisations also in the same stroke without a need for alot of hard code changes to zypper or yast to achive it