http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Michiel Janssens <michiel@nexigon.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |michiel@nexigon.net -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c3 Philip Raets <philip.raets@telenet.be> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |philip.raets@telenet.be --- Comment #3 from Philip Raets <philip.raets@telenet.be> --- Same here: Dell Optiplex 790 Running GNOME on XOrg Tried running without extensions, but it is the same. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Bruce Rogers <brogers@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |brogers@suse.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c5 --- Comment #5 from Robert Munteanu <rombert@apache.org> --- I installed the following packages and the responsiveness has greatly improved: gjs-1.54.2-163.1.x86_64.rpm gnome-shell-3.30.1-393.1.x86_64.rpm gnome-shell-browser-plugin-3.30.1-393.1.x86_64.rpm gnome-shell-calendar-3.30.1-393.1.x86_64.rpm gnome-shell-devel-3.30.1-393.1.x86_64.rpm gnome-shell-lang-3.30.1-393.1.noarch.rpm libgjs0-1.54.2-163.1.x86_64.rpm The CPU spikes are gone from the gnome-shell process as monitored via top -p $(pidof gnome-shell| tr ' ' ',') . I'm intentionally not removing the needinfo flag as my laptop did not have smooth animations before the update either so I'm not sure if the problem is entirely gone. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c7 Atri Bhattacharya <badshah400@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |badshah400@gmail.com --- Comment #7 from Atri Bhattacharya <badshah400@gmail.com> --- Installing gnome-shell, libgjs0, and gjs helps (a lot!) but I still see severe choppiness when the shell is in application overview mode. For instance, when I try clicking on the YaST2 Modules, it takes of the order of 5 seconds to unfold. When in application overview mode, mouse movement also becomes choppy generally. Plus, video playing is still choppy -- not as much as it was with TW 20181018 but nowhere near as smooth as 3.28.x levels before that. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c9 --- Comment #9 from Michiel Janssens <michiel@nexigon.net> --- I can confirm comment #7. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Oliver Kurz <okurz@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |okurz@suse.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Olivier Nicolas <o.nicolas@skynet.be> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |o.nicolas@skynet.be -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c11 --- Comment #11 from Olivier Nicolas <o.nicolas@skynet.be> --- Running GNOME on XOrg with Nvidia blob After installing packages from comment 4 gjs-1.54.2-163.1.x86_64.rpm gnome-shell-3.30.1-393.1.x86_64.rpm gnome-shell-calendar-3.30.1-393.1.x86_64.rpm gnome-shell-lang-3.30.1-393.1.noarch.rpm libgjs0-1.54.2-163.1.x86_64.rpm the performance is restored. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c13 --- Comment #13 from Larry Dewey <ldewey@suse.com> --- After installing the suggested packages, there seems to be a major performance increase in the standard UI, but when using the activities overview, there is still a lot of lag and slowness. It makes it much more usable, but it still needs work. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c14 Nicolás Luciano Bértolo <nicolasbertolo@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nicolasbertolo@gmail.com --- Comment #14 from Nicolás Luciano Bértolo <nicolasbertolo@gmail.com> --- The Wayland session is still unusable, though much better than before, it is significantly worse than 3.28. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Dingzhong Chen <wsxy162@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |wsxy162@gmail.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c17 Ivan Petrovic <ivanpetrovic96.dev@icloud.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ivanpetrovic96.dev@icloud.c | |om --- Comment #17 from Ivan Petrovic <ivanpetrovic96.dev@icloud.com> --- Now in 20181022 update, nautillus is too slow to open but everything else is fine. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c19 --- Comment #19 from Dead Mozay <windowskaput@gmail.com> --- I observe the same problems even with 20181029 snapshot https://youtu.be/l_fZvjqnuYM https://youtu.be/6vMaKK346No http://paste.opensuse.org/34989715 -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Michael Puls <michael.puls@gmx.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |michael.puls@gmx.de -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c22 --- Comment #22 from Nicolás Luciano Bértolo <nicolasbertolo@gmail.com> --- I think it is worth noting that this does not happen in Ubuntu 18.10 or Fedora 29, which use gnome 3.30. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c24 --- Comment #24 from Robert Munteanu <rombert@apache.org> --- (In reply to Dead Mozay from comment #23)

It seems the problem is only with Intel processors, only openSUSE, there is no problem with other distributions, I checked

Does that mean Intel CPUs have the problem and AMD CPUs don't? Or does it mean systems using the Intel iGPU have the problem and systems using discrete AMD / NVidia GPUs don't? -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c26 --- Comment #26 from Dead Mozay <windowskaput@gmail.com> --- I also reassembled all the components that could have a negative impact on the performance of the gnome, and 3d, xorg, mesa, gnome-shell, mutter, gjs with patches fedora, even compile the kernel from the source code is fresher without patches, it seems to be better, but sometimes it slowed down anyway, apparently somewhere else there is a problem, I could not find anything else -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c28 --- Comment #28 from Robert Munteanu <rombert@apache.org> --- (In reply to Dead Mozay from comment #27)

in general, as far as I can tell, the problem is not in the gnome, but somewhere in the depths of the system packages. after compiling the kernel from the source code, the problem with hyper-threading disappeared, the gnome with the enabled hyper-threading showed slideshows instead of animations

Interesting. Is this the same kernel version as in Tumbleweed? Did you use the same config? This might help tracking down the root cause. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Yifan Jiang <yfjiang@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |gwang@suse.com, | |yfjiang@suse.com, | |zcjia@suse.com, | |zgao@suse.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c31 --- Comment #31 from Robert Munteanu <rombert@apache.org> --- (In reply to Dead Mozay from comment #30)

Created attachment 790294 [details] kernel config

It seems that it was possible to solve the problem, at least on the Intel HD animations, they became smooth, so that someone else would check it, you need to rebuild the kernel with the attached config

Thanks for the config! I tried to compare it with the one shipped in TW, but unfortunately there are too many differences: $ zcat /proc/config.gz | diff - kconfig-suggested | wc -l 4621 I'll try and test your assumption that this is specific to Intel CPUs soon though, I have an AMD machine that I have not updated yet. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c32 --- Comment #32 from Dead Mozay <windowskaput@gmail.com> --- (In reply to Robert Munteanu from comment #31)

(In reply to Dead Mozay from comment #30)

...

Created attachment 790294 [details] kernel config

It seems that it was possible to solve the problem, at least on the Intel HD animations, they became smooth, so that someone else would check it, you need to rebuild the kernel with the attached config

Thanks for the config! I tried to compare it with the one shipped in TW, but unfortunately there are too many differences:

$ zcat /proc/config.gz | diff - kconfig-suggested | wc -l 4621

I'll try and test your assumption that this is specific to Intel CPUs soon though, I have an AMD machine that I have not updated yet.

It's my pleasure I just concluded, everyone who said that they had no problems with the gnome, everyone CPU AMD. There is also a possibility that a patch is guilty, and everything is fine with the config, I compiled it manually without patches. PS And yes, the problem really disappeared, I launched several programs, a browser with a lot of tabs, the animations remained smooth, there are jerks, but usual for a gnome and not like they were before -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c34 --- Comment #34 from Atri Bhattacharya <badshah400@gmail.com> --- (In reply to Michiel Janssens from comment #33)

There is a possibility that the spectre mitigations have something to do with all this, and or hyperthreading/smt.

3. boot with kernel parameters: spectre_v2=off Transition is smooth This parameter seems to have the biggest impact of all the tested parameters.

Early days -- minutes, actually -- but this seems to help a lot in my case too (Intel i7-7*** processors). There are still some animation choppiness on Application overview, but nothing like before. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c36 --- Comment #36 from Robert Munteanu <rombert@apache.org> --- (In reply to Michiel Janssens from comment #33)

There is a possibility that the spectre mitigations have something to do with all this, and or hyperthreading/smt. I tested my system on the impact of those mitigations. As a reference I used a mouse click on "Show applications" item on the gnome-shell menu, running wayland. Watching if the application icons transition smoothly from the corner into the main desktop area. Hardware: Dell XPS 13 9360 Snapshot 20181118 Kernel 4.19.2-1

My subjective results 1. boot with default kernel parameters Transition is barely noticeable, not smooth for sure

2. boot with kernel parameters: pti=off spectre_v2=off l1tf=off nospec_store_bypass_disable no_stf_barrier Transition is very smooth

3. boot with kernel parameters: spectre_v2=off Transition is smooth This parameter seems to have the biggest impact of all the tested parameters.

Now I read about the new recently disclosed spectre v2 vulnerabilities and the STIBP mitigation. Some voices say that we might as well disable hyperthreading/smt instead of these new mitigations.

4. boot with default kernel parameters, and hyperthreading disabled in BIOS Transitions are smooth.

For the moment I will leave all mitigations enabled and hyperthreading disabled. For the record, this is an Intel Kabylake cpu

Some more data points: - for my Lenovo W541 laptop - i7-4810MQ with iGPU, none of these helped. The animations are maybe slightly faster, but still laggy - after updating my desktop ( Ryzen 5 2600X, Nvidia 1070, Samsung EVO 970 NVME SSD ) from Gnome 3.28, the animations are smooth, but I feel I slight stutter when pressing buttons for the first time, e.g. the applications button. Maybe I'm imagining, I don't now. What I think would be useful is for those affected and for which the various boot options worked to clarify whether they were using a discrete GPU or the integrated GPU. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c38 --- Comment #38 from Atri Bhattacharya <badshah400@gmail.com> --- Oddly, I just now booted to kernel 4.18.12 -- the version of kernel with which GNOME 3.28.x was performing really well. With no changes to the default boot parameters, GNOME 3.30.2's performance is again awful, stuttering. It seems GNOME 3.30.x + recent kernel = problem. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c40 --- Comment #40 from Dead Mozay <windowskaput@gmail.com> --- disabled pti=off spectre_v2=off l1tf=off the problem does not solve, I checked it first. Here is the animation with rebuild kernel https://youtu.be/MdX2GM79w30, Protection against these vulnerabilities is enabled. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c42 --- Comment #42 from Dead Mozay <windowskaput@gmail.com> --- (In reply to Zhiyuan Gao from comment #41)

(In reply to Martin Wilck from comment #35)

...

(In reply to Michiel Janssens from comment #33)

...

3. boot with kernel parameters: spectre_v2=off Transition is smooth This parameter seems to have the biggest impact of all the tested parameters.

Thanks a LOT for figuring this out. But reading above that only openSUSE is affected - I'd assume that Ubuntu and Fedora are also shipping with these mitigations enabled by default, no?

So I had a look upon fedora29. - kernel version 4.18.17 - gnome version 3.30.2 hyperthreading on and it seems spectre protection is also turned on

So presumably the only difference we have is that kernel version is different.

What do you think?

Fedora kernel 4.19.2 there are no problems either -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Scott Reeves <sreeves@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |sreeves@suse.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c45 --- Comment #45 from Dead Mozay <windowskaput@gmail.com> --- After several days of tests with a compiled kernel without patches, I can say that this partially solved the problem, the application review is smooth, but the activites still sometimes slows down, just when I launch the application from Dash, there are also signatures sometimes, while reading Twitter yesterday I came across several posts about The fact that the brakes appeared everywhere when the activities was activated. perhaps this is due to patches spectre v2, now they are just deciding what to do with them, Linus suggests that they be removed from the code at all, he considers it unwise to use them in the kernel, referring to the fact that it will be necessary for someone to disable the SMT in the bios. Corrections promise in GNOME 3.31, backports may be done, but this is not accurate. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c47 --- Comment #47 from Martin Wilck <martin.wilck@suse.com> --- (In reply to Dead Mozay from comment #46)

and also found out that there are no patches STIBP in Fedora, in openSUSE this patch is there, maybe that's the problem

This would make sense if the performance regressions were only observed with 4.19.2 and newer, where STIBP was first enabled in openSUSE. But people have started reporting this with earlier kernels. If people are running 4.19.2, STIBP likely contributes to the slowness. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c49 --- Comment #49 from Dead Mozay <windowskaput@gmail.com> --- (In reply to Benjamin Sabatini from comment #48)

So what info/logs would be helpful to fix this?

problem fixed? I just have a self-assembly kernel, at the same time there was an update to the gnome, now I’m not sure what exactly improved the performance -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c51 --- Comment #51 from Dead Mozay <windowskaput@gmail.com> --- (In reply to Nicolás Luciano Bértolo from comment #50)

I tried kernel-vanilla and I found that it runs even better than kernel-default with spectre mitigations off. Can anybody replicate my findings?

in 4.19.4 a patch has already been adopted that disables stibp, by the way stibp was enabled in 4.14, but yes, performance improves with vanilla kernel -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c53 --- Comment #53 from Michiel Janssens <michiel@nexigon.net> --- (In reply to Nicolás Luciano Bértolo from comment #50)

I tried kernel-vanilla and I found that it runs even better than kernel-default with spectre mitigations off. Can anybody replicate my findings?

For my system they give similar behavior, so a lot better then just kernel-default. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c55 David Mulder <david.mulder@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david.mulder@suse.com --- Comment #55 from David Mulder <david.mulder@suse.com> --- (In reply to Nicolás Luciano Bértolo from comment #50)

I tried kernel-vanilla and I found that it runs even better than kernel-default with spectre mitigations off. Can anybody replicate my findings?

vanilla kernel + spectre fixes enabled works well for me. Disabling spectre is even better. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Tomasz Witowski <tomasz.witowski@outlook.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomasz.witowski@outlook.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c57 --- Comment #57 from Dead Mozay <windowskaput@gmail.com> --- Is there any news on this issue? With each update it gets worse on some hardware configurations. Nothing else helps, even disabling hyper-threading also partially solves the problem. Unfortunately, not everywhere you can turn it off. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c60 --- Comment #60 from Dead Mozay <windowskaput@gmail.com> --- (In reply to Borislav Petkov from comment #59)

Can anyone of the gazillion people on this bugzilla build the upstream v4.20-rc5 kernel and test with it?

We have disabled STIBP et al upstream and will backport this to our kernels eventually.

Thx.

disabling stibp does not solve the problem, I tried, I also deleted a number of patches that could affect performance, nothing changes. I also found that the problem also exists in kernel 4.12, and problems started with it -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c67 --- Comment #67 from Dead Mozay <windowskaput@gmail.com> --- (In reply to Borislav Petkov from comment #64)

(In reply to Dominique Leuenberger from comment #63)

...

yes, it seems to be an interaction between GNOME and the kernel

If the performance regression goes away when you use a different desktop env than gnome, then it is clearly a gnome issue.

Otherwise, you'd have to be more specific about this "interaction". Maybe perf-profile a typical workload which causes the regression, etc.

Thx.

it is possible, and it is also possible if the CPU will not work correctly, problems will also be observed in the gnome, the gnome is very dependent on how the CPU works. At the beginning of the search for a problem, I did synthetic tests, they showed a drop in multi-core tests, which in turn could well cause a similar problem in the GNOME -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c73 --- Comment #73 from Michiel Janssens <michiel@nexigon.net> --- (In reply to Borislav Petkov from comment #72)

(In reply to Nicolás Luciano Bértolo from comment #70)

...

I can confirm that those parameters help a lot. But in my tests I couldn't find a difference between 4.19 and 4.20. 4.20 with mitigations enabled is just as bad as 4.19-default.

Well, the mitigations do cost and we've tried to make them as unpunishing as possible. For example, my workstation with 20-rc5 has:

/sys/devices/system/cpu/vulnerabilities/l1tf:Not affected /sys/devices/system/cpu/vulnerabilities/meltdown:Not affected /sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Mitigation: Speculative Store Bypass disabled via prctl and seccomp /sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: __user pointer sanitization /sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full AMD retpoline, IBPB: conditional, STIBP: disabled, RSB filling

which means that SSB will be enabled only for applications which request it (prctl and seccomp), spectre_v2 is mitigated by retpolines (a lot cheaper than IBRS) and the indirect branch predictor barrier happens when switching between different user processes of which one can be a malicious one.

And this is the default setting.

The whole idea behind having all those cmdline options was for people who don't want to sacrifice performance and would like to disable the security mitigations. So the ultimate decision will be with the user.

In any case, the default case does enable a *sensible* set of the mitigations but they are not for free(!) and depend on the workload.

I hope I'm making sense here.

To me you are making sense, thank you for looking into this. I have the same result as https://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c70 However, I also tested kernel-vanilla, from Head and current 4.19.5. Kernel-vanilla gives similar or almost similar result as with mitigations off as you mentioned in https://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c66 But in my understanding kernel-vanilla has all default mitigations on, as running latest spectre-meltdown-checker says my system is not vulnerable. So it seems to me there must be a difference in the kernels. 4.19.5-1-vanilla: /sys/devices/system/cpu/vulnerabilities/l1tf Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable /sys/devices/system/cpu/vulnerabilities/meltdown Mitigation: PTI /sys/devices/system/cpu/vulnerabilities/spec_store_bypass Mitigation: Speculative Store Bypass disabled via prctl and seccomp /sys/devices/system/cpu/vulnerabilities/spectre_v1 Mitigation: __user pointer sanitization /sys/devices/system/cpu/vulnerabilities/spectre_v2 Mitigation: Full generic retpoline, IBPB, IBRS_FW 4.19.5-1-default: /sys/devices/system/cpu/vulnerabilities/l1tf Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable /sys/devices/system/cpu/vulnerabilities/meltdown Mitigation: PTI /sys/devices/system/cpu/vulnerabilities/spec_store_bypass Mitigation: Speculative Store Bypass disabled via prctl and seccomp /sys/devices/system/cpu/vulnerabilities/spectre_v1 Mitigation: __user pointer sanitization /sys/devices/system/cpu/vulnerabilities/spectre_v2 Mitigation: Indirect Branch Restricted Speculation, IBPB, IBRS_FW Apparently on my Intel system both kernels have different spectre_v2 mitigations. Kernel-default is using IBRS, which as you say is more expensive than retpoline, which is used by kernel-vanilla. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c78 --- Comment #78 from Dead Mozay <windowskaput@gmail.com> --- (In reply to Borislav Petkov from comment #76)

(In reply to Michiel Janssens from comment #73)

...

Apparently on my Intel system both kernels have different spectre_v2 mitigations. Kernel-default is using IBRS, which as you say is more expensive than retpoline, which is used by kernel-vanilla.

Yes, kernel-default has our SUSE patches which are part of SLE and have this additional IBRS enablement which is not upstream and thus vanilla doesn't have it.

IBRS is a heavy hammer and didn't get accepted upstream but we took it. Which is going to be replaced by enhanced IBRS which should be lighter but it is still being rolled out and I don't know whether older, already released machines can even get it through microcode. For details, see:

https://software.intel.com/sites/default/files/managed/c5/63/336996- Speculative-Execution-Side-Channel-Mitigations.pdf

where all the different mitigation mechanisms are explained.

Now, it is debatable whether a Skylake class machine which needs IBRS to be fully mitigated is even exploitable when only retpolines are enabled. It has been said that running a spectre v2 exploit on a machine only with retpolines and not IBRS is very very hard to do. Thus, many people are unwilling to pay the performance penalty and revert to retpolines. IOW, if you boot with spectre_v2=retpoline on kernel-default, you should be getting close to vanilla.

All IMHO, of course.

I build a kernel 4.19.5 without IBRS patches, nothing has changed, although I do not exclude the possibility that I could do something wrong, or skip. https://build.opensuse.org/package/show/home:Dead_Mozay:Kernel/kernel-defaul... -- You are receiving this mail because: You are on the CC list for the bug.

If you want to compare security mitigations, you need to take the same kernel and do two runs, once with the mitigation enabled and once with the mitigation > disabled. I performed tests with enabled and disabled parameters, the result in fedora is

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c83 --- Comment #83 from Dead Mozay <windowskaput@gmail.com> --- practically the same, in openSUSE the result differs, but it was at the time of the release of Leap 15, you need to make new tests, but I can only on the current kernels of both systems -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c85 --- Comment #85 from Takashi Iwai <tiwai@suse.com> --- (In reply to Dead Mozay from comment #84)

...

Are you testing Fedora kernel with openSUSE user-space stuff? Or are you testing Fedora user-space?

If Fedora kernel works better with openSUSE user-space, then the point should be either Fedora's downstream patch or the difference of the kernel configuration.

For the latter case, you can try to build the upstream kernel with Fedora kernel config and see whether it works.

I tried to build the kernel with the fedora config, I even wrote about it somewhere above, it works better, but there are still some problems,

If you still have the same performance problem with a self-built upstream kernel using the Fedora config, then it's really something else than kernel. If something is *improved* by the Fedora kernel config, though, we'd like to see the difference. You can try the following: - Boot with Fedora config kernel, copy /proc/config.gz to .config on 4.19.x Linux kernel tree, and run "make localmodconfig". This will give you a minimal set of kernel config for the currently running kernel. Save this config to somewhere. - Similarly, boot with openSUSE kernel, and do the same. Now you get two kernel configs you can directly compare. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c87 --- Comment #87 from Takashi Iwai <tiwai@suse.com> --- That config contains all stuff, so it's not clear which different matters. That's why I asked to create minimal configs so that we can compare 1:1. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c89 --- Comment #89 from Dead Mozay <windowskaput@gmail.com> --- Created attachment 792088 --> http://bugzilla.opensuse.org/attachment.cgi?id=792088&action=edit New tests performed the same tests, and so about since the release openSUSE Leap 15.0, only in Leap does disabling SMT in BIOS solve the problem -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c96 --- Comment #96 from Takashi Iwai <tiwai@suse.com> --- (In reply to Dead Mozay from comment #88)

(In reply to Takashi Iwai from comment #87)

...

That config contains all stuff, so it's not clear which different matters. That's why I asked to create minimal configs so that we can compare 1:1.

unfortunately in Fedora CONFIG_IKCONFIG_PROC=n

Then copy Fedora kernel config manually to .config on kernel source directory, then run "make localmodconfig". It's only about copying from the proper source. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Dennis Irrgang <me@dennis-irrgang.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |me@dennis-irrgang.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c99 --- Comment #99 from Dead Mozay <windowskaput@gmail.com> --- (In reply to Dennis Irrgang from comment #98)

Any updates on this? I thought I was going crazy because my fresh install was sluggish as hell.

Also running TW on a T480 with an Intel i5 8250u.

not yet, so nobody understood what the problem -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c102 --- Comment #102 from Takashi Iwai <tiwai@suse.com> --- Quickly looking at the logs of comment 97, the main differences between two kernel configs are: CONFIG_NO_HZ_FULL=y for Fedora CONFIG_NO_HZ_IDLE=y for TW CONFIG_PREEMPT_VOLUNTARY=y for Fedora CONFIG_PREEMPT=y for TW CONFIG_SCHED_AUTOGROUP=y for Fedora CONFIG_SCHED_AUTOGROUP=n for TW CONFIG_SLUB=y for Fedora CONFIG_SLAB=y for TW The last one is unlikely influencing on the performance, though. The rest are subtle drivers and debugging configs. So I'm building a TW kernel with the config changes above to follow Fedora in OBS home:tiwai:bsc1112824 repo. The build should finish in an hour or so. Please give it a try and see whether if this really improves things as advertised here. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c106 --- Comment #106 from Timo Jyrinki <tjyrinki@suse.com> --- (In reply to Takashi Iwai from comment #102)

So I'm building a TW kernel with the config changes above to follow Fedora in OBS home:tiwai:bsc1112824 repo. The build should finish in an hour or so.

Please give it a try and see whether if this really improves things as advertised here.

I don't see difference on my Intel machine with the 4.19.8-1.g8bc7ab5-default nosmt=force ie disabling hyperthreading is something that does seem to make a real tangible difference for me. With it the most annoying lags are gone. The effect is the same both with 4.19.7 and the one from home:tiwai:bsc1112824, so in other words the changed configuration there does not seem to affect this behavior. This is based on 10+ boots. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c108 --- Comment #108 from Timo Jyrinki <tjyrinki@suse.com> --- (In reply to Takashi Iwai from comment #107)

Just to be sure: did you already test the kernel in OBS Kernel:HEAD, which is based on the latest 4.20-rc?

Now I have (4.20.0-rc6-2.g91eea17-default). It seems the situation remains the same even there. With hyperthreading enabled overall user experience is bad due to lagginess, with nosmt=force it's usable. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c111 --- Comment #111 from Nicolás Luciano Bértolo <nicolasbertolo@gmail.com> ---

disabling patches on the 4.19.7 kernel no longer affects anything, just made several reboots with different patch shutdown options, and without shutting down at all, the result is the same. At the moment, my performance is only affected by disabling hyper-threading.

This does not match my experience. Using spectre_v2=retpoline makes the default kernel perform just as well as the vanilla kernel. I'm using kernel 4.19.7-1-default. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c113 --- Comment #113 from Dead Mozay <windowskaput@gmail.com> --- (In reply to Nicolás Luciano Bértolo from comment #111)

...

disabling patches on the 4.19.7 kernel no longer affects anything, just made several reboots with different patch shutdown options, and without shutting down at all, the result is the same. At the moment, my performance is only affected by disabling hyper-threading.

This does not match my experience. Using spectre_v2=retpoline makes the default kernel perform just as well as the vanilla kernel.

I'm using kernel 4.19.7-1-default.

Checked on my laptop http://paste.opensuse.org/view//92547497, the difference really is with the parameter spectre_v2=retpoline, but still this is not enough, a conflict of patches may have occurred somewhere, I saw a package with a bunch of patches somewhere, and a few were very old, 6 years ago were created, most likely they have not been needed for a long time, and now such patches can cause only problems -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c114 --- Comment #114 from Dennis Irrgang <me@dennis-irrgang.com> --- Probably related to this bug. I noticed that when simply moving a window around gnoem-shell uses 100% of its current core. Interestingly the same happens simply while typing in Telegram. I noticed the latter because every time I typed, a video running in a separate window would begin to stutter. This is while using the X-Server. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c116 --- Comment #116 from Dead Mozay <windowskaput@gmail.com> --- (In reply to Dennis Irrgang from comment #115)

There may also be a time component to this. It seems like performance is degrading over time. After a reboot performance is 'usable', but after a few hours of working it becomes notably worse.

At the expense of overall performance, I'm not sure, but the GNOME starts to fail more, if the computer is running for a long time without rebooting. What I had enough of my mind, I already checked, it remains to hope for real gurus. but in general, similar problems with animations appeared in Fedora, but in openSUSE, they are more noticeable and annoying, for many, but I do not exclude the possibility that the lags are due to the fact that I have Fedora installed on the old HDD, and openSUSE on NVME SSD, but problems are still more on openSUSE. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c118 --- Comment #118 from Adam Szyszko <szyszko.86@gmail.com> --- I'm testing 4.20 kernel now. For me nothing has changed. Still animations are choppy, windows are opening delayed. My specs: Intel i7-7500U, Intel HD graphics 620, 8GB ram. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c121 --- Comment #121 from Dead Mozay <dead_mozay@opensuse.org> --- (In reply to Borislav Petkov from comment #120)

And now can anyone try with a different desktop environment besides gnome?

some people say that everything is ok there, perhaps because they have nothing to compare with, but when I checked the bug I checked with plasma5 I also noticed some problems in animations there, but I’ll not say that the root of the problems in the gnome will be, in plasma5, through the release, they break something, into this the moment there is something broken too, when choosing the engine, the opengl3 animations work poorly, changing the version to the old one helps a little, but problems remain anyway, this problem is in plasma5, at the moment it is not possible to conduct such a check, you need to wait when the problem is corrected in plasma5. And there are no more sane desktop environments where you can see how graphics acceleration works and other things. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c124 Atri Bhattacharya <badshah400@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(dead_mozay@opensu | |se.org) --- Comment #124 from Atri Bhattacharya <badshah400@gmail.com> --- Are you using GNOME on a wayland session or an XOrg session? -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c126 Wolfgang Rosenauer <wolfgang@rosenauer.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(dead_mozay@opensu | |se.org) | --- Comment #126 from Wolfgang Rosenauer <wolfgang@rosenauer.org> --- I've lost track about this issue during the last weeks but I'm still facing Gnome performance which is really hard to work with on my laptop also with latest TW snapshot. I'm supposed to run Gnome under Xorg. At least that is what my session selector says but on the other hand when I look at the process list I see Xwayland running and not Xorg. (Looks like a bug to me) My system is a Core-i7-7500 (Quad) but the Gnome shell performance is really hard to work with. All window/desktop actions (moving windows, switches workspaces etc) are quite laggy and not smooth at all. This started in October and only got better a bit. Compared to before it's really slow. Considering switching to Xfce or something for the hope it'll be better. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c128 --- Comment #128 from Wolfgang Rosenauer <wolfgang@rosenauer.org> --- XDG_SESSION_TYPE is "x11" wolfgang@ox:~> ps ax | grep X 1834 tty7 Sl+ 0:00 /usr/bin/Xwayland :1024 -rootless -terminate -accessx -core -auth /run/user/464/mutter/Xauthority -listen 4 -listen 5 -displayfd 6 3393 tty2 Sl+ 0:02 /usr/bin/X vt2 -displayfd 3 -auth /run/user/1000/gdm/Xauthority -background none -noreset -keeptty -verbose 3 wolfgang@ox:~> grep -i wayland /etc/gdm/custom.conf #WaylandEnable=false What is the verdict for that output? Wayland or Xorg? I guess "wayland" but XDG_SESSION_TYPE does not reflect that. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c130 --- Comment #130 from Wolfgang Rosenauer <wolfgang@rosenauer.org> --- Did that now. Now there is no Xwayland running anymore. And the desktop animations are now certainly smoother than with wayland running but but also still stutter more than expected. gnome-shell CPU consumption still very visible. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c132 Michael G <opensuse.com@spaul.at> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |opensuse.com@spaul.at --- Comment #132 from Michael G <opensuse.com@spaul.at> --- I am on latest Tumbleweed with a T460s - i7 6600 The only thing that works for me to resolve the performance issues is to disable hyperthreading by adding nosmt=force boot parameter for now. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Victor Zhestkov <victor.zhestkov@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |victor.zhestkov@suse.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c135 --- Comment #135 from Dead Mozay <dead_mozay@opensuse.org> --- (In reply to simon izor from comment #134)

I'm really confused why were are assuming this is a kernel issue? What is GNOME doing with the kernel that could cause this?

I just encountered two other users in a Discord server that I frequent that have this same exact issue (GNOME is extremely sluggish and they have Intel CPUs), but only with GNOME, and only on Tumbleweed. All other DEs work fine. All other distros with GNOME 3.30 work fine.

There's gotta be some patch we apply to GNOME or *something* along those lines causing this... it seems extremely unlikely that it would be the kernel as these users have no issue using KDE Plasma or Xfce on the latest Tumbleweed snapshot (20190126). GNOME runs much slower for both of them on Tumbleweed than on Arch or Fedora, but only GNOME. As I said above, what would could be going with the kernel that would *only* affect GNOME?

Everything is very difficult, as I wrote above, the gnome is very dependent on how well the processor works in the operating system, any error in the kernel can affect its operation, which in turn can lead to different regressions, so the possibility of such exclusion was it is impossible, for example, by disabling the spectre protection, it started to work better, patches were applied to the kernel, not the gnome, there were such conclusions from here, currently disabling patches does not change anything at all, and he again started to lag after the latest updates.

GPU <- [ [ Cogl + Cairo ] <- [ GDK + Clutter ] <- GTK+ ] <- application So GNOM works, need to check these packages, I don’t have enough skills to check it

I tried plasma 5, it also lags me, not like a gnome, but it’s still noticeable -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c137 --- Comment #137 from Victor Zhestkov <victor.zhestkov@suse.com> --- Simon, I agree with you it's not just a kernel issue. And Dead Mozay said it's quite complicated. I've tried to analyze, but not yet get a full view. But basically speaking - there are a lot of issues inside GNOME (gjs, gnome-shell and probably mutter are affected), but these issues become more visible with kernel issue. spectre_v2=retpoline boot option really helps, but some lags still there. Finally I gave up with tumbleweed, there are too many packages updated the same time and it makes it harder to find out the exact root cause, especially because as I said I'm sure it's not only one issue. I just finished building some packages related to GNOME with the fixes I think should help, but I did it for Leap 15 and still testing it. Leap 15 has different issues in GNOME, but it seems easier to fix. For example one of the patch fixing tweener.js errors in journal, not just suppressing the messages as it was done with gdm options in some builds, but really fix the behavior of using objects. I'm still testing on my system, but if somebody wants to check it too, I'll be happy. But please note, the fixes is for Leap 15.0 only. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c140 --- Comment #140 from Dead Mozay <dead_mozay@opensuse.org> --- It seems it was possible to partially solve the problem, it does not fix everything, but the GNOME has become more responsive.

sudo zypper addrepo https://download.opensuse.org/repositories/home:Dead_Mozay:GNOME/openSUSE_Tu... sudo zypper refresh sudo zypper install -f libmozjs-60 reboot Who will check, please write

-- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c142 --- Comment #142 from Dead Mozay <dead_mozay@opensuse.org> --- Created attachment 795858 --> http://bugzilla.opensuse.org/attachment.cgi?id=795858&action=edit Without my fixes It seems this will not end, corrected one, got out another, need to rebuild all the components of GNOME, and not only GNOME, there is a whole lot, including some of the kernel. If anyone has a desire, you can start sysperf yourself, perform all operations where there are lags, and see the result -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c144 --- Comment #144 from Victor Zhestkov <victor.zhestkov@suse.com> --- Hi Richard. It seems nothing has changed with your kernel. I found that it's quite easy to check if the issue presents on the system. Just to use geekbench. The single core score is very dependent on spectre_v2=retpoline option. Here is the result with spectre_v2=retpoline: https://browser.geekbench.com/v4/cpu/11952293 And here is the result with default mitigation for my CPU (IBRS): https://browser.geekbench.com/v4/cpu/11952353 3981 vs 3140 in single core score. Multicore difference is insignificant. The behavior with micro lags in GNOME is the same with the default kernel from Tumbleweed and from your repo. Microlags are present with IBRS only. On my other system Lenovo T470p i5-7440HQ there is no difference in geekbench score with IBRS or retpoline options. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c146 --- Comment #146 from Takashi Iwai <tiwai@suse.com> --- Obviously there is no silver bullet. This is no single bug, but attributed by different factors. We couldn't address all of them so far, and it's enough to cause the significant problem... For TW, I guess the majority of regressions would be fixed by CONFIG_VOLUNTARY_PREEMPTY and the user-space side fixes. But there must be many remaining machines that still suffer from the performance issue, unfortunately. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Libor Pechacek <lpechacek@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |http://bugzilla.opensuse.or | |g/show_bug.cgi?id=1117833 -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c152 --- Comment #152 from Timo Jyrinki <tjyrinki@suse.com> --- Thanks everyone for your continued investigations. Let's not despair, the solutions are there somewhere :) (In reply to Dead Mozay from comment #151)

there is definitely progress, but visually almost nothing changes.

In your going through GNOME stack patches, did you find anything suspicious we're carrying that might be causing performance drops and should be dropped if not upstreamed? And dropping non-upstreamable patches is a good idea overall. I wouldn't (necessarily, unless having fun of course) spend time on doing things other distributions aren't doing, but finding the differences.

I also stumbled upon such tests https://www.phoronix.com/scan.php?page=article&item=dellxps-9380-linux&num=1 Perhaps the author of the article is right, and the problem lies in energy consumption.

Surely that's yet another part of the equation, but I've also tweaked my performance policies to be balance_performance which matches other distributions. And I've disabled hyperthreading, and using spectre_v2=retpoline. All of that makes things better, but not comparable to other distros. In addition to /sys/devices/system/cpu/*/*/energy_performance_preference (permanent via /etc/default/tlp) it'd be welcome to know if there are other knobs to turn that could be affecting what Phoronix found, but previously I haven't bumped to others that would be as important. I'd also be interested in trying out the voluntary preemption, to not rule out that and since that's clearly something that is not as easy to test (no runtime switch). Optimizing for smooth desktop usability seems to be really complex these days, and it seems that differing from what others do always has a negative impact due to that complexity. That's a bit sad state of affairs, and lack of performance work resources put to the desktop, but what can we do. It's nice though that Phoronix confirms that we're not all crazy. Sometimes user perceptions are biased or wrong, especially when it comes to "did it help?" if it's not a huge difference. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c154 --- Comment #154 from Nicolás Luciano Bértolo <nicolasbertolo@gmail.com> --- One thing that is different in TW compared to Fedora, Ubuntu, etc. is btrfs as root filesystem. It is known that gnome-shell likes to perform io operations in the rendering thread, so btrfs may be too slow for that? This is just speculation. I wonder if anyone that is using ext4 as root fs can repro this behaviour. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c156 --- Comment #156 from Victor Zhestkov <victor.zhestkov@suse.com> --- I have ext4 on all af my systems. This issue doesn't depend on file system at all. I have my solution for Leap 15.0 I'll be happy if it could help for somebody other than just me: https://download.opensuse.org/repositories/home:/vzhestkov:/leap15-gnome-upd... Just add the repo and make dist upgrade. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c158 --- Comment #158 from Dead Mozay <dead_mozay@opensuse.org> --- (In reply to Bruno Friedmann from comment #157)

Invited to join as a non impacted users :-)

I share some informations : I'm using plasma5 with xorg and proprio nvidia drivers I didn't find or perceive any slowdown on my desktop in the last months.

I've tested geekbench and the result are there with a plain normal TW (up to date) https://browser.geekbench.com/v4/cpu/12014162

What I always apply is patch all hardware firmware for all component on my hardware some have direct relation with spectre/meldown and as my cpu is a xeon it is also preboot patched with kernel-firmware when the lastest Dell bios doesn't contain all fixes.

This is true, there are no such problems in plasma5, it worked for almost 2 days, everything was fine. but it's still not clear what causes problems in GNOME -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c161 --- Comment #161 from Nicolás Luciano Bértolo <nicolasbertolo@gmail.com> --- Earlier in this thread we thought the problem was caused by a difference in the kernel configuration between Fedora and openSUSE. We got as far as diffing the /proc/config files. I wanted to test that hypothesis. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c163 --- Comment #163 from Dead Mozay <dead_mozay@opensuse.org> --- Now disconnected the discrete graphics, and booted from the integrated Intel HD, and there is no difference at all, that with GTX970 that with Intel HD, the behavior is absolutely the same -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Artem Tim <ego.cordatus@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ego.cordatus@gmail.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Timo Jyrinki <tjyrinki@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #798388|video comparison of Ubuntu |slow motion (4x) video description|and openSUSE with and |comparison of Ubuntu and |without special parameters |openSUSE with and without | |special parameters -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c165 --- Comment #165 from Timo Jyrinki <tjyrinki@suse.com> --- And adding that both machines were set to use same resolution (1920x1080) and have essentially same GPU+CPU - SUSE machine has slightly faster i7-8650 while the Ubuntu machines has i7-8550. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c167 --- Comment #167 from Timo Jyrinki <tjyrinki@suse.com> --- With confirming I largely mean the same you do by supporting :) I mean it's confirmed there is a problem somewhere, and like I said I'm aware that this video does not bring anything new to the table regarding fixing the problem. Sometimes a video is just easier to consume than subjective observations over multiple reboots, so I thought it'd be nice to have a direct comparison video. My current best but not necessarily very good guess about the root cause for the main performance degradation is some openSUSE specific patch(es) somewhere in GNOME stack / libraries it uses. I'm also interested to see if there's anything meaningful changing once transition from libmozjs52 to libmozjs60 is complete - but I do see my TW is already using the latter for gnome-shell at least. Regarding some of the different factors you mentioned, I believe it's not something Ubuntu has added, since Fedora (reportedly, I should test at some point myself) performs as well as Ubuntu, and it's unlikely both Fedora and Ubuntu would carry a non-upstreamed critical patch for such a long time. The problem has also persisted for several months, so there has been ample time for all the development versions of the distributions to at least somewhat get "in sync", and all of them have had eg the same upstream GNOME 3.30 stack that is currently in openSUSE Tumbleweed. Some person in this thread has, in addition to the kernel parameters mentioned, tried running Fedora kernel directly in openSUSE without a big help, so it seems the kernel alone does not fix the lagginess. Surely the hyper-threading disabling and IBRS -> retpoline help, but as seen on the video not enough to get on par and no other distribution of course needs to disable hyper-threading. So here's just my summary of my current thinking. When I have extra time I always try to think of something new to try or study, but my main requirement was to get a usable desktop for doing other work with SUSE, which I now have with the workarounds. Something on my mind is to check that 20181015 <-> 20181018 snapshot delta mentioned in comment #160 and see if the difference is as big as what I have there on the video between distributions. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c169 --- Comment #169 from Dead Mozay <dead_mozay@opensuse.org> --- (In reply to Robert Munteanu from comment #168)

(In reply to Timo Jyrinki from comment #167)

...

Something on my mind is to check that 20181015 <-> 20181018 snapshot delta mentioned in comment #160 and see if the difference is as big as what I have there on the video between distributions.

I think this is the first step that can be done. I unfortunately don't have a spare machine and the time to run the tests, but I think a binary search in the updates that were applied between 20181015 and 20181018 would help pinpoint the root cause.

Assuming that the 20181018 snapshot has broken performance, there are N package updates between 20181015 and 20181018. One can split these N package updates in N1 and N2. It should happen that one of these contains the problematic update, so applying N1 (for instance) should reproduce the problem but not N2. And so on and so forth.

If anyone has the possibility to do this I think it would be a worthwhile exercise.

It is not necessary to have another PC, you can use the virtualbox 6.x version and choose VMSVGA as the graphic controller, there is very good performance in GNOME, there is a list of updates here

http://mirror.tspu.ru/opensuse/tumbleweed/iso/Changes.20181018.txt

-- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c171 --- Comment #171 from Dead Mozay <dead_mozay@opensuse.org> --- Anyone can check

sudo zypper ar -f obs://home:Dead_Mozay:branches:openSUSE:Factory home:Dead_Mozay:branches:openSUSE:Factory and update with vendor change sudo zypper dup --from home:Dead_Mozay:branches:openSUSE:Factory --allow-vendor-change

-- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c173 --- Comment #173 from Robert Munteanu <rombert@apache.org> --- Created attachment 798904 --> http://bugzilla.opensuse.org/attachment.cgi?id=798904&action=edit VIdeo of behaviour after installing patched mutter from home:Dead_Mozay:branches:openSUSE:Factory -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c175 --- Comment #175 from Robert Munteanu <rombert@apache.org> --- (In reply to Dead Mozay from comment #174)

(In reply to Robert Munteanu from comment #172)

...

(In reply to Dead Mozay from comment #171)

...

Anyone can check

...

sudo zypper ar -f obs://home:Dead_Mozay:branches:openSUSE:Factory home:Dead_Mozay:branches:openSUSE:Factory and update with vendor change sudo zypper dup --from home:Dead_Mozay:branches:openSUSE:Factory --allow-vendor-change

Cool, thanks for setting this up!

For the record, these are the instructions that worked for me

$ sudo zypper ar -r https://download.opensuse.org/repositories/home:/Dead_Mozay:/branches:/ openSUSE:/Factory/openSUSE_Factory/home:Dead_Mozay:branches:openSUSE:Factory. repo $ sudo zypper dup --from home_Dead_Mozay_branches_openSUSE_Factory --allow-vendor-change

And even though the package is a branch of openSUSE_Factory, which sounds scary, it only brings in mutter with 4 additional patches:

The following 4 packages are going to be upgraded: libmutter-3-0 mutter mutter-data mutter-lang

In my testing the lag situation seems to be improved, although not completely smooth yet:

- when hitting the 'super' key to open the applications menu, the delay is much smaller, but still noticeable - when pressings the 'show applications' button there is a definite stutter/delay when the application icons should be flying in

So a definite improvement, but still not as it should be.

Host info:

$ inxi -G -v 1 System: Host: rombert.corp.adobe.com Kernel: 4.20.12-1-default x86_64 bits: 64 Desktop: Gnome 3.30.2 Distro: openSUSE Tumbleweed 20190301 CPU: 6-Core: Intel Core i7-8850H type: MT MCP speed: 800 MHz min/max: 800/4300 MHz Graphics: Device-1: Intel UHD Graphics 630 driver: i915 v: kernel Device-2: NVIDIA GP107GLM [Quadro P2000 Mobile] driver: nvidia v: 418.43 Display: x11 server: X.Org 1.20.4 driver: modesetting,nvidia resolution: 3840x2160~60Hz OpenGL: renderer: Quadro P2000/PCIe/SSE2 v: 4.6.0 NVIDIA 418.43 Drives: Local Storage: total: 961.19 GiB used: 97.53 GiB (10.1%) Info: Processes: 367 Uptime: 4h 16m Memory: 31.03 GiB used: 5.61 GiB (18.1%) Shell: bash inxi: 3.0.32

that's why I wrote that partially solves the problem, if hyper threading is enabled, try disabling

Yes, with: - your patches to mutter - hyper-threading disabled in BIOS - the latest TW kernel with the updated default config # CONFIG_PREEMPT_NONE is not set CONFIG_PREEMPT_VOLUNTARY=y # CONFIG_PREEMPT is not set Then responsiveness is spot on. I'll post another screencap for comparison. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c177 --- Comment #177 from Michael G <opensuse.com@spaul.at> --- Follow up comment 132 - On my T460s TW 20190310 Wayland Session (ongoing 5 hours intensive use) there were no noticable lags. Everything is smooth enough for me. There might be occasional hickups well below 300ms but no key repitition phantoms or similar disturbances. Boot param only retpoline. HT is ENABLED. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Juan Ramos <juan.ramos@outlook.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |juan.ramos@outlook.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 Jeffrey Cheung <jcheung@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jcheung@suse.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c184 Dead Mozay <dead_mozay@opensuse.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(hhetter@suse.com) | --- Comment #184 from Dead Mozay <dead_mozay@opensuse.org> --- Today the problem is still present. `spectre_v2=retpoline,generic` partially solves the problem, but not completely -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1112824 http://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c193 --- Comment #193 from Wolfgang Rosenauer <wolfgang@rosenauer.org> --- Removing all above mentioned configs brought back some visible lag (mainly "latency" in UI animations, less (but existing) stuttering in animations). -- You are receiving this mail because: You are on the CC list for the bug.