On Wed, 27 Nov 2024, Alessio Biancalana via openSUSE Factory wrote:
Il giorno mer, 27/11/2024 alle 04.54 +0100, Aaron Puchert ha scritto:
But at least right now I don't think they have dedicated people for performance, and I'm not aware of anyone regularly doing this kind of work in the community.
If they exist, please step up.
Howdy! The very first reply to this thread was mine, and it was pretty idiotic from my side not to introduce myself. I'm Alessio, I hack things on openSUSE when I have some free time, I maintain some packages and I try to champion all-things-observability both in the community _and_ in my day by day at SUSE.
When I first saw this proposal coming to Fedora I was extremely thrilled, and when this expanded to other communities like Arch Linux I was very much fascinated by how the discussion evolved. I'm very thrilled as well that this topic is reaching openSUSE at last. I think advanced diagnostics are something in the scope of every operating system, and hacking on a really wide variety of software in my free time led me to have some of this analysis using both sysprof _and_ eBPF . I think one of the greatest performance slowdowns in GTK was discovered thanks to system-wide framepointers availability.
This means to me that other distros are currently more ideal than openSUSE to suit these needs about the development of desktop environments and in general low level system software. I would love openSUSE to really stand up as "the makers' choice", as that is our claim and I think we have a lot of edge on that front to contribute to the ecosystem.
As Neal pointed out, right now eBPF profiling isn't really a thing because eBPF unwinding itself isn't really a thing.
Coming to GNOME, as Neal was reporting, Christian Hergert wrote a couple posts that were really eye-opening about this kind of system- wide profiling even being possible.
https://fedoramagazine.org/performance-profiling-in-fedora-linux/ https://blogs.gnome.org/chergert/2022/12/31/frame-pointers-and-other-practic...
On the contrary, he also wrote something about system-wide profiling actually _being_ possible even without frame pointers _but_ the overhead on the observed system IMHO makes it almost pointless (sorry for the blunt judgement here):
Where do you see it being "pointless"? Because 10% of the samples (on his system) are from unwinding? I've just tested doing a whole system profile on SLE15 SP6 and I see at most 5% overhead from perf (perf report isn't the most useful tool here. What's the perf overhead with a system with FPs? (does it reliably show the kernel side doing unwinding?) It will very much still show that greatest GTK slowdown, I bet even --call-graph lbr might have shown it. Richard.
https://blogs.gnome.org/chergert/2024/11/03/profiling-w-o-frame-pointers/
Obviously I don't spend all my time just watching flame graphs on my monitor. At the same time I think dismissing this as something totally unneeded wouldn't pay honmage to my favorite distro.
I hope my contribution was valuable to you in some way.
Alessio
-- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)