On Tue, 26 Nov 2024, Andrii Nakryiko wrote:
On Tue, Nov 26, 2024 at 6:43 PM Aaron Puchert <aaronpuchert@alice-dsl.net> wrote:
Am 25.11.24 um 23:59 schrieb Andrii Nakryiko:
[...] DWARF's .eh_frame-based stack unwinding tables. The latter is actually doesn't land itself well to system-wide profiling, and involved compromises in memory usage, CPU usage, and just plain reliability (no one will guarantee you that you captured all of the stack, for example).
The first two arguments are problematic: remember that we're talking about frame pointers, which cost CPU for all users. On one hand you're
[...]
To summarize: DWARF profiling doesn't work perfectly, but it might work well enough in a large enough number of cases, and in the remaining cases it's not made impossible. You just have to invest a bit more time.
There is an entire world of tools that use BPF and capture stack traces not for profiling, but to pinpoint which user code path causes some "issue", where the issue itself can be multitude of things (resource utilization, suspicious calling patterns, lock misuses, whatever you can come up with). Stack traces answer the questions like "where in the code" and "how did we get to that place", regardless of what's the actual question you are trying to answer.
So how does FP unwinding not run into the same issue as DWARF unwinding? If the actual unwinding happens at sampling time (to save you from capturing the very same stack and register state) then with deep call stacks you spend large time in the kernel and hit reliability issues when you fault. You can do DWARF unwinding in the kernel as well (as well as stack capturing for FP based unwinding).
Even more broad and ad-hoc are bpftrace-based scripts, they are written quickly, often as one-time throw-aways, to pinpoint or debug specific issues. They very often capture and use stack traces to pinpoint the source of the problem in the application code. While comprehensive tools like perf might afford to try to utilize DWARF-based stack unwinding (however unreliable and expensive), all of the more lightweight tooling just can't afford to even attempt to tackle this problem in any reasonable way.
Why so? Because they are not helped by appropriate tooling? I think this is all because of FUD and NIH. Frame pointers are not a requirement for unwinding and they are not in any way "better" than DWARF. Unwinding IP with a frame chain might be cheaper, but I doubt it would matter much for your ad-hoc bpftrace scripts.
Stack traces have uses way beyond profiling. Making them effortless is extremely useful to the disto ecosystem itself, IMO.
backtrace() is effortless stack tracing, and it uses DWARF. There's libbacktrace as well (used by GCC go which uses DWARF and not frame pointers). Richard. -- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)