[Bug 1201884] New: Massive augeas slowdown with MALLOC_PERTURB_
![](https://seccdn.libravatar.org/avatar/a895f78a81a109471893519443e4d933.jpg?s=120&d=mm&r=g)
https://bugzilla.suse.com/show_bug.cgi?id=1201884 Bug ID: 1201884 Summary: Massive augeas slowdown with MALLOC_PERTURB_ Classification: openSUSE Product: openSUSE Tumbleweed Version: Current Hardware: Other OS: Other Status: NEW Severity: Normal Priority: P5 - None Component: YaST2 Assignee: jsikes@suse.com Reporter: fvogt@suse.com QA Contact: jsrain@suse.com CC: pm.mullaney@gmail.com, yast2-maintainers@suse.de Found By: --- Blocker: --- I reported this upstream: https://github.com/hercules-team/augeas/issues/768 During package builds and during development phase, openSUSE sets MALLOC_CHECK_=3 and MALLOC_PERTURB_=69 in the environment. This causes the glibc heap to be less "forgiving" to catch some bugs. However, the MALLOC_PERTURB_=69 setting, which causes glibc to explicitly fill newly allocated and freed memory with that value, has a massive negative effect on augeas performance. The testsuite actually runs more than 20x slower! Effectively, the package build takes >2000s instead of 99s because of this. FWICT, the bottleneck is that add_new_trans calls realloc very often, and glibc treats that like a new allocation + deallocation to be able to overwrite the old data. Maybe that can be optimized somehow? perf report shows: - 95,52% 0,00% augparse libaugeas.so.0.25.0 [.] typecheck typecheck typecheck_iter - ambig_iter_check - 94,70% ambig_check - fa_ambig_example - 84,24% expand_alphabet (inlined) - 84,23% fa_clone - 83,90% add_new_trans - mem_realloc_n - 83,88% __realloc (inlined) - _int_realloc - 83,87% _int_free __memset_avx2_unaligned_erms needinfo on glibc to confirm that the suspicion is true. Maybe we should just unset _MALLOC_PERTURB in augeas.spec's %check section? -- You are receiving this mail because: You are on the CC list for the bug.
![](https://seccdn.libravatar.org/avatar/a895f78a81a109471893519443e4d933.jpg?s=120&d=mm&r=g)
https://bugzilla.suse.com/show_bug.cgi?id=1201884
Fabian Vogt
![](https://seccdn.libravatar.org/avatar/a895f78a81a109471893519443e4d933.jpg?s=120&d=mm&r=g)
https://bugzilla.suse.com/show_bug.cgi?id=1201884
https://bugzilla.suse.com/show_bug.cgi?id=1201884#c1
Andreas Schwab
![](https://seccdn.libravatar.org/avatar/a895f78a81a109471893519443e4d933.jpg?s=120&d=mm&r=g)
https://bugzilla.suse.com/show_bug.cgi?id=1201884
https://bugzilla.suse.com/show_bug.cgi?id=1201884#c2
Fabian Vogt
MALLOC_PERTURB_ will always touch every newly allocated and every freed memory. That's the whole point of it.
Yes, but I'm wondering whether it goes even further than that:
FWICT, the bottleneck is that add_new_trans calls realloc very often, and glibc treats that like a new allocation + deallocation to be able to overwrite the old data. Maybe that can be optimized somehow?
i.e. does glibc effectively transform realloc into malloc + free or does it realloc like before but just memset the additional space? Because according to the perf report, pretty much all of the realloc cost is spent in _int_free, while I'd expect that a noticable percentage of realloc calls could be expanded in-place. -- You are receiving this mail because: You are on the CC list for the bug.
![](https://seccdn.libravatar.org/avatar/a895f78a81a109471893519443e4d933.jpg?s=120&d=mm&r=g)
https://bugzilla.suse.com/show_bug.cgi?id=1201884
https://bugzilla.suse.com/show_bug.cgi?id=1201884#c3
Andreas Schwab
![](https://seccdn.libravatar.org/avatar/a895f78a81a109471893519443e4d933.jpg?s=120&d=mm&r=g)
https://bugzilla.suse.com/show_bug.cgi?id=1201884
https://bugzilla.suse.com/show_bug.cgi?id=1201884#c4
Jason Sikes
------------------------------------------------------------------- Tue Jul 26 15:49:09 UTC 2022 - Fabian Vogt
- Unset MALLOC_PERTURB_ to speed up %check significantly (boo#1201884, gh#hercules-team#768)
-------------------------------------------------------------------
With this line added to the spec file:
unset MALLOC_PERTURB_
Closing as fixed. Thank you, Fabian. -- You are receiving this mail because: You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply@suse.com