[opensuse-kernel] Why is CONFIG_SLAB the default in SUSE kernels ?
Hi: Just out of curiosity, why SUSE kernels default to CONFIG_SLAB and not CONFIG_SLUB ? I just checked the rest of current distribution world and everyone else takes the kernel default. I am assuming there is a reason for this, that it is intentional and not a oversight where somebody forgot changing the configuration... -- Cristian "I don't know the key to success, but the key to failure is trying to please everybody." -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
Hi Cristian, Le Tuesday 27 May 2014 à 11:49 -0400, Cristian Rodríguez a écrit :
Just out of curiosity, why SUSE kernels default to CONFIG_SLAB and not CONFIG_SLUB ? I just checked the rest of current distribution world and everyone else takes the kernel default.
I am assuming there is a reason for this, that it is intentional and not a oversight where somebody forgot changing the configuration...
The topic was already brought up for discussion in July 2008: http://lists.opensuse.org/opensuse-kernel/2008-07/msg00014.html While SLUB was already the default for one year, Jeff did not think it was the right time to switch. The lack of explanation in the upstream commit that changed the default may explain it :-( Almost 6 years later, I guess we can revisit this. While I am not familiar with SLAB vs. SLUB, I see that: * There must be a reason why SLUB is the default, and there is nothing extraordinary about openSUSE that I can think of that would justify not sticking to that default. * Using what the majority is using should make it less likely to hit a bug. And if there is a bug, it is likely to be fixed much faster. So unless someone can provide a good reason why we should stick to SLAB, I vote for switching to SLUB. -- Jean Delvare SUSE L3 Support -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On 06/04/2014 11:07 AM, Jean Delvare wrote:
Hi Cristian,
Le Tuesday 27 May 2014 à 11:49 -0400, Cristian Rodríguez a écrit :
Just out of curiosity, why SUSE kernels default to CONFIG_SLAB and not CONFIG_SLUB ? I just checked the rest of current distribution world and everyone else takes the kernel default.
I am assuming there is a reason for this, that it is intentional and not a oversight where somebody forgot changing the configuration...
The topic was already brought up for discussion in July 2008: http://lists.opensuse.org/opensuse-kernel/2008-07/msg00014.html
While SLUB was already the default for one year, Jeff did not think it was the right time to switch. The lack of explanation in the upstream commit that changed the default may explain it :-(
Almost 6 years later, I guess we can revisit this. While I am not familiar with SLAB vs. SLUB, I see that: * There must be a reason why SLUB is the default, and there is nothing extraordinary about openSUSE that I can think of that would justify not sticking to that default. * Using what the majority is using should make it less likely to hit a bug. And if there is a bug, it is likely to be fixed much faster.
So unless someone can provide a good reason why we should stick to SLAB, I vote for switching to SLUB.
+1 I build my own kernels, and I have used SLUB without problems for several years with openSUSE. As I run new kernel versions, I have hit some SLUB bugs in the -git or -rc1, but those are always fixed long before the stable release. Larry -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 6/4/14, 12:07 PM, Jean Delvare wrote:
Hi Cristian,
Le Tuesday 27 May 2014 à 11:49 -0400, Cristian Rodríguez a écrit :
Just out of curiosity, why SUSE kernels default to CONFIG_SLAB and not CONFIG_SLUB ? I just checked the rest of current distribution world and everyone else takes the kernel default.
I am assuming there is a reason for this, that it is intentional and not a oversight where somebody forgot changing the configuration...
The topic was already brought up for discussion in July 2008: http://lists.opensuse.org/opensuse-kernel/2008-07/msg00014.html
While SLUB was already the default for one year, Jeff did not think it was the right time to switch. The lack of explanation in the upstream commit that changed the default may explain it :-(
Almost 6 years later, I guess we can revisit this. While I am not familiar with SLAB vs. SLUB, I see that: * There must be a reason why SLUB is the default, and there is nothing extraordinary about openSUSE that I can think of that would justify not sticking to that default. * Using what the majority is using should make it less likely to hit a bug. And if there is a bug, it is likely to be fixed much faster.
So unless someone can provide a good reason why we should stick to SLAB, I vote for switching to SLUB.
I vote against unless there's a compelling reason to change it. Sometimes the default changes upstream because it's new and shiny and more or less working. It's a way to improve test coverage. The problem is that test coverage in most cases just means that it doesn't crash. I asked Mel about this on IRC a few days ago and the response was along the lines of "it's the devil we know." He also mentioned that he recalled the last time he looked into it there were still network performance regressions that hadn't been fixed yet. That may have changed since the last time he looked at it, but without confirmation, I'm going to assume they're still there. So far I haven't seen a compelling reason to change it other than "it's the default." I'd suggest that anyone pushing to change the choice in allocator be willing to actually do that testing and post the numbers and breadth of the test coverage before we do change it. - -Jeff - -- Jeff Mahoney SUSE Labs -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.19 (Darwin) iQIcBAEBAgAGBQJTj0cVAAoJEB57S2MheeWyV24P/1ar+lKArQM70E+cv+73rSb0 C1PDWell1STTIoE6yWOnDGoi+E59RpIvJ9bg+HEyzrmr6g0rUc5SpvfZarlBezur JkrekwbjSRBKMwaTBKmw/kZP2QvocYHu9eMvCFioD1fuckrZA8sZ2Uq5EF450eIB YDLSvb5oxh37GwSUoZnN0NPdt1s0CsVqSjRiQR+BQ+61Kt4us+VrxOxa+ZtWT08a BVfi0X15LCPKkVdsjED65l16zU+Y5iiaY1KNF6XioZc9wB8kgwya0cD2hNy44YaL nEI10p7TwiwbcqHPLkFJRMj5k+TCgsIhy7CPi/geCLil98MbhKcs1DRU9WXzSSnt PY3cOGCvsFxN6DrYDO95KCzuMrLrqE2FSU5wIn/M6/KTibtRk3h0Hmg8el80Tpz7 mC1qH3W+e65Plh3UO1lFrtj4jRsvdOQiEC86wwfmjGftRc2bjMUx3HlrNN0kIo/C tfnM4EJoe97yxFeZcE75QX+Eqg9aoMICr5xexzXvxq0v5CbFgSLxPW+NJTWq1VWK Pg69C/12wBrmFDWrBVdBtlAIfzlsuPGkDCEa+hP573jjobQACsR8N3gQB698zkhX 4m58oCUX/yK34cEIb7jBvtM3pzW8LxBpt1jFw4gQMJhWNbYxN4/3MDqJG32TgSR8 VfGCTEE6dyO0P0bFHsAn =zzp9 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Wed 04-06-14 12:19:33, Jeff Mahoney wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 6/4/14, 12:07 PM, Jean Delvare wrote:
Hi Cristian,
Le Tuesday 27 May 2014 à 11:49 -0400, Cristian Rodríguez a écrit :
Just out of curiosity, why SUSE kernels default to CONFIG_SLAB and not CONFIG_SLUB ? I just checked the rest of current distribution world and everyone else takes the kernel default.
I am assuming there is a reason for this, that it is intentional and not a oversight where somebody forgot changing the configuration...
The topic was already brought up for discussion in July 2008: http://lists.opensuse.org/opensuse-kernel/2008-07/msg00014.html
While SLUB was already the default for one year, Jeff did not think it was the right time to switch. The lack of explanation in the upstream commit that changed the default may explain it :-(
Almost 6 years later, I guess we can revisit this. While I am not familiar with SLAB vs. SLUB, I see that: * There must be a reason why SLUB is the default, and there is nothing extraordinary about openSUSE that I can think of that would justify not sticking to that default. * Using what the majority is using should make it less likely to hit a bug. And if there is a bug, it is likely to be fixed much faster.
I really do not remember any bug in SLAB during last years. The code is pretty much stable.
So unless someone can provide a good reason why we should stick to SLAB, I vote for switching to SLUB.
I vote against unless there's a compelling reason to change it.
Agreed.
Sometimes the default changes upstream because it's new and shiny and more or less working. It's a way to improve test coverage. The problem is that test coverage in most cases just means that it doesn't crash.
I asked Mel about this on IRC a few days ago and the response was along the lines of "it's the devil we know." He also mentioned that he recalled the last time he looked into it there were still network performance regressions that hadn't been fixed yet.
That may have changed since the last time he looked at it, but without confirmation, I'm going to assume they're still there. So far I haven't seen a compelling reason to change it other than "it's the default." I'd suggest that anyone pushing to change the choice in allocator be willing to actually do that testing and post the numbers and breadth of the test coverage before we do change it.
Slub allows debugging which can be turned on during runtime which is much better than in slab which has to compile debugging in which is quite impractical. But this doesn't sound like a real reason to switch our default I guess. -- Michal Hocko SUSE Labs -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Thu, Jun 05, 2014 at 11:53 Michal Hocko wrote: | On Wed 04-06-14 12:19:33, Jeff Mahoney wrote: | | > That may have changed since the last time he looked at it, but without | > confirmation, I'm going to assume they're still there. So far I | > haven't seen a compelling reason to change it other than "it's the | > default." I'd suggest that anyone pushing to change the choice in | > allocator be willing to actually do that testing and post the numbers | > and breadth of the test coverage before we do change it. | | Slub allows debugging which can be turned on during runtime which is | much better than in slab which has to compile debugging in which is | quite impractical. But this doesn't sound like a real reason to switch | our default I guess. Actually that might be a good enough reason to *consider* switching to SLUB, consider as in do the necessary regression testing, switch if no problems pop up and get uncovered issues on the radar. Chasing SLAB memory corruption and leaks in the {real world, field} is (extremely) painful as it requires recompiling the kernel along out of tree kernel modules. Cheers, Hedi. -- Hedi Berriche SGI::http://www.sgi.com -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Tue, May 27, 2014 at 11:49:08AM -0400, Cristian Rodríguez wrote:
Hi:
Just out of curiosity, why SUSE kernels default to CONFIG_SLAB and not CONFIG_SLUB ? I just checked the rest of current distribution world and everyone else takes the kernel default.
The upstream default to use SLUB was not based on performance data. This is the original commit that set the default http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=a0... To the best of my recollection it was set as a default so that it would be tested by the wider community. In the subsequent kernel releases there were a large number of patches to address performance concerns in SLUB. It has been some time since an upstream evaluation was done and at the time it was found to have problems on some workloads -- networking intensive ones in particular which are sl[a|u]b intensive. Performance changes to SLUB are now relatively rare but I'm not aware of any recent SLUB vs SLAB evaluation that would decide the matter one way or the other. It's true that a number of distributions have changed their default to SLUB but to the best of my knowledge this was not based on any performance evaluation either. The distributions changed their config to match the upstream default on the mistaken belief that it must have been made for performance reasons. For better or worse, SLAB has known performance characteristics. I'm not aware of any performance bug reports in openSUSE that identified SLAB as the bottleneck that SLUB would address. While there are some design considerations in SLAB such as the fact that alien caches can grow to a large size, it also has been found the performance of some workloads depended on those large remote caches. In some circumstances it is easier to debug problems using SLUB but that in itself is not a justification for the change. SLUB is also not without its performance concerns. IIRC, to achieve the best performance of SLUB requires the use of large contiguous pages to avoid list locks. While this will show better results for benchmarks that fit in memory, the performance can decay if the machine is under memory pressure and the ability of SLUB to use contiguous pages may decay the longer the system is up. To the best of my knowledge, this potential problem has never been properly analysed. Furthermore while SLUBs extensive use of atomic operations makes intuitive sense it is also not guaranteed to be a universal win in all cases meaning the performance profile of it is harder to predict. Due to the lack of concrete performance data identifying bottlenecks in SLAB there has been little motivation to evaluate SLUB vs SLAB in the openSUSE context. Clearly identifying a sensible workload (not a microbenchmark) that shows SLAB as the bottleneck on a range of machine size that is fundamental to its design may be grounds for revisiting this. However, the bar that would convince me to do a full evaluation would be relatively high -- much higher than "just because". -- Mel Gorman SUSE Labs -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
Le Tuesday 10 June 2014 à 14:14 +0100, Mel Gorman a écrit :
The upstream default to use SLUB was not based on performance data. This is the original commit that set the default http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=a0... To the best of my recollection it was set as a default so that it would be tested by the wider community. In the subsequent kernel releases there were a large number of patches to address performance concerns in SLUB. It has been some time since an upstream evaluation was done and at the time it was found to have problems on some workloads -- networking intensive ones in particular which are sl[a|u]b intensive. Performance changes to SLUB are now relatively rare but I'm not aware of any recent SLUB vs SLAB evaluation that would decide the matter one way or the other.
It's true that a number of distributions have changed their default to SLUB but to the best of my knowledge this was not based on any performance evaluation either. The distributions changed their config to match the upstream default on the mistaken belief that it must have been made for performance reasons.
For better or worse, SLAB has known performance characteristics. I'm not aware of any performance bug reports in openSUSE that identified SLAB as the bottleneck that SLUB would address. While there are some design considerations in SLAB such as the fact that alien caches can grow to a large size, it also has been found the performance of some workloads depended on those large remote caches. In some circumstances it is easier to debug problems using SLUB but that in itself is not a justification for the change.
SLUB is also not without its performance concerns. IIRC, to achieve the best performance of SLUB requires the use of large contiguous pages to avoid list locks. While this will show better results for benchmarks that fit in memory, the performance can decay if the machine is under memory pressure and the ability of SLUB to use contiguous pages may decay the longer the system is up. To the best of my knowledge, this potential problem has never been properly analysed. Furthermore while SLUBs extensive use of atomic operations makes intuitive sense it is also not guaranteed to be a universal win in all cases meaning the performance profile of it is harder to predict.
Due to the lack of concrete performance data identifying bottlenecks in SLAB there has been little motivation to evaluate SLUB vs SLAB in the openSUSE context. Clearly identifying a sensible workload (not a microbenchmark) that shows SLAB as the bottleneck on a range of machine size that is fundamental to its design may be grounds for revisiting this. However, the bar that would convince me to do a full evaluation would be relatively high -- much higher than "just because".
Thanks Mel for the thorough explanation. I think this closes the topic quite nicely. -- Jean Delvare SUSE L3 Support -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
participants (7)
-
Cristian Rodríguez -
Hedi Berriche -
Jean Delvare -
Jeff Mahoney -
Larry Finger -
Mel Gorman -
Michal Hocko