[mongodb-dev] tcmalloc vs glibc 2.26

[m***@redhat.com]

Recent glibc improved malloc implementation.

NEWS for version 2.26

=====================
* A per-thread cache has been added to malloc. Access to the cache requires
no locks and therefore significantly accelerates the fast path to allocate
and free small amounts of memory. Refilling an empty cache requires
locking
the underlying arena. Performance measurements show significant gains in a
wide variety of user workloads. Workloads were captured using a special
instrumented malloc and analyzed with a malloc simulator. Contributed by
DJ Delorie with the help of Florian Weimer, and Carlos O'Donell.
(https://sourceware.org/ml/libc-alpha/2017-08/msg00010.html)

As I understand mongodb suggest using libtcmalloc allocator. Right?

How do you measure allocator performance? Any chance to switch to glibc
allocator?

Thanks for answers,
Marek

['Andrew Morrow' via mongodb-dev]

Hi Marek -

I think it is unlikely that we would change the default allocator anytime
soon. We know tcmalloc well, and we surface instrumentation data that it
provides in our always-on diagnostic data. Even if we were to find that the
glibc allocator offered superior performance over tcmalloc, we would first
need to understand what allocator instrumentation glibc provides and
integrate with it in a way that did not degrade our diagnostic capabilities
before we would considering switching.

Definitely something we will keep an eye on, of course, as it would allow
us to shed a dependency.

Thanks,
Andrew

Post by m***@redhat.com
Recent glibc improved malloc implementation.
NEWS for version 2.26

=====================
* A per-thread cache has been added to malloc. Access to the cache requires
no locks and therefore significantly accelerates the fast path to allocate
and free small amounts of memory. Refilling an empty cache requires
locking
the underlying arena. Performance measurements show significant gains in a
wide variety of user workloads. Workloads were captured using a special
instrumented malloc and analyzed with a malloc simulator. Contributed by
DJ Delorie with the help of Florian Weimer, and Carlos O'Donell.
(https://sourceware.org/ml/libc-alpha/2017-08/msg00010.html)

As I understand mongodb suggest using libtcmalloc allocator. Right?
How do you measure allocator performance? Any chance to switch to glibc
allocator?
Thanks for answers,
Marek
--
You received this message because you are subscribed to the Google Groups
"mongodb-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/mongodb-dev.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-dev/25e9f1cb-6250-45c5-b152-3d232203b4d0%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-dev/25e9f1cb-6250-45c5-b152-3d232203b4d0%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[MARK CALLAGHAN]

Andrew - I don't disagree with you, change is expensive. Regardless I look
forward to a better glibc malloc. In the past it provided slightly worse
throughput and much worse fragmentation compared to tcmalloc and jemalloc
in my tests. From what I read the throughput problems might have been fixed
but I have yet to learn of improvements to avoiding fragmentation.
*
http://smalldatum.blogspot.com/2017/11/concurrent-large-allocations-glibc.html
*
http://smalldatum.blogspot.com/2015/10/myrocks-versus-allocators-glibc.html
* http://smalldatum.blogspot.com/2014/12/malloc-and-mongodb-performance.html

On Mon, Apr 2, 2018 at 7:43 AM, 'Andrew Morrow' via mongodb-dev <

Post by 'Andrew Morrow' via mongodb-dev
Hi Marek -
I think it is unlikely that we would change the default allocator anytime
soon. We know tcmalloc well, and we surface instrumentation data that it
provides in our always-on diagnostic data. Even if we were to find that the
glibc allocator offered superior performance over tcmalloc, we would first
need to understand what allocator instrumentation glibc provides and
integrate with it in a way that did not degrade our diagnostic capabilities
before we would considering switching.
Definitely something we will keep an eye on, of course, as it would allow
us to shed a dependency.
Thanks,
Andrew

Post by m***@redhat.com
Recent glibc improved malloc implementation.
NEWS for version 2.26

=====================
* A per-thread cache has been added to malloc. Access to the cache requires
no locks and therefore significantly accelerates the fast path to allocate
and free small amounts of memory. Refilling an empty cache requires
locking
the underlying arena. Performance measurements show significant gains in a
wide variety of user workloads. Workloads were captured using a special
instrumented malloc and analyzed with a malloc simulator. Contributed by
DJ Delorie with the help of Florian Weimer, and Carlos O'Donell.
(https://sourceware.org/ml/libc-alpha/2017-08/msg00010.html)

As I understand mongodb suggest using libtcmalloc allocator. Right?
How do you measure allocator performance? Any chance to switch to glibc
allocator?
Thanks for answers,
Marek
--
You received this message because you are subscribed to the Google Groups
"mongodb-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/mongodb-dev.
To view this discussion on the web visit https://groups.google.com/d/ms
gid/mongodb-dev/25e9f1cb-6250-45c5-b152-3d232203b4d0%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-dev/25e9f1cb-6250-45c5-b152-3d232203b4d0%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"mongodb-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/mongodb-dev.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-dev/CAHX05qFg-eouBu-h%3DB3iUXKwmW3T0i1%
2Bh6b-zEs_0QFYwTj6SA%40mail.gmail.com
<https://groups.google.com/d/msgid/mongodb-dev/CAHX05qFg-eouBu-h%3DB3iUXKwmW3T0i1%2Bh6b-zEs_0QFYwTj6SA%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--
Mark Callaghan
***@gmail.com