[illumos-Developer] [GSoC] libc: scalable malloc implementation

[Albert Lee]

On Wed, Apr 6, 2011 at 8:26 PM, Luca Longinotti <chtekk at longitekk.com> wrote:
> Hi, I'm very much interested in the above task for this year's Summer
> of Code. My name's Luca Longinotti, I'm a computer-science student at
> the University of Zürich, Switzerland. My main interests include
> multiprocessor scalability (ie. threading, especially lock-freedom and
> atomic operations, ...) and OS design, basically if it's kinda
> low-level I like it. I was thus happily excited, when checking the GSoC
> ideas of the various OS-es this year, to find the implementation of a
> scalable malloc, seeing as it is something I started to take a look at
> lately, because I was seeing horrible performance of glibc's malloc
> while benchmarking a lock-free data-structures library that I'm
> developing, and found the memory allocator to be the culprit.
> I've written up my proposal, and wouldn't mind some feedback before
> officially submitting it. ;) It can be downloaded at [1].
> I'm always available via E-Mail, and freenode's IRC is no stranger to
> me either (nick chtekk).
> Have a good day everyone and thanks!
>
> [1]
> https://chtekk.longitekk.com/illumos_gsoc2011_libc_scalable_malloc.pdf
>
> P.S.: I've tried to contact trisk (Albert Lee) off-list, as he's listed
> as possible mentor on the GSoC page, but haven't gotten any answer yet,
> hopefully he'll see this. :)

Hi!

I double-checked just now but I can't find any emails from you. I hope
other emails haven't been going to a black hole somewhere... :(

libumem is the current reasonable memory allocator in illumos, based
on the kernel slab allocator. It incorporates Jeff Bonwick and
Jonathan Adams' very important magazine layer in front of the slabs
for per-CPU allocations. Their "Magazines and Vmem" paper describes
the design, which like Bonwick's original slab allocator has been
incorporated into many modern OSes.

Something that might be useful to you is a port [1] of DragonFly BSD's
"nmalloc" allocator [2], which compares favourably with dlmalloc 2 and
jemalloc in threaded performance and fragmentation. It is also a
magazined slab allocator design and quite small and readable.

The author of nmalloc (who I've CC'ed, hi Venk!) also "strongly
recommends" "Features of a Multi-Threaded Memory Allocator" [4] and
Jason Evans' talk [5] for Facebook on jemalloc. :)

[1] http://hg.openindiana.org/nmalloc/
[2] http://leaf.dragonflybsd.org/mailarchive/users/2010-10/msg00109.html

[4] http://endeavour.zapto.org/src/malloc-thesis.pdf
[5] http://leaf.dragonflybsd.org/~vsrinivas/jemalloc-tech-talk.ogv

Thanks,

-Albert

> --
> Best regards, Luca Longinotti aka chtekk.
>
> LongiTEKK Networks Admin: chtekk at longitekk.com
> TILUG Member: chtekk at tilug.ch
>