[illumos-Developer] [GSoC] libc: scalable malloc implementation

[Albert Lee]

Oh, and our default malloc (the one in libc) is complete rubbish. It
can only use brk/sbrk. Best possible outcome is that it gets
completely replaced by the new implementation.

-Albert

On Wed, Apr 6, 2011 at 9:15 PM, Albert Lee <trisk at opensolaris.org> wrote:
> On Wed, Apr 6, 2011 at 8:26 PM, Luca Longinotti <chtekk at longitekk.com> wrote:
>> Hi, I'm very much interested in the above task for this year's Summer
>> of Code. My name's Luca Longinotti, I'm a computer-science student at
>> the University of Zürich, Switzerland. My main interests include
>> multiprocessor scalability (ie. threading, especially lock-freedom and
>> atomic operations, ...) and OS design, basically if it's kinda
>> low-level I like it. I was thus happily excited, when checking the GSoC
>> ideas of the various OS-es this year, to find the implementation of a
>> scalable malloc, seeing as it is something I started to take a look at
>> lately, because I was seeing horrible performance of glibc's malloc
>> while benchmarking a lock-free data-structures library that I'm
>> developing, and found the memory allocator to be the culprit.
>> I've written up my proposal, and wouldn't mind some feedback before
>> officially submitting it. ;) It can be downloaded at [1].
>> I'm always available via E-Mail, and freenode's IRC is no stranger to
>> me either (nick chtekk).
>> Have a good day everyone and thanks!
>>
>> [1]
>> https://chtekk.longitekk.com/illumos_gsoc2011_libc_scalable_malloc.pdf
>>
>> P.S.: I've tried to contact trisk (Albert Lee) off-list, as he's listed
>> as possible mentor on the GSoC page, but haven't gotten any answer yet,
>> hopefully he'll see this. :)
>
> Hi!
>
> I double-checked just now but I can't find any emails from you. I hope
> other emails haven't been going to a black hole somewhere... :(
>
> libumem is the current reasonable memory allocator in illumos, based
> on the kernel slab allocator. It incorporates Jeff Bonwick and
> Jonathan Adams' very important magazine layer in front of the slabs
> for per-CPU allocations. Their "Magazines and Vmem" paper describes
> the design, which like Bonwick's original slab allocator has been
> incorporated into many modern OSes.
>
> Something that might be useful to you is a port [1] of DragonFly BSD's
> "nmalloc" allocator [2], which compares favourably with dlmalloc 2 and
> jemalloc in threaded performance and fragmentation. It is also a
> magazined slab allocator design and quite small and readable.
>
> The author of nmalloc (who I've CC'ed, hi Venk!) also "strongly
> recommends" "Features of a Multi-Threaded Memory Allocator" [4] and
> Jason Evans' talk [5] for Facebook on jemalloc. :)
>
> [1] http://hg.openindiana.org/nmalloc/
> [2] http://leaf.dragonflybsd.org/mailarchive/users/2010-10/msg00109.html
>
> [4] http://endeavour.zapto.org/src/malloc-thesis.pdf
> [5] http://leaf.dragonflybsd.org/~vsrinivas/jemalloc-tech-talk.ogv
>
> Thanks,
>
> -Albert
>
>> --
>> Best regards, Luca Longinotti aka chtekk.
>>
>> LongiTEKK Networks Admin: chtekk at longitekk.com
>> TILUG Member: chtekk at tilug.ch
>>
>