[illumos-Developer] [GSoC] libc: scalable malloc implementation

[Luca Longinotti]

On Wed, 6 Apr 2011 21:15:46 -0400
Albert Lee <trisk at opensolaris.org> wrote:
> I double-checked just now but I can't find any emails from you. I hope
> other emails haven't been going to a black hole somewhere... :(

Hmm you might want to check your SPAM folder, a conservative
spam-filter might have thrown stuff in there, as I forgot to update the
SPF records for my domain to include the provider I pass mail through,
but I've fixed that now. That's the only explanation I have, as, as far
as I and Gmail can see, the mail was sent out correctly.

> libumem is the current reasonable memory allocator in illumos, based
> on the kernel slab allocator. It incorporates Jeff Bonwick and
> Jonathan Adams' very important magazine layer in front of the slabs
> for per-CPU allocations. Their "Magazines and Vmem" paper describes
> the design, which like Bonwick's original slab allocator has been
> incorporated into many modern OSes.
> 
> Something that might be useful to you is a port [1] of DragonFly BSD's
> "nmalloc" allocator [2], which compares favourably with dlmalloc 2 and
> jemalloc in threaded performance and fragmentation. It is also a
> magazined slab allocator design and quite small and readable.
> 
> The author of nmalloc (who I've CC'ed, hi Venk!) also "strongly
> recommends" "Features of a Multi-Threaded Memory Allocator" [4] and
> Jason Evans' talk [5] for Facebook on jemalloc. :)
> 
> [1] http://hg.openindiana.org/nmalloc/
> [2]
> http://leaf.dragonflybsd.org/mailarchive/users/2010-10/msg00109.html
> 
> [4] http://endeavour.zapto.org/src/malloc-thesis.pdf
> [5] http://leaf.dragonflybsd.org/~vsrinivas/jemalloc-tech-talk.ogv

Ok thanks a lot for all those resources, I've started reading through
them. The Magazines concept is something I too thought of.
I've incorporated Gordon's suggestions into my proposal and posted it
on the GSoC site, as I understand it, any feedback can be had directly
there now.
@Jörg: I'm aware that extreme care is needed here, and I'll strive to
be able to meet those expectations. Testing will of course be an
important part of the project. At the same time I don't expect to have
very much regards in being backwards-compatible to software that does
completely non-standard things and thus relies on the old memory
allocator, like directly accessing it's structures or something like
that.
And may the force be with them, in that case! ;)
-- 
Best regards, Luca Longinotti aka chtekk.

LongiTEKK Networks Admin: chtekk at longitekk.com
TILUG Member: chtekk at tilug.ch