Heh, after reading the proposed solution, it seems like such an obviously good way to do it... Funny how that always seems to work out.

i was just thinking about the memcached memory stuff last night, because i was reading that swarm book (i think the idea is it's a new approach for solving optimization problems-- i know toast did some toy apps with "ants" and i think he may have used in some chemistry research, too).

it seems that you could tune the slab sizes for a given dataset and that would give you the best of both worlds. it's "just" another optimization problem, no? i actually even cvs updated memcached to begin looking at how hard it would be to change the slab sizes based on some command-line parameters.

whatever happened to your analysis of that? did it ever go anywhere? the main drawback i saw was that weird-sized pieces might not fit together nicely in 1mb chunks or whatever. and, of course, that you'd have to reboot your memcaches whenever you tuned the parameters (though maybe you could do something really tricky at runtime, i'm not sure it'd be worth it).

as for the linked list: interesting idea. you definitely are sacrificing things you don't need (cpu) for things you do (memory efficiency). aside from cpu, you also lose all of your caches, but memory bandwidth probably won't be an issue until network bandwidth increases. making the chunk size tunable would also have the effect of one fixed-size slab-- if i know i'm always gonna have 1mb chunks, i could just tell memcached to make the list elements that big (well, with an adjustment for bookkeeping data).

I did the whole slab size optimization thing, and it was slow. And our application changed too often. Restarting memcached isn't an option. Plus, my goal for memcached is that administering it is dead simple: you shouldn't need to know about "slab caches". You should just start it and ignore it forever.

I don't think L1 caches are an issue here, either. Why do you think that?

my memory of caches is kinda hazy. i was under the impression cache lines were larger, on the order of like 1k, but i guess they're only 64 bytes anyway.

the reason i was worrying is that when you're pulling a k of data out of your linked-list scheme, you're (potentially) jumping all over memory, while sequential reads could hit your cache. but if you're using 64 byte chunks anyway, i guess my worry wasn't useful. (and in any case, memory bandwidth isn't your bottleneck...)

I had the same general impression as evan about cache line sizes. The reason I brought it up (on the list) was that for reads that require access to ram, the cpu has to sit around waiting for the ram, in sequential reads, pretty much only the first read is gonna have to wait for the ram, the cache is going to grab everything else before it's requested. This probably won't end up being a big performance hit, but it's likely to be noticable. The significant increase in memory allocation efficiency is probably more important than the marginal loss in performance.