The weather lately is so wonderful. Every few minutes today I just opened the door and walked outside for a bit and walked back in. Had all the windows open... gentle breeze... birds chirping.
I went for a bike ride & a run, then another run in the evening.
This morning (and when I say morning, I mean noon) I had a great idea in the shower:
A distributed memory cache daemon with a bounded memory limit. The LiveJournal server config file will just have an array of IPs and ports running cache daemons, and then the LJ API will be able intelligently hash a get/set request onto the right process.
And our idle machine with 12 GB? No problem... we just run 4 memcached processes with 2.5GB limits and put them all in the config. Any machine that has extra memory, we use it up, running a memcached.
Any process can go up or down at any time. The LJ API will consistently rehash the key until it hits a memcache process that’s alive. (and keep track of down servers, so it doesn’t keep trying to connect to them)
And if a process goes down and comes back up later? So now it’s empty, but gets refilled, and the stuff that got put on the other cache processes will quickly pass out of the cache without ever being promoted by an access. (I'm also considering making the client API know the uptime of the cache server, and if it's too low, assume it was recently dead and move onto the next hashed process if it didn't get what it wanted from the real one. If it ends up getting something from the second, put it in the first and delete it from the second.)
The data structure is just a dictionary. By default the hashing is over the key, but callers can optionally provide an alternate hashing value (say, username), you’ll get all their content together on the same cache process, making lookups take less connects later. (not that local connects are expensive... *shrug*)
Cache items can also have expiration times. The cache is LRU and everything is O(1). Accessed items get promoted to the top of the list. Neglected items fall of the bottom when the system needs to purge.
So, that’s what I built today.
The memcached process also provides commands to view stats: cache size in items/bytes, cache items, ages, hit rates.
The server is about 100 lines (thanks to the beauty of POE) and the client API is about 80.
Will it work? The question is: how many disk seeks can we avoid? I guess we’ll find that out tomorrow or so. I need to go use the API now to cache all the expensive/seeky DB calls.
Or sleep. I think it's getting light out, though... that makes it harder to sleep.