It was suggested that we have a post-mortem to discuss it, but I, being stubborn, said I wouldn't be attending, since we'd already had a suitable (I thought) post-mortem with the developers involved the night of the bug, and we'd recently had a long post-mortem for another unrelated issue that ended up covering tons of stuff, so I thought nothing new could come of this 3rd post-mortem that wasn't already covered in the first two, short of playing the blame game or something.
Ah, but I was wrong.
While I wanted to avoid a meeting and perhaps hack, it appears my time savings argument was fruitless as I've likely spent more time reading/writing emails about the meeting and the issue than the proposed meeting would've taken had I just attended.
So I admit defeat, and in addition to having spent time and my precious wrists/fingers writing emails, I will also be attending the meeting, if only to cut my losses and not type anymore.
But it might be fun as I'd love to discuss writing a test suite to cover the entirety of LiveJournal. Historically I've shunned tests, mostly because anything non-trivial I work with is distributed on lots of machines, deals with timing, and is just generally a bitch to test accurately. Lately, however, I've had success writing test suites of pretty complicated things, like LWPx::ParanoidAgent, OpenID, and just this weekend with Ben, Gearman (which Ben pretty much did).
So I'm warming up to automated testing, especially considering it'd be something the sysadmins could run first to feel better about code being pushed, and there'd be proof in the code repo that a test was or was not written. (which there would then be policy to include)
In conclusion: fun, fun.