Log in

No account? Create an account
Bugs, Meetings, Testing - brad's life — LiveJournal [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Bugs, Meetings, Testing [Jul. 10th, 2005|11:46 pm]
Brad Fitzpatrick
[Tags|, , , , ]

So the combination of like four of us teamed up to contribute seemingly benign patches which in combination produced a bug that was caught and fixed within hours of release.

It was suggested that we have a post-mortem to discuss it, but I, being stubborn, said I wouldn't be attending, since we'd already had a suitable (I thought) post-mortem with the developers involved the night of the bug, and we'd recently had a long post-mortem for another unrelated issue that ended up covering tons of stuff, so I thought nothing new could come of this 3rd post-mortem that wasn't already covered in the first two, short of playing the blame game or something.

Ah, but I was wrong.

While I wanted to avoid a meeting and perhaps hack, it appears my time savings argument was fruitless as I've likely spent more time reading/writing emails about the meeting and the issue than the proposed meeting would've taken had I just attended.

So I admit defeat, and in addition to having spent time and my precious wrists/fingers writing emails, I will also be attending the meeting, if only to cut my losses and not type anymore.

But it might be fun as I'd love to discuss writing a test suite to cover the entirety of LiveJournal. Historically I've shunned tests, mostly because anything non-trivial I work with is distributed on lots of machines, deals with timing, and is just generally a bitch to test accurately. Lately, however, I've had success writing test suites of pretty complicated things, like LWPx::ParanoidAgent, OpenID, and just this weekend with Ben, Gearman (which Ben pretty much did).

So I'm warming up to automated testing, especially considering it'd be something the sysadmins could run first to feel better about code being pushed, and there'd be proof in the code repo that a test was or was not written. (which there would then be policy to include)

In conclusion: fun, fun.

[User Picture]From: mart
2005-07-11 07:00 am (UTC)

Don't forget that the S2 compiler has had a regression test suite for a long time! It's perhaps not the most complete regression test suite in the world, but it was one you wrote a long time ago and it wouldn't be too hard to beef it up a little to test more cases.

Still, I'm not yet convinced that something like LiveJournal can really be tested as a whole. It's too big. The individual components could perhaps be tested, but given that there are so many of these the interactions between them are… many. Writing a test suite might be easier if some of the code were refactored to avoid the duplication that's present in a few places, such as sending HTML email and so on, but then without a test suite how would you test the refactored code? Fun, fun.

(Reply) (Thread)
[User Picture]From: brad
2005-07-11 07:40 am (UTC)
Don't forget that the S2 compiler has had a regression test suite for a long time!

I quite enjoyed writing that one, back in the day. So much easier than, say, testing Perlbal.
(Reply) (Parent) (Thread)