Log in

No account? Create an account
mailing lists - brad's life [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

mailing lists [Aug. 6th, 2007|10:11 pm]
Brad Fitzpatrick

Fun with sh....
$ cd /var/lib/mailman/lists
$ (for LIST in *;do (echo -n "$LIST ";list_members $LIST|wc -l); done)|sort -n -k2

mailman 1
brackup 13
memcached-commits 14
gearman 25
djabberd 123
perlbal 197
mogilefs 347
yadis 413
memcached 1120
Guess I know which projects are popular.

(yadis is the old name for OpenID, btw...)

I'm actually surprised that brackup has any subscribers, considering I just put up the mailing list and there's been like zero posts so far.

(playing with qpsmtpd... adding more fun plugins)

From: (Anonymous)
2007-08-07 06:57 am (UTC)

Three questions if you don't mind...

After leaving:

1) Will you continue to blog your adventures of hackerdom?

2) Will you continue to develop your open source projects?

3) Can you deal with giving up the awesomeness that is Perl?
(Reply) (Thread)
[User Picture]From: ciphergoth
2007-08-07 08:06 am (UTC)
Maybe brackup just works? I'm certainly planning on using it for backups RSN - I don't know of anything else that does encrypted backups properly.
(Reply) (Thread)
[User Picture]From: grumpy_sysadmin
2007-08-08 05:54 pm (UTC)
You mean brackup includes hardware doo-dads for my tape drives? Neat!

(Hint: all software encryption for backups is brain-damaged by design except in the rare case that you're also doing source dedupe.)
(Reply) (Parent) (Thread)
[User Picture]From: ciphergoth
2007-08-08 06:34 pm (UTC)
Brackup's use of crypto doesn't seem brain-damaged to me - explain further?
(Reply) (Parent) (Thread)
[User Picture]From: grumpy_sysadmin
2007-08-08 07:08 pm (UTC)
Doing encryption for (tape or disk) backups at the backup client side means that you lose out on compression, both for transfer over the wire and for transfer to storage medium (whatever it is).

Doing it on the server side is slightly better, but the backup server's got better things to do (like schedule backups and do I/O) than spend a bunch of cycles doing encryption.

Further, all modern tape drives have GOOD hardware compression... which is completely foiled by sending them gibberish (and tape remains the best place for long-term storage of data; don't let anybody trying to peddle optical media tell ya different).

The only right place for encryption of backups is the same place where the compression belongs: in a chip, on the tape drive, with removable smart cards for key data (the size of SIMs: that's exactly the format that Decru DataForts--now owned by NetApp--use for their key data, but they're just inline SCSI/FC, which means they still break compression).

I do, in fact, do this for a living.
(Reply) (Parent) (Thread)
[User Picture]From: brad
2007-08-08 07:15 pm (UTC)
I'm sorry, but...


For the rest of us that just don't care about special hardware, brackup is exactly what we want. Like I'd ever change a tape. S3 is perfect for me.
(Reply) (Parent) (Thread)
[User Picture]From: grumpy_sysadmin
2007-08-08 07:24 pm (UTC)
Okay, then the feature to go care more about is dedupe. (Preferably source side. You can still do that after encryption and dedupe across multiple clients if you store block-level pre-encryption hashes.)
(Reply) (Parent) (Thread)
[User Picture]From: grumpy_sysadmin
2007-08-08 07:27 pm (UTC)
Oh, and hardware encryption on affordable-by-human-beings tape drives isn't all that far off, really. Should be in the next Ultrium standard, and then the consumer market will actually chase after it. (Nobody really cares when SunTK does things, since they still want $45k a pop for a drive.)
(Reply) (Parent) (Thread)
[User Picture]From: grumpy_sysadmin
2007-08-08 07:11 pm (UTC)
PS, if memory serves, Bacula can also do encryption. (Not that I'm advocating it or brackup over the other; I've remained mostly lazy and one-off about backups on my home systems, but I've used all the major commercial options, so I'll get around to evaluating each and contributing to the one I like better eventually.)
(Reply) (Parent) (Thread)
[User Picture]From: grumpy_sysadmin
2007-08-09 12:17 am (UTC)
Wait, hang on, rewind.

I said this badly because I was dumbly reading lj at work to get away from being pissed off with frustration over bureaucracy and co-workers.

brackup isn't solving even remotely the same problem that I'm babbling about, and I should have kept my mouth shut, but I didn't so I'll do some cleanup now.

This is another (almost said "the other", and then realized I'd probably have to go apologize for that later) case where source-side encryption makes sense. It's also a place where dedupe is obvious. And brad is already doing all that. (I'd argue that it would be better to change the dedupe method a bit to deal with multiple-source dedupe, but without looking more closely at the implementation, I'm not even sure it won't do that just fine already.)

I had brackup confused with some other open source projects aimed at more traditional modes of (network) backup. Mea culpa.
(Reply) (Parent) (Thread)
From: jmason
2007-08-07 11:56 am (UTC)


qpsmtpd is indeed awesome. We use it to handle the SpamAssassin spamtraps, with great results.
(Reply) (Thread)
From: baudehlo
2007-08-07 01:39 pm (UTC)
playing with qpsmtpd... adding more fun plugins

Like what?
(Reply) (Thread)
[User Picture]From: matgb
2007-08-07 06:50 pm (UTC)
I'm guessing Yadis is popular because a lot of people (including non-coders like me) can see the potential of a distributed ID system and want to see it expand and be adopted, etc.

Memcache, otoh, is popular because, well, dude, Facebook. And Wikipedia. And Slashdot. Oh, and that other site that uses it, the one everyone's leaving at the moment...

On the subject of OpenID though; any chance you could get Blogspot/Google Accounts to be identity providers? That'd make life so much easier for those of us wanting to turn of anonymity on our various blogs.
(Reply) (Thread)