?

Log in

No account? Create an account
Brackup 1.00 - brad's life [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Brackup 1.00 [May. 21st, 2007|02:15 am]
Brad Fitzpatrick
[Tags|, ]

I just released Brackup 1.00:

http://danga.com/dist/Brackup/Brackup-1.00.tar.gz
(also uploaded to CPAN, appearing shortly)

Use, enjoy. ChangeLog.

As noted in the ChangeLog, I now have like 40 GB of data on Amazon, encrypted, so that's kinda a sign that I'm happy and things aren't changing incompatibly in the future.
LinkReply

Comments:
From: ext_3053
2007-05-21 10:55 am (UTC)

Time

Didn't it take a shockingly long time to upload all that? Or is your net connection a lot lot faster than mine?
(Reply) (Thread)
[User Picture]From: brad
2007-05-21 03:06 pm (UTC)

Re: Time

I put it on an external hard drive and uploaded it from a good connection.
(Reply) (Parent) (Thread)
From: (Anonymous)
2007-05-21 01:47 pm (UTC)

Very very nice

Brad,
there are some things I don't understand:

- how can I restore from amazon?
- how can I list contents of my backup?
- how is the incremental thing managed? i.e. how do I decide the number of days to keep and so on

Why don't you open a mailing list for the project? Brackup seems to be already quite popular and I'm sure that thousands of users are confused by lack of documentation.
(Reply) (Thread)
[User Picture]From: brad
2007-05-21 03:09 pm (UTC)

Re: Very very nice

- how can I restore from amazon?

http://search.cpan.org/~bradfitz/Brackup-1.00/brackup-restore

- how can I list contents of my backup?

$ cat foo-20070521.brackup

- how is the incremental thing managed? i.e. how do I decide the number of days to keep and so on

Incremental backups are free (except for the *.brackup file that's produced), so backup every day. You can even rename files and rearrange directory trees, and incremental backups are STILL free, since brackup is essentially a content-addressable filesystem like git/etc, so what's stored on Amazon is based on digests of the contents of your files, not their filenames.

Another good property is that if you have multiple copies of the same file, only one copy is stored on Amazon.
(Reply) (Parent) (Thread)
[User Picture]From: grahams
2007-05-21 05:13 pm (UTC)

Re: Very very nice

Another good property is that if you have multiple copies of the same file, only one copy is stored on Amazon.

Very nice. When brackup is restoring, does it still restore the multiple copies?
(Reply) (Parent) (Thread)
[User Picture]From: brad
2007-05-21 05:24 pm (UTC)

Re: Very very nice

Of course. And mtimes/atimes are restored as well.
(Reply) (Parent) (Thread)
[User Picture]From: grahams
2007-05-21 05:25 pm (UTC)

Re: Very very nice

Awesome. This should work much better than my current tar+s3cmd hacky backup...
(Reply) (Parent) (Thread)
From: legolas
2007-05-21 10:04 pm (UTC)

Re: Very very nice

Incremental backups are free

Not really, I think? The files that stay the same are free, but the new ones must be uploaded, no?
So what happens if a file changes, say, every day. Is there any way to control how many versions of it are stored? Or always just one? Or only all of of them? (And smae thing if a file gets deleted?)
(Reply) (Parent) (Thread)
[User Picture]From: brad
2007-05-22 03:53 am (UTC)

Re: Very very nice

Currently all versions get stored, forever. The Target garbage collector (delete all chunks not referenced in set of backups you care about) isn't yet done.
(Reply) (Parent) (Thread)
[User Picture]From: brad
2007-05-22 04:24 am (UTC)

Re: Very very nice

The files that stay the same are free, but the new ones must be uploaded, no?

Obviously. I thought that went without saying. :)
(Reply) (Parent) (Thread)
[User Picture]From: radven
2007-05-21 10:56 pm (UTC)

Re: Very very nice

"Another good property is that if you have multiple copies of the same file, only one copy is stored on Amazon."

Nice.

I've never been able to find a good tool to scan (multiple) HD's for "true duplicates" - but I've never felt fully motivated to write one of my own.

Do you know of a good duplicate scanner tool?

- chris
(Reply) (Parent) (Thread)
From: evan
2007-05-21 02:46 pm (UTC)
It's interesting that you just use sqlite as a key-value store, stuffing multiple fields into a single column. This sort of pattern sorta feels like a failure of the db, but I also see it at G with bigtable: columns in bigtable are just for independent "blobs" of data for fetching, and it's quite common to stick a (complicated, even recursive) data structure into a single column if you're usually going to need all the data together.
(Reply) (Thread)
[User Picture]From: brad
2007-05-21 03:05 pm (UTC)
I don't intend to use SQLite myself long-term, but I wanted an abstract interface that represented a dictionary, so people could drop in their own.. SQLite's just an obvious default case, since people understand/have it.
(Reply) (Parent) (Thread)
From: evan
2007-05-21 04:01 pm (UTC)
I guess Berkeley DB is a better fit for your usage pattern anyway. Might be more portable, too, unless Perl has the same confusing mess of different bdb-ish libraries found in other languages.
(Reply) (Parent) (Thread)
[User Picture]From: brad
2007-05-21 04:08 pm (UTC)
I think GDBM comes standard with Perl, so I should switch to that.
(Reply) (Parent) (Thread)
[User Picture]From: joshc
2007-05-21 09:57 pm (UTC)
How much does it cost to store/backup 40 GB on Amazon?
(Reply) (Thread)
[User Picture]From: brad
2007-05-22 03:56 am (UTC)
(Reply) (Parent) (Thread)