Brad Fitzpatrick (brad) wrote,
Brad Fitzpatrick

hacking update

Did a bunch of work on brackup this weekend:
  - make decryption --use-agent and --batch, and help out if env not set
    and gpg-agent probably not running
  - new --du-stats to command to act like the du(1) command, but
    based on a root in brackup.conf, and skipping ignored directories.
    good to let you know how big a backup will be.

  - walk directories smarter: jump over directories early which ignore
    patterns show as never matching.

  - deal w/ encryption better:  tell chunks when the backup target
    will need data, so it can forget cached digest/backlength
    ahead of time w/o errors/warnings later.

  - start of stats code (to give stats after a backup).  not done.
In regards to this previous post about me changing the format: don't worry (all two of you), I'm not. I realized all the metadata I need is (mostly) in the *.brackup backup meta files done at the end of each backup (and stored on the target), so I don't need to introduce more.

Also planned how compression will work for the not-using-gpg case. (but I also should explicitly set some gpg settings when having it do compression)

I'm planning on splitting the Brackup::Chunk class into 2 or 3 distinct-but-related classes.... Brackup::RawChunk, Brackup::StoredChunk, Brackup::ChunkHandle ... realized it's confusing how it is now, trying to pretend the RawChunk -> StoredChunk is a 1:1 mapping when it's not.

Then I want to separate out the "digest database" into separate data structures so they can be maintained and documented separately. For instance, the digest databased used to be called the "digest cache" when all it did was cache the digests of files and chunks of files... I want to get back to that, and the more important stuff move to per-target "What does this target have stored?" databases. And then make brackup have a --rebuild-target-inventory-dbs mode or something.

etc, etc.

But it's coming along well. Thinking this all through has taken more time than hacking it out will, so I feel like it's mostly done now. Fortunately I'm getting antsy to get my ~30GB of stuff uploaded, so that'll keep me from getting sidetracked.
Tags: brackup

  • Happy Birthday!

    Happy 20th Birthday, LiveJournal! 🐐🎂🎉

  • hi

    Posting from the iPhone app. Maybe I'm unblocked now.

  • Why, hello...

    Long time no see. How's my baby doing?

  • Post a new comment


    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.