For my purposes, this is a pain. Is there a way to disable it? I probably shouldn't, but I'm just curious. I realize if I disable it, it's possible for others to prove the existence of a file on my machine, if they have my public key (easy), and the original file they want to check for.
So yes, I really want to keep it enabled, but I didn't plan for it, and now I have to go back and re-design a bit.
I wanted to keep my digest caches as simply caches, with no harm but extra network latency and CPU if the digest cache is lost, but now I need to factor in the face that losing the local machine's digest cache will cause extra allocation of chunks on the server (because new encrypted file differs), which you pay for (for disk usage).
Which implies I need to store the digest cache on the server as well. Encrypted.
But that doesn't scale, since it'll only get bigger with time, and the uploads will suck more and more each day, not proportional to changed files.
Which implies storing iterative/delta digest caches, and having like a digest cache master index that lists all the digest caches on the server. But then there are race conditions if multiple backups are running.
Or I could just rely on the storage target to support enumeration of like objects. Like, "give me all digest cache files", but that removes the ability of a storage target to be purely key-value. So for the Filesystem, FTP, Scp, Amazon targets, this is no problem, but it might be a problem for other targets? MogileFS? well, mogile at least lets you enumerate keys by prefix, so I guess it's possible, if you design for it.
Okay, so that's fine. I'll go with that.
But I'm just annoyed that I have to go implement it all now.
Fuck all this: I just rename the "Digest Cache" to the "Digest Database" and make it required, documenting that you can't delete it. Just store it on the same filesystem as the files you're backing up. If you lose it, well... you probably lost all your other files anyway and need to restore. Okay, the show (er, hacking) goes on.... s/cache/database/.