Brad Fitzpatrick (brad) wrote,
Brad Fitzpatrick
brad

Parallel compression

Are there any multi-threaded compression algorithms, or at least wrappers/formats for interleaved compression, with variable interleave size?

It'd be nice to take advantage of multiple processors when gzipping 380 GB, while still doing sequential reads, even if the resultant file was non-standard.
Subscribe

  • Post a new comment

    Error

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.
    Help
  • 13 comments