This PR makes the oxipng binary process multiple files in parallel,
finally fulfilling #275. There seemed to be some debate about whether
oxipng _should_ do this or not but there's a couple of reasons I think
it makes sense:
1. The concern seemed mostly around the complexity of such a feature.
Not to worry, it was trivial* 🙂
2. Since then, oxipng has dropped from a max of something like 180
simultaneous compression trials down to 10, which is very much a good
thing but it does mean it's not utilising any more cores than that.
Some benchmarks on around 100 files on a machine with 8 cores:
Level | Master time | PR time
-|-|-
2 | 28.303 | 19.005
3 | 36.507 | 23.089
5 | 1:10.86 | 1:16.01
*Some additional changes were required in order to make sure sensible
output is printed to the terminal, since things won't be in order
anymore. Here's some example output from before:
```
Processing: tests/files/fully_optimized.png
file size = 67 bytes (0 bytes = 0.00% decrease)
File already optimized
Processing: tests/files/corrupted_header.png
Invalid PNG header detected
Processing: tests/files/verbose_mode.png
file size = 102480 bytes (12228 bytes = 10.66% decrease)
Output: tests/files/verbose_mode.png
```
And after:
```
Processing: tests/files/verbose_mode.png
Processing: tests/files/fully_optimized.png
Processing: tests/files/corrupted_header.png
tests/files/corrupted_header.png: Invalid PNG header detected
tests/files/fully_optimized.png: Could not optimize further, no change written
102480 bytes (10.66% smaller): tests/files/verbose_mode.png
```
Closes#275, #84, #169, #196 and #419.
[edit] This is the last thing I wanted to land before the next release 🥳
* Add .whitesource configuration file
* Experimental: allow Zopfli to use any size BufWriter
* Allow user to specify the output buffer size as well
* Allow user to specify maximum block splits
* Reformat and fix warnings
* Use deflater on iCCP chunk as well
* Bug fix: need to implement Zlib format
* Make functions const when possible
* Switch to using zopfli::Options in prep for https://github.com/zopfli-rs/zopfli/pull/21
* Switch to using zopfli::Options in prep for https://github.com/zopfli-rs/zopfli/pull/21
* Cargo fmt
* Fix compilation
* Fix tests
* Fix more lints
* Fix more lints
* Fix compilation more
---------
Co-authored-by: mend-bolt-for-github[bot] <42819689+mend-bolt-for-github[bot]@users.noreply.github.com>
Co-authored-by: Chris Hennick <hennickc@amazon.com>
Co-authored-by: Chris Hennick <4961925+Pr0methean@users.noreply.github.com>
* Keep track of number of pixels in each scanline
* Expand depth to 8-bit
* Attempt expand bit depth
* Simplify tracking of reduction_occurred
* Fix test
* Simplify depth handling in reductions
* Add tests for disabled reductions
* Update Zopfli and several other depenencies
I've just published a new Zopfli release, v0.7.3, which includes several
new features and internal refactors. Performance should be a tad bit
better, but I didn't test it throughly, so YMMV. Perhaps more
importantly for OxiPNG, its dependency tree is smaller, and
Gzip-exclusive compression code can be excluded at compile time thanks
to new feature switches. As I mentioned on
https://github.com/shssoichiro/oxipng/pull/495#issuecomment-1552669505,
I plan on delivering more significant Zopfli performance improvements at
some point, but for now I think it's good to give the new release more
real-world usage and testing 😄
While at it, I've upgraded other dependencies that are not
performance-critical to their latest semver-compatible versions. This
excludes `libdeflater` on purpose, as its performance characteristics
are said to be somewhat different.
* Update Zopfli to v0.7.4
v0.7.3 was superseeded shortly after v0.7.3 was released to address a
last minute change to the new API it introduced. OxiPNG is not affected
by this, but I think it's good practice to update Zopfli anyway.
* Fix bKGD conversion from gray to palette
* Allow grayscale reduction from 16 to 4 or less
* Refactor reduction evaluation sequence
* Separate palette into new file
* Refactor ColorType
Move transparency and palette data into the ColorType
* Fixup tests
* Make more use of helper functions
* Change BitDepth to u8 representation with TryFrom
* Fix clippy lints
* Don't use unstable language features
* Restore documentation on transparency/palette
* add `workflow_dispatch` for manual running
* add `fail-fast` false in matrix
* specify `persist-credentials: false` for actions/checkout
* update actions to the latest versions
* move cache before installing the toolchain
* cache more stuff
* limit deploy to shssoichiro/oxipng repository
* reindent
* Make dependency on `image` optional
After PR https://github.com/shssoichiro/oxipng/pull/481 was merged, the
`image` dependency became unused when building with debug assertions
disabled, as it is only used to implement output sanity checks when such
assertions are enabled.
The `image` crate transitively pulls a significant amount of
dependencies, so it's useful for OxiPNG users to get rid of them when
not needed.
[Cargo does not allow specifying dependencies that are only pulled when
debug assertions are
enabled](https://github.com/rust-lang/cargo/issues/7634), so the next
best way to give users some flexibility is to gate those debug
assertions behind a feature flag.
These changes add a `sanity-checks` feature flag that controls whether
the `image` crate and the related sanity checks are compiled in. This
feature is enabled by default to keep debug builds useful to catch
problems during development.
* Fix Clippy lints
* Run tests with new sanity-checks feature enabled
`crossbeam-channel` is only used if the `parallel` flag is enabled, so
it can be gated behind that feature flag to potentially reduce the size
of the build dependency tree for dependent projects.
While at it, I've fixed several warnings emmitted in tests/flags.rs.
The `itertools` dependency became unused at a3b104a2ed, so there is no point in declaring it as a dependency any longer.
This was detected with the help of the
[`cargo-udeps`](https://github.com/est31/cargo-udeps) tool. I then
verified that it was not a false positive.
