#648 may have been a bit hasty - I realised afterward that there's a
simpler way to achieve the same thing, and include the Brute filter as
well.
This reverts #648 and instead just picks None up front if the line is
all zeros. This is guaranteed to be the chosen filter for MinSum,
Entropy, Bigrams and BigEnt. It's almost certainly true for Brute as
well but this is harder to prove. I've tested this across hundreds of
images and found no change in output.
This adds a new palette sorting algorithm that attempts to minimise
entropy by an approximate solution to the Traveling Salesman Problem.
The algorithm comes from "An efficient Re-indexing algorithm for
color-mapped images" by Battiato et al
(https://ieeexplore.ieee.org/document/1344033).
It's fast and effective and works in addition to the luma sort (which
remains the single most effective sort). In order to keep lower presets
fast though, I've only enabled this for o3 and higher.
Results on a set of 190 indexed images at `-o5`:
18,932,727 bytes - master
18,578,306 bytes - PR
18,559,863 bytes - PR + #509
(These images may be particularly suited to alternative sorting methods
- the gains here are not necessarily what should be expected on average)
Note I looked into the 120 different palette sorting methods from
TruePNG, as mentioned in #74 (and seen in action in the Zopfli KrzYmod
fork). They're... largely ineffective. The combination of all 120
methods are outperformed by just the existing luma sort plus this new
one. That's not to say there's nothing further to be gained from them,
but trying to brute force all the combinations definitely seems like a
bad idea. There are other algorithms I hope to explore in future...
@ace-dent Thought this might interest you
UPDATE: I realised a quick tweak to alpha values in the luma sort can
provide a great improvement on images with transparency. The following
numbers were taken with PR #509 as base.
`-o2`:
19,065,549 bytes - base (luma sort)
18,949,747 bytes - modified luma sort
`-o5`:
18,922,165 bytes - base (luma sort)
18,559,863 bytes - new sorting algorithm + luma sort
18,544,813 bytes - new sorting algorithm + modified luma sort
* Add .whitesource configuration file
* Experimental: allow Zopfli to use any size BufWriter
* Allow user to specify the output buffer size as well
* Allow user to specify maximum block splits
* Reformat and fix warnings
* Use deflater on iCCP chunk as well
* Bug fix: need to implement Zlib format
* Make functions const when possible
* Switch to using zopfli::Options in prep for https://github.com/zopfli-rs/zopfli/pull/21
* Switch to using zopfli::Options in prep for https://github.com/zopfli-rs/zopfli/pull/21
* Cargo fmt
* Fix compilation
* Fix tests
* Fix more lints
* Fix more lints
* Fix compilation more
---------
Co-authored-by: mend-bolt-for-github[bot] <42819689+mend-bolt-for-github[bot]@users.noreply.github.com>
Co-authored-by: Chris Hennick <hennickc@amazon.com>
Co-authored-by: Chris Hennick <4961925+Pr0methean@users.noreply.github.com>
* Keep track of number of pixels in each scanline
* Expand depth to 8-bit
* Attempt expand bit depth
* Simplify tracking of reduction_occurred
* Fix test
* Simplify depth handling in reductions
* Add tests for disabled reductions
* Refactor ColorType
Move transparency and palette data into the ColorType
* Fixup tests
* Make more use of helper functions
* Change BitDepth to u8 representation with TryFrom
* Fix clippy lints
* Don't use unstable language features
* Restore documentation on transparency/palette
* Refactor filters as enum
* Include filter byte in filter output
* Add entropy filter
* Add bigrams filter
* Add bigram entropy filter
* Add brute filter
* Replace bit-vec
* Add tests and benches
* Show filters in help
* Use FxHasher in color to palette
* Use windows function for minor improvement
* Switch main compressor to libdeflate
* Use libdeflater in evaluate
* Use libdeflater to inflate
* Use libdeflater crc
* Tidy up
* Fix benches
* Allow libdeflater/freestanding feature
* Fix building without zopfli
* Update and optimize dependencies
These changes update the dependencies to their latest versions, fixing
some known issues that prevented doing so in the first place.
In addition, the direct dependency on byteorder was dropped in favor
of stdlib functions that have been stabilized for some time in Rust, and
the transitive dependency on chrono, pulled by stderrlog, was also
dropped, which had been affected by security issues and improperly
maintained in the past:
- https://github.com/cardoe/stderrlog-rs/issues/31
- https://www.reddit.com/r/rust/comments/ts84n4/chrono_or_time_03/
* Run rustfmt
* Bump MSRV to 1.56.1
Updating to this patch version should not be cumbersome for end-users,
and it is required by a transitive dependency.
* Bump MSRV to 1.57.0
os_str_bytes requires it.
* Improve performance consistency
Switch from HashMap / HashSet to IndexMap / IndexSet for consistent iteration order of various options and, as a result, more predictable performance.
* Fix verbose message
* No cloning when restoring original data
* Make reductions return a new uncompressed image
Partially fixes#145
* Async reduction evaluator
* Assert
* Faster bit depth check
* Also try 4-bit depth for small-depth images
* Skip test when using miniz
* Ensure palette is trimmed after depth reduction
Fixes#159
* Fudge factor for reductions to prefer better reductions even if gzip estimation says otherwise
* Move reductions to a module. Make copy instead of changing in-place.
* Alpha reductions
* Immutable color reductions
* Immutable interlace reductions
* Compress alpha with strategy sensitive to repetitions
* Update reductions flag when alpha changed
* Avoid needlessly cloning out-of-date idat_data and temp raw_data
