This adds a new `final_round` parameter to the evaluator, which allows
us to determine when to keep the `filtered` data and when to keep the
compressed `idat_data`. By only keeping one of these we can
significantly improve memory usage.
Ref #683
Two key changes here:
- Removed some defunct reduction benches and added some more alpha
reduction benches.
- Changed use of `chunks()` to `chunks_exact()` where appropriate to
improve performance. (As a byproduct, this also fixes a potential crash
if a malformed file has a PLTE chunk that isn't a multiple of 3.)
If no palette reduction occurred (or palette reductions were off), a
fully transparent palette entry may not have been zeroed to black. By
ensuring this gets cleaned in the `indexed_to_channels` transformation,
we may able to achieve a grayscale conversion that would otherwise have
been RGB.
This is the final piece of the puzzle in #649 to achieve better file
sizes on the first run and avoid further changes on a second run.
Results from the images in #649 (combining this PR and #673):
9.1.3 1st run: 2,191,773
9.1.3 2nd run: 2,191,629
PR 1st run: 2,191,380
PR 2nd run: 2,191,380
If multiple edge colours were tied for most popular, the one chosen
would depend on the current palette order, which could cause it to
change if run a second time.
This further improves determinism for #649. There is still one more
issue I've found affecting the images there which I'll try to address in
another PR.
The evaluator uses the `estimated_output_size` internally but we were
passing only the idat size in the fast filter evaluator. At level `o2`,
or higher levels when using `--fast`, this could cause a slight bias
towards the None and Bigrams filters for indexed images. The actual
impact to results is incredibly minor but it's definitely incorrect so
we should fix it.
This changes the evaluator to prefer a later image instead of an earlier
one in the case of a tie. This gives more consistent results since the
baseline is always added last.
I also removed bit depth from the key since the uncompressed size should
cover that.
Fixes#649.
(Mostly. I did discover another possible but very rare issue: if
multiple colours tie for "most popular edge colour" in the luma sort,
the sort may affect which one is actually picked and multiple runs may
flip-flop back and forth)
I don't know if this is a bug in the mangen but it seems to require
explicitly setting the value name to display correctly.
Before:
```
--zi [default: 15]
Set the number of iterations to use for Zopfli compression.
```
After:
```
--zi=iterations [default: 15]
Set the number of iterations to use for Zopfli compression.
```
This does not affect MANUAL.txt or the built-in help.
#648 may have been a bit hasty - I realised afterward that there's a
simpler way to achieve the same thing, and include the Brute filter as
well.
This reverts #648 and instead just picks None up front if the line is
all zeros. This is guaranteed to be the chosen filter for MinSum,
Entropy, Bigrams and BigEnt. It's almost certainly true for Brute as
well but this is harder to prove. I've tested this across hundreds of
images and found no change in output.
Allow the creation of PNGs with uncompressed deflate streams via level 0
of libdeflate. If you want a glorified BMP with delta filters, this
change will make your dreams come true 🙂.
This PR extends the `--zopfli` argument with an optional iteration
count. In my case, I have a bunch of very small images (a few kB or
less), and I often like to use hundreds of iterations to squeeze off the
last several bytes. (I know that this crate isn't intended for
brute-force optimization, but I've found that some of its
transformations and filter strategies can be more creative than
`zopflipng`.) But this is also useful in the opposite direction, for
allowing Zopfli compression on large images where 15 iterations would be
prohibitive.
I have identified two potential improvements for how we perform static analysis
on our code in our CI pipeline:
- The `giraffate/clippy-action` we currently use has not been updated to Node
20, and GitHub has repeatedly indicated that they will phase out actions that
do not support the latest Node versions. Despite my efforts to help with the
update by submitting a pull request upstream, it has been ignored for months
despite its perceived ease of review, raising concerns about the ongoing
maintenance of the action. This situation suggests we should explore
alternative methods for integrating Clippy with GitHub's UI.
- As evidenced by PR 632, thoroughly testing Rust crates for every possible
feature combination is often overlooked due to the tedious nature of the task.
Our current CI setup only checks two feature combinations, which is far from
comprehensive.
To address the first improvement, these changes drop `clippy-action` entirely in
favor of utilizing GitHub's native CodeQL SARIF (Static Analysis Results
Interchange Format) file integration. Since Clippy cannot directly output lints
in SARIF, `clippy-sarif` is used to convert Clippy's JSON output to SARIF.
Additionally, `sarif-fmt` is added to turn SARIF into a human-friendly display
format in the workflow run logs.
For the second improvement, let's use `cargo hack` with the `--feature-powerset`
flag to run Clippy for every possible feature combination. This approach strikes
a good balance between CI runtime and thoroughness, as the number of feature
combinations grows superlinearly with the number of features: running `cargo
nextest` for every powerset element would lead to excessively long CI times.
Fixes#619.
Interlacing still defaults to 0 when `--nx` is not specified and can
still be explicitly set even if it is (e.g. `-i 0 --nx` will still
deinterlace).
This PR adds the modified zeng ("mzeng") palette sorting method, in
addition to the existing luma and battiato methods. Speed is very
similar to the battiato method with slightly better results on average.
Resulting sizes from two different image sets (all indexed or able to be
indexed):
| | master | PR |
|-|-|-|
| Set 1 | 29,647,156 | 29,555,697 |
| Set 2 | 23,732,133 | 23,570,862 |
Additionally, I've added a new "first colour" heuristic for both the
mzeng and battiato methods: We use the most popular colour overall, but
only if it covers at least 15% of the image. This provided 13k savings
on Set 2 vs the edge colour heuristic (which is still used in the luma
sort).
This PR adds a build script to generate a man page using clap_mangen, as
per this example:
https://github.com/sondr3/clap-man-example/blob/main/build.rs
I'm not sure what to actually do with the man file from here, I guess
it's up to the packaging process to do something with it?
See
https://github.com/shssoichiro/oxipng/issues/69#issuecomment-1963352536
Note I couldn't see a way to include the `DISPLAY` chunk names from the
constant as we did before. They're now just hardcoded into the help and
will require manually updating if the list changes.
Closes#526
---------
Co-authored-by: Alejandro González <me@alegon.dev>
