Let me know if I should add these to the `Cargo.toml`.
All `cargo test` tests pass on the latest x86_64-unknown-linux-gnu
nightly.
---------
Co-authored-by: Alejandro González <me@alegon.dev>
This is a collection of all the BC breaks mentioned in #714, except for
11 which I'm not including for now.
Fixes#658.
Fixes#660.
It might be best to review each commit individually, referencing the
notes in #714 and #660 (I just didn't want to create a dozen separate
PRs).
This is an experiment I started a while ago before life happened. It
reduces memory usage of fast evaluation (-o2 and lower), bringing it
inline with normal (slow) evaluation. It does this by not retaining the
filtered image data of the evaluations, but instead retaining the row
filters that were used in each line so it can be quickly re-filtered
when required. This does incur a tiny performance penalty, but it's
negligible even at `-o0`.
Although this is fully functional, there are a few rough spots in the
code so I'm just opening it as a draft for now.
```
PR -sao6
70.08 real 614.36 user 2.66 sys
2024243200 maximum resident set size
1839339520 peak memory footprint
PR -sao2
11.16 real 60.85 user 1.33 sys
1982562304 maximum resident set size
1842173824 peak memory footprint
PR -sao2 -t1
55.11 real 53.85 user 0.76 sys
429457408 maximum resident set size
245008064 peak memory footprint
master -sao6
67.70 real 616.07 user 2.72 sys
2043379712 maximum resident set size
1838340416 peak memory footprint
master -sao2
11.53 real 60.63 user 1.25 sys
2753396736 maximum resident set size
2283741440 peak memory footprint
master -sao2 -t1
54.29 real 53.55 user 0.72 sys
626311168 maximum resident set size
305252544 peak memory footprint
```
Note that this involves some refactoring of `RowFilter` and the new
`FilterStrategy`. These are breaking changes so it will ultimately be
destined for v10. One advantage to this new structure is it opens the
door for future changes such as allowing the Brute strategy to take a
parameter for the number of lines.
If no palette reduction occurred (or palette reductions were off), a
fully transparent palette entry may not have been zeroed to black. By
ensuring this gets cleaned in the `indexed_to_channels` transformation,
we may able to achieve a grayscale conversion that would otherwise have
been RGB.
This is the final piece of the puzzle in #649 to achieve better file
sizes on the first run and avoid further changes on a second run.
Results from the images in #649 (combining this PR and #673):
9.1.3 1st run: 2,191,773
9.1.3 2nd run: 2,191,629
PR 1st run: 2,191,380
PR 2nd run: 2,191,380
I've replaced a few tests with synthetic images and removed a number
that I don't believe are relevant anymore.
I've also removed the #553 test without replacement as I don't know how
to synthesise a replacement and the issue is still open anyway.
Going forward I think we should avoid adding any new "issue" tests and
instead synthesise something suitable in one of the other collections.
Closes#655.
This PR is addressing #220. It's not super important but it's a breaking
change, so if it's something we want to do then I thought I should get
it in now before the next release.
- [x] pretend can become another variant of OutFile, probably
OutFile::None, as that's what it essentially is - just another output
destination and not a separate option
- [x] ~~backup and~~ preserve_attrs should become properties of
OutFile::Path variant (so that it would contain Path { path, ~~backup,~~
preserve_attrs }) as they don't have any effect on any other output and
so semantically belong there best
Closes#220
This adds a new palette sorting algorithm that attempts to minimise
entropy by an approximate solution to the Traveling Salesman Problem.
The algorithm comes from "An efficient Re-indexing algorithm for
color-mapped images" by Battiato et al
(https://ieeexplore.ieee.org/document/1344033).
It's fast and effective and works in addition to the luma sort (which
remains the single most effective sort). In order to keep lower presets
fast though, I've only enabled this for o3 and higher.
Results on a set of 190 indexed images at `-o5`:
18,932,727 bytes - master
18,578,306 bytes - PR
18,559,863 bytes - PR + #509
(These images may be particularly suited to alternative sorting methods
- the gains here are not necessarily what should be expected on average)
Note I looked into the 120 different palette sorting methods from
TruePNG, as mentioned in #74 (and seen in action in the Zopfli KrzYmod
fork). They're... largely ineffective. The combination of all 120
methods are outperformed by just the existing luma sort plus this new
one. That's not to say there's nothing further to be gained from them,
but trying to brute force all the combinations definitely seems like a
bad idea. There are other algorithms I hope to explore in future...
@ace-dent Thought this might interest you
UPDATE: I realised a quick tweak to alpha values in the luma sort can
provide a great improvement on images with transparency. The following
numbers were taken with PR #509 as base.
`-o2`:
19,065,549 bytes - base (luma sort)
18,949,747 bytes - modified luma sort
`-o5`:
18,922,165 bytes - base (luma sort)
18,559,863 bytes - new sorting algorithm + luma sort
18,544,813 bytes - new sorting algorithm + modified luma sort
* Fix bKGD conversion from gray to palette
* Allow grayscale reduction from 16 to 4 or less
* Refactor reduction evaluation sequence
* Separate palette into new file
* Refactor ColorType
Move transparency and palette data into the ColorType
* Fixup tests
* Make more use of helper functions
* Change BitDepth to u8 representation with TryFrom
* Fix clippy lints
* Don't use unstable language features
* Restore documentation on transparency/palette
* Make dependency on `image` optional
After PR https://github.com/shssoichiro/oxipng/pull/481 was merged, the
`image` dependency became unused when building with debug assertions
disabled, as it is only used to implement output sanity checks when such
assertions are enabled.
The `image` crate transitively pulls a significant amount of
dependencies, so it's useful for OxiPNG users to get rid of them when
not needed.
[Cargo does not allow specifying dependencies that are only pulled when
debug assertions are
enabled](https://github.com/rust-lang/cargo/issues/7634), so the next
best way to give users some flexibility is to gate those debug
assertions behind a feature flag.
These changes add a `sanity-checks` feature flag that controls whether
the `image` crate and the related sanity checks are compiled in. This
feature is enabled by default to keep debug builds useful to catch
problems during development.
* Fix Clippy lints
* Run tests with new sanity-checks feature enabled
* Refactor filters as enum
* Include filter byte in filter output
* Add entropy filter
* Add bigrams filter
* Add bigram entropy filter
* Add brute filter
* Replace bit-vec
* Add tests and benches
* Show filters in help
* Use FxHasher in color to palette
* Use windows function for minor improvement
This allows to configure or compile away logging in the library from a single place in Rust apps.
For the CLI side, the usage and output remained the same, except it's now colour-coded.
Fixes#217.
* Improve performance consistency
Switch from HashMap / HashSet to IndexMap / IndexSet for consistent iteration order of various options and, as a result, more predictable performance.
* Fix verbose message
* No cloning when restoring original data
* Make reductions return a new uncompressed image
Partially fixes#145
* Async reduction evaluator
* Assert
* Faster bit depth check
* Also try 4-bit depth for small-depth images
* Skip test when using miniz
* Ensure palette is trimmed after depth reduction
Fixes#159
* Fudge factor for reductions to prefer better reductions even if gzip estimation says otherwise
