Commit graph

7 commits

Author SHA1 Message Date
andrews05
aebddf443a
Allow brute filter configuration (#720)
This PR allows the Brute filter parameters to be configured rather than
using fixed constants. It also tweaks the presets to take advantage of
these new options:
o3: level=1, lines=3 (fractionally worse/faster)
o4: level=1, lines=4 (same as before)
o5: level=4, lines=4 (fractionally better/slower)
o6: level=5, lines=8 (slightly better/slower)

The parameters can be explicitly set in both the API and the CLI.
2025-09-25 23:20:29 +02:00
andrews05
71290acc62
Reduce memory usage of fast evaluation (#713)
This is an experiment I started a while ago before life happened. It
reduces memory usage of fast evaluation (-o2 and lower), bringing it
inline with normal (slow) evaluation. It does this by not retaining the
filtered image data of the evaluations, but instead retaining the row
filters that were used in each line so it can be quickly re-filtered
when required. This does incur a tiny performance penalty, but it's
negligible even at `-o0`.
Although this is fully functional, there are a few rough spots in the
code so I'm just opening it as a draft for now.

```
PR -sao6
       70.08 real       614.36 user         2.66 sys
          2024243200  maximum resident set size
          1839339520  peak memory footprint

PR -sao2
       11.16 real        60.85 user         1.33 sys
          1982562304  maximum resident set size
          1842173824  peak memory footprint

PR -sao2 -t1
       55.11 real        53.85 user         0.76 sys
           429457408  maximum resident set size
           245008064  peak memory footprint

master -sao6
       67.70 real       616.07 user         2.72 sys
          2043379712  maximum resident set size
          1838340416  peak memory footprint

master -sao2
       11.53 real        60.63 user         1.25 sys
          2753396736  maximum resident set size
          2283741440  peak memory footprint

master -sao2 -t1
       54.29 real        53.55 user         0.72 sys
           626311168  maximum resident set size
           305252544  peak memory footprint
```

Note that this involves some refactoring of `RowFilter` and the new
`FilterStrategy`. These are breaking changes so it will ultimately be
destined for v10. One advantage to this new structure is it opens the
door for future changes such as allowing the Brute strategy to take a
parameter for the number of lines.
2025-09-19 20:36:17 +02:00
Kornel
a72bc945fc Clippy 2025-01-02 09:15:24 +13:00
Alejandro González
0608ab9cab
Fix some new nightly Clippy lints and run rustfmt with prettier config 2024-02-20 19:12:35 +01:00
andrews05
ea5f1884be
Refactor aux chunk handling (#505) 2023-05-21 15:34:23 -04:00
andrews05
131a3c6af7
🚀 Revamp alpha optimisation (#475) 2022-12-10 21:14:50 -05:00
andrews05
a41d7de348
New filter strategies (#461)
* Refactor filters as enum

* Include filter byte in filter output

* Add entropy filter

* Add bigrams filter

* Add bigram entropy filter

* Add brute filter

* Replace bit-vec

* Add tests and benches

* Show filters in help

* Use FxHasher in color to palette

* Use windows function for minor improvement
2022-11-19 08:49:25 -05:00