4ae64c568b
This PR makes the oxipng binary process multiple files in parallel, finally fulfilling #275. There seemed to be some debate about whether oxipng _should_ do this or not but there's a couple of reasons I think it makes sense: 1. The concern seemed mostly around the complexity of such a feature. Not to worry, it was trivial* 🙂 2. Since then, oxipng has dropped from a max of something like 180 simultaneous compression trials down to 10, which is very much a good thing but it does mean it's not utilising any more cores than that. Some benchmarks on around 100 files on a machine with 8 cores: Level | Master time | PR time -|-|- 2 | 28.303 | 19.005 3 | 36.507 | 23.089 5 | 1:10.86 | 1:16.01 *Some additional changes were required in order to make sure sensible output is printed to the terminal, since things won't be in order anymore. Here's some example output from before: ``` Processing: tests/files/fully_optimized.png file size = 67 bytes (0 bytes = 0.00% decrease) File already optimized Processing: tests/files/corrupted_header.png Invalid PNG header detected Processing: tests/files/verbose_mode.png file size = 102480 bytes (12228 bytes = 10.66% decrease) Output: tests/files/verbose_mode.png ``` And after: ``` Processing: tests/files/verbose_mode.png Processing: tests/files/fully_optimized.png Processing: tests/files/corrupted_header.png tests/files/corrupted_header.png: Invalid PNG header detected tests/files/fully_optimized.png: Could not optimize further, no change written 102480 bytes (10.66% smaller): tests/files/verbose_mode.png ``` Closes #275, #84, #169, #196 and #419. [edit] This is the last thing I wanted to land before the next release 🥳
81 lines
1.9 KiB
Rust
81 lines
1.9 KiB
Rust
pub mod prelude {
|
|
pub use super::*;
|
|
}
|
|
|
|
pub trait ParallelIterator: Iterator + Sized {
|
|
fn with_max_len(self, _l: usize) -> Self {
|
|
self
|
|
}
|
|
fn reduce_with<OP>(mut self, op: OP) -> Option<Self::Item>
|
|
where
|
|
OP: Fn(Self::Item, Self::Item) -> Self::Item + Sync,
|
|
{
|
|
self.next().map(|a| self.fold(a, op))
|
|
}
|
|
}
|
|
|
|
pub trait IntoParallelIterator {
|
|
type Iter: Iterator<Item = Self::Item>;
|
|
type Item: Send;
|
|
fn into_par_iter(self) -> Self::Iter;
|
|
}
|
|
|
|
pub trait IntoParallelRefIterator<'data> {
|
|
type Iter: Iterator<Item = Self::Item>;
|
|
type Item: Send + 'data;
|
|
fn par_iter(&'data self) -> Self::Iter;
|
|
}
|
|
|
|
pub trait IntoParallelRefMutIterator<'data> {
|
|
type Iter: ParallelIterator<Item = Self::Item>;
|
|
type Item: Send + 'data;
|
|
fn par_iter_mut(&'data mut self) -> Self::Iter;
|
|
}
|
|
|
|
impl<I: IntoIterator> IntoParallelIterator for I
|
|
where
|
|
I::Item: Send,
|
|
{
|
|
type Iter = I::IntoIter;
|
|
type Item = I::Item;
|
|
|
|
fn into_par_iter(self) -> Self::Iter {
|
|
self.into_iter()
|
|
}
|
|
}
|
|
|
|
impl<'data, I: 'data + ?Sized> IntoParallelRefIterator<'data> for I
|
|
where
|
|
&'data I: IntoParallelIterator,
|
|
{
|
|
type Iter = <&'data I as IntoParallelIterator>::Iter;
|
|
type Item = <&'data I as IntoParallelIterator>::Item;
|
|
|
|
fn par_iter(&'data self) -> Self::Iter {
|
|
self.into_par_iter()
|
|
}
|
|
}
|
|
|
|
impl<'data, I: 'data + ?Sized> IntoParallelRefMutIterator<'data> for I
|
|
where
|
|
&'data mut I: IntoParallelIterator,
|
|
{
|
|
type Iter = <&'data mut I as IntoParallelIterator>::Iter;
|
|
type Item = <&'data mut I as IntoParallelIterator>::Item;
|
|
|
|
fn par_iter_mut(&'data mut self) -> Self::Iter {
|
|
self.into_par_iter()
|
|
}
|
|
}
|
|
|
|
impl<I: Iterator> ParallelIterator for I {}
|
|
|
|
#[allow(dead_code)]
|
|
pub fn join<A, B>(a: impl FnOnce() -> A, b: impl FnOnce() -> B) -> (A, B) {
|
|
(a(), b())
|
|
}
|
|
|
|
#[allow(dead_code)]
|
|
pub fn spawn<A>(a: impl FnOnce() -> A) -> A {
|
|
a()
|
|
}
|