Revamp CI workflow to upload artifacts, cross-compile ARM64 binaries, and more

As commented in issues #444 and #518, there is some user interest for
distributing binaries for each unstable commit, and target ARM64
platforms. Personally, I think both suggestions are useful for the
project, as uploading binary artifacts for each commit might help
interested users to catch regressions and give feedback earlier, and
powerful ARM64 platforms are becoming increasingly popular due to some
cloud services (e.g., Amazon EC2, Azure VMs, Oracle Cloud) offering
cheaper plans for this hardware, in addition to the well-known push for
ARM by Apple with their custom M1 chips.

These changes make the CI target ARM64 as a first-class citizen. Because
the public GitHub actions runners can only be hosted on x64 for now, I
resorted to cross-compilation, [Debian's
multiarch](https://elinux.org/images/d/d8/Multiarch_and_Why_You_Should_Care-_Running%2C_Installing_and_Crossbuilding_With_Multiple_Architectures.pdf),
and QEMU to build, get ARM64 C library dependencies, and run tests,
respectively.

When the CI workflow finishes, a release CLI binary artifact is now
uploaded, which can be downloaded from the workflow run page on the
GitHub web interface.

In addition, these changes also introduce some cleanup and miscellaneous
improvements and changes to the CI workflow:

- Tests are run using [`nextest`](https://nexte.st/) instead of `cargo
  test`, which substantially speeds up their execution. (On my
  development workstation, `cargo test --release` takes around 10.67 s,
  while `cargo nextest run --release` takes around 6.02 s.)
- The dependencies on unmaintained `actions-rs` actions were dropped in
  favor of running Cargo commands directly, or using
  `giraffate/clippy-action` for pretty inline annotations for Clippy.
  This gets rid of the deprecation warnings for each workflow run.
- Most CI steps are run with a nightly Rust toolchain now, which allows
  to take advantage of the latest Clippy lints and codegen improvements.
  In my experience, when not relying on specific nightly features or
  compiler internals, Rust does a pretty good job at making it possible
  to rely on a rolling-release compiler for CI, as breakage is extremely
  rare and thus offset by the improved features.
- The MSRV check was moved to a separate job with less steps, so that it
  takes less of a toll on total workflow run minutes.
This commit is contained in:
Alejandro González 2023-07-10 22:10:56 +02:00 committed by Josh Holmer
parent 02bd47ba29
commit 385f7758bb
2 changed files with 128 additions and 62 deletions

9
.cargo/config.toml Normal file
View file

@ -0,0 +1,9 @@
# Remove this if targeting AArch64 from an AArch64 Linux box
[target.'cfg(all(target_os = "linux", target_arch = "aarch64"))']
runner = 'qemu-aarch64'
[target.aarch64-unknown-linux-gnu]
linker = 'aarch64-linux-gnu-gcc'
[target.aarch64-unknown-linux-musl]
linker = 'aarch64-linux-musl-gcc'

View file

@ -2,100 +2,157 @@ name: oxipng
on: on:
push: push:
branches:
- master
pull_request: pull_request:
branches: types:
- master - opened
- synchronize
workflow_dispatch: workflow_dispatch:
jobs: jobs:
test: ci:
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
timeout-minutes: 60
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
target: target:
- x86_64-unknown-linux-gnu - x86_64-unknown-linux-gnu
- x86_64-unknown-linux-musl - x86_64-unknown-linux-musl
- aarch64-unknown-linux-gnu
- aarch64-unknown-linux-musl
- x86_64-pc-windows-msvc - x86_64-pc-windows-msvc
- x86_64-apple-darwin - x86_64-apple-darwin
toolchain: - aarch64-apple-darwin
# Minimum stable
- "1.65.0"
- stable
- beta
- nightly
include: include:
- target: x86_64-unknown-linux-gnu - target: x86_64-unknown-linux-gnu
os: ubuntu-latest os: ubuntu-latest
target-apt-arch: amd64
- target: x86_64-unknown-linux-musl - target: x86_64-unknown-linux-musl
os: ubuntu-latest os: ubuntu-latest
target-apt-arch: amd64
- target: aarch64-unknown-linux-gnu
os: ubuntu-latest
target-apt-arch: arm64
- target: aarch64-unknown-linux-musl
os: ubuntu-latest
target-apt-arch: arm64
- target: x86_64-pc-windows-msvc - target: x86_64-pc-windows-msvc
os: windows-latest os: windows-latest
- target: x86_64-apple-darwin - target: x86_64-apple-darwin
os: macOS-latest os: macos-latest
exclude: - target: aarch64-apple-darwin
- target: x86_64-pc-windows-msvc os: macos-latest
toolchain: beta
- target: x86_64-pc-windows-msvc env:
toolchain: nightly CARGO_BUILD_TARGET: ${{ matrix.target }}
- target: x86_64-apple-darwin
toolchain: beta
- target: x86_64-apple-darwin
toolchain: nightly
- target: x86_64-unknown-linux-musl
toolchain: beta
- target: x86_64-unknown-linux-musl
toolchain: nightly
steps: steps:
- uses: actions/checkout@v3 - name: Checkout source
uses: actions/checkout@v3
with: with:
persist-credentials: false persist-credentials: false
- name: Install musl tools
run: sudo apt-get install musl-tools - name: Set up Ubuntu multiarch
if: "contains(matrix.target, 'musl')" if: matrix.target-apt-arch != 'amd64'
- name: Cache cargo registry run: |
uses: actions/cache@v3 . /etc/os-release
dpkg --add-architecture "${{ matrix.target-apt-arch }}"
sed -i "s/^deb http/deb [arch=$(dpkg-architecture -q DEB_HOST_ARCH)] http/" /etc/apt/sources.list
for suite in '' '-updates' '-backports' '-security'; do
echo "deb [arch=${{ matrix.target-apt-arch }}] http://ports.ubuntu.com/ $DISTRIB_CODENAME$suite main universe multiverse" >> /etc/apt/sources.list
done
- name: Install musl development files
if: endsWith(matrix.target, '-musl')
run: |
apt-get -yq update
apt-get -yq install musl-dev:${{ matrix.target-apt-arch }}
- name: Install QEMU and AArch64 cross compiler
if: startsWith(matrix.target, 'aarch64-unknown-linux')
run: |
apt-get -yq update
# libc6 must be present to run executables dynamically linked
# against glibc for the target architecture
apt-get -yq install qemu-user gcc-aarch64-linux-gnu libc6:arm64
- name: Cache Cargo artifacts
uses: Swatinem/rust-cache@v2
- name: Install Rust toolchain (${{ matrix.toolchain }})
uses: dtolnay/rust-toolchain@v1
with: with:
path: | toolchain: nightly
~/.cargo/bin/ targets: ${{ env.CARGO_BUILD_TARGET }}
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
key: ${{ runner.os }}-${{ matrix.toolchain }}-cargo-registry-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-${{ matrix.toolchain }}-cargo-registry-
- name: Install ${{ matrix.toolchain }}
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: ${{ matrix.toolchain }}
override: true
components: clippy, rustfmt components: clippy, rustfmt
- name: Install nextest
uses: taiki-e/install-action@nextest
- name: Run rustfmt - name: Run rustfmt
if: matrix.toolchain == 'stable' if: matrix.target == 'x86_64-unknown-linux-gnu'
uses: actions-rs/cargo@v1 run: cargo fmt --check
- name: Run Clippy (no default features)
if: matrix.target == 'x86_64-unknown-linux-gnu'
uses: giraffate/clippy-action@v1
with: with:
command: fmt clippy_flags: --no-deps --all-targets --no-default-features -- -D warnings
args: -- --check reporter: github-check
- name: Run clippy fail_on_error: true
if: matrix.toolchain == 'stable'
uses: actions-rs/clippy-check@v1 - name: Run Clippy (all features)
if: matrix.target == 'x86_64-unknown-linux-gnu'
uses: giraffate/clippy-action@v1
with: with:
token: ${{ secrets.GITHUB_TOKEN }} clippy_flags: --no-deps --all-targets --all-features -- -D warnings
args: -- -D warnings reporter: github-check
fail_on_error: true
- name: Run tests - name: Run tests
run: cargo test --features sanity-checks run: |
cargo nextest run --release --features sanity-checks
cargo test --doc --release --features sanity-checks
- name: Build benchmarks - name: Build benchmarks
if: matrix.toolchain == 'nightly'
run: cargo bench --no-run run: cargo bench --no-run
- name: Build docs - name: Build docs
run: cargo doc --no-deps if: matrix.target == 'x86_64-unknown-linux-gnu'
- name: Check no default features run: cargo doc --release --no-deps
if: matrix.toolchain == 'stable'
uses: actions-rs/clippy-check@v1 - name: Build CLI binary
run: cargo build --release
- name: Upload CLI binary as artifact
uses: actions/upload-artifact@v3
with: with:
token: ${{ secrets.GITHUB_TOKEN }} name: Oxipng binary (${{ matrix.target }})
args: --no-default-features -- -D warnings path: target/${{ env.CARGO_BUILD_TARGET }}/release/oxipng
msrv-check:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout source
uses: actions/checkout@v3
with:
persist-credentials: false
- name: Cache Cargo artifacts
uses: Swatinem/rust-cache@v2
- name: Install MSRV Rust toolchain
uses: dtolnay/rust-toolchain@1.65.0
- name: Install nextest
uses: taiki-e/install-action@nextest
- name: Run tests
run: |
cargo nextest run --release --features sanity-checks
cargo test --doc --release --features sanity-checks