Use cached HuggingFace models offline in test job

The Qwen tokenizer and cross-encoder pre-downloads call the HF metadata
API to revalidate even on a cache hit; a 429 there propagates instead of
falling back to the cached files, failing CI on HF throttling.

Skip the pre-download steps when the cache is restored and run pytest
with HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE on a hit, so cached models are
used without any network revalidation; allow online on a miss so a fresh
cache key still populates. Rename the cache key so the snapshot
re-populates with every test model (the old key predated the
cross-encoder step and never cached it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Yiorgis Gozadinos 2026-06-03 12:01:43 +03:00
parent 651b22ddcf
commit a65757807e
No known key found for this signature in database

View file

@ -62,15 +62,21 @@ jobs:
- name: Install dependencies
run: uv sync --all-extras
- name: Cache HuggingFace models
id: hf-cache
uses: actions/cache@v4
with:
path: ~/.cache/huggingface
key: huggingface-${{ runner.os }}-qwen-tokenizer-v1
key: huggingface-${{ runner.os }}-test-models-v1
- name: Pre-download tokenizer
if: steps.hf-cache.outputs.cache-hit != 'true'
run: uv run python -c "from transformers import AutoTokenizer; AutoTokenizer.from_pretrained('Qwen/Qwen3-Embedding-0.6B')"
- name: Pre-download cross-encoder test model
if: steps.hf-cache.outputs.cache-hit != 'true'
run: uv run python -c "from sentence_transformers import CrossEncoder; CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')"
- name: Run tests with coverage
env:
HF_HUB_OFFLINE: ${{ steps.hf-cache.outputs.cache-hit == 'true' && '1' || '0' }}
TRANSFORMERS_OFFLINE: ${{ steps.hf-cache.outputs.cache-hit == 'true' && '1' || '0' }}
run: uv run pytest -m "not integration" --cov=haiku --cov-report=xml
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v5