da6cdfbc51
urlparse().path keeps the leading slash in front of a Windows drive, so file:///C:/docs/a.pdf read as \C:\docs\a.pdf and the ingester reported "File does not exist" for every file it discovered. url2pathname is the stdlib conversion that strips it, per platform. Four sites each decided both "is this local" and "what path is this": FSSource._uri_to_path and supports, resolve_adhoc_fetcher, create_document_from_source and check_source_accessible, and convert. is_local_uri and uri_to_path in haiku.rag.uri own those two decisions now, which closes two more cases of the same root cause. A bare C:\docs\a.pdf parses with scheme "c", so add-src raised "No source adapter for URI scheme 'c'" and convert silently treated the path as raw text. And convert and check_source_accessible never percent-decoded at all, so a file named a[b] c.md read as missing on Linux and macOS too. A file URI's host is reattached after conversion rather than passed to url2pathname, which as of 3.14 rejects a non-local authority off Windows. file:////server/share is the empty-authority spelling of a UNC path, its host being the first path segment, so that host is normalised into the authority before conversion. Output is identical on 3.12, 3.13 and 3.14. The ad-hoc FS fetcher roots at the path's own anchor rather than "/", which on Windows is only the current drive. test_uri.py runs on ubuntu, macos and windows across 3.13 and 3.14 without the project installed: --noconftest because the repo conftest imports dependencies that job does not need, and -o addopts= to drop the repository's -n auto. The Windows legs are what cover the drive conversion. Fixes #574.
105 lines
3.4 KiB
YAML
105 lines
3.4 KiB
YAML
name: Tests
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
pull_request:
|
|
branches: [main]
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
lint:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: astral-sh/setup-uv@v9.0.0
|
|
with:
|
|
enable-cache: true
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v5
|
|
with:
|
|
python-version-file: ".python-version"
|
|
- name: Install dependencies
|
|
run: uv sync --all-extras
|
|
- name: Lint
|
|
run: uv run ruff check
|
|
- name: Type check
|
|
run: uv run ty check
|
|
|
|
lint-frontend:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: pnpm/action-setup@v4
|
|
with:
|
|
version: 10
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v4
|
|
with:
|
|
node-version: "22"
|
|
cache: "pnpm"
|
|
cache-dependency-path: app/frontend/pnpm-lock.yaml
|
|
- name: Install dependencies
|
|
working-directory: app/frontend
|
|
run: pnpm install --frozen-lockfile
|
|
- name: Lint and format check
|
|
working-directory: app/frontend
|
|
run: pnpm run check
|
|
|
|
test-uri-platforms:
|
|
name: URI paths (${{ matrix.os }}, py${{ matrix.python }})
|
|
runs-on: ${{ matrix.os }}
|
|
strategy:
|
|
matrix:
|
|
os: [ubuntu-latest, macos-latest, windows-latest]
|
|
python: ["3.13", "3.14"]
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: astral-sh/setup-uv@v9.0.0
|
|
# --noconftest: tests/conftest.py imports the project's dependencies,
|
|
# which this job deliberately does not install. test_uri.py is stdlib-only.
|
|
- name: Test platform URI paths
|
|
env:
|
|
PYTHONPATH: haiku_rag_slim
|
|
run: >
|
|
uv run --no-project --python ${{ matrix.python }} --with pytest
|
|
pytest tests/test_uri.py -q --noconftest -o addopts=
|
|
|
|
test:
|
|
needs: [lint, lint-frontend]
|
|
runs-on: ubuntu-latest
|
|
env:
|
|
HF_TOKEN: ${{ secrets.HF_TOKEN }}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: astral-sh/setup-uv@v9.0.0
|
|
with:
|
|
enable-cache: true
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v5
|
|
with:
|
|
python-version-file: ".python-version"
|
|
- name: Install dependencies
|
|
run: uv sync --all-extras
|
|
- name: Cache HuggingFace models
|
|
id: hf-cache
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: huggingface-${{ runner.os }}-test-models-v2
|
|
- name: Pre-download tokenizer
|
|
if: steps.hf-cache.outputs.cache-hit != 'true'
|
|
run: uv run python -c "from transformers import AutoTokenizer; AutoTokenizer.from_pretrained('Qwen/Qwen3-Embedding-0.6B')"
|
|
- name: Pre-download cross-encoder test model
|
|
if: steps.hf-cache.outputs.cache-hit != 'true'
|
|
run: uv run python -c "from sentence_transformers import CrossEncoder; CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')"
|
|
- name: Run tests with coverage
|
|
env:
|
|
HF_HUB_OFFLINE: ${{ steps.hf-cache.outputs.cache-hit == 'true' && '1' || '0' }}
|
|
TRANSFORMERS_OFFLINE: ${{ steps.hf-cache.outputs.cache-hit == 'true' && '1' || '0' }}
|
|
run: uv run pytest -m "not integration" --cov --cov-report=xml --cov-report=term-missing:skip-covered
|
|
- name: Upload coverage to Codecov
|
|
uses: codecov/codecov-action@v5
|
|
with:
|
|
token: ${{ secrets.CODECOV_TOKEN }}
|
|
files: ./coverage.xml
|
|
fail_ci_if_error: false
|