Commit graph

2 commits

Author SHA1 Message Date
Broque Thomas
ab85c45785 Restore soulsync logger state between parallel-imports tests
Importing web_server fires utils.logging_config.setup_logging at
module-init, which clears + re-installs handlers on the shared
'soulsync' logger and pins its level to the user's configured
value. That mutation leaks across tests in the same pytest process.

This file runs alphabetically before test_library_reorganize_orchestrator,
so the leak broke test_watchdog_warns_about_stuck_workers downstream
— it relies on caplog capturing soulsync.library_reorganize warnings
via root-logger propagation, and the reconfigured logger's new
handler chain swallowed those records before they reached caplog
(caplog.records came back empty even though pytest's live-log
capture clearly showed the warning fired).

Adds an autouse fixture that snapshots the soulsync logger's
handlers, level, and propagate flag before each test in this file
and restores them afterwards. Pollution stays scoped to this file.

tests/test_tidal_auth_instructions.py also imports web_server but
runs alphabetically AFTER test_library_reorganize_orchestrator so
it never tripped this — fix is scoped here, not a project-wide
conftest, so we don't change behaviour for unrelated test files.
2026-05-01 08:45:17 -07:00
Broque Thomas
f339211654 Parallelize singles-import processing with a 3-worker executor
Discord-reported (fresh.dumbledore + maintainer ack): the
/api/import/singles/process route iterated staging files through a
plain Python for loop. Per-file work is dominated by metadata
search round-trips (Spotify/iTunes/Deezer/Discogs), so a multi-
track manual import on a typical home network was painfully slow.

Adds a dedicated import_singles_executor (3 workers) alongside the
existing executor pool, and refactors the route to submit every
file at once and aggregate results via as_completed. Worker count
balances throughput against any single provider's per-source rate
limits — the same shape used by missing_download_executor.

Extracts the per-file pipeline into _process_single_import_file
which returns a typed (status, payload) outcome:
- ("ok", final_title) on success
- ("error", message) for missing/malformed input or pipeline failure
The worker wraps its own exceptions so a single bad file can't
crash the batch; the route adds a belt-and-suspenders try/except
around future.result() for any worker-level surprises.

Pipeline thread-safety verified: post_process_matched_download
already serializes per-file via post_process_locks (one lock per
context_key — and each import gets a unique UUID context_key), DB
writes serialize through SQLite's WAL + busy_timeout, metadata
registry uses RLocks, no bare module-level mutable state.

Adds 9 regression tests:
- 4 worker-contract tests (missing file, malformed match, pipeline
  exception wrapping, happy-path return shape)
- 2 executor-config tests (worker count, thread name prefix)
- 1 integration test that proves the route actually parallelizes
  by checking wall-clock duration is well under sequential cost
- 1 mixed-outcome aggregation test
- 1 worker-crash recovery test

Doesn't address the related "stops on tab close" complaint —
that's a separate request-lifecycle issue that needs job_id +
polling, not just parallelism.
2026-04-30 22:41:04 -07:00