soulsync/core/imports
BoulderBadgeDad 79648a4f5f import: async staging scan so a large library doesn't time out the page (#947, phase 1 backend)
A whole-library migration (ramonskie copied his Lidarr library into /staging) makes the synchronous
staging scan walk + tag-read tens of thousands of files INSIDE the GET request, blowing past
gunicorn's 120s timeout — and because the killed request never warms the cache, every reload
re-times-out. Moves the SAME scan off the request thread; the page reports progress instead of
hanging.

- _scan_staging_records gains an optional `progress` param (additive; default None = unchanged).
  Refactored to two passes: a fast walk to collect the audio-file list (total), then the slow
  tag-read loop updating scanned. A generation guard stops a scan that finishes AFTER an import
  from committing stale records.
- ensure_background_staging_scan(path): idempotent background runner filling the existing cache.
- get_staging_records_or_status(): warm cache or a scan that finishes within a ~3s grace → records
  (so small/normal folders still answer in one request, no UX change); else ("scanning", progress).
  A scan error is re-raised so the endpoints log + return it exactly as before.
- /staging/files|groups|hints return {scanning, progress} when the scan is still running instead of
  blocking; new lightweight /staging/scan-status for cheap progress polling.

Single source preserved (same scan + cache, just off the request thread). 13 new tests (progress,
idempotent ensure, grace ready-vs-scanning, generation guard discards stale, endpoint scanning
shape, error contract, status ready/cold); full import suite 626 green; ruff clean.

Next: phase 2 — the React import page polls scan-status + shows a progress bar, then renders.
2026-06-29 09:24:41 -07:00
..
__init__.py Rehome import helpers into core/imports 2026-04-27 19:54:44 +03:00
album.py Show MusicBrainz release variants in import 2026-05-24 09:33:19 -07:00
album_grouping.py lint: log the skipped album source-id lookup instead of a bare try/except/pass (ruff S110) 2026-06-18 12:09:22 -07:00
album_matching.py #890: strip leading track-number prefix from filename-derived titles 2026-06-18 18:35:04 -07:00
album_naming.py Rehome import helpers into core/imports 2026-04-27 19:54:44 +03:00
album_position.py Auto-download: resolve an album track's real position from its album (not 1/1) 2026-06-22 13:32:24 -07:00
context.py Auto-download: resolve an album track's real position from its album (not 1/1) 2026-06-22 13:32:24 -07:00
file_integrity.py integrity: don't quarantine longer masters/versions as 'truncated' (#937) 2026-06-27 15:25:13 -07:00
file_ops.py lossy copy: support all lossless formats, not just FLAC (#941) 2026-06-28 12:13:05 -07:00
filename.py Fix album-bundle downloads landing every track as track 1 2026-05-26 21:04:27 -07:00
folder_artist.py Use app confirm modal for verification review actions 2026-06-11 01:53:05 +02:00
guards.py feat(import): master toggle for quality-filtering on import + collapsible tile 2026-06-15 15:29:29 +02:00
paths.py #890: strip leading track-number prefix from filename-derived titles 2026-06-18 18:35:04 -07:00
pipeline.py imports: never delete a file we couldn't quarantine — leave it for retry 2026-06-28 15:10:43 -07:00
quarantine.py fix(quarantine): use name-based group key across batches/sources 2026-06-24 12:48:04 +02:00
rematch_apply.py #889 Phase 4: the Re-identify modal + apply backend 2026-06-18 15:37:56 -07:00
rematch_hints.py #889: FIX data loss — re-identify to the same release no longer deletes the file 2026-06-18 17:13:46 -07:00
rematch_search.py #889 Phase 3: re-identify search — multi-source track→release lookup + API 2026-06-18 15:31:33 -07:00
resolution.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
routes.py import: async staging scan so a large library doesn't time out the page (#947, phase 1 backend) 2026-06-29 09:24:41 -07:00
side_effects.py merge: pull upstream/main (2.7.4) into feature/best-quality-search-mode 2026-06-19 16:54:51 +02:00
silence.py quality: recognize DSD (.dsf/.dff) as lossless + stop the false "truncated" flag (#939) 2026-06-28 11:45:16 -07:00
single_to_album.py Library: single->parent-album resolution core (pure selector + injected-I/O resolver) 2026-06-18 09:24:06 -07:00
staging.py #922: import search labelled Spotify Free users' primary source as 'Deezer' 2026-06-24 08:43:52 -07:00
tag_policy.py Import: never wipe a clean/matched import's tags when enhancement fails (#804) 2026-06-07 22:44:43 -07:00
track_number.py Multi-disc: file the track in the disc folder that matches its tag (Sokhi) 2026-06-22 10:56:53 -07:00
version_mismatch_fallback.py feat(quality): rank-based candidate ordering toggle for priority mode 2026-06-25 22:00:57 +02:00