enhance_file_metadata rebuilds tags from scratch: for FLAC it calls
clear_pictures(), for MP3/MP4 it clears the whole tag block — and it does
this UP FRONT, then saves the file, long before it tries to fetch and embed
the replacement art. So every way the re-embed could come up empty left the
file saved with the original art destroyed and nothing put back:
- extract_source_metadata returns nothing -> early save, no embed
- no album-art URL / art download fails / rejected by the min-size guard
-> embed_album_art_metadata returns early without adding a picture
- art embedding disabled in config -> embed skipped entirely
- embed raises mid-enrichment -> file left cleared on disk
This is the "cover art gets corrupted/destroyed during import" half of #764
(continuation of #755); distinct from #750's truncated-cache DISPLAY bug.
Fix: new core/metadata/art_preservation.py snapshots the existing art
(the live Picture / APIC / MP4Cover objects, so they re-apply verbatim)
BEFORE the clear, and restores it before each save IFF the file currently
has none. Wired into all three exit paths in enhance_file_metadata
(no-metadata early return, the final save, and the except handler). The
restore is a strict no-op when art is already present, so the happy path —
new art embedded — is byte-for-byte unchanged: it never clobbers or
duplicates a freshly-embedded cover. embed_album_art_metadata now returns a
bool so the intent (embedded / didn't) is explicit.
Tests:
- tests/test_art_preservation.py (5) — snapshot/restore round-trips through
real mutagen FLAC + ID3 objects; restore no-ops when new art is present.
- tests/test_enrichment_art_preservation.py (4) — runs the REAL
enhance_file_metadata over a real FLAC with embedded art and asserts the
art survives on disk for missing-metadata / failed-embed / embed-raises,
and is correctly REPLACED (exactly one picture, new bytes) on success.
1019 tests pass across the metadata/enrichment/imports/acoustid suites.
Follow-up to the preferred-art feature. Real test runs showed a source could
win on priority while handing back a small cover: Cover Art Archive is
volunteer-uploaded with no size floor, so CAA-first gave a 599x531 (Taylor
Swift) and a 600x600 (Kendrick GNX) -- front-1200 only caps the max, so a
~600px upload stays ~600px -- and Deezer/iTunes lower in the order never got a
turn.
Fix:
- Minimum-resolution guard: artwork._min_size_art_validator builds the
resolver's validate hook -- it fetches each candidate, caches the bytes (so
the winner isn't fetched twice), and accepts art only when its shortest side
>= metadata_enhancement.min_art_size (default 1000px; 0 disables). Art that's
too small is a miss, so the resolver falls through to the next source instead
of winning on priority. Unmeasurable images are accepted (don't over-reject;
fallback is still today's art). Wired into both embed_album_art_metadata and
download_cover_art.
- iTunes art upgraded to /3000x3000bb/ (was the 600px default) so it
contributes high-res when it wins.
- select_preferred_art_url gains a validate passthrough to the resolver.
- config default metadata_enhancement.min_art_size: 1000.
Effect: with an order like caa > deezer > spotify > itunes, a ~600px CAA upload
is now skipped and Deezer's ~1900px wins -- consistent big art. (Spotify art
often maxes ~640px, so it's skipped at the 1000 floor in favor of bigger
sources; lower min_art_size to ~640 to allow it.)
Tests: tests/metadata/test_art_min_size.py (6 -- incl. the real 599x531 and
600x600 cases, shortest-side logic, unmeasurable-accept, no-bytes-reject,
0-disables) + iTunes max-res upgrade test. Full metadata suite green (617).
Lets users pick which providers' cover art to use and in what priority,
generalizing the single prefer_caa_art toggle into an ordered, mix-and-match
list (Sokhi's request). Fully opt-in: default album_art_order is [], so every
existing install is byte-for-byte unchanged until the user enables sources.
How it works:
- Per album, walk the user's ordered sources top-to-bottom; the first source
that actually has THIS album's cover wins. A miss falls through to the next;
if all miss, the download's own art is kept (today's default). The worst case
is always exactly the cover you'd get today -- never wrong art, never an
error into the download.
- Connection-gated: a source is only tried when the user is connected to it
(free sources CAA/Deezer/iTunes/AudioDB always; Spotify only when
authenticated). Tidal/Qobuz/HiFi deferred (cover-URL construction + no clean
core accessor -- not shipping unverified extraction).
- Album-match validated: a source's art is used only when the album it returns
matches the requested artist+album (significant-token subset, tolerant of
Deluxe/Remastered/articles/feat./multi-artist). A loose top search hit for a
different record is treated as a miss -> guarantees no wrong-album art.
- The list supersedes the legacy prefer_caa_art toggle: when album_art_order is
non-empty it is the sole authority (add 'caa' to the list to use Cover Art
Archive), and prefer_caa_art is neutralized for both the embedded-tag art and
cover.jpg paths. With an empty list, prefer_caa_art behaves exactly as before.
Implementation:
- core/metadata/art_sources.py: pure resolver -- effective_art_order (config +
legacy back-compat) and resolve_cover_art (ordered walk + fallback,
exception-safe per source). No network/config/DB; fully unit-testable.
- core/metadata/art_lookup.py: availability gating, per-source lookups against
existing clients (Deezer/iTunes/AudioDB/Spotify search + CAA via MBID),
album-match validation, per-album caching, and select_preferred_art_url --
the single gate the pipeline calls (no-op unless an explicit list is set).
- core/metadata/artwork.py: wired into embed_album_art_metadata and
download_cover_art, gated so no configured list == current behavior.
- web_server.py: GET /api/metadata/art-sources (connected sources only).
- config/settings.py: default album_art_order: [].
- webui (index.html + settings.js): reorderable list in Core Features reusing
the hybrid-source-list pattern + real service logos (with emoji fallback);
load/save wired through the existing metadata_enhancement settings flow.
loadArtSourceOrder populates the saved order synchronously (filtered to known
sources, not availability) so a save before the availability fetch resolves,
or a temporarily-disconnected source, can never wipe the saved order.
Tests: 40 unit/seam tests (resolver ordering/fallback/back-compat, availability,
per-source extraction, album-match validation incl. wrong-album/wrong-artist
rejection, caching, exception-safety, the off-by-default gate). Full metadata
suite still green (610 passed) -- the gated integration changes nothing when no
list is configured.
Note: the settings UI (DOM-heavy, not unit-testable in the JS harness) and the
live per-source art-fetch quality are validated by manual testing.
Follow-up to the album-art resolution fix. That change upgraded MusicBrainz
Cover Art Archive thumbnails (/front-250) to the bare /front original — but
/front redirects to archive.org, which is unreliable: probing release-group
covers showed intermittent HTTP 500s (same URL 500s one second, serves the
next) and multi-MB originals (2.9 MB seen). The result was the user-reported
flakiness: cover art that "sometimes works, sometimes shows nothing", and a
huge image embedded into every track when it did work.
The sized thumbnails (/front-250, -500, -1200) are served by CAA's own CDN,
not the archive.org redirect — which is why /front-250 (240p) was always
reliable. Upgrade to /front-1200 instead: 1200x1200 is a massive jump from
240p, reliably CDN-served, and a sane ~40 KB instead of multi-MB.
Applied in all three CAA spots for consistency: the _upgrade_art_url helper
(embed + cover.jpg paths) and both prefer_caa ("CCA") blocks, which fetched
the bare /front directly with no fallback — so CCA-on users hit the same
flakiness. _fetch_art_bytes still falls back to the original /front-250 if
/front-1200 is ever refused.
Tests updated to assert the 1200px target, idempotency, and that the bare
/front original is intentionally left untouched.
User report: embedded album art came out ~600x600 while the cover.jpg in
the folder was high-res. The cover.jpg path upgraded the source CDN URL
to its highest resolution, but the tag-embed path fetched the raw URL —
so iTunes art embedded at its 600x600 default, Spotify at 640, Deezer at
1000. The "Write Tags to File" retag path had the same gap (Deezer-only
upgrade), and MusicBrainz art was worse still: every Cover Art Archive
URL is built as the /front-250 thumbnail, so MB-sourced downloads
embedded 250x250.
Factor the resolution upgrade + fetch into two shared helpers in
core/metadata/artwork.py and route every art path through them:
_upgrade_art_url(url) — bump to the source's highest resolution:
- Spotify (i.scdn.co) -> original master (~2000px+)
- iTunes (mzstatic.com) -> 3000x3000
- Deezer (dzcdn) -> 1900x1900
- Cover Art Archive -> /front original (was /front-250)
_fetch_art_bytes(url) — upgrade, fetch, and fall back once to the
original size if the CDN refuses the larger one (non-regressive).
Now consistent across: embed-into-tags (post-process), folder cover.jpg
(post-process), and the enhanced-library "Write Tags to File" retag flow.
The YouTube path already upgraded via Album.from_spotify_album, unchanged.
De-duplicates the per-source upgrade code that was copied across sites
and drops the now-unused urllib import from tag_writer.
Not covered (follow-up): Last.fm / Amazon / Tidal / Qobuz have no
explicit upgrade yet — some already serve full-res, others may hand over
a capped size that passes through unchanged.
Tests: new tests/metadata/test_artwork_resolution.py pins every upgrade
(Spotify 300/640->master, iTunes 100/600->3000, Deezer->1900, CAA
thumbnail->original, unrecognized/empty unchanged) and the fetch
fallback. Updated the two tag_writer fallback tests to patch the network
at its new home in artwork.
Add a disk-backed image cache with hashed browser URLs, SQLite metadata, size/type validation, stale fallback, and per-image fetch locking. Route normalized artwork through /api/image-cache while keeping /api/image-proxy as a compatibility shim, and align browser max-age with the image cache TTL. Add focused tests for cache behavior and image URL normalization.
The first token-leak fix scrubbed the artwork URL fixer's own log
calls. This catches three more sites that ALSO leaked tokens, plus
one upstream gap that let URL-encoded tokens slip through the
redactor.
Three sites in `web_server.py` (artist endpoint at line 8765-8773):
- "Artist image before fix: '...'" -- logged the raw image_url with
the auth token in plain form.
- "Artist image after fix: '...'" -- logged the URL-encoded form
after it had been wrapped in the image proxy
(`/api/image-proxy?url=<percent-encoded-token>`).
- "Final artist data being sent: {...}" -- dumped the entire
artist_info dict on every render, including the image_url field.
All three were dev-time debug noise. Removed entirely. The "No
artist image URL found" warning at line 8770 stays (no URL, just
the artist name).
One site in `core/discovery/sync.py:402`:
- "[PLAYLIST IMAGE] image_url=..." -- logged the playlist poster URL
during sync. Same auth-token leak risk for Plex / Jellyfin
playlists. Changed to log only `has_image=True/False`.
Upstream gap in `_redact_url_secrets`:
- The original regex only matched plain query params (`?key=value`).
When an auth-bearing URL gets wrapped inside another URL's query
string (our `/api/image-proxy?url=<encoded>` flow) the auth params
end up percent-encoded -- `%3FX-Plex-Token%3D...` -- and slipped
through.
- New second pattern catches the URL-encoded form. Both passes run
on every redact call; idempotent.
Verified manually:
/api/image-proxy?url=...%3FX-Plex-Token%3DABC...
-> /api/image-proxy?url=...%3FX-Plex-Token%3D***REDACTED***
6 artwork tests pass.
The artwork URL normalizer was logging the full constructed media-
server URL on every cover-art lookup at INFO level, including the
auth query params (X-Plex-Token / X-Emby-Token / Subsonic t+s+p).
Those lines pile up in app.log on disk -- anyone with read access to
the log file gains full read access to the user's media server.
Also dropped the noisy per-call "Plex/Jellyfin/Navidrome config -
base_url: ..., token: ..." INFO lines that fired on every thumbnail.
Even the truncated `token[:10]` form is enough partial-known-plaintext
to be uncomfortable to leak.
- New `_redact_url_secrets` helper masks the values of X-Plex-Token,
X-Emby-Token, api_key, apikey, Subsonic t / s / p, generic token /
password query params. Regex anchored on `?` or `&` boundary so
short keys like `t` don't false-match inside `format=Jpg`.
- "Fixed URL: ..." log calls moved from INFO to DEBUG so they don't
persist by default, and the URL passed in is run through the
redactor first.
- Per-call "Plex config - ..." / "Jellyfin config - ..." /
"Navidrome config - ..." INFO lines removed entirely. Config
inspection has dedicated UI; per-thumbnail spam belongs to no one.
- Error-path logging (line 149) also routed through the redactor in
case the failing URL had auth params attached.
Users with existing app.log files containing the leaked tokens
should rotate / wipe the log. Plex tokens can be regenerated by
signing out of all devices in Plex settings; Jellyfin api_keys can
be revoked from the dashboard; Navidrome users should rotate the
account password.
Defensive followup. If Deezer CDN ever refuses the upgraded
1900×1900 URL for a specific album (rare — empirically tested 4
albums and none hit it), pre-fix would have succeeded with the
1000×1000 URL and post-fix would have failed entirely.
Both download sites now retry with the original URL when the
upgraded URL fails:
- `core/metadata/artwork.py::download_cover_art` — auto post-process
flow. Resolves the original URL from album_info / context the same
way the existing path does.
- `core/tag_writer.py::download_cover_art` — captures the original
URL before upgrade so the retry has it without a second context
lookup.
Strictly non-regressive: worst plausible post-fix case is now
identical to pre-fix (cover at 1000×1000 succeeds). Fallback only
fires on the rare CDN-refusal edge.
Tests added (2):
- `test_tag_writer_retries_with_original_on_failure` — upgraded URL
raises, original succeeds, both attempts logged in call order
- `test_tag_writer_no_fallback_for_non_dzcdn_url` — non-Deezer URLs
go through unchanged, no fallback path triggered (single attempt)
Verification:
- 18/18 helper + integration tests pass
- 2561 full suite passes
- Ruff clean
Discord report (Tim): downloaded cover art via Deezer metadata
source came out visibly blurry in Navidrome / on phones — large
displays exposed the limited resolution.
# Cause
Deezer's API returns `cover_xl` URLs at 1000×1000. The underlying
CDN actually serves up to 1900×1900 by rewriting the size segment
in the URL path (same trick the iTunes mzstatic + Spotify scdn
upgrades already use). SoulSync wasn't doing the rewrite — every
Deezer-sourced cover got embedded at 1000×1000 regardless of how
much higher resolution the CDN had available.
# Verified empirically
```
$ for size in 1000 1400 1800 1900 2000; do curl -I "...{size}x{size}-..."; done
1000: 200 OK 106 KB
1400: 200 OK 198 KB
1800: 200 OK 331 KB
1900: 200 OK 371 KB
2000: 403 Forbidden
```
1900 is the safe ceiling. Above that the CDN returns 403. CDN
serves source-native bytes when source < target (smaller-source
albums get same bytes whether we ask for 1000 or 1900), so asking
for 1900 universally is safe.
# Fix
New `_upgrade_deezer_cover_url(url, target_size=1900)` helper in
`core/deezer_client.py`. Pure function, mirrors the
`_upgrade_spotify_image_url` pattern that already lives in
`core/spotify_client.py`. Defensive on every input shape:
- Empty / None → returned as-is
- Non-Deezer URL (no `dzcdn`) → returned as-is
- No size segment in URL → returned as-is
- Already at/above target → returned as-is (idempotent, never
downgrades)
Applied at both cover-download sites:
- `core/metadata/artwork.py::download_cover_art` — auto post-process
flow. Mirrors the existing iTunes mzstatic upgrade right above it.
- `core/tag_writer.py::download_cover_art` — enhanced library view's
"Write Tags to File" feature.
# Scope discipline
- Helper applied at the DOWNLOAD boundary, not the source extraction
point in `deezer_client.py`. Means cached entries in the metadata
cache + DB row `image_url` columns keep the original 1000×1000 URL
Deezer's API returned. Future CDN behavior changes only affect the
download path, not stored data.
- Pre-existing `prefer_caa_art` toggle (Settings → Library →
Post-Processing) untouched — orthogonal workaround for users who
want even higher quality (MusicBrainz Cover Art Archive, often
3000×3000+).
- iTunes / Spotify upgrade paths untouched — they already worked.
# Tests added (16)
`tests/metadata/test_deezer_cover_url_upgrade.py`:
- Standard upgrade: default target 1900 on cover URL, alternate
dzcdn host (`e-cdns-images.dzcdn.net` vs `cdn-images.dzcdn.net`),
artist picture URLs (same path pattern), 500×500 source upgrades
too
- Custom target size: smaller target = no-op (never downgrade),
larger target works
- Idempotent: already at/above target returned unchanged
- Defensive on non-Deezer URLs: parametrised across 5 hosts
(Spotify scdn, iTunes mzstatic, MB CAA, Last.fm, random) — all
returned untouched
- Defensive on malformed Deezer URL (no size segment) → returned
as-is
- Empty / None handling
# Verification
- 16/16 helper tests pass
- 560/560 metadata + imports tests pass (no regression)
- 2559 full suite passes
- Ruff clean
- keep existing /api/image-proxy URLs from being wrapped again
- reuse the shared metadata package instead of duplicating URL logic in web_server.py
- add regression coverage for proxy passthrough and internal URL normalization
- Relocate the shared metadata helper module from core/metadata_common.py into core/metadata/common.py.
- Update the new metadata package, the import pipeline, and the web entrypoint to use the package-scoped helper.
- Keep the shared config, mutagen, file-lock, and tag-writing helpers centralized without touching unrelated files.
- Keep existing metadata_cache and metadata_service at the top level for now
- Move the new branch-local metadata helpers under core/metadata
- Share MusicBrainz release cache state from core.metadata.source and update import sites
