danvics/soulsync
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

fix

Browse source
This commit is contained in:
Broque Thomas 2025-07-13 14:43:31 -07:00
parent d9c6a30a69
commit 60b62bef14
1 changed files with 208 additions and 207 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

415
SPOTIFY_MATCHING_SPEC.md
View file

@ -42,73 +42,56 @@ Now we need to incorporate this functionality into full album downloads by addin
If we fail to match an artist in the modal, treat the download as a normal downoad without any matching and keep it in the downloads folder. Also any matched downloads need to update the 'download queue' the same way a normal download would. The cancel button should remain functional on a matched download in the queue and clicking it should behave exaclty the same. a finished matched download should transfer to finished downloads as expected.
Remix should be handled elegantly. If artist A does a remix of Artist B song. The song artist will be Artist A with a contributting artist of Artist B.
---
## VERY IMPORTANT! DO NOT BREAK ANYTHING
## 🏗️ System Architecture Overview
### Core Components
```
┌─────────────────────────────────────────────────────────────────┐
│ Spotify Matched Download System │
Spotify Matched Download System - Technical Specification (v2 - Complete)📋 Document PurposeThis document provides comprehensive technical specifications for implementing the Spotify Matched Download System. It addresses the complexity of music metadata matching, file organization, and user interface design based on real-world challenges and requirements. This revised version incorporates explicit logic for batch processing, metadata writing, and album/single differentiation to ensure a robust and user-friendly implementation.🏗️ System Architecture OverviewCore Components┌─────────────────────────────────────────────────────────────────┐
│ Spotify Matched Download System │
├─────────────────────────────────────────────────────────────────┤
│ UI Layer │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Matched Download│ │ Matching Modal │ │ Progress Tracking│ │
│ │ Buttons │ │ │ │ │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Matched Download│ │ Matching Modal │ │ Batch Review UI │ │
│ │ Buttons │ │ (Single Track) │ │ (For Albums) │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
├─────────────────────────────────────────────────────────────────┤
│ Service Layer │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Metadata │ │ Spotify Matching│ │ File Organization│ │
│ │ Extraction │ │ Service │ │ Service │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Metadata │ │ Spotify Matching│ │ File Organization│ │
│ │ Extraction │ │ Service │ │ Service │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
│ ┌─────────────────┐ │
│ │ Metadata Writer │ │
│ │ Service │ │
│ └─────────────────┘ │
├─────────────────────────────────────────────────────────────────┤
│ Integration Layer │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Soulseek │ │ Spotify │ │ File System │ │
│ │ Client │ │ Client │ │ Manager │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Soulseek │ │ Spotify │ │ File System │ │
│ │ Client │ │ Client │ │ Manager │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
│ ┌─────────────────┐ │
│ │ Batch Matching │ │
│ │ Manager │ │
│ └─────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
```
---
## 🔍 Advanced Metadata Extraction System
### Problem Statement
Soulseek metadata is notoriously inconsistent:
- Filenames: `"Track 01.mp3"`, `"asdjklh.flac"`, `"Artist - Song (Remix).mp3"`
- Directory paths: `"/User/Music/Random Folder/"`
- Missing or incorrect artist/title/album information
### Solution: Multi-Tier Extraction Strategy
#### Tier 1: Leverage Existing TrackResult Fields (PRIMARY)
```python
class EnhancedMetadataExtractor:
🔍 Advanced Metadata Extraction SystemProblem StatementSoulseek metadata is notoriously inconsistent:Filenames: "Track 01.mp3", "asdjklh.flac", "Artist - Song (Remix).mp3"Directory paths: "/User/Music/Random Folder/"Missing or incorrect artist/title/album informationSolution: Multi-Tier Extraction StrategyTier 1: Leverage Existing TrackResult Fields (PRIMARY)class EnhancedMetadataExtractor:
def extract_from_track_result(self, track: TrackResult) -> TrackMetadata:
"""
Primary extraction using existing TrackResult fields.
These are already parsed by soulseek_client.py
"""
return TrackMetadata(
artist=track.artist, # Already extracted!
title=track.title, # Already extracted!
album=track.album, # Already extracted!
artist=track.artist, # Already extracted!
title=track.title, # Already extracted!
album=track.album, # Already extracted!
track_number=track.track_number, # Already extracted!
filename=track.filename,
confidence=self.calculate_field_confidence(track)
)
```
#### Tier 2: Enhanced Filename Parsing (SECONDARY)
```python
class AdvancedFilenameParser:
Tier 2: Enhanced Filename Parsing (SECONDARY)class AdvancedFilenameParser:
PATTERNS = [
# Pattern: "Artist - Title"
r'^(?P<artist>.+?)\s*[-–—]\s*(?P<title>.+?)(?:\s*\[(?P<extra>.*?)\])?(?:\s*\((?P<remix>.*?[Rr]emix.*?)\))?$',
@ -136,11 +119,7 @@ class AdvancedFilenameParser:
return self.create_metadata_from_match(match)
return None
```
#### Tier 3: Directory Context Analysis (TERTIARY)
```python
class DirectoryContextAnalyzer:
Tier 3: Directory Context Analysis (TERTIARY)class DirectoryContextAnalyzer:
def analyze_path_context(self, filepath: str) -> Optional[AlbumContext]:
"""
Extract album context from directory structure
@ -163,17 +142,7 @@ class DirectoryContextAnalyzer:
return AlbumContext(**match.groupdict())
return None
```
---
## 🎵 Sophisticated Matching Algorithms
### Multi-Stage Matching Pipeline
#### Stage 1: Exact Match Strategy
```python
class ExactMatcher:
🎵 Sophisticated Matching AlgorithmsMulti-Stage Matching PipelineStage 1: Exact Match Strategyclass ExactMatcher:
def find_exact_match(self, metadata: TrackMetadata) -> List[SpotifyMatch]:
"""
Highest confidence matching with exact metadata
@ -194,11 +163,7 @@ class ExactMatcher:
results = self.spotify_client.search_tracks(query, limit=5)
return [SpotifyMatch(track, confidence=0.95) for track in results[:3]]
```
#### Stage 2: Fuzzy Match Strategy
```python
class FuzzyMatcher:
Stage 2: Fuzzy Match Strategyclass FuzzyMatcher:
def find_fuzzy_matches(self, metadata: TrackMetadata) -> List[SpotifyMatch]:
"""
Similarity-based matching with confidence scoring
@ -258,11 +223,7 @@ class FuzzyMatcher:
confidence = (artist_sim * 0.4) + (title_sim * 0.5) + (album_sim * 0.1) + duration_bonus
return min(confidence, 1.0)
```
#### Stage 3: Remix Detection & Handling
```python
class RemixMatcher:
Stage 3: Remix Detection & Handlingclass RemixMatcher:
REMIX_PATTERNS = [
r'(?P<title>.+?)\s*\((?P<remix_artist>.+?)\s+[Rr]emix\)',
r'(?P<title>.+?)\s*\[(?P<remix_artist>.+?)\s+[Rr]emix\]',
@ -304,22 +265,7 @@ class RemixMatcher:
matches.append(SpotifyMatch(track, confidence, match_type="remix"))
return matches
```
---
## 🎨 Professional UI Architecture
### Responsive Modal Design
#### Problem with Previous Implementation
- "Squished" content with poor spacing
- Inflexible layouts that didn't adapt to content
- Poor user experience with cramped interface
#### Solution: Professional Modal Architecture
```python
class ResponsiveMatchingModal(QDialog):
🎨 Professional UI ArchitectureResponsive Modal Design (Single Track)The architecture for the single-track matching modal remains essential for manual corrections and one-off downloads.class ResponsiveMatchingModal(QDialog):
"""
Professional modal with responsive design and proper spacing
"""
@ -334,7 +280,7 @@ class ResponsiveMatchingModal(QDialog):
"""
# Modal sizing - responsive to screen size
screen = QApplication.primaryScreen().geometry()
modal_width = min(900, int(screen.width() * 0.7)) # 70% of screen width, max 900px
modal_width = min(900, int(screen.width() * 0.7)) # 70% of screen width, max 900px
modal_height = min(700, int(screen.height() * 0.8)) # 80% of screen height, max 700px
self.resize(modal_width, modal_height)
@ -457,9 +403,6 @@ class ResponsiveMatchingModal(QDialog):
results_layout.addWidget(scroll_area, 1) # Expand to fill space
parent_layout.addWidget(results_frame, 1) # Allow results section to expand
```
#### Individual Result Item Design
```python
class SpotifyMatchResultItem(QFrame):
"""
@ -499,7 +442,7 @@ class SpotifyMatchResultItem(QFrame):
info_layout.setSpacing(4)
# Track title
title_label = QLabel(spotify_track.name)
title_label = QLabel(self.spotify_track.name)
title_label.setStyleSheet("""
QLabel {
font-size: 16px;
@ -510,7 +453,7 @@ class SpotifyMatchResultItem(QFrame):
title_label.setWordWrap(True)
# Artist and album
artist_text = ", ".join(spotify_track.artists)
artist_text = ", ".join(self.spotify_track.artists)
details_label = QLabel(f"by {artist_text}")
details_label.setStyleSheet("""
QLabel {
@ -519,7 +462,7 @@ class SpotifyMatchResultItem(QFrame):
}
""")
album_label = QLabel(f"from {spotify_track.album}")
album_label = QLabel(f"from {self.spotify_track.album}")
album_label.setStyleSheet("""
QLabel {
font-size: 12px;
@ -613,36 +556,76 @@ class SpotifyMatchResultItem(QFrame):
layout.addWidget(percentage_label)
return confidence_widget
```
---
## 📁 Professional File Organization System
### Atomic File Operations
```python
class AtomicFileOrganizer:
🗂️ Batch Processing for Albums/FoldersProblem StatementProcessing an entire album by showing a modal for each track is inefficient and provides a poor user experience. The system must handle batch operations gracefully.Solution: Batch Matching Manager & Summary UIA dedicated manager will orchestrate the matching of multiple files, presenting a single, consolidated UI for user review.class BatchMatchingManager:
"""
Professional file organization with rollback capability
Orchestrates the matching process for a batch of tracks (e.g., an album).
"""
def __init__(self, file_paths: List[str], spotify_client, metadata_extractor):
self.file_paths = file_paths
self.spotify_client = spotify_client
self.metadata_extractor = metadata_extractor
self.batch_results = []
def run_automatic_matching(self):
"""
Processes all files in the batch, performing non-interactive matching.
"""
for path in self.file_paths:
# 1. Extract initial metadata from filename/path
initial_metadata = self.metadata_extractor.extract_from_path(path)
# 2. Find the best automatic match from Spotify
# This would use a combination of ExactMatcher and FuzzyMatcher
# to find the single most likely candidate (e.g., highest confidence > 0.85)
best_match = self.find_best_match(initial_metadata)
self.batch_results.append({
"original_path": path,
"initial_metadata": initial_metadata,
"proposed_match": best_match, # SpotifyTrack object or None
"confidence": best_match.confidence if best_match else 0.0
})
def present_review_ui(self):
"""
Displays a summary UI for the user to review all matches.
The UI should list all tracks, their proposed matches, and confidence scores.
Tracks with low confidence should be highlighted.
The user can click on a single track to open the ResponsiveMatchingModal
for manual correction.
"""
# This would instantiate a new QWidget/QDialog for the batch review
review_dialog = BatchReviewDialog(self.batch_results)
if review_dialog.exec_():
# User confirmed the matches
final_matches = review_dialog.get_final_matches()
self.process_confirmed_downloads(final_matches)
def process_confirmed_downloads(self, final_matches):
"""
Initiates the download and file organization for all confirmed tracks.
"""
# ... logic to queue downloads and trigger file organization on completion
📁 Professional File Organization SystemAlbum vs. Single Determination LogicTo correctly apply the specified folder structure, the system must differentiate between a standalone single and a track from a larger album or EP.Rule: The determination will be based on the album_type field provided by the Spotify API for the matched track's album.Album Structure (ARTIST/ARTIST - ALBUM_NAME/): Use if album.album_type is 'album', or if album.album_type is 'single' and the album contains more than one track (to correctly handle EPs).Single Structure (ARTIST/ARTIST - SINGLE_NAME/): Use only if album.album_type is 'single' and the album contains exactly one track.Atomic File Operationsclass AtomicFileOrganizer:
"""
Professional file organization with rollback capability.
This service is now responsible for moving, renaming, AND initiating metadata tagging.
"""
def __init__(self, transfer_base_path: str = "Transfer"):
self.transfer_base_path = Path(transfer_base_path)
self.operation_log = [] # Track operations for rollback
def organize_file(self, source_path: str, spotify_track: SpotifyTrack) -> FileOrganizationResult:
self.operation_log = []
# Inject the metadata writer service
self.metadata_writer = MetadataWriterService()
def organize_and_tag_file(self, source_path: str, spotify_track: SpotifyTrack) -> FileOrganizationResult:
"""
Atomically organize file with full rollback capability
Atomically organizes and tags a file with full rollback capability.
"""
try:
# Phase 1: Validation
source_file = Path(source_path)
if not source_file.exists():
return FileOrganizationResult(
success=False,
error="Source file does not exist",
source_path=source_path
)
return FileOrganizationResult(success=False, error="Source file does not exist", source_path=source_path)
# Phase 2: Destination planning
destination_path = self.calculate_destination_path(spotify_track, source_file.suffix)
@ -650,44 +633,52 @@ class AtomicFileOrganizer:
# Phase 3: Conflict resolution
final_destination = self.resolve_conflicts(destination_path)
# Phase 4: Atomic operation
return self.perform_atomic_move(source_file, final_destination)
# Phase 4: Atomic operation (Move & Tag)
move_result = self.perform_atomic_move(source_file, final_destination)
if not move_result.success:
raise Exception(f"File move failed: {move_result.error}")
# Tag the file in its new location
tag_result = self.metadata_writer.write_tags(
file_path=str(final_destination),
spotify_track=spotify_track
)
if not tag_result.success:
# If tagging fails, roll back the file move
self.rollback_operation(move_result.operation_id)
raise Exception(f"Metadata tagging failed: {tag_result.error}")
return FileOrganizationResult(success=True, destination_path=str(final_destination))
except Exception as e:
return FileOrganizationResult(
success=False,
error=f"Organization failed: {str(e)}",
source_path=source_path
)
# Rollback is handled within the try/except blocks
return FileOrganizationResult(success=False, error=str(e), source_path=source_path)
def calculate_destination_path(self, spotify_track: SpotifyTrack, file_extension: str) -> Path:
"""
Calculate organized file path following professional naming conventions
Calculates organized file path based on album_type and professional naming conventions.
"""
# Sanitize names for filesystem compatibility
artist_name = self.sanitize_filename(spotify_track.artists[0])
album_name = self.sanitize_filename(spotify_track.album)
# Use album artist for primary folder structure to keep albums together
album_artist = self.sanitize_filename(spotify_track.album.artists[0].name)
album_name = self.sanitize_filename(spotify_track.album.name)
track_name = self.sanitize_filename(spotify_track.name)
# Handle multi-artist tracks
if len(spotify_track.artists) > 1:
primary_artist = spotify_track.artists[0]
# Keep featured artists in track name
if "feat." in track_name.lower() or "featuring" in track_name.lower():
final_track_name = f"{primary_artist} - {track_name}"
else:
featured_artists = ", ".join(spotify_track.artists[1:])
final_track_name = f"{primary_artist} - {track_name} (feat. {featured_artists})"
track_number = spotify_track.track_number
# Determine if it's a single or album based on defined logic
is_true_single = (spotify_track.album.album_type == 'single' and
spotify_track.album.total_tracks == 1)
if is_true_single:
# Single folder structure: ARTIST/ARTIST - TRACK/TRACK.flac
album_folder_name = f"{album_artist} - {track_name}"
file_name = f"{track_name}{file_extension}"
else:
final_track_name = f"{artist_name} - {track_name}"
# Create path structure
artist_folder = artist_name
album_folder = f"{artist_name} - {album_name}"
filename = f"{final_track_name}{file_extension}"
return self.transfer_base_path / artist_folder / album_folder / filename
# Album folder structure: ARTIST/ARTIST - ALBUM/## - TRACK.flac
album_folder_name = f"{album_artist} - {album_name}"
file_name = f"{track_number:02d} - {track_name}{file_extension}"
return self.transfer_base_path / album_artist / album_folder_name / file_name
def resolve_conflicts(self, destination_path: Path) -> Path:
"""
Handle file naming conflicts professionally
@ -779,15 +770,54 @@ class AtomicFileOrganizer:
name = name[:200].rsplit(' ', 1)[0] # Break at word boundary
return name or "Unknown" # Fallback for empty names
```
✍️ Metadata Writer ServiceProblem StatementAfter a file is correctly named and placed, its internal metadata (tags) must be updated with the rich, accurate data from Spotify.Solution: Dedicated Metadata Writer ServiceA new service in the Service Layer will handle writing ID3v2 (for MP3) or Vorbis Comment (for FLAC) tags to the audio files using the mutagen library.import mutagen
---
class MetadataWriterService:
"""
Writes Spotify metadata to audio file tags.
"""
def write_tags(self, file_path: str, spotify_track: SpotifyTrack) -> TaggingResult:
try:
audio = mutagen.File(file_path, easy=True)
if audio is None:
raise Exception("Could not load audio file.")
## 🔄 Integration with Existing Download System
# Clear existing relevant tags
for key in ['title', 'artist', 'album', 'albumartist', 'tracknumber', 'date', 'genre']:
if key in audio:
del audio[key]
# Write new tags from Spotify data
audio['title'] = spotify_track.name
audio['album'] = spotify_track.album.name
# CRITICAL: Distinguish between Album Artist and Track Artist
# Album Artist: Used for grouping albums. Typically the primary artist of the album.
audio['albumartist'] = spotify_track.album.artists[0].name
# Track Artist: All artists featured on the specific track.
audio['artist'] = [artist.name for artist in spotify_track.artists]
audio['tracknumber'] = f"{spotify_track.track_number}/{spotify_track.album.total_tracks}"
audio['date'] = spotify_track.album.release_date
# Note: Spotify API genre data can be sparse. Fetch from artist if needed.
if spotify_track.album.genres:
audio['genre'] = spotify_track.album.genres
audio.save()
# Separately, handle downloading and embedding cover art
self.embed_cover_art(file_path, spotify_track.album.images[0].url)
### Download Completion Detection
```python
class DownloadCompletionMonitor:
return TaggingResult(success=True)
except Exception as e:
return TaggingResult(success=False, error=str(e))
def embed_cover_art(self, file_path: str, image_url: str):
# ... Logic to download image data and embed it into the file using mutagen ...
pass
🔄 Integration with Existing Download SystemDownload Completion Detectionclass DownloadCompletionMonitor:
"""
Monitor download completions and trigger matching process
"""
@ -810,7 +840,9 @@ class DownloadCompletionMonitor:
def on_download_completed(self, download_item):
"""
Handle download completion and trigger matching if needed
Handle download completion and trigger matching if needed.
This now triggers either the single modal or the file organization
step for pre-confirmed batch items.
"""
download_id = self.generate_download_id(download_item.search_result)
@ -835,15 +867,7 @@ class DownloadCompletionMonitor:
download_path=download_item.local_path
)
modal.show()
```
---
## 🧪 Testing Strategy
### Unit Tests
```python
class TestMetadataExtraction:
🧪 Testing StrategyUnit Testsclass TestMetadataExtraction:
"""Test metadata extraction with real-world examples"""
def test_common_filename_patterns(self):
@ -889,11 +913,24 @@ class TestFileOrganization:
def test_cross_platform_compatibility(self):
"""Test filename sanitization across platforms"""
pass
```
### Integration Tests
```python
class TestEndToEndWorkflow:
def test_single_vs_album_path_generation(self):
"""Test that paths are correctly generated for singles vs. albums."""
pass
class TestMetadataWriter:
def test_tag_writing_for_flac(self):
"""Verify Vorbis comments are written correctly."""
pass
def test_tag_writing_for_mp3(self):
"""Verify ID3 tags are written correctly."""
pass
def test_artist_vs_albumartist_tagging(self):
"""Ensure artist and albumartist are handled correctly for remixes/features."""
pass
Integration Testsclass TestEndToEndWorkflow:
"""Test complete matched download workflow"""
def test_single_track_workflow(self):
@ -907,45 +944,9 @@ class TestEndToEndWorkflow:
def test_ui_responsiveness(self):
"""Test UI behavior under various conditions"""
pass
```
---
## 📊 Performance Considerations
### Optimization Strategies
1. **Caching**: Cache Spotify search results to avoid duplicate API calls
2. **Batch Processing**: Group multiple searches for efficiency
3. **Lazy Loading**: Load UI elements as needed
4. **Background Processing**: Perform heavy operations in separate threads
5. **Memory Management**: Proper cleanup of modal dialogs and threads
### Monitoring & Metrics
- Track matching success rates
- Monitor API response times
- Log file organization errors
- Measure user interaction patterns
---
## 🎯 Implementation Priorities
### Phase 1: Core Foundation
1. Enhanced metadata extraction system
2. Basic matching algorithms
3. File organization framework
4. Professional UI architecture
### Phase 2: Advanced Features
1. Remix detection and handling
2. Confidence scoring system
3. Error handling and rollback
4. Performance optimizations
### Phase 3: Integration & Polish
1. Download system integration
2. Comprehensive testing
3. User experience refinements
4. Documentation and deployment
This specification provides a comprehensive foundation for implementing a professional-grade Spotify matching system that addresses real-world complexity and user experience requirements.
def test_batch_album_workflow(self):
"""Test the complete workflow for a matched album download,
including the review UI and final organization of all tracks."""
pass
📊 Performance ConsiderationsOptimization StrategiesCaching: Cache Spotify search results to avoid duplicate API callsBatch Processing: Group multiple searches for efficiencyLazy Loading: Load UI elements as neededBackground Processing: Perform heavy operations in separate threadsMemory Management: Proper cleanup of modal dialogs and threadsMonitoring & MetricsTrack matching success ratesMonitor API response timesLog file organization errorsMeasure user interaction patterns🎯 Implementation PrioritiesPhase 1: Core FoundationEnhanced metadata extraction systemBasic matching algorithmsFile organization frameworkProfessional UI architecturePhase 2: Advanced FeaturesRemix detection and handlingConfidence scoring systemError handling and rollbackPerformance optimizationsPhase 3: Integration & PolishDownload system integrationComprehensive testingUser experience refinementsDocumentation and deployment