Commit graph

39 commits

Author SHA1 Message Date
Kieran
e9f6b45953
[Enhancement] Add rate limiting to yt-dlp requests; prevent saving Media Items when throttled by YouTube (#559)
* Added sleep interval to settings

* Added new sleep setting to yt-dlp runner and added tests

* Added setting for form; updated setting name

* Updated form label

* Prevented saving/updating of media items if being throttled by youtube

* Added the bot message to the list of non-retryable errors

* Fixed typo
2025-01-14 11:38:40 -08:00
Kieran
9185f075ca
[Enhancement] Overhaul indexing to be more efficient (#540)
* WIP - created methods for breaking on existing media

* WIP - got everything hooked up for POC

* Add some docs, tests

* Refactors

* Updated TODO
2025-01-02 15:48:18 -08:00
Kieran
a2a70fcce2
[Bugfix] Respect cookies preference when performing download pre-check (#517)
* Updated get_downloadable_status to pass yt cookies

* Updated tests
2024-12-17 11:18:47 -08:00
Kieran
023f449dbe
[Housekeeping] Pass the current action when calling the yt-dlp runner (#514)
* Updated yt-dlp runner to take an action type

* Added actions to all callers of the yt-dlp runner

* [SQUASH] updated test files to use new mocking strategy

* Removed unneeded alias
2024-12-13 12:29:05 -08:00
Kieran
bfb27427ce
[Bugfix] Ensure livestreams aren't downloaded until they're finished processing (#485)
* Added logic to ignore downloads that aren't in the right live state

* Added tests for get_downloadable_status method

* Added tests for media downloader module

* Added tests to download worker modeule
2024-11-26 11:56:33 -08:00
Kieran
83c10b2b00
[Enhancement] Track the predicted final filepath for indexed media items (#461)
* Added ability to pass additional yt-dlp options to indexing step

* Added predicted_filename to media struct

* WIP added ability to predict filepath to source indexing

* renamed predicted_filepath

* Added the ability to predict filepath when fast indexing

* Add predicted_media_filepath to media items table

* Addressed TODOs
2024-11-08 09:42:59 -08:00
Kieran
a5afcf168b
Went back to using original_url over webpage_url to improve short detection (#447) 2024-10-30 14:09:28 -07:00
Kieran
108a141c65
Improve error messages when adding a non-valid source URL (#436) 2024-10-25 11:01:46 -07:00
Kieran
a6c61ccd0d
[Bugfix] Improve livestream detection (#362)
* Livestream status is now based off live_status instead of was_live

* Updated tests
2024-08-20 09:37:42 -07:00
Kieran
fd20ac5d84
[Enhancement] Use cookies on a per-source basis (#354)
* Added 'use_cookies' column to source

* Added cookies to the source form

* Updated every command to optionally include cookies

* LOTS of tests

* Made YT cookies off by default
2024-08-14 12:00:18 -07:00
Kieran
e06e050f95
[Enhancement] Capture playlist_index for use with output templates (#315)
* Started capturing playlist_index on indexing pass

* Added playlist_index as a media item field

* Added playlist index to output variable templates

* Improved the way playlist_indexes are rejected on update

* Updated docs

* Undid unneeded changes
2024-07-15 16:06:13 -07:00
Kieran
5a10015634
[Enhancement] Download image when using playlists with media center apps (#313)
* [WIP] started adding calls for downloading posters for playlists

* Updated source image parser to work with playlists
2024-07-15 12:04:57 -07:00
Kieran
af86ca1e0e
Switched thumbnail downloading to use yt-dlp (#281) 2024-06-04 10:39:54 -07:00
Kieran
22439e0273
[Housekeeping] Store the upload event as a datetime rather than a date (#269)
* [WIP] renamed and migrated upload_date column

* Refactored yt-dlp media module

* Refactored parse_upload_date

* Refactored media item upload_date_index

* Got media tests running

* Refactored media item table live

* Cleaned up the stragglers

* Fixed old oversight re: original_url
2024-05-28 13:01:40 -07:00
Kieran
dcf033f10a
Added support for ytdlp cache (#245) 2024-05-15 13:17:11 -07:00
Kieran
8d9bd11882
[Housekeeping] Improved logging for all executable calls (#244)
* Improved logging for all CLI calls

* CAptured more details in logging
2024-05-15 12:56:56 -07:00
Kieran
933daf8c78
[Housekeeping] Improve test stability (#232)
* Ensured all tests use at least the DataCase module

* addressed one flakey test

* More flakey tests

* Consolidated Mox usage
2024-05-08 09:57:36 -07:00
Kieran
f655a8ae01
Improved robustness of file downloads (#225) 2024-05-03 09:15:12 -07:00
Kieran
112c6a4f14
[Enhancement] Custom media lifecycle scripts (#219)
* Namespaced notification modules under lifecycle

* Added a JSON encoder for all the main model types

* Added startup task to create user script file

* Hook up user script event to media download

* Hooked up media deletion user script

* Added jq to docker deps

* Updated README
2024-05-02 08:43:37 -07:00
Kieran
88569640c9
[Bugfix] Ensure source metadata fetching uses windows-compatible filepaths (#193)
* Fixed underlying bug

* Refactored windows_filenames to be a global flag; added tests

* Added some random test coverage for fun
2024-04-17 12:51:52 -07:00
Kieran
8fbcc8b289 [Enhancement] Allow custom yt-dlp options (#176)
* Added option for yt-dlp config file usage

* renamed yt-dlp config file

* refactored to use a precedence-based approach

* Updated README
2024-04-10 20:17:22 -07:00
Kieran Eglin
0fcdd1df84 Renamed FS Helpers module to FS Utils 2024-04-10 18:02:26 -07:00
Kieran
26d457e656 [Enhancement] Add Apprise support (#170)
* [WIP] add settings sidebar entry and placeholder page

* [WIP] added placeholder UI and logic for settings form

* Added column and UI for apprise server

* Add some tests

* Added placeholder command runner for apprise

* [WIP] Adding apprise package

* Added apprise command runner

* Hooked up apprise notification module

* Ensured apprise was running in verbose mode

* Updated wording of apprise notification

* Added apprise to README
2024-04-09 17:45:39 -07:00
Kieran
f27323ffa3
[Ehnacement] More gracefully handle Sponsorblock failures (#169)
* Updated downloader and runner to handle sponsorblock failures more gracefully

* stopped download worker from running if a media item is preventing download
2024-04-06 13:23:36 -07:00
Kieran
c58e176619 Added yt-dlp version to sidebar (#155) 2024-04-01 17:54:51 -07:00
Kieran
fae7ba3304 [Enhancement] Start tracking a media item's duration (#146)
* Added duration field, started importing it during indexing and download

* Added duration to RSS feed
2024-03-28 21:51:04 -07:00
Kieran
fe641cfc49 Cookie auth (#116)
* Added new extras directory

* Automatically create cookie file on boot

* Added cookie file option to DL option builder

* Put cookie auth in correct spot

* Updated Docs

* Removed useless setter for config path
2024-03-24 20:23:25 -07:00
Kieran
37eb745d03 Add docs link (#103)
* Added docs link to README

* Added docs to sidebar in-app

* Updated some tests

* Updated docs in media profile form
2024-03-21 17:26:51 -07:00
Kieran
565a1d4972 Download images for sources (#96)
* [WIP] hooked up photo downloading to source metadata worker

* Added new attributes for source images

* Added option to profile form
2024-03-19 18:34:01 -07:00
Kieran
05f3deebfa Source NFO downloads (#95)
* Hooked up series directory finding to source metadata runner

* Fixed aired NFO tag for episodes

* Updated MI NFO builder to take in a filepath

* Hooked up NFO generation to the source worker

* Added NFO controls to form

* Improved the way the source metadata worker updates the source

* Consolidated NFO selection options in media profile instead of source
2024-03-18 17:27:28 -07:00
Kieran
b633d0f75c Misc refactors 2024-03-14 (#89)
* Adds method to improve cleanup of empty directories

* resolved bug where source metadata worker could call itself in an infinite loop

* Refactored file deletion for media items

* Removed useless filesystem data worker

* Updated task listing fns to take a record directly

* Refactored the way I call workers

* Improved some tests
2024-03-15 17:44:58 -07:00
Kieran
a135746c97 Improve episode-level compatability with media center apps (#86)
* Add media profile presets (#85)

* Added presets for output templates

* Added presets for the entire media profile form

* Append `-thumb` to thumbnails when downloading (#87)

* Appended -thumb to thumbnails when downloading

* Added code to compensate for yt-dlp bug

* Squash all the commits from the other branch bc I broke things (#88)
2024-03-14 18:30:08 -07:00
Kieran
1cc8731914 [Bugfix] Fix reddit bugs v2 (#82)
* Ensured thumbnail is converted to jpg before embedding

* Ensured media indexing doesn't fail if an upload date can't be parsed
2024-03-12 20:23:51 -07:00
Kieran
5b2d6b094d [Bugfix] Fixes issue with grabbing source/media details when first video is a premier (#79)
* Fixed issue with source details when first video is a premier

* Updated other occurance
2024-03-12 18:18:19 -07:00
Kieran
33baa99aae Refactor modules into contexts (#78)
* [WIP] break out a few contexts, start refactoring fast index modules

* [WIP] more contexts, this time around slow indexing and downloads

* [WIP] got all tests passing

* [WIP] Added moduledocs

* Built a genserver to rename old jobs on boot

* Added a module naming check; moved things around

* Fixed specs
2024-03-12 17:54:55 -07:00
Kieran
e1fc361e98 Improve audio-only download (#74)
* Improved quality from auto-only download; enables thumbnail and data embedding for audio

* Updated UI to reflect new audio behaviour
2024-03-11 19:33:27 -07:00
Kieran
e6b04bd675 Converts thumbnails to jpg for better compat (#70) 2024-03-11 18:03:40 -07:00
Kieran
f60ec4f49d Download cutoff date for sources (#69)
* Added new uploaded_at column to media items

* Updated indexer to pull upload date

* Updates media item creation to update on conflict

* Added download cutoff date to sources

* Applies cutoff date logic to pending media logic

* Updated docs
2024-03-10 21:24:01 -07:00
Kieran
e1f8e94686 Fast indexing (#58)
* Made method to getting singular media details; Renamed other related method

* Takes a fun and flirty digression to remove abstractions around yt-dlp since I'm 100% committed to using it exclusively

* Removed commented test code

* Lays the groundwork for fast indexing

* Added module for working with youtube RSS feed

* Added methods to kick off indexing workers from RSS response

* Improve short detection (#59)

* Made media attribute-related yt-dlp calls return a struct

* Added shorts attribute to media items

* Added ability to discern a short from yt-dlp response

* Updated search to use new shorts attribute

* Fast index UI (#63)

* Added fast_index field and adds it to source form

* Added fast indexing to source changeset operations

* Added fast indexing worker and updated other modules to start using it

* Handled fast index worker on source update

* Add support modals (#65)

* Added fast indexing upgrade modal

* Improved modal on smaller screens

* Updated links to work again

* Added donation modal

* Reverted source fast index to 15 minutes

* Removed unneeded HTML attributes from old alpine approach
2024-03-10 14:36:34 -07:00