Commit graph

175 commits

Author SHA1 Message Date
Kieran
e06e050f95
[Enhancement] Capture playlist_index for use with output templates (#315)
* Started capturing playlist_index on indexing pass

* Added playlist_index as a media item field

* Added playlist index to output variable templates

* Improved the way playlist_indexes are rejected on update

* Updated docs

* Undid unneeded changes
2024-07-15 16:06:13 -07:00
Kieran
5a10015634
[Enhancement] Download image when using playlists with media center apps (#313)
* [WIP] started adding calls for downloading posters for playlists

* Updated source image parser to work with playlists
2024-07-15 12:04:57 -07:00
Kieran
8f91c4e6a2
[Enhancement] Improve flow for adding many sources at once (#306)
* Added mechamism for using existing sources as a reference for adding new sources

* Added test
2024-07-12 08:53:26 -07:00
Kieran
a46cfd1888
[Bugfix] Check for regex errors before saving a source (#296)
* Added error if provided regex is invalid

* improved test
2024-06-20 12:48:52 -07:00
Kieran
132f8e8da6
[Dev] Upgrade to Elixir 1.17.0 (#289)
* upgraded to Elixir 1.17.0

* Addressed deprecation warnings

* Updated Credo
2024-06-13 10:05:03 -07:00
Kieran
f6708a327c
[Enhancement] Optionally use the YouTube API for improved fast indexing (#282)
* Started adding youtube API for fast indexing

* Hooked youtube API into fast indexing

* Added youtube_api_key to settings

* Added youtube api key to settings UI

* Added tests

* Refactored the youtube api module

* More refactor

* Changed editing mode name from basic to standard

* [WIP] started on copy changes

* Updated copy
2024-06-10 11:45:41 -07:00
Kieran
582eb53698
Add regex for validating source URL (#285) 2024-06-06 11:54:39 -07:00
Kieran
a20d06072f
Fixed upload_date_index callback from running unnecessarily (#284) 2024-06-05 17:04:49 -07:00
Kieran
af86ca1e0e
Switched thumbnail downloading to use yt-dlp (#281) 2024-06-04 10:39:54 -07:00
Kieran
4994e70652
[Bugfix] Partially revert custom codec selection (#279)
* Removed and re-added codec preference columns

* Removed custom codec work from download options builder

* Updated settings UI

* Made codec preferences non-optional fields
2024-06-04 09:19:37 -07:00
Kieran
3dd901ff3e
Prevented error when media item doesn't have description (#274) 2024-05-29 13:52:43 -07:00
Kieran
e751c65424
[Enhancement] Improve handling of large media collections (#272)
* Added compound index for updated media item table for performance

* Improved large number display on homepage

* Improved UI around large numbers

* Centered the content of homepage data cards

* Renamed migration
2024-05-29 12:38:03 -07:00
Kieran
22439e0273
[Housekeeping] Store the upload event as a datetime rather than a date (#269)
* [WIP] renamed and migrated upload_date column

* Refactored yt-dlp media module

* Refactored parse_upload_date

* Refactored media item upload_date_index

* Got media tests running

* Refactored media item table live

* Cleaned up the stragglers

* Fixed old oversight re: original_url
2024-05-28 13:01:40 -07:00
Kieran
10880e9699
[Enhancement] Added link to local stream on media item show page (#265)
* Adds link for opening local stream

* Put conditional on correct element
2024-05-27 11:07:23 -07:00
Kieran
b2e16d50cd
Added library size to homepage (#264) 2024-05-27 10:43:50 -07:00
Kieran
95a0c29358
[Enhancement] Added search to source forms (#259)
* Changed sqlite FTS to use a trigram tokenizer

* Added search UI to source tables

* Fix bug with special chars in search form

* improved main search results form

* Improved centering for media table header elements
2024-05-23 15:09:49 -07:00
Kieran
d2f91a8253
[Enhancement] Added "other" tab to see media that's not set for download (#258)
* Added tab for other (non-downloaded and non-pending) media

* Added column for tracking if media was manually ignored
2024-05-23 10:24:52 -07:00
Kieran
81d5efd4c1
[Enhancement] Allow setting preference for video/audio codec (#255)
* [WIP] started on codec parser

* Added codec preferences to settings model

* [WIP] added basic codec preference fields to settings form

* Hooked up the backend portion of the codec preference work

* Added codec settings to frontend

* Ensured you can remove codec data
2024-05-21 16:28:19 -07:00
Kieran
dec3938780
Added 8k support (#254) 2024-05-21 09:59:05 -07:00
Kieran
a2a0f48065
[Enhancement] Add media count to sources page (#251)
* [WIP] Got query kinda working, now to refactor other queries

* Refactored all query methods to use dynamic snippets

* Refactored tab layout to grab tabs by name

* Removed standalone show buttons from in-app tables

* Removed unneeded comment
2024-05-21 08:53:22 -07:00
Kieran
94c0cf9970
[Enhancement] Show currently executing jobs on homepage (#250)
* [WIP] Added PoC job table

* [WIP] hooked up tasks table with real data

* Added reload on websocket command

* Hooked it all up to a websocket
2024-05-17 10:55:27 -07:00
Kieran
b1a6a6a81f
[Bugfix] Fix yt-dlp illegal write issue that causes failure to fetch source details (#247)
* Bandaid fix for yt-dlp write issue

* Ensured tempfile directory on app boot
2024-05-15 17:32:56 -07:00
Kieran
dcf033f10a
Added support for ytdlp cache (#245) 2024-05-15 13:17:11 -07:00
Kieran
8d9bd11882
[Housekeeping] Improved logging for all executable calls (#244)
* Improved logging for all CLI calls

* CAptured more details in logging
2024-05-15 12:56:56 -07:00
Kieran
bdcb49185a
[Enhancement] Adds in-app diagnostics page (#243)
* Added improved sidebar menuing

* Added new view for getting diagnostic data

* Changed default log level to debug

* Disabled false-positive static analysis
2024-05-15 12:27:57 -07:00
Kieran
d575548226
Bumped RSS feed limit; fixed ordering (#241) 2024-05-15 10:15:00 -07:00
Kieran
a38ffbc55b
[Enhancement] Allow redownloading of files for existing media items (#239)
* Added ability to specify overwrite behaviour when downloading media

* Added helper for redownloading media items

* renamed media redownload worker to disambiguate it from similarly named methods

* Added new redownload option to source actions dropdown

* Refactored MediaQuery to use a __using__ macro

* docs
2024-05-13 14:25:39 -07:00
Kieran
933daf8c78
[Housekeeping] Improve test stability (#232)
* Ensured all tests use at least the DataCase module

* addressed one flakey test

* More flakey tests

* Consolidated Mox usage
2024-05-08 09:57:36 -07:00
Kieran
944d26b57d
Added test button to apprise input (#226) 2024-05-03 15:42:22 -07:00
Kieran
f655a8ae01
Improved robustness of file downloads (#225) 2024-05-03 09:15:12 -07:00
Kieran
04b14719ee
[Enhancement] Add Media Center support for videos uploaded on the same day (#221)
* Added upload date index field to media_items

* Added incrementing index for upload dates

* Added media item upload date index to download option builder

* Added new season_episode_index_from_date to UI; updated parser

* Improve support for channels

* Hopefully fixed flakey test
2024-05-02 11:06:10 -07:00
Kieran
112c6a4f14
[Enhancement] Custom media lifecycle scripts (#219)
* Namespaced notification modules under lifecycle

* Added a JSON encoder for all the main model types

* Added startup task to create user script file

* Hook up user script event to media download

* Hooked up media deletion user script

* Added jq to docker deps

* Updated README
2024-05-02 08:43:37 -07:00
Kieran
8051107d32
Stopped notifications from being sent if the source doesn't download (#218) 2024-05-01 08:56:15 -07:00
Kieran
3f74f199dc
Stopped worker from retrying if doing so wouldn't improve things (#210) 2024-04-29 14:06:12 -07:00
Kieran
09cac46e14
Adds thumbnails as episode-level images for podcasts (#201) 2024-04-26 12:10:12 -07:00
Kieran
98c2812ee8
[Enhancement] Allow forcing a refresh of source metadata (#194)
* Stopped sources from fetching metadata on every update

* Added action button to force a metadata refresh
2024-04-18 15:35:53 -07:00
Kieran
88569640c9
[Bugfix] Ensure source metadata fetching uses windows-compatible filepaths (#193)
* Fixed underlying bug

* Refactored windows_filenames to be a global flag; added tests

* Added some random test coverage for fun
2024-04-17 12:51:52 -07:00
Kieran
aea40a3f92
[Enhancement] Add pagination for a source's media (#190)
* [WIP] added first attempt at pagination component

* Hooked up pagination for downloaded media
2024-04-17 10:22:55 -07:00
Kieran
618711691b
[Bugfix]: Misc. bugfixes 2024-04-16 (#189)
* Manually installed mutagen

* Stopped upgrade form from submitting on enter

* Gracefully handle duplicate enqueued downloads

* Update metadata thumbnail fetcher to use the best jpg available
2024-04-16 17:37:39 -07:00
Kieran
4721957875
Added yt-dlp options for year formatting (#188) 2024-04-16 16:54:55 -07:00
Kieran
b2e5e9b880
Resolved bug where non-pending media would notify when fast indexing (#187) 2024-04-16 13:47:56 -07:00
Kieran
f2a7463ff3 [Enhancement] Improve support for 4k videos with Plex (#181)
* Added WIP 4k MP4 fix [skip ci]

* Added tests for new remux options
2024-04-10 22:22:17 -07:00
Kieran
c36e33e1fd [Housekeeping] Close system port when jobs are cancelled (#182)
* Created a test setup that works

* Refactored test setup into real-world fixes
2024-04-10 22:13:19 -07:00
Kieran
e984c05298 [Enhancement] Allow overriding output templates on a per-source basis (#179)
* Added output path override to table and download option builder

* Added output template override to UI
2024-04-10 22:02:19 -07:00
Kieran Eglin
25aaef7da4 Fixed bug with redownloading not forcing download of the video 2024-04-10 21:43:15 -07:00
Kieran
8fbcc8b289 [Enhancement] Allow custom yt-dlp options (#176)
* Added option for yt-dlp config file usage

* renamed yt-dlp config file

* refactored to use a precedence-based approach

* Updated README
2024-04-10 20:17:22 -07:00
Kieran Eglin
0fcdd1df84 Renamed FS Helpers module to FS Utils 2024-04-10 18:02:26 -07:00
Kieran
e841f39cf2 [Enhancement] Redownload new media after a delay (#173)
* Added redownload-related columns

* Added methods for fetching re-downloadable media items

* Filled out redownload worker + tests

* Added redownload worker to config.exs cron

* Added to UI and README
2024-04-10 17:54:45 -07:00
Kieran
26d457e656 [Enhancement] Add Apprise support (#170)
* [WIP] add settings sidebar entry and placeholder page

* [WIP] added placeholder UI and logic for settings form

* Added column and UI for apprise server

* Add some tests

* Added placeholder command runner for apprise

* [WIP] Adding apprise package

* Added apprise command runner

* Hooked up apprise notification module

* Ensured apprise was running in verbose mode

* Updated wording of apprise notification

* Added apprise to README
2024-04-09 17:45:39 -07:00
Kieran
f27323ffa3
[Ehnacement] More gracefully handle Sponsorblock failures (#169)
* Updated downloader and runner to handle sponsorblock failures more gracefully

* stopped download worker from running if a media item is preventing download
2024-04-06 13:23:36 -07:00
Kieran
81b49f55bf
[Bugfix] Properly escape NFO files (#168)
* Properly escaped NFO file contents

* Added an NFO backfill worker

* Added a try-catch to the backfill since I _really_ don't want failures to halt app boot
2024-04-06 11:48:28 -07:00
Kieran
24875eaeac
[Housekeeping] Refactor settings model (#165)
* [WIP] renamed current settings module and tables to have backup suffix

* Created new settings table, schema, and context

* Migrated from old settings module to new one

* Removed settings backup modules

* Added some tests and docs
2024-04-04 12:43:17 -07:00
Kieran
e55bcaddd0
[Enhancement] Improve ordering of models (#164)
* [WIP] ordering app queries

* Refactored media queries to be self-contained
2024-04-03 17:26:46 -07:00
Kieran
b872c5c20b
[Enhancement] Allow manual indexing/downloading (#162)
* Added controller actions and UI for forcing index and download actions

* Added forcing of downloads for media items
2024-04-03 14:21:10 -07:00
Kieran
79c61bca4f
[Enhancement] Delete media after "x" days (#160)
* [Enhancement] Adds ability to stop media from re-downloading (#159)

* Added column

* Added methods for ignoring media items from future download

* Added new deletion options to controller and UI

* Added controller actions and UI for editing a media item

* Added column to sources

* Added retention period to form

* [WIP] getting retention methods in place

* Hooked up retention worker

* Added column and UI to prevent automatic deletion

* Docs

* Removed unused backfill worker

* Added edit links to media item tabs on source view

* Clarified form wording

* Form wording (again)
2024-04-03 10:44:11 -07:00
Kieran
f9c2f7b8f2 [Enhancement] Improve Dockerfile permissions (#157)
* Updated dockerfile

* added a healthcheck
2024-04-01 18:56:03 -07:00
Kieran Eglin
3daf72a161 Improved index to respect nulls 2024-04-01 18:22:58 -07:00
Kieran Eglin
4e26253b33 Re-adds source uniqueness index 2024-04-01 18:08:46 -07:00
Kieran
c58e176619 Added yt-dlp version to sidebar (#155) 2024-04-01 17:54:51 -07:00
Kieran
89f4634d1f
[Enhancement] Misc fixes 2024-03-29 (#150)
* Removes header to allow embedding in frame

* Escaped links in podcast RSS builder

* Turned homepage sections into clickable links

* Enabled live dashboard in all ENVs

* Ensured download worker always returns a 2-member tuple
2024-03-29 18:01:09 -07:00
Kieran
fae7ba3304 [Enhancement] Start tracking a media item's duration (#146)
* Added duration field, started importing it during indexing and download

* Added duration to RSS feed
2024-03-28 21:51:04 -07:00
Kieran
964ab2f89b [Feature] Add SponsorBlock-based section removal (#144)
* Added sponsorblock columns to media profile

* Added sponsorblock options to profile form

* Added SB to download options builder
2024-03-28 19:02:13 -07:00
Kieran
216e05567f trying something new with queries (#142) 2024-03-28 17:59:53 -07:00
Kieran
d1fe87bdf3 Updated fast indexing to respect title filters (#136) 2024-03-27 19:39:50 -07:00
Kieran
320b25f5b6 RSS feed for sources (#110)
* Add media streaming (#108)

* [WIP] set up streaming route

* Added UUID to sources and media items

* Added media preview to MI show page

* Added plug to strip file extensions

* [VERY WIP] basic podcast RSS setup

* [WIP] got basic podcast RSS working

* [WIP] more expanding on RSS

* Comment

* [WIP] Working on refactoring feed

* Added UUID backfill to a migration

* [WIP] Moar refactoring

* [WIP] Adding UI for getting RSS feed

* Many tests

* Added conditional routing for feed URLs

* Removed the need for url_base to be set

* Updated preset name

* Rendered certain fields HTML-safe; Added logging to confirm range request support

* Fixed incorrect scheme issue

* Updated env var

* Updated other UI to use dropdown

* removed commented code

* Generate rss feeds (#123)

* Added plug to strip file extensions

* [VERY WIP] basic podcast RSS setup

* [WIP] got basic podcast RSS working

* [WIP] more expanding on RSS

* [WIP] Working on refactoring feed

* Added UUID backfill to a migration

* [WIP] Moar refactoring

* [WIP] Adding UI for getting RSS feed

* Many tests

* Added conditional routing for feed URLs

* Removed the need for url_base to be set

* Updated preset name

* Rendered certain fields HTML-safe; Added logging to confirm range request support

* Fixed incorrect scheme issue

* Updated env var

* Updated other UI to use dropdown

* removed commented code

* docs

* Added unique index to UUID fields
2024-03-27 18:11:25 -07:00
Kieran
7411177e7d Added description to source (#124) 2024-03-25 20:40:09 -07:00
Kieran
c08654714a Removed unneeded startup tasks (#121) 2024-03-25 18:52:02 -07:00
Kieran
5c5e7a8a91 Started storing source images in internal metadata folder (#120) 2024-03-25 18:36:03 -07:00
Kieran
af803b7081 Fixed filepath bug when parsing metadata (#118) 2024-03-25 17:20:29 -07:00
Kieran
fe641cfc49 Cookie auth (#116)
* Added new extras directory

* Automatically create cookie file on boot

* Added cookie file option to DL option builder

* Put cookie auth in correct spot

* Updated Docs

* Removed useless setter for config path
2024-03-24 20:23:25 -07:00
Kieran
a3d0927dfb Improve preflight file permissions check (#114)
* Added startup permissions check with more helpful message

* Updated README

* Fixes faulty test
2024-03-24 15:06:08 -07:00
Kieran
37eb745d03 Add docs link (#103)
* Added docs link to README

* Added docs to sidebar in-app

* Updated some tests

* Updated docs in media profile form
2024-03-21 17:26:51 -07:00
Kieran
9310b5e26e Misc indexing improvements (#99)
* increased file follower timeout to 10 minutes

* Removed unique index across a source's collection and its media profile

* Ensure source gets updating during slow indexing
2024-03-20 18:34:33 -07:00
Kieran
3c522d8d6b Source title regex filtering (#98)
* Added a basic/advanced mode to source form

* Added regex field and UI to source form

* Implemented title regex filtering
2024-03-20 17:51:14 -07:00
Kieran
e2385eccd6 Improve index tables and improved consistency (#97)
* Simplifies index pages

* Improved UX of more pages
2024-03-19 19:03:02 -07:00
Kieran
565a1d4972 Download images for sources (#96)
* [WIP] hooked up photo downloading to source metadata worker

* Added new attributes for source images

* Added option to profile form
2024-03-19 18:34:01 -07:00
Kieran
05f3deebfa Source NFO downloads (#95)
* Hooked up series directory finding to source metadata runner

* Fixed aired NFO tag for episodes

* Updated MI NFO builder to take in a filepath

* Hooked up NFO generation to the source worker

* Added NFO controls to form

* Improved the way the source metadata worker updates the source

* Consolidated NFO selection options in media profile instead of source
2024-03-18 17:27:28 -07:00
Kieran
6e7e8dc96c Misc refactors 2024-03-15 (#90)
* Fixed TV show output template

* Discard jobs if record not found; hopefully addressed timing-related source deletion bug
2024-03-15 18:29:02 -07:00
Kieran
b633d0f75c Misc refactors 2024-03-14 (#89)
* Adds method to improve cleanup of empty directories

* resolved bug where source metadata worker could call itself in an infinite loop

* Refactored file deletion for media items

* Removed useless filesystem data worker

* Updated task listing fns to take a record directly

* Refactored the way I call workers

* Improved some tests
2024-03-15 17:44:58 -07:00
Kieran
a135746c97 Improve episode-level compatability with media center apps (#86)
* Add media profile presets (#85)

* Added presets for output templates

* Added presets for the entire media profile form

* Append `-thumb` to thumbnails when downloading (#87)

* Appended -thumb to thumbnails when downloading

* Added code to compensate for yt-dlp bug

* Squash all the commits from the other branch bc I broke things (#88)
2024-03-14 18:30:08 -07:00
Kieran
c67278ab5c Add support for episode NFO files (#84)
* Added nfo builder for 'episodes'

* Added NFO download fields; hooked up NFO generation to downloading pipeline

* Added NFO option to media profile
2024-03-13 18:31:53 -07:00
Kieran
1cc8731914 [Bugfix] Fix reddit bugs v2 (#82)
* Ensured thumbnail is converted to jpg before embedding

* Ensured media indexing doesn't fail if an upload date can't be parsed
2024-03-12 20:23:51 -07:00
Kieran
5b2d6b094d [Bugfix] Fixes issue with grabbing source/media details when first video is a premier (#79)
* Fixed issue with source details when first video is a premier

* Updated other occurance
2024-03-12 18:18:19 -07:00
Kieran
33baa99aae Refactor modules into contexts (#78)
* [WIP] break out a few contexts, start refactoring fast index modules

* [WIP] more contexts, this time around slow indexing and downloads

* [WIP] got all tests passing

* [WIP] Added moduledocs

* Built a genserver to rename old jobs on boot

* Added a module naming check; moved things around

* Fixed specs
2024-03-12 17:54:55 -07:00
Kieran
993865909c Onboarding improvements (#76)
* Disabled pro modal during onboarding

* Restructured the way onboarding changes settings

* Added better upload date template placeholder
2024-03-11 20:19:10 -07:00
Kieran
e1fc361e98 Improve audio-only download (#74)
* Improved quality from auto-only download; enables thumbnail and data embedding for audio

* Updated UI to reflect new audio behaviour
2024-03-11 19:33:27 -07:00
Kieran
e6b04bd675 Converts thumbnails to jpg for better compat (#70) 2024-03-11 18:03:40 -07:00
Kieran
f60ec4f49d Download cutoff date for sources (#69)
* Added new uploaded_at column to media items

* Updated indexer to pull upload date

* Updates media item creation to update on conflict

* Added download cutoff date to sources

* Applies cutoff date logic to pending media logic

* Updated docs
2024-03-10 21:24:01 -07:00
Kieran
19ec13e64c Local data worker improvements (#66)
* Adds new worker for backfilling data

* Adds backfill job to startup tasks
2024-03-10 15:17:15 -07:00
Kieran
e1f8e94686 Fast indexing (#58)
* Made method to getting singular media details; Renamed other related method

* Takes a fun and flirty digression to remove abstractions around yt-dlp since I'm 100% committed to using it exclusively

* Removed commented test code

* Lays the groundwork for fast indexing

* Added module for working with youtube RSS feed

* Added methods to kick off indexing workers from RSS response

* Improve short detection (#59)

* Made media attribute-related yt-dlp calls return a struct

* Added shorts attribute to media items

* Added ability to discern a short from yt-dlp response

* Updated search to use new shorts attribute

* Fast index UI (#63)

* Added fast_index field and adds it to source form

* Added fast indexing to source changeset operations

* Added fast indexing worker and updated other modules to start using it

* Handled fast index worker on source update

* Add support modals (#65)

* Added fast indexing upgrade modal

* Improved modal on smaller screens

* Updated links to work again

* Added donation modal

* Reverted source fast index to 15 minutes

* Removed unneeded HTML attributes from old alpine approach
2024-03-10 14:36:34 -07:00
Kieran
128485c30a Resolved bug where null values in search terms would break search (#57) 2024-03-07 18:42:20 -08:00
Kieran
0948bebb9d Add settings model (#54)
* Adds a basic settings model

* Added more settings methods; Hooked up initial settings runner to app boot

* Update onboarding flow to use settings model instead of session data
2024-03-07 17:32:12 -08:00
Kieran
82eb12b983 [Bugfix] Fix basic auth in deployed apps (#52)
* Renamed auth env vars and updated them to work in prod env

* Updated basic auth parsing to ignore empty strings
2024-03-04 18:29:26 -08:00
Kieran
a4f4afeac9 Updated README (#51) 2024-03-04 18:25:33 -08:00
Kieran
90a0022d85 Rename video-related modules to be media-related (#50)
* Added test from before that I forgor

* Renamed video-related modules to media
2024-03-04 17:46:33 -08:00
Kieran
f55cdc80dd Streaming media item creation during indexing (#49)
* Implemented streaming during indexing

* Updated file watcher to enqueue download; refactored download worker methods

* Updated File Follower Server timeout
2024-03-04 17:14:02 -08:00
Kieran
b370c97bfb Added way to track media filesize (#48) 2024-03-02 19:19:10 -08:00
Kieran
c37157d644 Support audio only downloads (#46)
* Adds option to only download audio

* Improves helper text for profile options
2024-03-02 13:01:47 -08:00
Kieran
09448e3fcc Source attributes in output template (#45)
* Updated download options to take a media item; allowed for specifying custom output path template options

* Fixed bug where indexing job wouldn't run for the first time

* Renamed friendly_name to custom_name

* Added new options to default template and UI

* Improved explainer UI
2024-03-02 12:15:10 -08:00
Kieran
be8bcf0eb2 Improve relationship UI handling (#42)
* Added tabbed view; improved relationships for media profile

* Adds new maybe_limit method for queries

* Improves UI for media items, associated tasks
2024-02-29 18:35:51 -08:00