Commit graph

30 commits

Author SHA1 Message Date
Kieran
a2a0f48065
[Enhancement] Add media count to sources page (#251)
* [WIP] Got query kinda working, now to refactor other queries

* Refactored all query methods to use dynamic snippets

* Refactored tab layout to grab tabs by name

* Removed standalone show buttons from in-app tables

* Removed unneeded comment
2024-05-21 08:53:22 -07:00
Kieran
933daf8c78
[Housekeeping] Improve test stability (#232)
* Ensured all tests use at least the DataCase module

* addressed one flakey test

* More flakey tests

* Consolidated Mox usage
2024-05-08 09:57:36 -07:00
Kieran
04b14719ee
[Enhancement] Add Media Center support for videos uploaded on the same day (#221)
* Added upload date index field to media_items

* Added incrementing index for upload dates

* Added media item upload date index to download option builder

* Added new season_episode_index_from_date to UI; updated parser

* Improve support for channels

* Hopefully fixed flakey test
2024-05-02 11:06:10 -07:00
Kieran
112c6a4f14
[Enhancement] Custom media lifecycle scripts (#219)
* Namespaced notification modules under lifecycle

* Added a JSON encoder for all the main model types

* Added startup task to create user script file

* Hook up user script event to media download

* Hooked up media deletion user script

* Added jq to docker deps

* Updated README
2024-05-02 08:43:37 -07:00
Kieran
618711691b
[Bugfix]: Misc. bugfixes 2024-04-16 (#189)
* Manually installed mutagen

* Stopped upgrade form from submitting on enter

* Gracefully handle duplicate enqueued downloads

* Update metadata thumbnail fetcher to use the best jpg available
2024-04-16 17:37:39 -07:00
Kieran
e841f39cf2 [Enhancement] Redownload new media after a delay (#173)
* Added redownload-related columns

* Added methods for fetching re-downloadable media items

* Filled out redownload worker + tests

* Added redownload worker to config.exs cron

* Added to UI and README
2024-04-10 17:54:45 -07:00
Kieran
e55bcaddd0
[Enhancement] Improve ordering of models (#164)
* [WIP] ordering app queries

* Refactored media queries to be self-contained
2024-04-03 17:26:46 -07:00
Kieran
79c61bca4f
[Enhancement] Delete media after "x" days (#160)
* [Enhancement] Adds ability to stop media from re-downloading (#159)

* Added column

* Added methods for ignoring media items from future download

* Added new deletion options to controller and UI

* Added controller actions and UI for editing a media item

* Added column to sources

* Added retention period to form

* [WIP] getting retention methods in place

* Hooked up retention worker

* Added column and UI to prevent automatic deletion

* Docs

* Removed unused backfill worker

* Added edit links to media item tabs on source view

* Clarified form wording

* Form wording (again)
2024-04-03 10:44:11 -07:00
Kieran
216e05567f trying something new with queries (#142) 2024-03-28 17:59:53 -07:00
Kieran
d1fe87bdf3 Updated fast indexing to respect title filters (#136) 2024-03-27 19:39:50 -07:00
Kieran
320b25f5b6 RSS feed for sources (#110)
* Add media streaming (#108)

* [WIP] set up streaming route

* Added UUID to sources and media items

* Added media preview to MI show page

* Added plug to strip file extensions

* [VERY WIP] basic podcast RSS setup

* [WIP] got basic podcast RSS working

* [WIP] more expanding on RSS

* Comment

* [WIP] Working on refactoring feed

* Added UUID backfill to a migration

* [WIP] Moar refactoring

* [WIP] Adding UI for getting RSS feed

* Many tests

* Added conditional routing for feed URLs

* Removed the need for url_base to be set

* Updated preset name

* Rendered certain fields HTML-safe; Added logging to confirm range request support

* Fixed incorrect scheme issue

* Updated env var

* Updated other UI to use dropdown

* removed commented code

* Generate rss feeds (#123)

* Added plug to strip file extensions

* [VERY WIP] basic podcast RSS setup

* [WIP] got basic podcast RSS working

* [WIP] more expanding on RSS

* [WIP] Working on refactoring feed

* Added UUID backfill to a migration

* [WIP] Moar refactoring

* [WIP] Adding UI for getting RSS feed

* Many tests

* Added conditional routing for feed URLs

* Removed the need for url_base to be set

* Updated preset name

* Rendered certain fields HTML-safe; Added logging to confirm range request support

* Fixed incorrect scheme issue

* Updated env var

* Updated other UI to use dropdown

* removed commented code

* docs

* Added unique index to UUID fields
2024-03-27 18:11:25 -07:00
Kieran
3c522d8d6b Source title regex filtering (#98)
* Added a basic/advanced mode to source form

* Added regex field and UI to source form

* Implemented title regex filtering
2024-03-20 17:51:14 -07:00
Kieran
b633d0f75c Misc refactors 2024-03-14 (#89)
* Adds method to improve cleanup of empty directories

* resolved bug where source metadata worker could call itself in an infinite loop

* Refactored file deletion for media items

* Removed useless filesystem data worker

* Updated task listing fns to take a record directly

* Refactored the way I call workers

* Improved some tests
2024-03-15 17:44:58 -07:00
Kieran
33baa99aae Refactor modules into contexts (#78)
* [WIP] break out a few contexts, start refactoring fast index modules

* [WIP] more contexts, this time around slow indexing and downloads

* [WIP] got all tests passing

* [WIP] Added moduledocs

* Built a genserver to rename old jobs on boot

* Added a module naming check; moved things around

* Fixed specs
2024-03-12 17:54:55 -07:00
Kieran
f60ec4f49d Download cutoff date for sources (#69)
* Added new uploaded_at column to media items

* Updated indexer to pull upload date

* Updates media item creation to update on conflict

* Added download cutoff date to sources

* Applies cutoff date logic to pending media logic

* Updated docs
2024-03-10 21:24:01 -07:00
Kieran
e1f8e94686 Fast indexing (#58)
* Made method to getting singular media details; Renamed other related method

* Takes a fun and flirty digression to remove abstractions around yt-dlp since I'm 100% committed to using it exclusively

* Removed commented test code

* Lays the groundwork for fast indexing

* Added module for working with youtube RSS feed

* Added methods to kick off indexing workers from RSS response

* Improve short detection (#59)

* Made media attribute-related yt-dlp calls return a struct

* Added shorts attribute to media items

* Added ability to discern a short from yt-dlp response

* Updated search to use new shorts attribute

* Fast index UI (#63)

* Added fast_index field and adds it to source form

* Added fast indexing to source changeset operations

* Added fast indexing worker and updated other modules to start using it

* Handled fast index worker on source update

* Add support modals (#65)

* Added fast indexing upgrade modal

* Improved modal on smaller screens

* Updated links to work again

* Added donation modal

* Reverted source fast index to 15 minutes

* Removed unneeded HTML attributes from old alpine approach
2024-03-10 14:36:34 -07:00
Kieran
128485c30a Resolved bug where null values in search terms would break search (#57) 2024-03-07 18:42:20 -08:00
Kieran
f55cdc80dd Streaming media item creation during indexing (#49)
* Implemented streaming during indexing

* Updated file watcher to enqueue download; refactored download worker methods

* Updated File Follower Server timeout
2024-03-04 17:14:02 -08:00
Kieran
be8bcf0eb2 Improve relationship UI handling (#42)
* Added tabbed view; improved relationships for media profile

* Adds new maybe_limit method for queries

* Improves UI for media items, associated tasks
2024-02-29 18:35:51 -08:00
Kieran
1f4823bda9 UX improvements v1 (#39)
* Removed collection_type user input instead inferring from yt-dlp response

* Updated docs

* Added delete buttons for source; Refactored the way deletion methods work

* Update source to always run an initial index

* Added deletion to the last models

* Improved clarity around deletion operation

* Improved task fetching and deletion methods

* More tests
2024-02-28 18:54:01 -08:00
Kieran
c77047951f Improve metadata storage (#37)
* Updated config path to specify metadata storage path

* Removed metadata column from DB, replacing it with filepath columns

* Updated app to store compressed metadata; automatically download thumbnails

* Ensured metadata is deleted when other files are deleted

* Updated docs

* Added inets to application start so http calls work in prod
2024-02-24 13:40:05 -08:00
Kieran
af7e76c1d1 Rename media sources (#25)
* Renamed MediaSource to Sources

* Renamed MediaItem controller namespace
2024-02-20 17:48:20 -08:00
Kieran
69c47c2c65 Basic onboading flow (#24)
* removed unused file

* [WIP] Created onboarding form

* Finished basic onboarding flow; added tests

* Improved onboarding language
2024-02-20 17:37:17 -08:00
Kieran
b81c8d64b3 Switch to sqlite (#22)
* Updated project to use sqlite

* Edited migrations directly because I fear no man

* Updated search functions and tests to work with sqlite

* Updated build tools to remove postgres
2024-02-16 17:11:41 -08:00
Kieran
c5fcb8fe12 Delete media items (#20)
* Added method for deleting media files and their content

* Adds controllers and methods for deleting media and files

* Improved tmpfile setup and teardown for tests

* Actually got tmpfile cleanup running once per suite run

* Finally fixed flash messages
2024-02-15 17:51:19 -08:00
Kieran
58771ee37b Search v1 (#19)
* Adds description to media items; hooks it up to indexing/media downloading

* Added a search method using postgres fulltext search

* Hooked up search functionality to the search form

* Added persistence to the search form when on search page
2024-02-13 17:46:14 -08:00
Kieran
e0637331bb Redo indexing mechanism (#16)
* Bumped up the line length because I fear no man

* Refactored indexing

Previously, indexing worked by collecting the video IDs of only videos
that matched indexing criteria. This new model instead stores ALL videos
for a given source, but will only _download_ videos that meet that criteria.
This lets us backfill without indexing, makes it easier to add in other
backends, lets us download one-off videos for a source that don't quite
meet criteria, you name it.

* Updated media finders to respect format filters; Added credo file
2024-02-09 18:23:37 -08:00
Kieran
366fd80213 Change Channel to Source (#12)
* Ensure channel detail lookup doesn't download the video - oops

* Ran the needful migrations for replacing channels with sources

* got media source test back working

* channel tasks test

* Media indexing worker test

* media tests

* Got all tests working except controller tests

* got all tests working

* Renamed Channel struct to Source

* renamed ChannelTasks

* More renaming; renamed channel details module

* Removed Channel yt-dlp module, instead throwing things in the VideoCollection module

* Renamed what looks like the last of the outstanding data
2024-02-02 17:45:21 -08:00
Kieran
977b69b7c3 Allow subtitle downloading (#11)
* Added subtitle options to media profile model

* Updated media profile form

* Adds subtitle-based options in options builder

* Updates metadata parser to include subtitles

* Adds subtitle_filepaths to media_item

* renamed video_filepath to media_filepath

* Added more fields to media profile show page
2024-01-31 18:55:02 -08:00
Kieran
4dd9d837a3 Download video(s) (first iteration) (#5)
* Updated package.json (and made an excuse to make a branch)

* Video filepath parser (#6)

* Restructured files; Added parser placeholder

* More restructuring

* Added basic parser for hydrating template strings

* Improved docs

* More docs

* Initial implementation of media profiles (#7)

* [WIP] Added basic video download method

* [WIP] Very-WIP first steps at parsing options and downloading

* Made my options safe by default and removed special safe versions

* Ran html generator for mediaprofile model - leaving as-is for now

* Addressed a bunch of TODO comments

* Add "channel" type Media Source (#8)

* [WIP] Working on fetching channel metadata in yt-dlp backend

* Finished first draft of methods to do with querying channels

* Renamed CommandRunnerMock to have a more descriptive name

* Ran the phx generator for the channel model

* Renamed Downloader namespace to MediaClient

* [WIP] saving before attempting LiveView

* LiveView did not work out but here's a working controller how about

* Index a channel (#9)

* Ran a MediaItem generator; Reformatted to my liking

* [WIP] added basic index function

* setup oban

* Added basic Oban job for indexing

* Added in workers for indexing; hooked them into record creation flow

* Added a task model with a phx generator

* Tied together tasks with jobs and channels

* Download indexed videos (#10)

* Clarified documentation

* more comments

* [WIP] hooked up basic video downloading; starting work on metadata

* Added metadata model and parsing

Adding the metadata model made me realize that, in many cases, yt-dlp
returns undesired input in stdout, breaking parsing. In order to get
the metadata model working, I had to change the way in which the app
interacts with yt-dlp. Now, output is written as a file to disk which
is immediately re-read and returned.

* Added tests for video download worker

* Hooked up video downloading to the channel indexing pipeline

* Adds tasks for media items

* Updated video metadata parser to extract the title

* Ran linting
2024-01-30 20:15:59 -08:00