Commit graph

54 commits

Author SHA1 Message Date
Kieran
fbe21cb304
Misc refactors 2024-03-14 (#89)
* Adds method to improve cleanup of empty directories

* resolved bug where source metadata worker could call itself in an infinite loop

* Refactored file deletion for media items

* Removed useless filesystem data worker

* Updated task listing fns to take a record directly

* Refactored the way I call workers

* Improved some tests
2024-03-15 10:44:58 -07:00
Kieran
0f3329e97d
Improve episode-level compatability with media center apps (#86)
* Add media profile presets (#85)

* Added presets for output templates

* Added presets for the entire media profile form

* Append `-thumb` to thumbnails when downloading (#87)

* Appended -thumb to thumbnails when downloading

* Added code to compensate for yt-dlp bug

* Squash all the commits from the other branch bc I broke things (#88)
2024-03-14 12:30:08 -07:00
Kieran
25eb772896
Add support for episode NFO files (#84)
* Added nfo builder for 'episodes'

* Added NFO download fields; hooked up NFO generation to downloading pipeline

* Added NFO option to media profile
2024-03-13 11:31:53 -07:00
Kieran
cf59bf99cd
More onboarding improvements (#83)
* Improved custom YYYY-MM-DD output template option

* Clarified embedding vs. downloading on media profile form

* Improved form helpers more; Added a helper to every field
2024-03-13 08:51:40 -07:00
Kieran
a7af6a9125
[Bugfix] Fix reddit bugs v2 (#82)
* Ensured thumbnail is converted to jpg before embedding

* Ensured media indexing doesn't fail if an upload date can't be parsed
2024-03-12 16:23:51 -07:00
Kieran
8f9d18dc71
Added input validation to help with cutoff date usage (#80) 2024-03-12 15:31:50 -07:00
Kieran
5ec2c92a0c
[Bugfix] Fixes issue with grabbing source/media details when first video is a premier (#79)
* Fixed issue with source details when first video is a premier

* Updated other occurance
2024-03-12 12:18:19 -07:00
Kieran
3c897e96e6
Refactor modules into contexts (#78)
* [WIP] break out a few contexts, start refactoring fast index modules

* [WIP] more contexts, this time around slow indexing and downloads

* [WIP] got all tests passing

* [WIP] Added moduledocs

* Built a genserver to rename old jobs on boot

* Added a module naming check; moved things around

* Fixed specs
2024-03-12 10:54:55 -07:00
Kieran
b2f39a7b7f
Onboarding improvements (#76)
* Disabled pro modal during onboarding

* Restructured the way onboarding changes settings

* Added better upload date template placeholder
2024-03-11 15:19:10 -07:00
Kieran
7e4f1a8412
Improve audio-only download (#74)
* Improved quality from auto-only download; enables thumbnail and data embedding for audio

* Updated UI to reflect new audio behaviour
2024-03-11 14:33:27 -07:00
Kieran
afe41d2e36
Converts thumbnails to jpg for better compat (#70) 2024-03-11 09:03:40 -07:00
Kieran
ccdcf8eec5
Download cutoff date for sources (#69)
* Added new uploaded_at column to media items

* Updated indexer to pull upload date

* Updates media item creation to update on conflict

* Added download cutoff date to sources

* Applies cutoff date logic to pending media logic

* Updated docs
2024-03-10 21:24:01 -07:00
Kieran
09a4bcb36b
Local data worker improvements (#66)
* Adds new worker for backfilling data

* Adds backfill job to startup tasks
2024-03-10 15:17:15 -07:00
Kieran
dc0313d875
Fast indexing (#58)
* Made method to getting singular media details; Renamed other related method

* Takes a fun and flirty digression to remove abstractions around yt-dlp since I'm 100% committed to using it exclusively

* Removed commented test code

* Lays the groundwork for fast indexing

* Added module for working with youtube RSS feed

* Added methods to kick off indexing workers from RSS response

* Improve short detection (#59)

* Made media attribute-related yt-dlp calls return a struct

* Added shorts attribute to media items

* Added ability to discern a short from yt-dlp response

* Updated search to use new shorts attribute

* Fast index UI (#63)

* Added fast_index field and adds it to source form

* Added fast indexing to source changeset operations

* Added fast indexing worker and updated other modules to start using it

* Handled fast index worker on source update

* Add support modals (#65)

* Added fast indexing upgrade modal

* Improved modal on smaller screens

* Updated links to work again

* Added donation modal

* Reverted source fast index to 15 minutes

* Removed unneeded HTML attributes from old alpine approach
2024-03-10 14:36:34 -07:00
Kieran
1caddd86c7
Resolved bug where null values in search terms would break search (#57) 2024-03-07 12:24:20 -08:00
Kieran
d0f55cd463
Add settings model (#54)
* Adds a basic settings model

* Added more settings methods; Hooked up initial settings runner to app boot

* Update onboarding flow to use settings model instead of session data
2024-03-07 10:32:12 -08:00
Kieran
254054c0c1
[Bugfix] Fix basic auth in deployed apps (#52)
* Renamed auth env vars and updated them to work in prod env

* Updated basic auth parsing to ignore empty strings
2024-03-04 13:29:26 -08:00
Kieran
cbade0dd02
Rename video-related modules to be media-related (#50)
* Added test from before that I forgor

* Renamed video-related modules to media
2024-03-04 10:46:33 -08:00
Kieran
63e5058365
Streaming media item creation during indexing (#49)
* Implemented streaming during indexing

* Updated file watcher to enqueue download; refactored download worker methods

* Updated File Follower Server timeout
2024-03-04 10:14:02 -08:00
Kieran
a46dd19ec7
Added way to track media filesize (#48) 2024-03-02 19:19:10 -08:00
Kieran
55f9676bb7
Clarifies wording of source form for index frequency (#47) 2024-03-02 13:09:58 -08:00
Kieran
fc52f16475
Support audio only downloads (#46)
* Adds option to only download audio

* Improves helper text for profile options
2024-03-02 13:01:47 -08:00
Kieran
8cc36928f1
Source attributes in output template (#45)
* Updated download options to take a media item; allowed for specifying custom output path template options

* Fixed bug where indexing job wouldn't run for the first time

* Renamed friendly_name to custom_name

* Added new options to default template and UI

* Improved explainer UI
2024-03-02 12:15:10 -08:00
Kieran
6738071613
Useful data on homepage v1 (#43)
* Made the homepage _slightly_ more useful

* Very minor refactor in alpine body-level object
2024-03-01 16:25:55 -08:00
Kieran
b54e69c609
Improve relationship UI handling (#42)
* Added tabbed view; improved relationships for media profile

* Adds new maybe_limit method for queries

* Improves UI for media items, associated tasks
2024-02-29 15:35:51 -08:00
Kieran
5b29f5036a
Responsive UI improvements (#41)
* Improved responsive UX for media profiles

* Improved responsiveness for other pages

* Added docs; fixed up stragglers
2024-02-29 12:24:54 -08:00
Kieran
9fc2edfd4a
Added some logos up in this (#40) 2024-02-29 11:37:14 -08:00
Kieran
01b8856f22
UX improvements v1 (#39)
* Removed collection_type user input instead inferring from yt-dlp response

* Updated docs

* Added delete buttons for source; Refactored the way deletion methods work

* Update source to always run an initial index

* Added deletion to the last models

* Improved clarity around deletion operation

* Improved task fetching and deletion methods

* More tests
2024-02-28 09:54:01 -08:00
Kieran
3bb7edba0b
Resolved windows filesystem bug (#38) 2024-02-28 08:56:25 -08:00
Kieran
52c0a1742a
Improve metadata storage (#37)
* Updated config path to specify metadata storage path

* Removed metadata column from DB, replacing it with filepath columns

* Updated app to store compressed metadata; automatically download thumbnails

* Ensured metadata is deleted when other files are deleted

* Updated docs

* Added inets to application start so http calls work in prod
2024-02-24 13:40:05 -08:00
Kieran
e33a46bddc
Small form improvements (#36)
* Clarified intent of custom name field

* More copy improvements
2024-02-22 15:48:25 -08:00
Kieran
2ca395a5bf
Updated video downloader to use full URL; improved docs (#35) 2024-02-22 12:46:08 -08:00
Kieran
0d29f70a28
Added log rotation for prod config (#32) 2024-02-21 12:01:18 -08:00
Kieran
c1f30dbd2c
Added optional basic auth (#31) 2024-02-21 11:25:25 -08:00
Kieran
94f777d1ed
Updated source's friendly_name to default to the collection_name (#29) 2024-02-20 14:24:47 -08:00
Kieran
47f0fd533f
Refactor and document output path (#28)
* Updated output path builder to better use yt-dlp variables

* Added UI explaining output template options

* Fixed outdated reference in dockerfile
2024-02-20 13:40:13 -08:00
Kieran
7d62f7c7df
Rename media sources (#25)
* Renamed MediaSource to Sources

* Renamed MediaItem controller namespace
2024-02-20 09:48:20 -08:00
Kieran
d4d653398b
Basic onboading flow (#24)
* removed unused file

* [WIP] Created onboarding form

* Finished basic onboarding flow; added tests

* Improved onboarding language
2024-02-20 09:37:17 -08:00
Kieran
fab77f4deb
Prod packaging v1 (#23)
* Sets media directory for prod

* Improved logging; Added docker files for prod

* Improved fetching SECRET_KEY_BASE for selfhosted instances
2024-02-16 11:50:00 -08:00
Kieran
8fcd41f45d
Switch to sqlite (#22)
* Updated project to use sqlite

* Edited migrations directly because I fear no man

* Updated search functions and tests to work with sqlite

* Updated build tools to remove postgres
2024-02-16 10:11:41 -08:00
Kieran
44bd105e80
Add resolution option (#21)
* added preferred resolution to profiles

* Add resolution to download option builder
2024-02-15 14:21:41 -08:00
Kieran
9d3b9cc063
Delete media items (#20)
* Added method for deleting media files and their content

* Adds controllers and methods for deleting media and files

* Improved tmpfile setup and teardown for tests

* Actually got tmpfile cleanup running once per suite run

* Finally fixed flash messages
2024-02-15 13:51:19 -08:00
Kieran
060e340558
Search v1 (#19)
* Adds description to media items; hooks it up to indexing/media downloading

* Added a search method using postgres fulltext search

* Hooked up search functionality to the search form

* Added persistence to the search form when on search page
2024-02-13 14:46:14 -08:00
Kieran
219320ce11
Media item lifecycle improvements (#18)
* Added jobs for enqueuing/dequeuing media based on source's status

* adds lifecycle columns to source and media_item tables
2024-02-09 20:51:23 -08:00
Kieran
8bdd189fa5
Media UI v1 (#17)
* Updated comments

* Added basic media tables to sources#show

* Adds a very basic listing and show page for media
2024-02-09 16:21:51 -08:00
Kieran
dabec62e99
Redo indexing mechanism (#16)
* Bumped up the line length because I fear no man

* Refactored indexing

Previously, indexing worked by collecting the video IDs of only videos
that matched indexing criteria. This new model instead stores ALL videos
for a given source, but will only _download_ videos that meet that criteria.
This lets us backfill without indexing, makes it easier to add in other
backends, lets us download one-off videos for a source that don't quite
meet criteria, you name it.

* Updated media finders to respect format filters; Added credo file
2024-02-09 13:23:37 -08:00
Kieran
89497c4c6b
First UI Pass (#15)
* Added scratchpad dir

* Installed Alpine

* [WIP] began integrating Tailwind and accompanying theme

* [WIP] Set up basic views for sources

* Adds new UI to media profiles page

* Removes unneeded view
2024-02-08 11:03:11 -08:00
Kieran
013b2ccc21
Media downloading indexing options (round 2) (#14)
* Adds options + option builder + metadata parsing for media thumbnails

* Added release-type options to media profile; built option parser for indexing operations

* Added new media_profile options to creation form; made show helper for rendering database items

* Added options for downloading/embedding metadata

* Adds option on sources to not download media (index only)

* reformatted docs
2024-02-06 15:36:41 -08:00
Kieran
ca01f17b58
Start integrating playlist support (#13)
* [WIP] updated the output of VideoCollection to include playlists

* Updated source's name to collection_name; supported playlist ID/name fetching

* Hooked up collection_type to form; refactored enqueue_pending_media_downloads

* Added friendly_name to form

* Added media profile link to source view

* Updates comment
2024-02-05 11:38:19 -08:00
Kieran
bdef6c75bb
Change Channel to Source (#12)
* Ensure channel detail lookup doesn't download the video - oops

* Ran the needful migrations for replacing channels with sources

* got media source test back working

* channel tasks test

* Media indexing worker test

* media tests

* Got all tests working except controller tests

* got all tests working

* Renamed Channel struct to Source

* renamed ChannelTasks

* More renaming; renamed channel details module

* Removed Channel yt-dlp module, instead throwing things in the VideoCollection module

* Renamed what looks like the last of the outstanding data
2024-02-02 10:45:21 -08:00