Commit graph

24 commits

Author SHA1 Message Date
Kieran
e7adc9d68f
[Enhancement] Record and display errors related to downloading (#610)
* Added last_error to media item table

* Error messages are now persisted to the last_error field

* Minor layout updates

* Added help tooltip to source content view

* Added error information to homepage tables

* Remove unneeded index

* Added docs to tooltip component
2025-02-12 10:17:24 -08:00
Kieran
e9f6b45953
[Enhancement] Add rate limiting to yt-dlp requests; prevent saving Media Items when throttled by YouTube (#559)
* Added sleep interval to settings

* Added new sleep setting to yt-dlp runner and added tests

* Added setting for form; updated setting name

* Updated form label

* Prevented saving/updating of media items if being throttled by youtube

* Added the bot message to the list of non-retryable errors

* Fixed typo
2025-01-14 11:38:40 -08:00
Robert Kleinschuster
e9d365ee9e
use css truncate everywhere and remove StringUtils.truncate function (#513)
Co-authored-by: robs <git@robs.social>
2024-12-13 12:23:12 -08:00
Kieran
a5b65061f0
[Housekeeping] Bump Phoenix LiveView to 1.0.0 (#495)
* bumped liveview to 1.0.0

* Converted interpolation to new syntax
2024-12-06 10:37:15 -08:00
Kieran
0163e85e76
[Bugfix] prevent duplicate videos from being downloaded if the video's name changes (#396)
* Added methods for deleting outdated files

* Hooked up outdated file deletion to media download worker
2024-09-23 15:59:48 -07:00
Kieran
22439e0273
[Housekeeping] Store the upload event as a datetime rather than a date (#269)
* [WIP] renamed and migrated upload_date column

* Refactored yt-dlp media module

* Refactored parse_upload_date

* Refactored media item upload_date_index

* Got media tests running

* Refactored media item table live

* Cleaned up the stragglers

* Fixed old oversight re: original_url
2024-05-28 13:01:40 -07:00
Kieran
b2e16d50cd
Added library size to homepage (#264) 2024-05-27 10:43:50 -07:00
Kieran
b1a6a6a81f
[Bugfix] Fix yt-dlp illegal write issue that causes failure to fetch source details (#247)
* Bandaid fix for yt-dlp write issue

* Ensured tempfile directory on app boot
2024-05-15 17:32:56 -07:00
Kieran
933daf8c78
[Housekeeping] Improve test stability (#232)
* Ensured all tests use at least the DataCase module

* addressed one flakey test

* More flakey tests

* Consolidated Mox usage
2024-05-08 09:57:36 -07:00
Kieran
112c6a4f14
[Enhancement] Custom media lifecycle scripts (#219)
* Namespaced notification modules under lifecycle

* Added a JSON encoder for all the main model types

* Added startup task to create user script file

* Hook up user script event to media download

* Hooked up media deletion user script

* Added jq to docker deps

* Updated README
2024-05-02 08:43:37 -07:00
Kieran
aea40a3f92
[Enhancement] Add pagination for a source's media (#190)
* [WIP] added first attempt at pagination component

* Hooked up pagination for downloaded media
2024-04-17 10:22:55 -07:00
Kieran
c36e33e1fd [Housekeeping] Close system port when jobs are cancelled (#182)
* Created a test setup that works

* Refactored test setup into real-world fixes
2024-04-10 22:13:19 -07:00
Kieran Eglin
0fcdd1df84 Renamed FS Helpers module to FS Utils 2024-04-10 18:02:26 -07:00
Kieran
26d457e656 [Enhancement] Add Apprise support (#170)
* [WIP] add settings sidebar entry and placeholder page

* [WIP] added placeholder UI and logic for settings form

* Added column and UI for apprise server

* Add some tests

* Added placeholder command runner for apprise

* [WIP] Adding apprise package

* Added apprise command runner

* Hooked up apprise notification module

* Ensured apprise was running in verbose mode

* Updated wording of apprise notification

* Added apprise to README
2024-04-09 17:45:39 -07:00
Kieran
81b49f55bf
[Bugfix] Properly escape NFO files (#168)
* Properly escaped NFO file contents

* Added an NFO backfill worker

* Added a try-catch to the backfill since I _really_ don't want failures to halt app boot
2024-04-06 11:48:28 -07:00
Kieran
320b25f5b6 RSS feed for sources (#110)
* Add media streaming (#108)

* [WIP] set up streaming route

* Added UUID to sources and media items

* Added media preview to MI show page

* Added plug to strip file extensions

* [VERY WIP] basic podcast RSS setup

* [WIP] got basic podcast RSS working

* [WIP] more expanding on RSS

* Comment

* [WIP] Working on refactoring feed

* Added UUID backfill to a migration

* [WIP] Moar refactoring

* [WIP] Adding UI for getting RSS feed

* Many tests

* Added conditional routing for feed URLs

* Removed the need for url_base to be set

* Updated preset name

* Rendered certain fields HTML-safe; Added logging to confirm range request support

* Fixed incorrect scheme issue

* Updated env var

* Updated other UI to use dropdown

* removed commented code

* Generate rss feeds (#123)

* Added plug to strip file extensions

* [VERY WIP] basic podcast RSS setup

* [WIP] got basic podcast RSS working

* [WIP] more expanding on RSS

* [WIP] Working on refactoring feed

* Added UUID backfill to a migration

* [WIP] Moar refactoring

* [WIP] Adding UI for getting RSS feed

* Many tests

* Added conditional routing for feed URLs

* Removed the need for url_base to be set

* Updated preset name

* Rendered certain fields HTML-safe; Added logging to confirm range request support

* Fixed incorrect scheme issue

* Updated env var

* Updated other UI to use dropdown

* removed commented code

* docs

* Added unique index to UUID fields
2024-03-27 18:11:25 -07:00
Kieran
33baa99aae Refactor modules into contexts (#78)
* [WIP] break out a few contexts, start refactoring fast index modules

* [WIP] more contexts, this time around slow indexing and downloads

* [WIP] got all tests passing

* [WIP] Added moduledocs

* Built a genserver to rename old jobs on boot

* Added a module naming check; moved things around

* Fixed specs
2024-03-12 17:54:55 -07:00
Kieran
a4f4afeac9 Updated README (#51) 2024-03-04 18:25:33 -08:00
Kieran
f55cdc80dd Streaming media item creation during indexing (#49)
* Implemented streaming during indexing

* Updated file watcher to enqueue download; refactored download worker methods

* Updated File Follower Server timeout
2024-03-04 17:14:02 -08:00
Kieran
01042ebd1b Updated source's friendly_name to default to the collection_name (#29) 2024-02-20 19:24:47 -08:00
Kieran
58771ee37b Search v1 (#19)
* Adds description to media items; hooks it up to indexing/media downloading

* Added a search method using postgres fulltext search

* Hooked up search functionality to the search form

* Added persistence to the search form when on search page
2024-02-13 17:46:14 -08:00
Kieran
e0637331bb Redo indexing mechanism (#16)
* Bumped up the line length because I fear no man

* Refactored indexing

Previously, indexing worked by collecting the video IDs of only videos
that matched indexing criteria. This new model instead stores ALL videos
for a given source, but will only _download_ videos that meet that criteria.
This lets us backfill without indexing, makes it easier to add in other
backends, lets us download one-off videos for a source that don't quite
meet criteria, you name it.

* Updated media finders to respect format filters; Added credo file
2024-02-09 18:23:37 -08:00
Kieran
4dd9d837a3 Download video(s) (first iteration) (#5)
* Updated package.json (and made an excuse to make a branch)

* Video filepath parser (#6)

* Restructured files; Added parser placeholder

* More restructuring

* Added basic parser for hydrating template strings

* Improved docs

* More docs

* Initial implementation of media profiles (#7)

* [WIP] Added basic video download method

* [WIP] Very-WIP first steps at parsing options and downloading

* Made my options safe by default and removed special safe versions

* Ran html generator for mediaprofile model - leaving as-is for now

* Addressed a bunch of TODO comments

* Add "channel" type Media Source (#8)

* [WIP] Working on fetching channel metadata in yt-dlp backend

* Finished first draft of methods to do with querying channels

* Renamed CommandRunnerMock to have a more descriptive name

* Ran the phx generator for the channel model

* Renamed Downloader namespace to MediaClient

* [WIP] saving before attempting LiveView

* LiveView did not work out but here's a working controller how about

* Index a channel (#9)

* Ran a MediaItem generator; Reformatted to my liking

* [WIP] added basic index function

* setup oban

* Added basic Oban job for indexing

* Added in workers for indexing; hooked them into record creation flow

* Added a task model with a phx generator

* Tied together tasks with jobs and channels

* Download indexed videos (#10)

* Clarified documentation

* more comments

* [WIP] hooked up basic video downloading; starting work on metadata

* Added metadata model and parsing

Adding the metadata model made me realize that, in many cases, yt-dlp
returns undesired input in stdout, breaking parsing. In order to get
the metadata model working, I had to change the way in which the app
interacts with yt-dlp. Now, output is written as a file to disk which
is immediately re-read and returned.

* Added tests for video download worker

* Hooked up video downloading to the channel indexing pipeline

* Adds tasks for media items

* Updated video metadata parser to extract the title

* Ran linting
2024-01-30 20:15:59 -08:00
Kieran
89b1640eb6 Initial yt-dlp implementation (#3)
* Added basic runner for yt-dlp commands

* Updated formatting rules

* Added JSON runner; Added test script for CLI interactions

* Reworked yt-dlp runner; added proper test mocks
2024-01-19 22:26:32 -08:00