Commit graph

322 commits

Author SHA1 Message Date
Jesse Bannon
e3c662bc35
[REFACTOR] Shared plugin code (#664)
* [REFACTOR] Shared plugin code

* arg names

* all plugins have priority

* split abstract method

* lint

* lint
2023-07-25 12:11:50 -07:00
Jesse Bannon
dcd636192c
[REFACTOR] download abstraction overhaul (#663)
* [REFACTOR] `download` abstraction overhaul

* remove plugin options
2023-07-25 00:36:51 -07:00
Jesse Bannon
bb03c91656
[BACKEND] Links to deprecation (#662) 2023-07-25 00:10:54 -07:00
Jesse Bannon
77487aee86
[FEATURE] Simplify `video_tags` (#661)
* [FEATURE] Simplify ``video_tags``

* video tags old

* change for episode

* deprecation notice
2023-07-24 18:18:53 -07:00
Jesse Bannon
cc38db9c7a
[FEATURE] Simplify `music_tags` plugin (#660)
* [FEATURE] Simplify ``music_tags`` plugin

* docs

* more compatible

* check old format better

* better dict instantiate
2023-07-24 18:05:54 -07:00
Jesse Bannon
0cbd2af4d3
[FEATURE] embed_thumbnail plugin (#659)
* [REFACTOR] OptionsValidator + OptionsDictValidator

* more

* [FEATURE] Dedicated embed_thumbnail plugin

* tests

* fix test

* docs

* deprecation notices

* priority to run after file convert

* naming

* fixtures
2023-07-24 12:07:36 -07:00
Jesse Bannon
7e997f304d
[REFACTOR] OptionsValidator + OptionsDictValidator (#657)
* [REFACTOR] OptionsValidator + OptionsDictValidator

* more
2023-07-24 10:12:38 -07:00
Jesse Bannon
b8d1f1edbb
[FEATURE] Ability to change download archive name (#643) 2023-06-30 11:51:52 -07:00
Jesse Bannon
7a43b0a90b
[FEATURE] Add __value__ to subscription files to be able to create one-liner subs (#641)
* [FEATURE] Add __value__ to subscription files to create one-liner subs

* docs
2023-06-29 23:57:54 -07:00
Jesse Bannon
557854fc83
[FEATURE] release_date source variables (#638)
* [FEATURE] `release_date` source variables

* lint
2023-06-24 23:01:18 -07:00
Jesse Bannon
a8841bf150
[BUGFIX] Ignore extra data streams with -dn in ffmpeg (#637) 2023-06-24 15:42:29 -07:00
Jesse Bannon
134882a17d
[DOCS] Fix url + multi_url docs (#635) 2023-06-17 13:45:22 -07:00
Jesse Bannon
0f39b38dc3
[DOCS] Actionable message for file lock errors (#634)
* [DOCS] Actionable message for file lock errors"

* pylint
2023-06-17 13:42:36 -07:00
Jesse Bannon
acc7bb306b
[DOCS] Clean up match-filter plugin docs (#633) 2023-06-17 13:32:12 -07:00
Jesse Bannon
a993beff0e
[FEATURE] Ability to use overrides in regex plugin (#631)
* [FEATURE] Ability to use overrides in regex plugin

* lint

* more tests

* more docs

* docs
2023-06-17 13:02:35 -07:00
Jesse Bannon
ac5561cb2e
[FEATURE] Content rating override variable (#626)
* [FEATURE] Content rating override variable

* docs
2023-06-03 07:42:16 -07:00
Jesse Bannon
7816c2fe59
[BUGFIX] Add check_formats on download retry (#623) 2023-05-30 23:20:40 -07:00
costaht
465a290f8b
[BUGFIX] Add line break to transaction logs (#622)
* add line break to  logs

* Update main.py
2023-05-27 22:03:45 -07:00
Alex Leahu
ac5080f273
[DOCS] Fix --update-with-info-json message (#618)
* Fix --update-with-info-json message

* Run black with --preview to fix pylint error
2023-05-22 18:43:12 -07:00
Jesse Bannon
ec447cfbd7
[FEATURE] exclude field in regex plugin (#620)
* [FEATURE] `exclude` field in regex plugin

* tests

* excludes doc
2023-05-22 10:23:00 -07:00
Jesse Bannon
2f1756dea7
[FEATURE] More tv show collection seasons (#615) 2023-05-19 09:05:51 -07:00
Jesse Bannon
0497f2b749
[FEATURE] Perform match-filter on download, not metadata pull (#612)
* [FEATURE] Apply match-filter on download instead of metadata

* correct ytdl_option ordering

* lint

* test that downloads
2023-05-18 09:52:24 -07:00
Jesse Bannon
38e64a5bd5
[FEATURE] Perform regex at both metadata and post-download (#611)
* [FEATURE] Perform regex at both metadata and post-download

* sadf
2023-05-17 22:33:42 -07:00
Jesse Bannon
a2f1a2ba94
[FEATURE] Add subscription_name override variable by default (#599)
* [FEATURE] Add `subscription_name` override variable by default

* docs for it

* lint

* only test non-cli video test
2023-04-28 19:36:20 -07:00
Jesse Bannon
d5d9978cdb
[BUGFIX] Fix splitting chapters extracted from comments (#593) 2023-04-19 19:27:27 -07:00
Jesse Bannon
e1242eaa61
[BUGFIX] Fix parsing chapters from comments (#592) 2023-04-19 18:44:25 -07:00
Jesse Bannon
1588725bb1
[BUGFIX] Fix Windows split by chapters not reading chapter titles correctly (#591) 2023-04-19 18:32:13 -07:00
catduckgnaf
9a5ee716cf
[BACKEND] Fix Plex MP4 (#586)
* Fix Plex MP4

No WebM, Best quality, No AV1 either

* Fixed because I am idiot

Forcing Plex to download only MP4 and 1080P. More efficient then downloading Webm and converting. Plex does not direct Play AV1 currently anyway.

* Plex Update

Default will get best video in H265 or H264, if neither are available, it will get next best video. All MP4, no limit on resolution.

Plex_tv_show_av1 will allow for absolute best possible resolution in any codec as long as its MP4, including Av1 which is not currently supported in Plex.

Without AV1 is considered default, for maximum compatibility at this time.

* EBUATK

Incorrect setting on Av1, fixed.

* Removed Av1 Preset

---------

Co-authored-by: catduckgnaf <128985786+cellardoor452@users.noreply.github.com>
2023-04-12 09:30:59 -07:00
Benjamin Hsieh
14c7865165
[FEATURE] channel_id source variable (#575) 2023-03-31 21:25:13 -07:00
Benjamin Hsieh
e1f2e37858
[FEATURE] creator_sanitized source variable (#574) 2023-03-31 20:47:44 -07:00
Jesse Bannon
fa6eae7423
[FEATURE] creator source variable (#572) 2023-03-31 15:37:06 -07:00
Jesse Bannon
1965bc0a40
[FEATURE] Multiple URLs supported for tv_show_by_date preset (#568)
* [BACKEND] Support empty urls for multi_url, add overrides for extra urls in tv_show_by_date preset

* need to check for missing urls in downloader

* use override in test

* docs about multi url

* wording
2023-03-30 09:28:32 -07:00
Jesse Bannon
c8de12833e
[BUGFIX] Read parent preset download strategy when partial validating (#566) 2023-03-28 20:35:07 -07:00
Jesse Bannon
4b9709aeab
[FEATURE] Configurable file_name_max_bytes (#562)
* config for max file size, need to move to defaults file

* wroking
2023-03-25 20:17:38 -07:00
Jesse Bannon
84cd611f89
[BUGFIX] File name too long with -thumb.jpg, upgrade docker alpine image (#560) 2023-03-24 11:15:54 -07:00
Jesse Bannon
70d81bf20f
[BACKEND] Mark entry as deleted if all entries deleted in update (#559)
* [BACKEND] Mark entry as deleted if all entries deleted in update

* make subtitles not found debug
2023-03-21 14:21:02 -07:00
Jesse Bannon
54b8b76034
[BACKEND] Handle case when collection url is missing from entry (#558)
* [BACKEND] Handle case when collection url is missing from entry

* hide experimental docs
2023-03-21 13:39:41 -07:00
Jesse Bannon
7385aac1ee
[FEATURE][EXPERIMENTAL] --update-with-info-json (#557)
* [FEATURE
] Reformat existing downloads

* working?!?!

* reformat in output dir

* close, need info json enabled by default

* asdfadsfsaf

* lint

* closer

* multi created variables do not exist for update command!

* info json downloader

* [REFACTOR] Shared plugin and download options class

* more refactoring

* lint

* put in initialize plugins

* changes

* [REFACTOR] simplify enhanced download archive

* working!

* in main

* simplified, many url tests working

* ready?
2023-03-21 08:05:08 -07:00
Jesse Bannon
baebd9cba7
[BUGFIX] Add timeout for bad ffmpeg image conversions (#554)
* [BUGFIX] Add timeout for bad ffmpeg image conversions

* add -y

* lint
2023-03-16 23:45:11 -07:00
Jesse Bannon
ecdae163f6
[FEATURE] tv_show_by_date preset to support multiple URLs (#552) 2023-03-16 20:25:10 -07:00
Jesse Bannon
ac2bb4c018
[REFACTOR] simplify enhanced download archive (#551) 2023-03-15 23:57:12 -07:00
Jesse Bannon
14ddd750d9
[REFACTOR] Shared OptionsValidator, downloader plugins (#550) 2023-03-15 23:06:55 -07:00
Jesse Bannon
52fdc9a213
[DOCKER] Add ubuntu docker image for device passthrough (#545) 2023-03-15 14:10:32 -07:00
Jesse Bannon
7c692cda6e
[REFACTOR] BaseDownloader (#542)
* [REFACTOR] BaseDownloader

* doh

* fix mock
2023-03-14 23:42:50 -07:00
Jesse Bannon
f4807aa9a4
[BUGFIX] Do not load YAML unless dict is returned (#541) 2023-03-14 23:06:51 -07:00
Jesse Bannon
e69933d525
[REFACTOR] Split metadata and download into two separate steps (#540) 2023-03-13 22:41:09 -07:00
Jesse Bannon
1922296c31
[REFACTOR] BaseDownloader class (#539) 2023-03-13 00:42:32 -07:00
Jesse Bannon
c546ce89df
[DOCS] Empty value docs for thumbnail + info json (#535) 2023-03-12 20:50:12 -07:00
Jesse Bannon
7502a76b99
[BACKEND] Enable info.json by default for TV show presets (#534) 2023-03-12 18:31:29 -07:00
Jesse Bannon
45244889ae
[FEATURE] chunk_initial_download preset (#532) 2023-03-10 19:18:37 -08:00
Jesse Bannon
f42973cf2f
[FEATURE] download_reverse in URL download strategy (#531)
* [FEATURE] download_reverse in URL download strategy

* doc fix
2023-03-10 16:13:02 -08:00
Jesse Bannon
0cd8df6e11
[BUGFIX] Fix extractor IDs in yt-dlp download archive (#528) 2023-03-10 12:48:53 -08:00
Jesse Bannon
11f00abaf6
[FEATURE] best_video_quality prebuilt preset (#526)
* [FEATURE] best_video_quality prebuilt preset

* common
2023-03-10 11:48:14 -08:00
Jesse Bannon
2d2fff9a40
[FEATURE] Prebuilt presets to include title in their filename (#525) 2023-03-10 10:30:35 -08:00
Jesse Bannon
f9318b9d07
[BUGFIX] Handle truncating file names if too long (#524)
* [BUGFIX] Handle truncating file names if too long

* always import os

* test fix

* fix for windows

* lint
2023-03-10 07:20:43 -08:00
Jesse Bannon
d46829b4ab
[BUGFIX] Properly align summary numbers (#517) 2023-03-08 18:35:03 -08:00
Jesse Bannon
ce877a32f9
[BUGFIX] Fix quiet log level not showing warning logs (#516) 2023-03-08 16:30:42 -08:00
Jesse Bannon
91e9fe7602
[FEATURE] Add summary to console output (#515) 2023-03-08 15:34:58 -08:00
Jesse Bannon
fddf3d3088
[FEATURE] Ability to suppress or write transaction log to file (#514) 2023-03-08 13:32:07 -08:00
Jesse Bannon
fd1934b821
[BUGFIX] Create persist log directory if it does not exist (#513) 2023-03-08 10:12:52 -08:00
Jesse Bannon
648027204f
[FEATURE] Persisted subscription logs (#512) 2023-03-08 00:17:11 -08:00
Jesse Bannon
e6a9b3ca0b
[BUILD] New yt-dlp version (#503) 2023-03-03 17:21:51 -08:00
Jesse Bannon
3c6fd0b4e8
[BUGFIX] Fix global options not working after sub (#500)
* [BACKEND] Error if arg after positional arg

* working

* dry-run default

* remove unused code
2023-03-03 11:07:11 -08:00
Jesse Bannon
7d7777c119
[REFACTOR] init ffmpeg paths in config initialize function (#501)
* [REFACTOR] init ffmpeg paths in config initialize function

* init within init
2023-03-03 10:56:18 -08:00
Jesse Bannon
7b7a4f8ab1
[BACKEND] Add --version arg (#499) 2023-03-02 19:32:11 -08:00
Jesse Bannon
896a9e2073
[BACKEND] Reset debug log on every subscription (#497) 2023-03-02 18:57:13 -08:00
Jesse Bannon
628c0f1f43
[BACKEND] Add more collection seasons (#498) 2023-03-02 18:33:25 -08:00
Jesse Bannon
5d40d02eb3
[BACKEND] Make break message debug level and more clear (#496) 2023-03-02 13:01:01 -08:00
Jesse Bannon
6146762571
[DOCS] readthedocs makeover (#493) 2023-03-01 23:45:16 -08:00
Jesse Bannon
265dae105d
[DOCS] More verbose error when ffmpeg/ffprobe not found (#491) 2023-03-01 23:40:39 -08:00
Jesse Bannon
7cfd14ed71
[DOCS] Add ffmpeg path docs (#487) 2023-03-01 09:41:34 -08:00
Jesse Bannon
798bbbc62c
[BACKEND] Windows support (#476) 2023-02-28 17:07:03 -08:00
Jesse Bannon
a2808fa7fe
[BACKEND] Properly sanitize recursive variables (#477)
* [BACKEND] Properly sanitize recursive variables

* lint

* update tests
2023-02-25 09:35:41 -08:00
Jesse Bannon
c153185b33
[DOCKER] Relax ffmpeg version requirement (#465) 2023-02-22 09:53:24 -08:00
Jesse Bannon
1423142d07
[RELEASE] Fix commit count issue in release post (#460) 2023-02-20 19:46:51 -08:00
Jesse Bannon
c39393e69a
[RELEASE] Package ytdl-sub to PyPI (#457)
* post only if > 0

* since todays date

* pypi workflow

* test token

* fix test

* long description

* simpler names

* more simple

* no test pypi in master

* publish only on master
2023-02-20 16:39:33 -08:00
Jesse Bannon
2231bd43c8
[BACKEND] Set __version__ as date, dynamically update it in make (#456)
* [BACKEND] Set python __version__ as date, dynamically update it in make

* set version dynamically

* keep empty

* git ignore __init__

* keep init

* keep

* fix unit test
2023-02-20 14:18:30 -08:00
Jesse Bannon
251cd541ec
[BACKEND] File lock directory config parameter (#455)
* [BACKEND] File lock directory config parameter

* [BACKEND] File lock directory config parameter
2023-02-20 14:05:19 -08:00
Jesse Bannon
741e4825a9
[DOCS] Automatic sanitized override variable docs (#453)
* [DOCS] Automatic sanitized override variable docs

* formatting
2023-02-20 11:01:43 -08:00
Jesse Bannon
9b90c367a5
[DOCS] Better error message for strict override variable fields (#452)
* [DOCS] Better error message for strict override variable fields

* fix unit test
2023-02-20 10:57:40 -08:00
Jesse Bannon
4ba41c757c
[BACKEND] More verbose logging when breaking (#450)
* [BACKEND] More verbose logging when breaking

* make thumbnail id a debugger log

* mock for info log

* info
2023-02-20 10:50:04 -08:00
Jesse Bannon
17e5ca3904
[DOCS] Lowest date_range granularity is day (#451) 2023-02-20 10:40:28 -08:00
Jesse Bannon
73094e79bb
[FEATURE] Plex sanitized title and uid. Add into prebuilt preset (#449)
* [FEATURE] Plex sanitized title and uid. Add into prebuilt preset

* update fixtures, add fixture test

* lint
2023-02-20 10:19:44 -08:00
Joshua M. Boniface
69df95e0e9
[BACKEND] Always perform copy/delete on OSError (#446)
* Always perform copy/delete on OSError

Extends the fix in #400 to handle all potential OSError errno values and
attempt a copy/delete in all cases. For example, a combination
cross-device move and a permissions change would throw errno 1 instead
of errno 18, which wasn't caught here. Instead of messing with
individual errno numbers, just always attempt a copy/delete and let
those throw their own exceptions if they don't work either.

* Remove unused os_error_exc variable
2023-02-19 21:06:48 -08:00
Jesse Bannon
3106861a42
[BACKEND] Additional filtering on root parent search (#445)
* [BACKEND] Additional filtering on root parent search

* update failing test
2023-02-17 23:32:02 -08:00
Jesse Bannon
e5b5c5e638
[BACKEND] Bandcamp e2e test, update music example (#393)
* [BUGFIX] Fix missing contains function when checking entries

* remove children func

* [BACKEND] Bandcamp e2e test, update music example

* WIP

* combine soundcloud and all music into music_audio config

* sc fixtures

* updated tests and examples

* bandcamp e2e test

* regen off

* remove url validators

* use bandcamp artist url for more parent coverage

* lint
2023-01-14 16:48:27 -08:00
Jesse Bannon
c828ca5978
[BUGFIX] Fix TV show collection season thumbnail path for Plex (#419) 2023-01-14 16:46:40 -08:00
Jesse Bannon
32f1398d60
[BUGFIX] Fix playlist season thumbnails not appearing (#416) 2023-01-13 14:05:56 -08:00
Jesse Bannon
ab52ebbfe5
[DOCS] Extractor args example for english metadata (#415) 2023-01-13 13:59:24 -08:00
Jesse Bannon
30c77565bc
[BACKEND] Add playlist_uploader_sanitized var (#414) 2023-01-13 10:47:43 -08:00
Jesse Bannon
6020f27f12
[BACKEND] Better error message for music_tags recieving a video (#413)
* [BACKEND] Better error message for music_tags recieving a video

* test and better message
2023-01-13 10:25:17 -08:00
Jesse Bannon
c95c0f6aa4
[DOCS] Document additional backslash for regex (#412)
* [DOCS] Document additional backslash for regex

* double-backslash
2023-01-11 22:51:01 -08:00
Jesse Bannon
d9584dca63
[DOCS] Add popular ytdl_options to docs (#411) 2023-01-11 22:14:29 -08:00
Jesse Bannon
053042b4b9
[BACKEND] Purge youtube + soundcloud download strategies (#408)
* [BACKEND] Purge youtube + soundcloud download strategies

* update unit tests

* fix missing for ffmpeg;

* try update
2023-01-11 22:05:50 -08:00
Jesse Bannon
c84496910f
[BUGFIX] Cross-device patch for ffmpeg file writes (#407) 2023-01-10 14:19:45 -08:00
Jesse Bannon
c3d9d45c18
[BUGFIX] Invalid cross-device link workaround (#400) 2023-01-04 15:34:26 -08:00
Jesse Bannon
beaa34a367
[BUGFIX] Fix entry missing __contains__ function (#392)
* [BUGFIX] Fix missing contains function when checking entries

* remove children func

* Update failing hash
2022-12-13 00:23:16 -08:00
Jesse Bannon
c47b504cb4
[BUGFIX] Clear working directory before starting any downloads (#382)
* [BUGFIX] Clear working directory before starting any downloads

* update fixture

* fix prebuilt tests

* check if dir exists before del in fixture
2022-11-30 15:34:17 -08:00
Jesse Bannon
c8c66b5660
[BUGFIX] Create dir before converting channel thumbnails (#381) 2022-11-30 15:06:56 -08:00
Jesse Bannon
96e741c71d
[BACKEND] Proactively clean working directory (#379)
* fix playlist last thumbnail move to copy

* playlist working

* channel working

* working directory getting cleaned

* working dir e2e fixture

* cleanup files in any case

* clean up if pre-split returns None
2022-11-30 10:36:13 -08:00