Commit graph

26 commits

Author SHA1 Message Date
Jesse Bannon
e3158583ca
[FEATURE] Toggleable plugin field enable for all dict-based plugins (#897)
For key/value-based plugins, add an `enable` field to make it easier for child presets to disable their functionality
2024-01-11 00:40:07 -08:00
Jesse Bannon
db5f095a51
[BACKEND] Set max_downloads to keep_max_files if present (#871)
Implements: https://github.com/jmbannon/ytdl-sub/issues/829

Optimization to not scrape more than what you will keep
2024-01-04 16:38:20 -08:00
Jesse Bannon
e92b1cd12a
[BACKEND][HUGE] Function Support in variable syntax (#838)
A complete gutting of the internals of ytdl-sub to support functions in our variable syntax, in addition to being able to access a yt-dlp entry's .info.json fields using functions. Functionally, ytdl-sub should still look and behave the same from a user-perspective.

With so many lines of code changed (+8927, -2708), no doubt there will be new issues. Please make a GH issue or reach out on Discord if your config/subscriptions break in any way/shape/form.

Details on how to use function support will come soon in the form of proper documentation in our readthedocs.
2023-12-18 16:08:15 -08:00
Jesse Bannon
544af2207b
[FEATURE] Add output_options.keep_max_files to limit number of downloads per subscription (#820)
Adds a new mechanism to limit subscriptions' number of videos to an explicit number (only supported date ranges prior to this).

Example usage:
```
output_options:
  keep_max_files: 10
```
will only keep 10 videos at max.

In addition:
- `keep_max_files` is bundled into the `Only Recent` preset and can be used via setting the `only_recent_max_files` override variable (see examples or README for usage).
- Changed the `date_range` variable name to `only_recent_date_range` for more clarity.  Usage of `date_range` will continue to work.
2023-11-17 09:01:21 -08:00
Jesse Bannon
33a8da9afc
[BACKEND] Log full subscription errors (#793) 2023-11-03 10:12:15 -07:00
Jesse Bannon
83ecd19c77
[FEATURE] Process all subscriptions even if one or more error (#771)
Closes GH Issue #766, partially closes #520

Prior to this release, no other subscriptions would download if a subscription before it had an error. Now, subscriptions will continue to download even if one has an error, and will be reported in the output summary.
2023-10-21 12:25:38 -07:00
Jesse Bannon
6615e1d1ac
[BACKEND] Always filter out live, post_live, and upcoming videos on download (#742)
Closes Issue: https://github.com/jmbannon/ytdl-sub/issues/729

Add a new field to the match_filter plugin: `download_match_filters`. This set of filters will be applied at the download stage (as opposed to `filters` which get applied at the metadata stage). By default, always set `"!is_live & !is_upcoming & !post_live"` as a download filter. This prevents ytdl-sub from hanging when trying to grab live and upcoming videos.

Docs on match-filters: https://ytdl-sub.readthedocs.io/en/latest/config.html#match-filters
2023-09-24 06:24:21 -07:00
Jesse Bannon
5f9b8afc4b
[BUGFIX] Do not hard-fail if the thumbnail fails to download/convert (#675)
* [BUGFIX] Do not fail with thumbnail issues

* helper

* refactor

* dry-run

* regen fixture

* download arhcive txt -> json in docs, fixtures

* return if no download thumb path

* better test

* more test coverage
2023-07-28 13:12:46 -07:00
Jesse Bannon
d24048bb01
[REFACTOR] No more download strategy (#669)
* [REFACTOR] No more download strategy

* remove more
2023-07-25 19:51:50 -07:00
Jesse Bannon
bf3c61213b
[REFACTOR] SourcePluginExtension (#667)
* [REFACTOR] SourcePluginExtension

* param name

* priority to plugin extensions

* info json
2023-07-25 14:04:18 -07:00
Jesse Bannon
1236af270d
[REFACTOR] SourcePlugin (#666)
* [REFACTOR] SourcePlugin

* param name
2023-07-25 12:37:56 -07:00
Jesse Bannon
e3c662bc35
[REFACTOR] Shared plugin code (#664)
* [REFACTOR] Shared plugin code

* arg names

* all plugins have priority

* split abstract method

* lint

* lint
2023-07-25 12:11:50 -07:00
Jesse Bannon
b8d1f1edbb
[FEATURE] Ability to change download archive name (#643) 2023-06-30 11:51:52 -07:00
Jesse Bannon
7385aac1ee
[FEATURE][EXPERIMENTAL] --update-with-info-json (#557)
* [FEATURE
] Reformat existing downloads

* working?!?!

* reformat in output dir

* close, need info json enabled by default

* asdfadsfsaf

* lint

* closer

* multi created variables do not exist for update command!

* info json downloader

* [REFACTOR] Shared plugin and download options class

* more refactoring

* lint

* put in initialize plugins

* changes

* [REFACTOR] simplify enhanced download archive

* working!

* in main

* simplified, many url tests working

* ready?
2023-03-21 08:05:08 -07:00
Jesse Bannon
14ddd750d9
[REFACTOR] Shared OptionsValidator, downloader plugins (#550) 2023-03-15 23:06:55 -07:00
Jesse Bannon
e69933d525
[REFACTOR] Split metadata and download into two separate steps (#540) 2023-03-13 22:41:09 -07:00
Jesse Bannon
7d7777c119
[REFACTOR] init ffmpeg paths in config initialize function (#501)
* [REFACTOR] init ffmpeg paths in config initialize function

* init within init
2023-03-03 10:56:18 -08:00
Jesse Bannon
798bbbc62c
[BACKEND] Windows support (#476) 2023-02-28 17:07:03 -08:00
Jesse Bannon
c47b504cb4
[BUGFIX] Clear working directory before starting any downloads (#382)
* [BUGFIX] Clear working directory before starting any downloads

* update fixture

* fix prebuilt tests

* check if dir exists before del in fixture
2022-11-30 15:34:17 -08:00
Jesse Bannon
96e741c71d
[BACKEND] Proactively clean working directory (#379)
* fix playlist last thumbnail move to copy

* playlist working

* channel working

* working directory getting cleaned

* working dir e2e fixture

* cleanup files in any case

* clean up if pre-split returns None
2022-11-30 10:36:13 -08:00
Jesse Bannon
5705a28842
[FEATURE] Add ytdl-sub view URL to view source variables (#340)
* [FEATURE] Add `ytdl-sub view URL` to view source variables

* update file path
2022-11-19 16:56:45 -08:00
Jesse Bannon
978dc36713
[BACKEND] Optimize metadata fetching (#335)
* [BACKEND] Do not fetch unnecessary metadata when grabbing .info.json files"

* update with more optimized ytdl options
2022-11-16 12:54:18 -08:00
Jesse Bannon
839192059b
[FEATURE] Add playlist thumbnail support (#250) 2022-09-27 23:48:18 -07:00
Jesse Bannon
3d0ade1141
[BACKEND] Collection thumbnail support (#241) 2022-09-18 11:16:25 -07:00
Jesse Bannon
5782445366
[FEATURE] Add ability for downloaders to add new override variables. Add source_title, source_uploader, and source_description for YouTube channels and playlists (#203) 2022-09-01 00:25:01 -07:00
Jesse Bannon
ca0ac787b0
[FEATURE] Ability to write .info.json files (#201) 2022-08-29 22:50:42 -07:00