Commit graph

112 commits

Author SHA1 Message Date
Jesse Bannon
fc2da4d525
[FEATURE] square_thumbnail plugin (#1244)
Adds the ability to make thumbnails square, both file and embedded. Usage:

```
my_preset:
  square_thumbnail: True
```

Will soon enable this by default for all music-based presets with a toggle to disable.
Huge thanks to @Kentaro1043 for crafting the ffmpeg command for this!

Closes https://github.com/jmbannon/ytdl-sub/issues/383
2025-06-18 22:37:06 -07:00
Jesse Bannon
a68d5e4539
[FEATURE] Emby prebuilt presets (#1232)
Gives Emby first class support by creating prebuilt presets for each variant, including
- `Emby TV Show by Date`
- `Emby TV Show Collection`
- `Emby Music Videos`

Emby is identical to Jellyfin except in TV Show Collection, where it requires a `season.nfo`. A new plugin `StaticNfoTags` has been created to create those.

Huge thanks to @Kamaroth92 for the initial code for this feature!
2025-06-01 09:56:48 -07:00
Jesse Bannon
0bd97dbe96
[FEATURE] release_date support for date_range (#1231)
Adds the ability to toggle whether to use `upload_date` or `release_date` in the date_range plugin, which is used to cull older videos.

```
date_range:
  type: "upload_date"
```

If using the Only Recent preset, it will be automatically applied based on whether your season/episode ordering uses release date or upload date.

Closes https://github.com/jmbannon/ytdl-sub/issues/1182
2025-06-01 09:28:42 -07:00
Jesse Bannon
bc15fe09bd
[FEATURE] Throttle protection per request (#1229)
Allows adding a sleep inbetween every request during metadata scrape.

```
throttle_protection:
  sleep_per_request_s:
    min: 4
    max: 6
```

Closes https://github.com/jmbannon/ytdl-sub/issues/1222
2025-05-31 09:41:14 -07:00
Jesse Bannon
dfcbadb69d
[BACKEND] Make kodi_safe nfo overrideable (#1216)
For an upcoming change to make preset selection easier
2025-05-03 21:46:31 -07:00
Jesse Bannon
01e1f46b86
[DEV] Remaining throttle unit tests (#1075) 2024-09-30 22:04:32 -07:00
Jesse Bannon
f2ec26d77e
[BUGFIX] Fix variable collision when disabling plugins (#1072)
Fixes https://github.com/jmbannon/ytdl-sub/issues/1071
2024-09-29 22:38:26 -07:00
Jesse Bannon
1b57d79c8f
[BREAKING CHANGE] Remove regex plugin (#1067)
Regex plugin has been removed in favor of scripting. The function [regex_capture_many](https://ytdl-sub.readthedocs.io/en/latest/config_reference/scripting/scripting_functions.html#regex-capture-many) has been created to replicate the plugin's behavior. See the following converted example:

`regex, now deprecated`
```
    regex:
      from:
        title:
          match:
            - ".*? - (.*)"  # Captures 'Song' from 'Emily Hopkins - Some - Song'
          capture_group_names:
            - "captured_track_title"
          capture_group_defaults:
            - "{title}"
    overrides:
      track_title: "{captured_track_title}"
```

`scripting equivalent`
```
    overrides:
      # Captures 'Song' from 'Emily Hopkins - Some - Song'
      captured_track_title: >-
        {
          %regex_capture_many(
            title,
            [ ".*? - (.*)" ],
            [ title ]
          )
        }
      track_title: "{%array_at(captured_track_title, 1)}"
```

## Motivation:
Regex was a unique plugin that added custom variables based on user input. This made the backend need to support both `overrides` and 'plugin user variables'. Removing `regex` plugin will consolidate this logic into only `overrides`, making it much easier to maintain.
2024-09-29 08:41:09 -07:00
Jesse Bannon
79ba60021c
[BACKEND] Proper functions with docstrings (#1061)
Reimplement some of the 'hidden' functions, including `%regex_capture_many`, as standard functions with docstrings
2024-09-25 23:24:49 -07:00
Jesse Bannon
0ed8aec83d
[BACKEND] Show ffmpeg conversion errors (#1016) 2024-07-04 10:03:34 -07:00
dependabot[bot]
ec58a80660
[DEV] Bump black from 22.3.0 to 24.4.2 (#973)
* Bump black from 22.3.0 to 24.4.2

Bumps [black](https://github.com/psf/black) from 22.3.0 to 24.4.2.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/22.3.0...24.4.2)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* run linter

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jesse Bannon <jbann1994@gmail.com>
2024-04-27 15:46:47 -07:00
Jesse Bannon
0beeeb464d
[FEATURE] Music tag dates (#938)
Adds proper support for setting `date` and `original_date` fields via `music_tag` plugin using a standardized date format.
2024-02-28 13:48:42 -08:00
Jesse Bannon
248e9a15a6
[DOCS] Various doc cleanups (#928) 2024-02-17 08:53:06 -08:00
Jesse Bannon
7c217db843
[BUGFIX] Fix usage of chapter_title_sanitized (#924)
Usage of `chapter_title_sanitized` would sometimes result in an error. This should hopefully fix it
2024-02-12 16:05:50 -08:00
Jesse Bannon
f0b3991e3e
[FEATURE] Toggle date_range to break or not (#904)
Adds a the new field `breaking` to the `date_range` plugin, to toggle whether an entry breaks subsequent metadata pulls. This is useful to disable if you are grabbing a playlist that may have videos out of order, but still want to apply a date range to it.
2024-01-18 16:23:02 -08:00
Jesse Bannon
d2e733f2e7
[BACKEND] Only download audio when using audio_extract plugin (#902)
This addition optimizes the `audio_extract` plugin by preemptively telling yt-dlp to discard the video and only download the audio stream
2024-01-18 15:00:17 -08:00
Jesse Bannon
c0ca3c3945
[BACKEND] Remove old format for video_tags, music_tags, download_strategy (#899)
Now that sufficient time has passed, we can formally deprecate the following:
- https://ytdl-sub.readthedocs.io/en/latest/deprecation_notices.html#video-tags
- https://ytdl-sub.readthedocs.io/en/latest/deprecation_notices.html#music-tags
- usage of `download_strategy` (simply remove it if you're getting an error about it)
2024-01-15 00:04:32 -08:00
Jesse Bannon
e3158583ca
[FEATURE] Toggleable plugin field enable for all dict-based plugins (#897)
For key/value-based plugins, add an `enable` field to make it easier for child presets to disable their functionality
2024-01-11 00:40:07 -08:00
Jesse Bannon
5e384ba96e
[BACKEND] Info log for filter_include and filter_exclude plugins (#873)
Adds more verbose logging when entries get filtered using the `filter_include` and `filter_exclude` plugins
2024-01-05 14:44:16 -08:00
Jesse Bannon
2dbe426ced
[BUGFIX] Properly handle UIDs that have misbehaving chars that get sanitized (#870)
Should hopefuly fix both:
- https://github.com/jmbannon/ytdl-sub/issues/843
- https://github.com/jmbannon/ytdl-sub/issues/867

Various sites have unique IDs with characters that get sanitized (i.e. `/`, `:`). `ytdl-sub` was not using sanitized UIDs when looking for files, and would throw an error saying download was missing. This should resolve that issue.
2024-01-04 15:01:17 -08:00
Jesse Bannon
731d4e444c
[DOCS] Tooling to auto-generate custom sphinx docs (#849)
Tooling + updating docs
2023-12-28 22:39:59 -08:00
Jesse Bannon
308c76a8f2
[FEATURE] Filter plugins, regex helper script functions (#848)
Adds generic `filter_include` and `filter_exclude` plugins in anticipation for function script usage. Will detail it more with official docs at a later time!
2023-12-20 09:52:12 -08:00
Jesse Bannon
e92b1cd12a
[BACKEND][HUGE] Function Support in variable syntax (#838)
A complete gutting of the internals of ytdl-sub to support functions in our variable syntax, in addition to being able to access a yt-dlp entry's .info.json fields using functions. Functionally, ytdl-sub should still look and behave the same from a user-perspective.

With so many lines of code changed (+8927, -2708), no doubt there will be new issues. Please make a GH issue or reach out on Discord if your config/subscriptions break in any way/shape/form.

Details on how to use function support will come soon in the form of proper documentation in our readthedocs.
2023-12-18 16:08:15 -08:00
OmarAhmadyar
8998861b49
[BUGFIX] file_convert output file deleted (#822)
file_convert.py:modify_entry() deletes the converted output file if the output and input file names are identical.  Reordered the file deletion to avoid.

Thanks @OmarAhmadyar !
2023-11-25 00:23:46 -08:00
Jesse Bannon
395691f2b9
[FEATURE] Create output_directory_nfo_tags on first entry instead of last (#809) 2023-11-05 23:10:29 -08:00
Jesse Bannon
f1a2187a17
[FEATURE] Add throttle_protection plugin (#799)
Provides options to make ytdl-sub look more 'human-like' to protect from throttling. For range-based values, a random number will be chosen within the range to avoid sleeps looking scripted.

Usage:
```
    throttle_protection:
      sleep_per_download_s:
        min: 2.2
        max: 10.8
      sleep_per_subscription_s:
        min: 9.0
        max: 14.1
      max_downloads_per_subscription:
        min: 10
        max: 36
      subscription_download_probability: 1.0
```
2023-11-05 08:23:12 -08:00
Jesse Bannon
056b111be6
[FEATURE] Prebuilt Music Videos (#787)
Creates standardized prebuilt presets for music videos. Preset names are:

```
"Kodi Music Videos"
"Jellyfin Music Videos"
"Plex Music Videos"
```

Usage can be found in the `examples/` directory
2023-10-28 00:12:22 -07:00
Jesse Bannon
d810291e7c
[BUGFIX] Write music multi-tags correctly (#782)
Music multi-tags (artists, albumartists, etc) were not writing correctly. Fix it by explicitly checking to see if the tag is multi or not in the codebase.
2023-10-25 10:37:24 -07:00
Jesse Bannon
c119e6f644
[BUGFIX] Fix match-filter edge case and date_range.before (#750)
Fixes yet another bug that is match-filter related. The `date_range.before` value would get ignored if you specified any other match-filters. That is fixed by making them `&` behind-the-scenes.
2023-10-03 11:46:42 -07:00
Jesse Bannon
6615e1d1ac
[BACKEND] Always filter out live, post_live, and upcoming videos on download (#742)
Closes Issue: https://github.com/jmbannon/ytdl-sub/issues/729

Add a new field to the match_filter plugin: `download_match_filters`. This set of filters will be applied at the download stage (as opposed to `filters` which get applied at the metadata stage). By default, always set `"!is_live & !is_upcoming & !post_live"` as a download filter. This prevents ytdl-sub from hanging when trying to grab live and upcoming videos.

Docs on match-filters: https://ytdl-sub.readthedocs.io/en/latest/config.html#match-filters
2023-09-24 06:24:21 -07:00
Jesse Bannon
3c87dc9f8e
[BUGFIX] Fix date_range plugin as a match-filter (#726)
Bugfix to the prior release. Convert yt-dlp date formats (i.e. `today-2months`) into a date string with format `YYYYMMDD`. This will make it compatible with match-filter syntax
2023-09-20 08:30:29 -07:00
Jesse Bannon
1a118e225a
[BACKEND] Perform match-filters at metadata stage, use match-filters for date_range plugin (#725)
Solved Issue: https://github.com/jmbannon/ytdl-sub/issues/706

From our discussion with yt-dlp folks (https://github.com/yt-dlp/yt-dlp/issues/8108), we learned that best practices for date--range filtering is done using match-filters. We now set the `date_range` plugin values to be a 'breaking' match filter, and any custom ones specified from users in the `match_filters` plugin as non-breaking. This means match-filters can be performed at the metadata stage, which should yield a significant speedup

Relevant docs:
- date_range: https://ytdl-sub.readthedocs.io/en/latest/config.html#date-range
- match_filters: https://ytdl-sub.readthedocs.io/en/latest/config.html#match-filters
2023-09-19 11:19:15 -07:00
Jesse Bannon
0b62cc13d0
[DOCS] Mention YAML single vs double quotes in regex docs (#719)
Improve regex plugin docs by mentioning difference between using single-quotes and double-quotes in YAML: https://ytdl-sub.readthedocs.io/en/latest/config.html#regex
2023-09-15 13:54:08 -07:00
Jesse Bannon
a48efdc84c
[FEATURE] format plugin to set yt-dlp format easier (#714)
Makes setting yt-dlp's `format` field easier by giving it its own documented plugin.

Old:
```
my_format_preset:
  ytdl_options:
    format: "best"
```

New:
```
my_format_preset:
  format: "best"
```

The old method will still work, so do not worry about updating configs ASAP. However, the option is available to save a few lines 😉
2023-09-13 00:02:19 -07:00
Jesse Bannon
792da1ba86
[BACKEND] Set merge_output_format to file_convert file type (#699)
Adds a small optimization to potentially avoid an additional remux by setting yt-dlp's merge output format to the desired format in the `file_convert` plugin (https://ytdl-sub.readthedocs.io/en/latest/config.html#file-convert)
2023-09-02 11:18:04 -07:00
Jesse Bannon
da70c725fc
[FEATURE] Support best codec type in audio_extract (#700)
Adds support to the `audio_extract` plugin (https://ytdl-sub.readthedocs.io/en/latest/config.html#audio-extract) to the following to grab the best audio format available:

```
audio_extract:
  codec: "best"
```
2023-09-02 00:50:26 -07:00
Jesse Bannon
5f9b8afc4b
[BUGFIX] Do not hard-fail if the thumbnail fails to download/convert (#675)
* [BUGFIX] Do not fail with thumbnail issues

* helper

* refactor

* dry-run

* regen fixture

* download arhcive txt -> json in docs, fixtures

* return if no download thumb path

* better test

* more test coverage
2023-07-28 13:12:46 -07:00
Jesse Bannon
e3c662bc35
[REFACTOR] Shared plugin code (#664)
* [REFACTOR] Shared plugin code

* arg names

* all plugins have priority

* split abstract method

* lint

* lint
2023-07-25 12:11:50 -07:00
Jesse Bannon
dcd636192c
[REFACTOR] download abstraction overhaul (#663)
* [REFACTOR] `download` abstraction overhaul

* remove plugin options
2023-07-25 00:36:51 -07:00
Jesse Bannon
bb03c91656
[BACKEND] Links to deprecation (#662) 2023-07-25 00:10:54 -07:00
Jesse Bannon
77487aee86
[FEATURE] Simplify `video_tags` (#661)
* [FEATURE] Simplify ``video_tags``

* video tags old

* change for episode

* deprecation notice
2023-07-24 18:18:53 -07:00
Jesse Bannon
cc38db9c7a
[FEATURE] Simplify `music_tags` plugin (#660)
* [FEATURE] Simplify ``music_tags`` plugin

* docs

* more compatible

* check old format better

* better dict instantiate
2023-07-24 18:05:54 -07:00
Jesse Bannon
0cbd2af4d3
[FEATURE] embed_thumbnail plugin (#659)
* [REFACTOR] OptionsValidator + OptionsDictValidator

* more

* [FEATURE] Dedicated embed_thumbnail plugin

* tests

* fix test

* docs

* deprecation notices

* priority to run after file convert

* naming

* fixtures
2023-07-24 12:07:36 -07:00
Jesse Bannon
7e997f304d
[REFACTOR] OptionsValidator + OptionsDictValidator (#657)
* [REFACTOR] OptionsValidator + OptionsDictValidator

* more
2023-07-24 10:12:38 -07:00
Jesse Bannon
acc7bb306b
[DOCS] Clean up match-filter plugin docs (#633) 2023-06-17 13:32:12 -07:00
Jesse Bannon
a993beff0e
[FEATURE] Ability to use overrides in regex plugin (#631)
* [FEATURE] Ability to use overrides in regex plugin

* lint

* more tests

* more docs

* docs
2023-06-17 13:02:35 -07:00
Jesse Bannon
ec447cfbd7
[FEATURE] exclude field in regex plugin (#620)
* [FEATURE] `exclude` field in regex plugin

* tests

* excludes doc
2023-05-22 10:23:00 -07:00
Jesse Bannon
0497f2b749
[FEATURE] Perform match-filter on download, not metadata pull (#612)
* [FEATURE] Apply match-filter on download instead of metadata

* correct ytdl_option ordering

* lint

* test that downloads
2023-05-18 09:52:24 -07:00
Jesse Bannon
38e64a5bd5
[FEATURE] Perform regex at both metadata and post-download (#611)
* [FEATURE] Perform regex at both metadata and post-download

* sadf
2023-05-17 22:33:42 -07:00
Jesse Bannon
1588725bb1
[BUGFIX] Fix Windows split by chapters not reading chapter titles correctly (#591) 2023-04-19 18:32:13 -07:00