danvics/ytdl-sub
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
684 commits 10 branches 373 tags 34 MiB
Commit graph

684 commits

This branch
This branch
All branches
Author SHA1 Message Date
Jesse Bannon
c69c18f623 ubuntu working, headless next 2024-06-19 10:48:46 -07:00
Jesse Bannon
c2383614f3 default cron script 2024-06-19 00:44:51 -07:00
Jesse Bannon
94db90f113 [FEATURE] Orchestrate cron using Docker Env variables" 2024-06-19 00:00:04 -07:00
dependabot[bot]
f7305ebf8a
[DEV] Bump pylint from 3.2.2 to 3.2.3 (#1001) 
Bumps [pylint](https://github.com/pylint-dev/pylint) from 3.2.2 to 3.2.3.
- [Release notes](https://github.com/pylint-dev/pylint/releases)
- [Commits](https://github.com/pylint-dev/pylint/compare/v3.2.2...v3.2.3)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-06-06 15:30:36 -07:00
Jesse Bannon
5e9472986a
[FEATURE] Enhance music video presets using specialized map syntax (#808) 
Enhances the music video presets by supporting the following syntax:

`subscriptions.yaml`
```
__preset__:
  overrides:
    music_video_directory: "/music_videos"

# Choose between Jellyfin/Kodi/Plex Music Videos preset:
#   - Plex Music Videos:
#   - Jellyfin Music Videos:
#   - Kodi Music Videos:
#
"Plex Music Videos":

  = Pop:  # Sets genre tag to "Pop"
    "Rick Astley": "https://www.youtube.com/playlist?list=PLlaN88a7y2_plecYoJxvRFTLHVbIVAOoc"
    "Michael Jackson": "https://www.youtube.com/playlist?list=OLAK5uy_mnY03zP6abNWH929q2XhGzWD_2uKJ_n8E"

  = Rock:
    # Prefixing with '+' puts the subscription into 'map-mode'.
    # Music video presets in map-mode support grouping videos into different
    # categories, which get set on the album field.
    #
    # URLs can either be strings, or maps that can overload title, year, date
    "+ Guns N' Roses":
      Music Videos:
        - "https://www.youtube.com/playlist?list=PLOTK54q5K4INNXaHKtmXYr6J7CajWjqeJ"
      Concerts:
        - title: "Live at The Ritz - New York City"
          year: "1988"
          url: "https://www.youtube.com/watch?v=OldpIhHPsbs"
        - title: "Live at The Hollywood Bowl"
          date: "2023-01-11"
          url: "https://www.youtube.com/watch?v=Z7hutGlvq9I"
```
The 'map-mode' (denoted by `+`) lets you specify a map of music video categories. Under each category, a URL can be specified as-is or with additional metadata (title, year/date, url).

Music videos and concerts, especially older ones, are typically uploaded by random users with inconsistencies in their titles. This syntax aims to make it easy to specify individual URLs with the ability to overwrite their title and year, and group them by category.
 2024-06-06 11:50:33 -07:00
Jesse Bannon
5866c12104
[BUGFIX] Do not run all branches of if functions in scripts (#999) 
Will only run branch script code of `if` statements if the condition evaluates to that branch
 2024-06-03 15:02:59 -07:00
Jesse Bannon
5e335b195c
[FEATURE] Allow YAML maps and lists in overrides, convert to script format (#956) 
Adds the ability to create map and list-based override variables. 

For example, you can now create lists like this:
```
overrides:
  urls:
    - "https://...1"
    - "https://...2"
```
which is equivalent to:
```
overrides:
  urls: >-
    {
      [
        "https://...1",
        "https://...2",
      ]
    }
```

Likewise, maps can now look like:
```
overrides:
  music_video_category:
    concerts:
      - "https://...1"
      - "https://...2"
    interviews:
      - "https://...3"
```
which is equivalent to:
```
overrides:
  music_video_category: >-
    {
      "concerts": [
        "https://...1",
        "https://...2"
      ],
      "interviews": [
        "https://...3"
      ]
    }
```
 2024-06-02 20:02:48 -07:00
Jesse Bannon
6e31ad3600
[BACKEND] Include .info.json files for Music video presets (#996) 
Music video presets now include .info.json files by default
 2024-06-02 10:57:32 -07:00
Jesse Bannon
30a2ad7a63
[BUGFIX] Prevent corrupt writes to download archive (#983) 
Attempts to make writes to the download archive safer (https://github.com/jmbannon/ytdl-sub/issues/982).

`ytdl-sub` will now *copy* the download archive from the working directory to the output directory with a temp name, then perform a *move* to store it with its final expected name. This will drastically lower the window of time where  the process could die mid-write and corrupt it on the next read.
 2024-06-02 10:53:19 -07:00
Jesse Bannon
74625c293d
[DEV] Update fixtures (#995) 
* [DEV] Update fixtures

* thumbs fixed, longer timeout

* ugh

* fanart
 2024-06-02 00:05:44 -07:00
dependabot[bot]
ed969b84d2
[DEV] Bump pylint from 3.2.0 to 3.2.2 (#991) 
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-29 17:04:26 -07:00
dependabot[bot]
87f9378d33
[BACKEND] Bump yt-dlp from 2024.04.09 to 2024.5.27 (#994) 
Bumps [yt-dlp](https://github.com/yt-dlp/yt-dlp) from 2024.04.09 to 2024.5.27.
- [Release notes](https://github.com/yt-dlp/yt-dlp/releases)
- [Changelog](https://github.com/yt-dlp/yt-dlp/blob/master/Changelog.md)
- [Commits](https://github.com/yt-dlp/yt-dlp/compare/2024.04.09...2024.05.27)

---
updated-dependencies:
- dependency-name: yt-dlp
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-28 19:48:22 -07:00
dependabot[bot]
02db0c41a5
[DEV] Bump pylint from 3.1.1 to 3.2.0 (#988) 
Bumps [pylint](https://github.com/pylint-dev/pylint) from 3.1.1 to 3.2.0.
- [Release notes](https://github.com/pylint-dev/pylint/releases)
- [Commits](https://github.com/pylint-dev/pylint/compare/v3.1.1...v3.2.0)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-14 16:17:21 -07:00
dependabot[bot]
8461f5f229
[DEV] Bump pylint from 3.1.0 to 3.1.1 (#987) 
Bumps [pylint](https://github.com/pylint-dev/pylint) from 3.1.0 to 3.1.1.
- [Release notes](https://github.com/pylint-dev/pylint/releases)
- [Commits](https://github.com/pylint-dev/pylint/compare/v3.1.0...v3.1.1)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-13 19:40:17 -07:00
Jesse Bannon
325d229061
[BUGFIX] Fix usage of ~ in paths (#981) 
Fixes path with tildes in them, i.e. `~/videos/youtube`
 2024-05-10 19:57:48 -07:00
dependabot[bot]
64d3082a8a
[DEV] Bump pylint from 2.13.5 to 3.1.0 (#972) 
* Bump pylint from 2.13.5 to 3.1.0

Bumps [pylint](https://github.com/pylint-dev/pylint) from 2.13.5 to 3.1.0.
- [Release notes](https://github.com/pylint-dev/pylint/releases)
- [Commits](https://github.com/pylint-dev/pylint/compare/v2.13.5...v3.1.0)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* bend knee to pylint

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jesse Bannon <jbann1994@gmail.com>
 2024-04-28 10:40:15 -07:00
Tomas Babej
1d176050a7
[FEATURE] %regex_sub built-in script function (#971) 
This implements %regex_sub built-in function to enhance string-processing capabilities, allowing users to perform substitutes like:

- removing non-ascii characters
- replacing subsequent whitespace characters with a single whitespace

Thanks @tbabej !
 2024-04-27 22:47:29 -07:00
dependabot[bot]
ec58a80660
[DEV] Bump black from 22.3.0 to 24.4.2 (#973) 
* Bump black from 22.3.0 to 24.4.2

Bumps [black](https://github.com/psf/black) from 22.3.0 to 24.4.2.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/22.3.0...24.4.2)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* run linter

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jesse Bannon <jbann1994@gmail.com>
 2024-04-27 15:46:47 -07:00
Jesse Bannon
caad4598fc
[DEV] Regen function doc strings on make docs (#977) 
* [DEV] Regen docs with `make docs`

* default 0
 2024-04-27 14:06:16 -07:00
dependabot[bot]
aa637d12dc
[DEV] Bump isort from 5.10.1 to 5.13.2 (#974) 
Bumps [isort](https://github.com/pycqa/isort) from 5.10.1 to 5.13.2.
- [Release notes](https://github.com/pycqa/isort/releases)
- [Changelog](https://github.com/PyCQA/isort/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pycqa/isort/compare/5.10.1...5.13.2)

---
updated-dependencies:
- dependency-name: isort
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-04-27 13:37:20 -07:00
dependabot[bot]
10387c925d
[DEV] Update coverage[toml] requirement from ~=6.3 to >=6.3,<8.0 (#975) 
Updates the requirements on [coverage[toml]](https://github.com/nedbat/coveragepy) to permit the latest version.
- [Release notes](https://github.com/nedbat/coveragepy/releases)
- [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst)
- [Commits](https://github.com/nedbat/coveragepy/compare/6.3...7.5.0)

---
updated-dependencies:
- dependency-name: coverage[toml]
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-04-27 10:32:38 -07:00
dependabot[bot]
f3a979c818
[DEV] Update pytest requirement from ~=7.2 to >=7.2,<9.0 (#976) 
Updates the requirements on [pytest](https://github.com/pytest-dev/pytest) to permit the latest version.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.2.0...8.1.2)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-04-27 10:03:14 -07:00
Jesse Bannon
5b34df4588
[DEV] Update fixtures (#970) 
* [DEV] Update fixtures

* update yt-dlp

* fanart
 2024-04-27 09:20:30 -07:00
Nathaniel Barragan
232c0acdfa
[BACKEND] Update yt-dlp, move to pyproject.toml (#965) 
- Updates yt-dlp to 2024.4.9
- Moves setup.cfg to pyproject.toml
- Updates other various backend packages

Thanks @Noodlez1232 !
 2024-04-26 00:09:02 -07:00
Noah Kiss
156c788689
[DOCS] Add Plex Agent Sources Information (#969) 
Co-authored-by: Noah Kiss <noah@nkmk.co>
 2024-04-25 23:12:08 -07:00
Jesse Bannon
ad72e1a6de
[BACKEND] Debug log exception on retry (#957) 
Help diagnose retry errors better
 2024-04-02 17:27:21 -07:00
Jesse Bannon
c622cf68b5
[DEV] Escapable curly braces (#955) 2024-04-01 22:40:56 -07:00
Jesse Bannon
598c574b8a
[DOCS] Show ytdl_options per url (#954) 2024-04-01 04:41:12 -07:00
Jesse Bannon
017db953bf
[FEATURE] Automatically handle playlists ordered in reverse (#948) 
Playlists have always been a pain-point with ytdl-sub. If an author adds new videos to the end of a playlist, as opposed to the front, it breaks ytdl-sub's intuition of incremental scraping by breaking on the first (oldest) video. This update now makes it possible to handle this in the prebuilt TV Show presets:
- Add each URL variable (`url`, `url2`, ...) into the `download` portion of the subscription twice
- First definition is what we all know and use, simply scrapes first-to-last, then downloads last-to-first
- Second definition does the following:
  - Check to see if a URL is a YouTube playlist URL, if so...
    - Set the field to download, but with modifications to scrape last-to-first, then download first-to-last
  - Otherwise...
    - Set the field to an empty string, which means ytdl-sub will skip it
 2024-04-01 04:24:41 -07:00
Jesse Bannon
083db0d9dc
[FEATURE] Simplify TV Show Collection Preset (#953) 
Simplifies TV Show Collection presets, drastically. Old version:
```
  rick_a_tv_show_collection:
    preset:
      - "jellyfin_tv_show_collection"
      - "season_by_collection__episode_by_year_month_day_reversed"
      - "collection_season_1"
      - "collection_season_2"
    overrides:
      tv_show_name: "Rick A"
      collection_season_1_url: "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
      collection_season_1_name: "All Videos"
      collection_season_2_url: "https://www.youtube.com/playlist?list=PLlaN88a7y2_plecYoJxvRFTLHVbIVAOoc"
      collection_season_2_name: "Official Music Videos"
```

New version:
```
  Jellyfin TV Show Collection:
    "~Rick A":
      s01_name: "All Videos"
      s01_url: "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
      s02_name: "Official Music Videos"
      s02_url: "https://www.youtube.com/playlist?list=PLlaN88a7y2_plecYoJxvRFTLHVbIVAOoc"
```
 2024-03-30 22:04:45 -07:00
Jesse Bannon
d5e647554e
[DEV] Fix resolve_once script bug with custom functions (#952) 
* [DEV] Fix resolve_once script bug with custom functions

* function name

* fix adding custom functions
 2024-03-30 00:53:05 -07:00
Jesse Bannon
82c503c515
[DEV] Update fixtures (#947) 
* [DEV] Update fixtures

* bandcamp
 2024-03-25 00:25:28 -07:00
Jesse Bannon
f940d3ec3e
[BACKEND] yt-dlp 2024.3.10 (#943) 2024-03-18 22:29:35 -07:00
Jesse Bannon
a02e44f09d
[FEATURE] Add dedicated track_date and track_original_date variables (#939) 
As title, to make it easier to override
 2024-02-28 15:05:39 -08:00
Jesse Bannon
0beeeb464d
[FEATURE] Music tag dates (#938) 
Adds proper support for setting `date` and `original_date` fields via `music_tag` plugin using a standardized date format.
 2024-02-28 13:48:42 -08:00
Jesse Bannon
8e6a2cb98e
[DOCS] Rewrite first subscription documentation (#932) 2024-02-20 21:04:13 -08:00
Jesse Bannon
f346b0ef52
[BUGFIX] Handle case when yt-dlp returns LazyList (#929) 
Fixes https://github.com/jmbannon/ytdl-sub/issues/910 , when yt-dlp sometimes returns a non-serializable LazyList
 2024-02-17 09:51:21 -08:00
Jesse Bannon
248e9a15a6
[DOCS] Various doc cleanups (#928) 2024-02-17 08:53:06 -08:00
Jesse Bannon
b95ba86279
[BUGFIX] Use epoch_date if upload_date is missing (#927) 
Fixes https://github.com/jmbannon/ytdl-sub/issues/912

If a required variable (like uid) is missing, ytdl-sub will error. This bugfix prevents sites that do not provide an upload date from erroring.
 2024-02-17 08:19:21 -08:00
Jesse Bannon
7c217db843
[BUGFIX] Fix usage of chapter_title_sanitized (#924) 
Usage of `chapter_title_sanitized` would sometimes result in an error. This should hopefully fix it
 2024-02-12 16:05:50 -08:00
Jesse Bannon
cea21ca47f
[DOCS] Add more configuration docs (#916) 2024-02-02 22:52:17 -08:00
Jesse Bannon
29616144b4
[BUGFIX] Fix multiple top-level parents for linked VEVO YouTube channels (#909) 
Fixes https://github.com/jmbannon/ytdl-sub/issues/908 - when channel URLs return videos from other channels
 2024-01-20 11:16:17 -08:00
Jesse Bannon
b3374cb4d5
[DEV] %map_extend scripting function (#906) 
* [DEV] `%map_extend` scripting function

* lint
 2024-01-19 00:51:44 -08:00
Jesse Bannon
213580ee84
[DEV] Add type-check functions (#905) 2024-01-18 23:24:10 -08:00
Jesse Bannon
29398862da
[BUGFIX] Fix YouTube channels iterating 2x entries (#903) 
A recent feature to grab channel artwork for playlists caused downloading channels to misreport the number of entries it was downloading. This change fixes that
 2024-01-18 19:00:19 -08:00
Jesse Bannon
f0b3991e3e
[FEATURE] Toggle date_range to break or not (#904) 
Adds a the new field `breaking` to the `date_range` plugin, to toggle whether an entry breaks subsequent metadata pulls. This is useful to disable if you are grabbing a playlist that may have videos out of order, but still want to apply a date range to it.
 2024-01-18 16:23:02 -08:00
Qualis Svagtlys
c7823a40ad
[DEV] Prebuilt Presets Revamp (#864) 
Modifies some of the prebuilt preset logic in the backend
 2024-01-18 15:01:29 -08:00
Jesse Bannon
d2e733f2e7
[BACKEND] Only download audio when using audio_extract plugin (#902) 
This addition optimizes the `audio_extract` plugin by preemptively telling yt-dlp to discard the video and only download the audio stream
 2024-01-18 15:00:17 -08:00
Jesse Bannon
c0ca3c3945
[BACKEND] Remove old format for video_tags, music_tags, download_strategy (#899) 
Now that sufficient time has passed, we can formally deprecate the following:
- https://ytdl-sub.readthedocs.io/en/latest/deprecation_notices.html#video-tags
- https://ytdl-sub.readthedocs.io/en/latest/deprecation_notices.html#music-tags
- usage of `download_strategy` (simply remove it if you're getting an error about it)
 2024-01-15 00:04:32 -08:00
Jesse Bannon
e3158583ca
[FEATURE] Toggleable plugin field enable for all dict-based plugins (#897) 
For key/value-based plugins, add an `enable` field to make it easier for child presets to disable their functionality
 2024-01-11 00:40:07 -08:00