danvics/ytdl-sub
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
912 commits 10 branches 373 tags 34 MiB
Commit graph

912 commits

This branch
This branch
All branches
Author SHA1 Message Date
Jesse Bannon
5b34df4588
[DEV] Update fixtures (#970) 
* [DEV] Update fixtures

* update yt-dlp

* fanart
 2024-04-27 09:20:30 -07:00
Nathaniel Barragan
232c0acdfa
[BACKEND] Update yt-dlp, move to pyproject.toml (#965) 
- Updates yt-dlp to 2024.4.9
- Moves setup.cfg to pyproject.toml
- Updates other various backend packages

Thanks @Noodlez1232 !
 2024-04-26 00:09:02 -07:00
Noah Kiss
156c788689
[DOCS] Add Plex Agent Sources Information (#969) 
Co-authored-by: Noah Kiss <noah@nkmk.co>
 2024-04-25 23:12:08 -07:00
Jesse Bannon
ad72e1a6de
[BACKEND] Debug log exception on retry (#957) 
Help diagnose retry errors better
 2024-04-02 17:27:21 -07:00
Jesse Bannon
c622cf68b5
[DEV] Escapable curly braces (#955) 2024-04-01 22:40:56 -07:00
Jesse Bannon
598c574b8a
[DOCS] Show ytdl_options per url (#954) 2024-04-01 04:41:12 -07:00
Jesse Bannon
017db953bf
[FEATURE] Automatically handle playlists ordered in reverse (#948) 
Playlists have always been a pain-point with ytdl-sub. If an author adds new videos to the end of a playlist, as opposed to the front, it breaks ytdl-sub's intuition of incremental scraping by breaking on the first (oldest) video. This update now makes it possible to handle this in the prebuilt TV Show presets:
- Add each URL variable (`url`, `url2`, ...) into the `download` portion of the subscription twice
- First definition is what we all know and use, simply scrapes first-to-last, then downloads last-to-first
- Second definition does the following:
  - Check to see if a URL is a YouTube playlist URL, if so...
    - Set the field to download, but with modifications to scrape last-to-first, then download first-to-last
  - Otherwise...
    - Set the field to an empty string, which means ytdl-sub will skip it
 2024-04-01 04:24:41 -07:00
Jesse Bannon
083db0d9dc
[FEATURE] Simplify TV Show Collection Preset (#953) 
Simplifies TV Show Collection presets, drastically. Old version:
```
  rick_a_tv_show_collection:
    preset:
      - "jellyfin_tv_show_collection"
      - "season_by_collection__episode_by_year_month_day_reversed"
      - "collection_season_1"
      - "collection_season_2"
    overrides:
      tv_show_name: "Rick A"
      collection_season_1_url: "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
      collection_season_1_name: "All Videos"
      collection_season_2_url: "https://www.youtube.com/playlist?list=PLlaN88a7y2_plecYoJxvRFTLHVbIVAOoc"
      collection_season_2_name: "Official Music Videos"
```

New version:
```
  Jellyfin TV Show Collection:
    "~Rick A":
      s01_name: "All Videos"
      s01_url: "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
      s02_name: "Official Music Videos"
      s02_url: "https://www.youtube.com/playlist?list=PLlaN88a7y2_plecYoJxvRFTLHVbIVAOoc"
```
 2024-03-30 22:04:45 -07:00
Jesse Bannon
d5e647554e
[DEV] Fix resolve_once script bug with custom functions (#952) 
* [DEV] Fix resolve_once script bug with custom functions

* function name

* fix adding custom functions
 2024-03-30 00:53:05 -07:00
Jesse Bannon
82c503c515
[DEV] Update fixtures (#947) 
* [DEV] Update fixtures

* bandcamp
 2024-03-25 00:25:28 -07:00
Jesse Bannon
f940d3ec3e
[BACKEND] yt-dlp 2024.3.10 (#943) 2024-03-18 22:29:35 -07:00
Jesse Bannon
a02e44f09d
[FEATURE] Add dedicated track_date and track_original_date variables (#939) 
As title, to make it easier to override
 2024-02-28 15:05:39 -08:00
Jesse Bannon
0beeeb464d
[FEATURE] Music tag dates (#938) 
Adds proper support for setting `date` and `original_date` fields via `music_tag` plugin using a standardized date format.
 2024-02-28 13:48:42 -08:00
Jesse Bannon
8e6a2cb98e
[DOCS] Rewrite first subscription documentation (#932) 2024-02-20 21:04:13 -08:00
Jesse Bannon
f346b0ef52
[BUGFIX] Handle case when yt-dlp returns LazyList (#929) 
Fixes https://github.com/jmbannon/ytdl-sub/issues/910 , when yt-dlp sometimes returns a non-serializable LazyList
 2024-02-17 09:51:21 -08:00
Jesse Bannon
248e9a15a6
[DOCS] Various doc cleanups (#928) 2024-02-17 08:53:06 -08:00
Jesse Bannon
b95ba86279
[BUGFIX] Use epoch_date if upload_date is missing (#927) 
Fixes https://github.com/jmbannon/ytdl-sub/issues/912

If a required variable (like uid) is missing, ytdl-sub will error. This bugfix prevents sites that do not provide an upload date from erroring.
 2024-02-17 08:19:21 -08:00
Jesse Bannon
7c217db843
[BUGFIX] Fix usage of chapter_title_sanitized (#924) 
Usage of `chapter_title_sanitized` would sometimes result in an error. This should hopefully fix it
 2024-02-12 16:05:50 -08:00
Jesse Bannon
cea21ca47f
[DOCS] Add more configuration docs (#916) 2024-02-02 22:52:17 -08:00
Jesse Bannon
29616144b4
[BUGFIX] Fix multiple top-level parents for linked VEVO YouTube channels (#909) 
Fixes https://github.com/jmbannon/ytdl-sub/issues/908 - when channel URLs return videos from other channels
 2024-01-20 11:16:17 -08:00
Jesse Bannon
b3374cb4d5
[DEV] %map_extend scripting function (#906) 
* [DEV] `%map_extend` scripting function

* lint
 2024-01-19 00:51:44 -08:00
Jesse Bannon
213580ee84
[DEV] Add type-check functions (#905) 2024-01-18 23:24:10 -08:00
Jesse Bannon
29398862da
[BUGFIX] Fix YouTube channels iterating 2x entries (#903) 
A recent feature to grab channel artwork for playlists caused downloading channels to misreport the number of entries it was downloading. This change fixes that
 2024-01-18 19:00:19 -08:00
Jesse Bannon
f0b3991e3e
[FEATURE] Toggle date_range to break or not (#904) 
Adds a the new field `breaking` to the `date_range` plugin, to toggle whether an entry breaks subsequent metadata pulls. This is useful to disable if you are grabbing a playlist that may have videos out of order, but still want to apply a date range to it.
 2024-01-18 16:23:02 -08:00
Qualis Svagtlys
c7823a40ad
[DEV] Prebuilt Presets Revamp (#864) 
Modifies some of the prebuilt preset logic in the backend
 2024-01-18 15:01:29 -08:00
Jesse Bannon
d2e733f2e7
[BACKEND] Only download audio when using audio_extract plugin (#902) 
This addition optimizes the `audio_extract` plugin by preemptively telling yt-dlp to discard the video and only download the audio stream
 2024-01-18 15:00:17 -08:00
Jesse Bannon
c0ca3c3945
[BACKEND] Remove old format for video_tags, music_tags, download_strategy (#899) 
Now that sufficient time has passed, we can formally deprecate the following:
- https://ytdl-sub.readthedocs.io/en/latest/deprecation_notices.html#video-tags
- https://ytdl-sub.readthedocs.io/en/latest/deprecation_notices.html#music-tags
- usage of `download_strategy` (simply remove it if you're getting an error about it)
 2024-01-15 00:04:32 -08:00
Jesse Bannon
e3158583ca
[FEATURE] Toggleable plugin field enable for all dict-based plugins (#897) 
For key/value-based plugins, add an `enable` field to make it easier for child presets to disable their functionality
 2024-01-11 00:40:07 -08:00
Jesse Bannon
2076de3074
[BUGFIX] Smarter subscription validation (#895) 
With the right functions, it was possible for subscription validation to raise a false-positive error. This should hopefully resolve that issue and give validation a small performance increase
 2024-01-10 18:16:12 -08:00
Jesse Bannon
00e4cd8541
[BUGFIX] Fix rare case when yt-dlp returns None and is downloaded (#894) 2024-01-10 09:33:03 -08:00
Jesse Bannon
b9d05c5b8f
[BUGFIX] keep_max_files=0 downloading 2 (#893) 
Fixes https://github.com/jmbannon/ytdl-sub/issues/892

Does not set max_downloads if keep_max_files = 0
 2024-01-10 09:31:37 -08:00
Jesse Bannon
305d084a6d
[BUGFIX] Only apply --dl-override to subscriptions in --match (#890) 
As title, to not accidently add breaking changes for subscriptions that are not going to run
 2024-01-09 20:50:31 -08:00
Jesse Bannon
3b36ffd050
[BUGFIX] Fix source metadata getting fetched when yt-dlp breaks (#889) 
Prior PR only got channel metadata for a playlist if it never 'broke' (i.e. MaxDownloads, BreakOnExisting). Will now fetch it regardless.
 2024-01-09 19:25:09 -08:00
Jesse Bannon
be0f6f2d2d
[BUGFIX] Parent entry is not a dict (#888) 
Fixes an AttributeError bug
 2024-01-09 14:35:19 -08:00
Jesse Bannon
c29cca2b3d
[FEATURE] YouTube playlists to fetch playlist uploader's poster + fanart (#887) 
Implements https://github.com/jmbannon/ytdl-sub/issues/756

TV Show presets whose URL is a playlist will try to get the playlist uploader's poster and fanart.
 2024-01-08 22:51:42 -08:00
Jesse Bannon
915e29379a
[BACKEND] Try to fetch source metadata from uploader_url (#886) 
Makes it possible to fix https://github.com/jmbannon/ytdl-sub/issues/756

In the case that uploader_url does not equal the input url, keep trying to fetch uploader_url to get all possible metadata that yt-dlp does not provide in a single API call
 2024-01-08 18:53:00 -08:00
Jesse Bannon
961e04d027
[DOCS] Update Usage page (#883) 2024-01-08 16:20:08 -08:00
Jesse Bannon
78cdcce464
[FEATURE] Add --dl-override to pass dl arg overrides to all subscriptions (#882) 
With both `--match` and `--dl-override`, it is now possible to more easily experiment with subscriptions without needing to create a separate file.

For example, suppose you are changing some values in your subscription "Rick A" and want to test them out. You can now run:
```
ytdl-sub sub --dry-run --match Rick --dl-override '--ytdl-options.max_downloads 3'
```
This will
1. Dry-run
2. Only run for the subscription "Rick A"
3. Apply setting max_downloads to 3
 2024-01-08 15:19:01 -08:00
Jesse Bannon
9382be5c42
[FEATURE] --match flag to only run subset of subscriptions (#881) 
Implements https://github.com/jmbannon/ytdl-sub/issues/880
and maybe fixes https://github.com/jmbannon/ytdl-sub/issues/827

Can now do `ytdl-sub sub --match SubA SubB` or `ytdl-sub sub --match SubA --match SubB`, which will only run subscriptions that contain `SubA` or `SubB` in their names
 2024-01-08 13:37:07 -08:00
Jesse Bannon
d80a63631a
[BACKEND] String split scripting function (#877) 2024-01-07 00:13:32 -08:00
Jesse Bannon
ba71abc466
[BACKEND] Optional default in %array_at scripting function (#879) 
As title
 2024-01-06 23:47:09 -08:00
Jesse Bannon
dc74c6a52a
[BACKEND] elif scripting function (#876) 
As titled
 2024-01-05 17:48:59 -08:00
Jesse Bannon
8de9a5b6da
[BACKEND] Update yt-dlp to 2023.12.30 (#875) 
As titled
 2024-01-05 15:59:53 -08:00
Jesse Bannon
d96bf11b12
[BACKEND] More url variables in helper preset (#874) 
Have `url` variables go up to 100
 2024-01-05 14:45:03 -08:00
Jesse Bannon
5e384ba96e
[BACKEND] Info log for filter_include and filter_exclude plugins (#873) 
Adds more verbose logging when entries get filtered using the `filter_include` and `filter_exclude` plugins
 2024-01-05 14:44:16 -08:00
Jesse Bannon
4d0123c8b0
[BACKEND] Optimize memory usage (#872) 
Reduce memory usage significantly when pulling large channels all at once by
- Not loading entry script until it is being processed
- Deleting the entry memory once fully processed
- Disable yt-dlp from keeping entry info in memory since ytdl-sub reads it from file
 2024-01-05 13:04:01 -08:00
Qualis Svagtlys
891b951625
[DOCS] Unraid and FAQ Doc Updates (#869) 2024-01-05 11:12:09 -08:00
Jesse Bannon
db5f095a51
[BACKEND] Set max_downloads to keep_max_files if present (#871) 
Implements: https://github.com/jmbannon/ytdl-sub/issues/829

Optimization to not scrape more than what you will keep
 2024-01-04 16:38:20 -08:00
Jesse Bannon
2dbe426ced
[BUGFIX] Properly handle UIDs that have misbehaving chars that get sanitized (#870) 
Should hopefuly fix both:
- https://github.com/jmbannon/ytdl-sub/issues/843
- https://github.com/jmbannon/ytdl-sub/issues/867

Various sites have unique IDs with characters that get sanitized (i.e. `/`, `:`). `ytdl-sub` was not using sanitized UIDs when looking for files, and would throw an error saying download was missing. This should resolve that issue.
 2024-01-04 15:01:17 -08:00
Jesse Bannon
b3d298b664
[DOCS] Advanced scripting usage, rename Overrides to Static (#866) 2024-01-01 00:01:46 -08:00