danvics/ytdl-sub
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

[DOCS] Scripting reference start (#862)

Browse source
This commit is contained in:
Jesse Bannon 2023-12-29 22:34:36 -08:00 committed by GitHub
parent 735da8b647
commit 12430ecf1e
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 481 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

164
docs/source/config_reference/scripting/index.rst
View file

@ -2,11 +2,171 @@
Scripting
=========
Work in progress! Explanation of how to define scripts/variables will be added here.
``ytdl-sub`` fields (file-names, tags, etc) are defined using variables and scripts. The links below
contain reference documentation for each built-in variable and scripting function.
.. toctree::
:maxdepth: 1
entry_variables
override_variables
scripting_functions
scripting_functions
scripting_types
How it Works
------------
Fields in the config that support ``formatters`` mean they support scripting, and will
*format* the field using its defined script.
In its most basic form, a script is a string comprised of variables and/or functions.
Static String
~~~~~~~~~~~~~
The following example sets ``ytdl-sub``'s output directory. It is
considered *static* because it does not depend on anything from an entry.
.. code-block:: yaml
output_options:
output_directory: "Custom YTDL-SUB TV Show"
Static Variables
~~~~~~~~~~~~~~~~
``ytdl-sub`` offers a few built-in static variables, including ``subscription_name``.
We can use this instead of hard-coding it above:
.. code-block:: yaml
output_options:
output_directory: "{subscription_name}"
The syntax for variable usage is curly-braces with the variable name within it. Assuming
our subscription is actually named "Custom YTDL-SUB TV Show", then ``ytdl-sub``
will actually write to that directory.
Entry Variables
~~~~~~~~~~~~~~~
For context, an *entry* is a video or audio file downloaded from ``yt-dlp``.
*Entry variables* are variables that are derived from an entry's ``info.json`` file. This file
comes from ``yt-dlp`` and contains every piece of metadata that it scraped.
These variables are not considered static since they change per entry download. There are a
few fields in ``ytdl-sub`` (i.e. ``output_directory``) that must be static. For others,
we are free to use values that derive from an entry.
Suppose we want to customize the name of an entry's output file and thumbnail to include its
title in its name. We can do that using entry variables:
.. code-block:: yaml
output_options:
output_directory: "{subscription_name}"
file_name: "{title}.{ext}"
thumbnail_name: "{title}.{thumbnail_ext}"
Creating Custom Variables
~~~~~~~~~~~~~~~~~~~~~~~~~
Suppose we want to include the date in our file names. This means we'd need to update
both the ``file_name`` and ``thumbnail_name`` fields to include it.
Instead, we can create a custom *override variable*. This is ``ytdl-sub``'s method
for creating and overriding custom variables.
These are created in the ``overrides`` section. Let's take our above example and create
a ``custom_file_name`` variable to use for the entry file and thumbnail fields:
.. code-block:: yaml
output_options:
output_directory: "{subscription_name}"
file_name: "{custom_file_name}.{ext}"
thumbnail_name: "{custom_file_name}.{thumbnail_ext}"
overrides:
custom_file_name: "{upload_date_standardized} {title}"
Sanitizing Variables
~~~~~~~~~~~~~~~~~~~~
For experienced ``yt-dlp`` scrapers, you may be thinking:
- What if the title has characters that do not play nice with my operating system?
``ytdl-sub`` is able to *sanitize* any variable, meaning it replaces any problematic characters
with safe alternatives that can be used in file names. We can ensure our file names and directories
are safe by using:
.. code-block:: yaml
output_options:
output_directory: "{subscription_name_sanitized}"
file_name: "{custom_file_name}.{ext}"
thumbnail_name: "{custom_file_name}.{thumbnail_ext}"
overrides:
custom_file_name: "{upload_date_standardized} {title_sanitized}"
Simply add a ``_sanitized`` suffix to any variable name to make it sanitized.
.. note::
Make sure you do not sanitize custom variables that intentionally create directories, otherwise
they will... be sanitized and not resolve to directories!
Using Scripting Functions
~~~~~~~~~~~~~~~~~~~~~~~~~
Let's suppose you are an avid command-line user, and like all of your file names to be
``snake_cased_with_no_spaces``. We can use the
`replace <https://ytdl-sub.readthedocs.io/en/latest/config_reference/scripting/scripting_functions.html#replace>`_
*scripting function* to create and use a snake-cased title.
.. code-block:: yaml
output_options:
output_directory: "{subscription_name_sanitized}"
file_name: "{custom_file_name}.{ext}"
thumbnail_name: "{custom_file_name}.{thumbnail_ext}"
overrides:
snake_cased_title: >-
{
%replace( title, ' ', '_' )
}
custom_file_name: "{upload_date_standardized}_{snake_cased_title_sanitized}"
Scripting functions are similar to variables - they must be used within curly-braces.
It is good practice to use ``>-`` when defining variables that use functions. It is YAML's way of
saying:
- Allow a string to be multi-lined, and do not include newlines before or after it.
See for yourself `here <https://yaml-online-parser.appspot.com/?yaml=output_options%3A%0A%20%20output_directory%3A%20%22%7Bsubscription_name_sanitized%7D%22%0A%20%20file_name%3A%20%22%7Bcustom_file_name%7D.%7Bext%7D%22%0A%20%20thumbnail_name%3A%20%22%7Bcustom_file_name%7D.%7Bthumbnail_ext%7D%22%0A%0Aoverrides%3A%0A%20%20snake_cased_title%3A%20%3E-%0A%20%20%20%20%7B%0A%20%20%20%20%20%20%25replace%28%20title%2C%20%27%20%27%2C%20%27_%27%20%29%0A%20%20%20%20%7D%0A%20%20custom_file_name%3A%20%22%7Bupload_date_standardized%7D%20%7Bsnake_cased_title_sanitized%7D%22&type=canonical_yaml>`_.
Any whitespace within curly-braces is okay since it will be parsed out. This is needed to make
scripting function usage readable.
.. important::
It is important to use ``>-`` over other YAML new-line directives like ``>`` because they
add newlines before or after curly-braces, and will be included in your variable's output string.
Advanced Scripting
------------------
Accessing ``info.json`` Fields
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
WIP
Creating Custom Functions
~~~~~~~~~~~~~~~~~~~~~~~~~
WIP
Parsing Maps and Arrays
~~~~~~~~~~~~~~~~~~~~~~~
WIP

319
docs/source/config_reference/scripting/scripting_types.rst Normal file
View file

@ -0,0 +1,319 @@
Scripting Types
===============
Types
-----
String
~~~~~~
Strings are a series of characters surrounded by quotes and can be defined in a few ways, including:
.. tab-set::
.. tab-item:: Literal
.. code-block:: yaml
string_variable: "This is a String variable"
.. tab-item:: In-Line
.. code-block:: yaml
string_variable: "{ %string('This is a String variable') }"
.. tab-item:: Single Quote
.. code-block:: yaml
string_variable: >-
{
%string('This is a String variable')
}
.. tab-item:: Double Quote
.. code-block:: yaml
string_variable: >-
{
%string("This is a String variable")
}
.. tab-item:: Triple Quote
.. code-block:: yaml
string_variable: >-
{
%string('''This is a String variable''')
}
.. tab-item:: Triple-Double Quote
.. code-block:: yaml
string_variable: >-
{
%string("""This is a String variable""")
}
.. note::
For non-String types, they must be defined as parameters to scripting functions. This is because
anything in a variable definition that is not within curly-braces gets evaluated as a String.
Integer
~~~~~~~
Integers are whole numbers with no decimal.
.. tab-set::
.. tab-item:: Literal
.. code-block:: yaml
int_variable: >-
{
%int(2022)
}
.. tab-item:: In-Line
.. code-block:: yaml
int_variable: "{ %int(2022) }"
Float
~~~~~
Floats are floating-point decimals numbers.
.. tab-set::
.. tab-item:: Literal
.. code-block:: yaml
float_variable: >-
{
%float(3.14)
}
.. tab-item:: In-Line
.. code-block:: yaml
float_variable: "{ %float(3.14) }"
Boolean
~~~~~~~
A type is considered boolean if it spells out ``True`` or ``False``, case-insensitive.
.. tab-set::
.. tab-item:: Literal
.. code-block:: yaml
bool_variable: >-
{
%bool(True)
}
.. tab-item:: In-Line
.. code-block:: yaml
bool_variable: "{ %bool(FALSE) }"
Array
~~~~~
An Array contains multiple types of any kind, including nested Arrays and Maps.
Arrays are defined using brackets (``[ ]``), and are accessed using zero-based indexing.
.. tab-set::
.. tab-item:: Literal
.. code-block:: yaml
array_variable: >-
{
[
"element with index 0",
1,
2.0,
[ "Nested Array 3" ]
]
}
element_0: >-
{
%array_at(array_variable, 0)
}
.. tab-item:: In-Line
.. code-block:: yaml
array_variable: "{ ['element with index 0', 1, 2.0, ['Nested Array 3' ]] }"
element_0: "{ %array_at(array_variable, 0) }"
Map
~~~
A Map is a key-value store, containing mappings between keys and values.
Maps are defined using curley-braces (``{ }``), and are accessed using their keys.
.. tab-set::
.. tab-item:: Literal
.. code-block:: yaml
map_variable: >-
{
{
"string_key": "string_value",
1: "int_key",
"list_value": [ "elem0", 1, 2.0 ]
}
}
string_value: >-
{
%map_get(map_variable, "string_key")
}
.. tab-item:: In-Line
.. code-block:: yaml
map_variable: "{ {'string_key': 'string_value', 1: 'int_key', 'list_value': [ 'elem0', 1, 2.0 ]} }"
string_value: "{ %map_get(map_variable, 'string_key') }"
Null
~~~~
Null is represented by an empty String, and can be conveyed by spelling out ``null``,
case-insensitive.
.. tab-set::
.. tab-item:: Literal
.. code-block:: yaml
null_variable: ""
.. tab-item:: In-Line
.. code-block:: yaml
null_variable: "{ %string(null) }"
Function Type-Hints
-------------------
AnyArgument
~~~~~~~~~~~
AnyArgument means any of the above Types are valid as input or output to a scripting function.
.. note::
Strict typing is enforced. For functions that return ``AnyArgument`` need to be casted before
passing into functions that expect a particular type.
Numeric
~~~~~~~
Numeric refers to either an Integer or Float.
Optional
~~~~~~~~
Optional means a particular scripting function argument can be either provided or not included.
For example, the function
`map_get <https://ytdl-sub.readthedocs.io/en/latest/config_reference/scripting/scripting_functions.html#map-get>`_
has an optional default value. Both of these usages are valid:
.. tab-set::
.. tab-item:: Map Get
.. code-block:: yaml
will_throw_key_does_not_exist_error: "{ %map_get( {}, 'key' ) }"
.. tab-item:: Map Get with Optional Default Value
.. code-block:: yaml
will_return_default: "{ %map_get( {}, 'key', 'default value' ) }"
Lambda
~~~~~~
Lambda parameters are a reference to a function, and will call that lambda function
on the input. In this example,
.. code-block:: yaml
lambda_array_numeric_to_string: >-
{
%array_apply( [ 1, 2, 3, 4], %string )
}
We apply ``%string`` as a lambda function to
`array_apply <https://ytdl-sub.readthedocs.io/en/latest/config_reference/scripting/scripting_functions.html#array-apply>`_,
which is called on every element in the input array. The output becomes ``["1", "2", "3", "4"]``.
This example has one input-argument being passed into the lambda. For other lambda-based functions
like `array_enumerate <https://ytdl-sub.readthedocs.io/en/latest/config_reference/scripting/scripting_functions.html#array-enumerate>`_,
it expects the lambda function to have two input arguments. These are denoted using
``LambdaTwo``, ``LambdaThree``, etc within the function spec.
LambdaReduce
~~~~~~~~~~~~
LambdaReduce is special type of lambda that reduces an Array to a single value by calling the
LabmdaReduce function repeatedly on two elements in the Array until it is reduced to a single value.
In this example,
.. code-block:: yaml
lambda_reduce_sum: >-
{
%array_reduce( [ 1, 2, 3, 4], %add )
}
We call
`array_reduce <https://ytdl-sub.readthedocs.io/en/latest/config_reference/scripting/scripting_functions.html#array-reduce>`_
on the input array, using
`add <https://ytdl-sub.readthedocs.io/en/latest/config_reference/scripting/scripting_functions.html#add>`_
as the LambdaReduce function. This will reduce the Array to a single value by internally calling
.. code-block::
- %add(1, 2) = 3
- %add(3, 3) = 6
- %add(6, 4) = 10
And evaluate to ``10``.
ReturnableArguments
~~~~~~~~~~~~~~~~~~~
Returnable arguments are used in conditional functions like
`if <https://ytdl-sub.readthedocs.io/en/latest/config_reference/scripting/scripting_functions.html#if>`_,
which implies the argument passed into the function is the function's output. For example,
.. code-block:: yaml
conditional_function: >-
{
%if( True, "Return this if True", "Return this if False" )
}
is going to return ``"Return this if True"`` since the condition parameter is ``True``.