diff --git a/docs/source/config_reference/scripting/index.rst b/docs/source/config_reference/scripting/index.rst index c99105f8..bee9a4e8 100644 --- a/docs/source/config_reference/scripting/index.rst +++ b/docs/source/config_reference/scripting/index.rst @@ -2,11 +2,171 @@ Scripting ========= -Work in progress! Explanation of how to define scripts/variables will be added here. +``ytdl-sub`` fields (file-names, tags, etc) are defined using variables and scripts. The links below +contain reference documentation for each built-in variable and scripting function. .. toctree:: :maxdepth: 1 entry_variables override_variables - scripting_functions \ No newline at end of file + scripting_functions + scripting_types + +How it Works +------------ + +Fields in the config that support ``formatters`` mean they support scripting, and will +*format* the field using its defined script. + +In its most basic form, a script is a string comprised of variables and/or functions. + +Static String +~~~~~~~~~~~~~ + +The following example sets ``ytdl-sub``'s output directory. It is +considered *static* because it does not depend on anything from an entry. + +.. code-block:: yaml + + output_options: + output_directory: "Custom YTDL-SUB TV Show" + +Static Variables +~~~~~~~~~~~~~~~~ + +``ytdl-sub`` offers a few built-in static variables, including ``subscription_name``. +We can use this instead of hard-coding it above: + +.. code-block:: yaml + + output_options: + output_directory: "{subscription_name}" + +The syntax for variable usage is curly-braces with the variable name within it. Assuming +our subscription is actually named "Custom YTDL-SUB TV Show", then ``ytdl-sub`` +will actually write to that directory. + +Entry Variables +~~~~~~~~~~~~~~~ + +For context, an *entry* is a video or audio file downloaded from ``yt-dlp``. +*Entry variables* are variables that are derived from an entry's ``info.json`` file. This file +comes from ``yt-dlp`` and contains every piece of metadata that it scraped. + +These variables are not considered static since they change per entry download. There are a +few fields in ``ytdl-sub`` (i.e. ``output_directory``) that must be static. For others, +we are free to use values that derive from an entry. + +Suppose we want to customize the name of an entry's output file and thumbnail to include its +title in its name. We can do that using entry variables: + +.. code-block:: yaml + + output_options: + output_directory: "{subscription_name}" + file_name: "{title}.{ext}" + thumbnail_name: "{title}.{thumbnail_ext}" + +Creating Custom Variables +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Suppose we want to include the date in our file names. This means we'd need to update +both the ``file_name`` and ``thumbnail_name`` fields to include it. + +Instead, we can create a custom *override variable*. This is ``ytdl-sub``'s method +for creating and overriding custom variables. + +These are created in the ``overrides`` section. Let's take our above example and create +a ``custom_file_name`` variable to use for the entry file and thumbnail fields: + +.. code-block:: yaml + + output_options: + output_directory: "{subscription_name}" + file_name: "{custom_file_name}.{ext}" + thumbnail_name: "{custom_file_name}.{thumbnail_ext}" + + overrides: + custom_file_name: "{upload_date_standardized} {title}" + +Sanitizing Variables +~~~~~~~~~~~~~~~~~~~~ + +For experienced ``yt-dlp`` scrapers, you may be thinking: + +- What if the title has characters that do not play nice with my operating system? + +``ytdl-sub`` is able to *sanitize* any variable, meaning it replaces any problematic characters +with safe alternatives that can be used in file names. We can ensure our file names and directories +are safe by using: + +.. code-block:: yaml + + output_options: + output_directory: "{subscription_name_sanitized}" + file_name: "{custom_file_name}.{ext}" + thumbnail_name: "{custom_file_name}.{thumbnail_ext}" + + overrides: + custom_file_name: "{upload_date_standardized} {title_sanitized}" + +Simply add a ``_sanitized`` suffix to any variable name to make it sanitized. + +.. note:: + + Make sure you do not sanitize custom variables that intentionally create directories, otherwise + they will... be sanitized and not resolve to directories! + + +Using Scripting Functions +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Let's suppose you are an avid command-line user, and like all of your file names to be +``snake_cased_with_no_spaces``. We can use the +`replace `_ +*scripting function* to create and use a snake-cased title. + +.. code-block:: yaml + + output_options: + output_directory: "{subscription_name_sanitized}" + file_name: "{custom_file_name}.{ext}" + thumbnail_name: "{custom_file_name}.{thumbnail_ext}" + + overrides: + snake_cased_title: >- + { + %replace( title, ' ', '_' ) + } + custom_file_name: "{upload_date_standardized}_{snake_cased_title_sanitized}" + +Scripting functions are similar to variables - they must be used within curly-braces. +It is good practice to use ``>-`` when defining variables that use functions. It is YAML's way of +saying: + +- Allow a string to be multi-lined, and do not include newlines before or after it. + +See for yourself `here `_. +Any whitespace within curly-braces is okay since it will be parsed out. This is needed to make +scripting function usage readable. + +.. important:: + + It is important to use ``>-`` over other YAML new-line directives like ``>`` because they + add newlines before or after curly-braces, and will be included in your variable's output string. + +Advanced Scripting +------------------ + +Accessing ``info.json`` Fields +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +WIP + +Creating Custom Functions +~~~~~~~~~~~~~~~~~~~~~~~~~ +WIP + +Parsing Maps and Arrays +~~~~~~~~~~~~~~~~~~~~~~~ +WIP diff --git a/docs/source/config_reference/scripting/scripting_types.rst b/docs/source/config_reference/scripting/scripting_types.rst new file mode 100644 index 00000000..d389c2a9 --- /dev/null +++ b/docs/source/config_reference/scripting/scripting_types.rst @@ -0,0 +1,319 @@ + +Scripting Types +=============== + +Types +----- + +String +~~~~~~ + +Strings are a series of characters surrounded by quotes and can be defined in a few ways, including: + +.. tab-set:: + + .. tab-item:: Literal + + .. code-block:: yaml + + string_variable: "This is a String variable" + + .. tab-item:: In-Line + + .. code-block:: yaml + + string_variable: "{ %string('This is a String variable') }" + + .. tab-item:: Single Quote + + .. code-block:: yaml + + string_variable: >- + { + %string('This is a String variable') + } + + .. tab-item:: Double Quote + + .. code-block:: yaml + + string_variable: >- + { + %string("This is a String variable") + } + + .. tab-item:: Triple Quote + + .. code-block:: yaml + + string_variable: >- + { + %string('''This is a String variable''') + } + + .. tab-item:: Triple-Double Quote + + .. code-block:: yaml + + string_variable: >- + { + %string("""This is a String variable""") + } + +.. note:: + + For non-String types, they must be defined as parameters to scripting functions. This is because + anything in a variable definition that is not within curly-braces gets evaluated as a String. + +Integer +~~~~~~~ + +Integers are whole numbers with no decimal. + +.. tab-set:: + + .. tab-item:: Literal + + .. code-block:: yaml + + int_variable: >- + { + %int(2022) + } + + .. tab-item:: In-Line + + .. code-block:: yaml + + int_variable: "{ %int(2022) }" + +Float +~~~~~ + +Floats are floating-point decimals numbers. + +.. tab-set:: + + .. tab-item:: Literal + + .. code-block:: yaml + + float_variable: >- + { + %float(3.14) + } + + .. tab-item:: In-Line + + .. code-block:: yaml + + float_variable: "{ %float(3.14) }" + +Boolean +~~~~~~~ + +A type is considered boolean if it spells out ``True`` or ``False``, case-insensitive. + +.. tab-set:: + + .. tab-item:: Literal + + .. code-block:: yaml + + bool_variable: >- + { + %bool(True) + } + + .. tab-item:: In-Line + + .. code-block:: yaml + + bool_variable: "{ %bool(FALSE) }" + +Array +~~~~~ + +An Array contains multiple types of any kind, including nested Arrays and Maps. +Arrays are defined using brackets (``[ ]``), and are accessed using zero-based indexing. + +.. tab-set:: + + .. tab-item:: Literal + + .. code-block:: yaml + + array_variable: >- + { + [ + "element with index 0", + 1, + 2.0, + [ "Nested Array 3" ] + ] + } + element_0: >- + { + %array_at(array_variable, 0) + } + + .. tab-item:: In-Line + + .. code-block:: yaml + + array_variable: "{ ['element with index 0', 1, 2.0, ['Nested Array 3' ]] }" + element_0: "{ %array_at(array_variable, 0) }" + +Map +~~~ + +A Map is a key-value store, containing mappings between keys and values. +Maps are defined using curley-braces (``{ }``), and are accessed using their keys. + +.. tab-set:: + + .. tab-item:: Literal + + .. code-block:: yaml + + map_variable: >- + { + { + "string_key": "string_value", + 1: "int_key", + "list_value": [ "elem0", 1, 2.0 ] + } + } + string_value: >- + { + %map_get(map_variable, "string_key") + } + + .. tab-item:: In-Line + + .. code-block:: yaml + + map_variable: "{ {'string_key': 'string_value', 1: 'int_key', 'list_value': [ 'elem0', 1, 2.0 ]} }" + string_value: "{ %map_get(map_variable, 'string_key') }" + +Null +~~~~ +Null is represented by an empty String, and can be conveyed by spelling out ``null``, +case-insensitive. + +.. tab-set:: + + .. tab-item:: Literal + + .. code-block:: yaml + + null_variable: "" + + .. tab-item:: In-Line + + .. code-block:: yaml + + null_variable: "{ %string(null) }" + + +Function Type-Hints +------------------- + +AnyArgument +~~~~~~~~~~~ +AnyArgument means any of the above Types are valid as input or output to a scripting function. + +.. note:: + + Strict typing is enforced. For functions that return ``AnyArgument`` need to be casted before + passing into functions that expect a particular type. + +Numeric +~~~~~~~ +Numeric refers to either an Integer or Float. + +Optional +~~~~~~~~ +Optional means a particular scripting function argument can be either provided or not included. +For example, the function +`map_get `_ +has an optional default value. Both of these usages are valid: + +.. tab-set:: + + .. tab-item:: Map Get + + .. code-block:: yaml + + will_throw_key_does_not_exist_error: "{ %map_get( {}, 'key' ) }" + + .. tab-item:: Map Get with Optional Default Value + + .. code-block:: yaml + + will_return_default: "{ %map_get( {}, 'key', 'default value' ) }" + +Lambda +~~~~~~ +Lambda parameters are a reference to a function, and will call that lambda function +on the input. In this example, + +.. code-block:: yaml + + lambda_array_numeric_to_string: >- + { + %array_apply( [ 1, 2, 3, 4], %string ) + } + +We apply ``%string`` as a lambda function to +`array_apply `_, +which is called on every element in the input array. The output becomes ``["1", "2", "3", "4"]``. + +This example has one input-argument being passed into the lambda. For other lambda-based functions +like `array_enumerate `_, +it expects the lambda function to have two input arguments. These are denoted using +``LambdaTwo``, ``LambdaThree``, etc within the function spec. + +LambdaReduce +~~~~~~~~~~~~ +LambdaReduce is special type of lambda that reduces an Array to a single value by calling the +LabmdaReduce function repeatedly on two elements in the Array until it is reduced to a single value. + +In this example, + +.. code-block:: yaml + + lambda_reduce_sum: >- + { + %array_reduce( [ 1, 2, 3, 4], %add ) + } + +We call +`array_reduce `_ +on the input array, using +`add `_ +as the LambdaReduce function. This will reduce the Array to a single value by internally calling + +.. code-block:: + + - %add(1, 2) = 3 + - %add(3, 3) = 6 + - %add(6, 4) = 10 + +And evaluate to ``10``. + +ReturnableArguments +~~~~~~~~~~~~~~~~~~~ + +Returnable arguments are used in conditional functions like +`if `_, +which implies the argument passed into the function is the function's output. For example, + +.. code-block:: yaml + + conditional_function: >- + { + %if( True, "Return this if True", "Return this if False" ) + } + +is going to return ``"Return this if True"`` since the condition parameter is ``True``. \ No newline at end of file