[DOCS] Scripting reference start (#862)

docs/source/config_reference/scripting/index.rst

@@ -2,7 +2,8 @@
 Scripting
 =========
 
-Work in progress! Explanation of how to define scripts/variables will be added here.
+``ytdl-sub`` fields (file-names, tags, etc) are defined using variables and scripts. The links below
+contain reference documentation for each built-in variable and scripting function.
 
 .. toctree::
   :maxdepth: 1
@@ -10,3 +11,162 @@ Work in progress! Explanation of how to define scripts/variables will be added h
   entry_variables
   override_variables
   scripting_functions
+  scripting_types
+
+How it Works
+------------
+
+Fields in the config that support ``formatters`` mean they support scripting, and will
+*format* the field using its defined script.
+
+In its most basic form, a script is a string comprised of variables and/or functions.
+
+Static String
+~~~~~~~~~~~~~
+
+The following example sets ``ytdl-sub``'s output directory. It is
+considered *static* because it does not depend on anything from an entry.
+
+.. code-block:: yaml
+
+   output_options:
+     output_directory: "Custom YTDL-SUB TV Show"
+
+Static Variables
+~~~~~~~~~~~~~~~~
+
+``ytdl-sub`` offers a few built-in static variables, including ``subscription_name``.
+We can use this instead of hard-coding it above:
+
+.. code-block:: yaml
+
+   output_options:
+     output_directory: "{subscription_name}"
+
+The syntax for variable usage is curly-braces with the variable name within it. Assuming
+our subscription is actually named "Custom YTDL-SUB TV Show", then ``ytdl-sub``
+will actually write to that directory.
+
+Entry Variables
+~~~~~~~~~~~~~~~
+
+For context, an *entry* is a video or audio file downloaded from ``yt-dlp``.
+*Entry variables* are variables that are derived from an entry's ``info.json`` file. This file
+comes from ``yt-dlp`` and contains every piece of metadata that it scraped.
+
+These variables are not considered static since they change per entry download. There are a
+few fields in ``ytdl-sub`` (i.e. ``output_directory``) that must be static. For others,
+we are free to use values that derive from an entry.
+
+Suppose we want to customize the name of an entry's output file and thumbnail to include its
+title in its name. We can do that using entry variables:
+
+.. code-block:: yaml
+
+   output_options:
+     output_directory: "{subscription_name}"
+     file_name: "{title}.{ext}"
+     thumbnail_name: "{title}.{thumbnail_ext}"
+
+Creating Custom Variables
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Suppose we want to include the date in our file names. This means we'd need to update
+both the ``file_name`` and ``thumbnail_name`` fields to include it.
+
+Instead, we can create a custom *override variable*. This is ``ytdl-sub``'s method
+for creating and overriding custom variables.
+
+These are created in the ``overrides`` section. Let's take our above example and create
+a ``custom_file_name`` variable to use for the entry file and thumbnail fields:
+
+.. code-block:: yaml
+
+   output_options:
+     output_directory: "{subscription_name}"
+     file_name: "{custom_file_name}.{ext}"
+     thumbnail_name: "{custom_file_name}.{thumbnail_ext}"
+
+   overrides:
+     custom_file_name: "{upload_date_standardized} {title}"
+
+Sanitizing Variables
+~~~~~~~~~~~~~~~~~~~~
+
+For experienced ``yt-dlp`` scrapers, you may be thinking:
+
+- What if the title has characters that do not play nice with my operating system?
+
+``ytdl-sub`` is able to *sanitize* any variable, meaning it replaces any problematic characters
+with safe alternatives that can be used in file names. We can ensure our file names and directories
+are safe by using:
+
+.. code-block:: yaml
+
+   output_options:
+     output_directory: "{subscription_name_sanitized}"
+     file_name: "{custom_file_name}.{ext}"
+     thumbnail_name: "{custom_file_name}.{thumbnail_ext}"
+
+   overrides:
+     custom_file_name: "{upload_date_standardized} {title_sanitized}"
+
+Simply add a ``_sanitized`` suffix to any variable name to make it sanitized.
+
+.. note::
+
+   Make sure you do not sanitize custom variables that intentionally create directories, otherwise
+   they will... be sanitized and not resolve to directories!
+
+
+Using Scripting Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Let's suppose you are an avid command-line user, and like all of your file names to be
+``snake_cased_with_no_spaces``. We can use the
+`replace <https://ytdl-sub.readthedocs.io/en/latest/config_reference/scripting/scripting_functions.html#replace>`_
+*scripting function* to create and use a snake-cased title.
+
+.. code-block:: yaml
+
+   output_options:
+     output_directory: "{subscription_name_sanitized}"
+     file_name: "{custom_file_name}.{ext}"
+     thumbnail_name: "{custom_file_name}.{thumbnail_ext}"
+
+   overrides:
+     snake_cased_title: >-
+       {
+         %replace( title, ' ', '_' )
+       }
+     custom_file_name: "{upload_date_standardized}_{snake_cased_title_sanitized}"
+
+Scripting functions are similar to variables - they must be used within curly-braces.
+It is good practice to use ``>-`` when defining variables that use functions. It is YAML's way of
+saying:
+
+- Allow a string to be multi-lined, and do not include newlines before or after it.
+
+See for yourself `here <https://yaml-online-parser.appspot.com/?yaml=output_options%3A%0A%20%20output_directory%3A%20%22%7Bsubscription_name_sanitized%7D%22%0A%20%20file_name%3A%20%22%7Bcustom_file_name%7D.%7Bext%7D%22%0A%20%20thumbnail_name%3A%20%22%7Bcustom_file_name%7D.%7Bthumbnail_ext%7D%22%0A%0Aoverrides%3A%0A%20%20snake_cased_title%3A%20%3E-%0A%20%20%20%20%7B%0A%20%20%20%20%20%20%25replace%28%20title%2C%20%27%20%27%2C%20%27_%27%20%29%0A%20%20%20%20%7D%0A%20%20custom_file_name%3A%20%22%7Bupload_date_standardized%7D%20%7Bsnake_cased_title_sanitized%7D%22&type=canonical_yaml>`_.
+Any whitespace within curly-braces is okay since it will be parsed out. This is needed to make
+scripting function usage readable.
+
+.. important::
+
+   It is important to use ``>-`` over other YAML new-line directives like ``>`` because they
+   add newlines before or after curly-braces, and will be included in your variable's output string.
+
+Advanced Scripting
+------------------
+
+Accessing ``info.json`` Fields
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+WIP
+
+Creating Custom Functions
+~~~~~~~~~~~~~~~~~~~~~~~~~
+WIP
+
+Parsing Maps and Arrays
+~~~~~~~~~~~~~~~~~~~~~~~
+WIP

docs/source/config_reference/scripting/scripting_types.rst

@@ -0,0 +1,319 @@
+
+Scripting Types
+===============
+
+Types
+-----
+
+String
+~~~~~~
+
+Strings are a series of characters surrounded by quotes and can be defined in a few ways, including:
+
+.. tab-set::
+
+  .. tab-item:: Literal
+
+    .. code-block:: yaml
+
+      string_variable: "This is a String variable"
+
+  .. tab-item:: In-Line
+
+    .. code-block:: yaml
+
+      string_variable: "{ %string('This is a String variable') }"
+
+  .. tab-item:: Single Quote
+
+    .. code-block:: yaml
+
+      string_variable: >-
+        {
+          %string('This is a String variable')
+        }
+
+  .. tab-item:: Double Quote
+
+    .. code-block:: yaml
+
+      string_variable: >-
+        {
+          %string("This is a String variable")
+        }
+
+  .. tab-item:: Triple Quote
+
+    .. code-block:: yaml
+
+      string_variable: >-
+        {
+          %string('''This is a String variable''')
+        }
+
+  .. tab-item:: Triple-Double Quote
+
+    .. code-block:: yaml
+
+      string_variable: >-
+        {
+          %string("""This is a String variable""")
+        }
+
+.. note::
+
+   For non-String types, they must be defined as parameters to scripting functions. This is because
+   anything in a variable definition that is not within curly-braces gets evaluated as a String.
+
+Integer
+~~~~~~~
+
+Integers are whole numbers with no decimal.
+
+.. tab-set::
+
+  .. tab-item:: Literal
+
+    .. code-block:: yaml
+
+       int_variable: >-
+         {
+           %int(2022)
+         }
+
+  .. tab-item:: In-Line
+
+    .. code-block:: yaml
+
+       int_variable: "{ %int(2022) }"
+
+Float
+~~~~~
+
+Floats are floating-point decimals numbers.
+
+.. tab-set::
+
+  .. tab-item:: Literal
+
+    .. code-block:: yaml
+
+       float_variable: >-
+         {
+           %float(3.14)
+         }
+
+  .. tab-item:: In-Line
+
+    .. code-block:: yaml
+
+       float_variable: "{ %float(3.14) }"
+
+Boolean
+~~~~~~~
+
+A type is considered boolean if it spells out ``True`` or ``False``, case-insensitive.
+
+.. tab-set::
+
+  .. tab-item:: Literal
+
+    .. code-block:: yaml
+
+       bool_variable: >-
+         {
+           %bool(True)
+         }
+
+  .. tab-item:: In-Line
+
+    .. code-block:: yaml
+
+       bool_variable: "{ %bool(FALSE) }"
+
+Array
+~~~~~
+
+An Array contains multiple types of any kind, including nested Arrays and Maps.
+Arrays are defined using brackets (``[ ]``), and are accessed using zero-based indexing.
+
+.. tab-set::
+
+  .. tab-item:: Literal
+
+    .. code-block:: yaml
+
+       array_variable: >-
+         {
+           [
+             "element with index 0",
+             1,
+             2.0,
+             [ "Nested Array 3" ]
+           ]
+         }
+       element_0: >-
+         {
+           %array_at(array_variable, 0)
+         }
+
+  .. tab-item:: In-Line
+
+    .. code-block:: yaml
+
+       array_variable: "{ ['element with index 0', 1, 2.0, ['Nested Array 3' ]] }"
+       element_0: "{ %array_at(array_variable, 0) }"
+
+Map
+~~~
+
+A Map is a key-value store, containing mappings between keys and values.
+Maps are defined using curley-braces (``{ }``), and are accessed using their keys.
+
+.. tab-set::
+
+  .. tab-item:: Literal
+
+    .. code-block:: yaml
+
+       map_variable: >-
+         {
+           {
+             "string_key": "string_value",
+             1: "int_key",
+             "list_value": [ "elem0", 1, 2.0 ]
+           }
+         }
+       string_value: >-
+         {
+           %map_get(map_variable, "string_key")
+         }
+
+  .. tab-item:: In-Line
+
+    .. code-block:: yaml
+
+       map_variable: "{ {'string_key': 'string_value', 1: 'int_key', 'list_value': [ 'elem0', 1, 2.0 ]} }"
+       string_value: "{ %map_get(map_variable, 'string_key') }"
+
+Null
+~~~~
+Null is represented by an empty String, and can be conveyed by spelling out ``null``,
+case-insensitive.
+
+.. tab-set::
+
+  .. tab-item:: Literal
+
+    .. code-block:: yaml
+
+       null_variable: ""
+
+  .. tab-item:: In-Line
+
+    .. code-block:: yaml
+
+      null_variable: "{ %string(null) }"
+
+
+Function Type-Hints
+-------------------
+
+AnyArgument
+~~~~~~~~~~~
+AnyArgument means any of the above Types are valid as input or output to a scripting function.
+
+.. note::
+
+   Strict typing is enforced. For functions that return ``AnyArgument`` need to be casted before
+   passing into functions that expect a particular type.
+
+Numeric
+~~~~~~~
+Numeric refers to either an Integer or Float.
+
+Optional
+~~~~~~~~
+Optional means a particular scripting function argument can be either provided or not included.
+For example, the function
+`map_get <https://ytdl-sub.readthedocs.io/en/latest/config_reference/scripting/scripting_functions.html#map-get>`_
+has an optional default value. Both of these usages are valid:
+
+.. tab-set::
+
+  .. tab-item:: Map Get
+
+    .. code-block:: yaml
+
+       will_throw_key_does_not_exist_error: "{ %map_get( {}, 'key' ) }"
+
+  .. tab-item:: Map Get with Optional Default Value
+
+    .. code-block:: yaml
+
+      will_return_default: "{ %map_get( {}, 'key', 'default value' ) }"
+
+Lambda
+~~~~~~
+Lambda parameters are a reference to a function, and will call that lambda function
+on the input. In this example,
+
+.. code-block:: yaml
+
+   lambda_array_numeric_to_string: >-
+     {
+       %array_apply( [ 1, 2, 3, 4], %string )
+     }
+
+We apply ``%string`` as a lambda function to
+`array_apply <https://ytdl-sub.readthedocs.io/en/latest/config_reference/scripting/scripting_functions.html#array-apply>`_,
+which is called on every element in the input array. The output becomes ``["1", "2", "3", "4"]``.
+
+This example has one input-argument being passed into the lambda. For other lambda-based functions
+like `array_enumerate <https://ytdl-sub.readthedocs.io/en/latest/config_reference/scripting/scripting_functions.html#array-enumerate>`_,
+it expects the lambda function to have two input arguments. These are denoted using
+``LambdaTwo``, ``LambdaThree``, etc within the function spec.
+
+LambdaReduce
+~~~~~~~~~~~~
+LambdaReduce is special type of lambda that reduces an Array to a single value by calling the
+LabmdaReduce function repeatedly on two elements in the Array until it is reduced to a single value.
+
+In this example,
+
+.. code-block:: yaml
+
+   lambda_reduce_sum: >-
+     {
+       %array_reduce( [ 1, 2, 3, 4], %add )
+     }
+
+We call
+`array_reduce <https://ytdl-sub.readthedocs.io/en/latest/config_reference/scripting/scripting_functions.html#array-reduce>`_
+on the input array, using
+`add <https://ytdl-sub.readthedocs.io/en/latest/config_reference/scripting/scripting_functions.html#add>`_
+as the LambdaReduce function. This will reduce the Array to a single value by internally calling
+
+.. code-block::
+
+   - %add(1, 2) = 3
+   - %add(3, 3) = 6
+   - %add(6, 4) = 10
+
+And evaluate to ``10``.
+
+ReturnableArguments
+~~~~~~~~~~~~~~~~~~~
+
+Returnable arguments are used in conditional functions like
+`if <https://ytdl-sub.readthedocs.io/en/latest/config_reference/scripting/scripting_functions.html#if>`_,
+which implies the argument passed into the function is the function's output. For example,
+
+.. code-block:: yaml
+
+   conditional_function: >-
+     {
+       %if( True, "Return this if True", "Return this if False" )
+     }
+
+is going to return ``"Return this if True"`` since the condition parameter is ``True``.