Improve transcode workflows code

[hbomb79]

Transcoder Profiles

Top-Level Overview

Currently, I'm at the stage of development where ffmpeg integration is the next logical step; however, it raises the problems of what exactly the user wants ffmpeg to do. This includes transcoder properties, thread control, etc.

It would be in the best interest of everyone if instead of hardcoding the way that ffmpeg operates, we could instead provide each user with an easy UI to control exactly how ffmpeg operates. Sounds easy right? Well, let's take it one step further.

It would be even better if a set of these profiles could be created, each customized to a particular type of content (1080p, 4k, movie, tv show, etc) and they could be edited on the fly via the web UI, and applied to different types of content manually/automatically.

Profiles

Certain types of content may require different types of transcoding. For instance, perhaps we'd want ffmpeg to target a different resolution depending on the resolution of the source. Perhaps we'd like ffmpeg to output the files to a directory tree if it's a tv-show, but a movie should go to a different file server. Profiles with advanced features such as these could be implemented to service these requirements.

A very rough overview of each major component of a profile can be found below.

Configuration w/ Interpolation

Profiles will require a wealth of options to control exactly how ffmpeg operates. To minimize the complexity for the user, a string interpolation method will be used such that the user is provided a set of keywords, such as %DEFAULT_THREADS%, %TITLE%, %SEASON_NUMBER%, %EPISODE_NUMBER%, %SOURCE_RESOLUTION%.

These variables (and many more) can be inserted into any of the inputs provided when configuring a profile. As an example, perhaps we're specifying the output path of files from a certain profile; we could set it is as "/mnt/my_nas/movies/%TITLE%_%YEAR%/%SOURCE_RESOLUTION%.%EXTENSION%" which would place a 4k render of "Joker 2019" in /mnt/my_nas/movies/Joker_2019/4k.mp4

Of course, the specifics of these examples are irrelevant; it's only to demonstrate that a profile could control fully exactly how ffmpeg operates. Some preset options could be provided (thread count, output directly, export format), but an advanced text input could be provided that simply allows the user to type in the command-line ffmpeg arguments manually (still featuring the above variable interpolation).

Multiple Targets

To accommodate users of slower internet connections, or those of us who don't wish to stream 4k content, the option for a profile to specify multiple 'targets' is a nice-to-have. This would mean, as an example, a movie ingested at 4k may have two or even three ffmpeg targets at different resolutions.

However, at this level of customization, there is only so much TPA can do to assist the end-user. When configuring a target (each profile will have one by default) TPA will simply provide the user with some basic options, and then an advanced text entry for command-line arguments.

While we'd love to be able to provide a discrete option for each and every eventuality, the reality is that ffmpeg is a huge and complex piece of software, and end-users will be better served by TPA if there is full transparency with regards to what ffmpeg is being provided as arguments.

Threading and Blocking

One of the features of TPA is that it's fully parallelized. This means that our profiles will need to have a concept of threads, more specifically, how many they want. To follow on from the above Options, each profile target must be able to specify an explicit thread count as well as whether or not the target is blocking.

For instance, a profile may have 3 targets however the first target should be fully completed before the next targets are started in parallel with each other. To achieve this, target 1 must be the first in the list of the profiles targets, and it should be configured as 'blocking'. Additionally, each target could specify how many threads it wants. Target 1 may want 4 threads, while targets 2 and 3 are happy transcoding more slowly with 1 thread as the target resolution is low.

Applying

These profiles provide a huge degree of flexibility, but what makes them truly useful is that these are not server-level settings. They are individual groups of settings that can be applied to a particular item in the queue. Rather than the server having a concept of these settings, we instead have profiles (one of which is default) that can be selected for an item and all targets inside of that profile will be run against the item.

During the transcoding of this item, the server will lookup the profile that has been selected for this item and retrieve the configuration for the ffmpeg instances (how many, how the threading should be achieved, and what arguments for each ffmpeg instance).

Automatically Applying Profiles

Applying these profiles sounds good on paper, however, TPA is designed to be configured and then left to run with no user intervention. Therefore it's only natural that profiles be fitted with some form of auto-application logic. For instance, a regex match against a title, a specific resolution, perhaps only movies, etc. Each profile in the server's config will be tried for a match against these settings, and the first to match will be applied to the item.

[hbomb79]

So some work has been done against this ticket, but I'm not really very happy with where it's at right now, so I'm kicking this back to selected for dev and intend to have another try and coming up with an intuitive, simple, yet flexible system for managing the above.