Pickup existing subtitle country code better

[Rouzax]

So here is what I'm trying to accomplish.
I download a lot of 1080p WEB-DL episodes and most of the times there are no good English subtitles available from the existing services. Most of the time the episode will have embedded subs but those contain Hearing Impaired additions.

I have been tinkering with https://github.com/SubtitleEdit/subtitleedit to automatically export those subs and strip them of Hearing Impaired and other issues.
It works with a simple command like SubtitleEdit /convert *.mkv subrip /inputfolder:"C:\TEMP" /removetextforhi /fixcommonerrors
That will give me an output of subtitles from the episode all cleaned up and all HI removed.
Like this:

The problem when importing these files with Medusa it does not understand the country codes that are in the files and will only import the last one and completely omit the country code.
This is the output of the post processor:

Processing path: C:\DATA\Private\Torrent\PROCD\Unsorted
Processing folder: C:\DATA\Private\Torrent\PROCD\Unsorted
Post-processing files: [u'The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife).mkv', u'The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)_3_eng.srt', u'The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)_4_dut.srt', u'The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)_5_spa.srt', u'The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)_6_fre.srt', u'The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)_7_ger.srt']
Post-processing video files: [u'The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife).mkv']
Processing C:\DATA\Private\Torrent\PROCD\Unsorted\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife).mkv
Show information parsed from file name
Retrieving episode object for The Good Place S02E10
The episode file has a quality in it, using that: 1080p WEB-DL
Snatch in history: False
Manually snatched: False
Info hash: None
NZB: False
Current quality: 1080p WEB-DL
New quality: 1080p WEB-DL
Proper: False
This episode is a priority download: True
New file: C:\DATA\Private\Torrent\PROCD\Unsorted\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife).mkv
New size: 605.87 MB
There is no existing file so there's no worries about replacing it
This download is marked a priority download so I'm going to replace an existing file if I find one
No associated files were found for C:\DATA\TVSeries\The Good Place\Season 02\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife).mkv
Found release name The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)
Destination folder for this episode: C:\DATA\TVSeries\The Good Place\Season 02
Found the following associated files for C:\DATA\Private\Torrent\PROCD\Unsorted\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife).mkv: set([u'C:\\DATA\\Private\\Torrent\\PROCD\\Unsorted\\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)_5_spa.srt', u'C:\\DATA\\Private\\Torrent\\PROCD\\Unsorted\\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)_4_dut.srt', u'C:\\DATA\\Private\\Torrent\\PROCD\\Unsorted\\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)_3_eng.srt', u'C:\\DATA\\Private\\Torrent\\PROCD\\Unsorted\\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)_6_fre.srt', u'C:\\DATA\\Private\\Torrent\\PROCD\\Unsorted\\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)_7_ger.srt'])
Copying file from C:\DATA\Private\Torrent\PROCD\Unsorted\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife).mkv to C:\DATA\TVSeries\The Good Place\Season 02\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife).mkv
Copying file from C:\DATA\Private\Torrent\PROCD\Unsorted\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)_5_spa.srt to C:\DATA\TVSeries\The Good Place\Season 02\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife).srt
Copying file from C:\DATA\Private\Torrent\PROCD\Unsorted\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)_3_eng.srt to C:\DATA\TVSeries\The Good Place\Season 02\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife).srt
Copying file from C:\DATA\Private\Torrent\PROCD\Unsorted\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)_4_dut.srt to C:\DATA\TVSeries\The Good Place\Season 02\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife).srt
Copying file from C:\DATA\Private\Torrent\PROCD\Unsorted\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)_6_fre.srt to C:\DATA\TVSeries\The Good Place\Season 02\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife).srt
Copying file from C:\DATA\Private\Torrent\PROCD\Unsorted\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife)_7_ger.srt to C:\DATA\TVSeries\The Good Place\Season 02\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife).srt
Processing succeeded for C:\DATA\Private\Torrent\PROCD\Unsorted\The Good Place - S02E10 - Rhonda, Diana, Jake, and Trent (1080p.WEB-DL-strife).mkv
Successfully processed. 

This is what I end up with

Subtitle Settings:

Post Process Settings

[medariox]

Replace the last _ with . and you should be good.

[Rouzax]

I don't have control over the naming output but that can be solved with an additional rename script.
After changing the name it seems to work but it is not removing the undesired languages.
I do have the setting enabled to delete unwanted languages

[fernandog]

Do you have postponed feature enabled?

[Rouzax]

No because I only initiate manual post process using the API.

[fernandog]

As per your screenshots that's feature is only for postponed feature

[Rouzax]

Could you elaborate a bit, I cant figure out how that would work or what the use case is.

[fernandog]

the feature to delete unwanted subtitles, only works when feature postpone PP when no subtitles feature is enabled

[Rouzax]

Why are those linked to each other? Would it not make sense to be able to use that option standalone?

I can think of 3 scenarios:

1. Video file with no subtitles
2. Video file with 1 external subtitle
3. Video file with multiple external subtitles

With these two options linked scenario 1 would never post process.

[fernandog]

You could ask @medariox to use the function def delete_unwanted_subtitles(dirpath, filename): in normal PP

[Rouzax]

@fernandog, sorry I do have Postpone post processing enabled.

[medariox]

That's the setting @fernandog means:

I think it's exactly what you are looking for as far as I could understand.

[fernandog]

So it's no replacing because the last "_"

[Rouzax]

The _ issue has been resolved, the creator of SubtitleEdit has changed his output to . instead of _ so no the only thing is that I want Medussa to ignore and remove unwanted subtitle languages.
So my flow is now as follows.

1. I download a WEB-DL from torrent.
2. After the download completes I have a script that will unpack torrents and put them in a temporary process location. When all the files are unpacked the script will extract all subtitles and fix Common Errors and remove HI
3. When that is all done the same script will call the Medusa API to start Post Processing from the temporary process location.

Hope that it better explains my process and wishes.

[medariox]

What doesn't work for you with Postpone if no subtitle + Delete unwanted subtitles features enabled?

[Rouzax]

What would happen if I have those two options enabled and there are no subtitles at all?
Since my Post Process only runs once as I call the API for the temporary folder where the episode is unpacked.

[fernandog]

Please send the API command with all params used.

[Rouzax]

params = {'cmd': "postprocess", 'force_replace': 1, 'delete_files': 1, 'path': dir_to_process, 'return_data': 1, 'process_method': "copy", 'type': "manual"}

[fernandog]

I don't know if this API command bypass the postpone code and go directly to the PP code.
I can check monday.

[medariox]

It doesn't bypass it afaik.

@Rouzax
It won't process the episode until there is at least one subtitle you want.

[Rouzax]

In my case I don't want that, If there are no subtitles it should process normally and if there are it should ignore/remove unwanted languages.
If it won't post process without subtitles it could hold up my post processing indefinitely or skip the episode in question depending on the way Medusa works.
I have shows that don't get any subtitles as they are for instance in my native language.

[fernandog]

Disable postpone feature then

[medariox]

@Rouzax
You can disable subtitles for those shows, so they will be post-processed even without subtitles.

[Rouzax]

@fernandog 😄 but than it won't delete unwanted subtitle languages.
@medariox What would happen if I would call the API to Post Process but there are no subtitles. Would it hang in the API call until it would find one or would the API call be ended without actually Post Processing the episode?

I'm sorry for annoying you guys with this.
On a side note, it is picking up the subtitles now but it is not renaming the 3 letter country code to a 2 letter country code that is the default for Medusa.

[medariox]

@Rouzax
The API call would be ended without actually post-processing the episode.
No problem 😃

[Rouzax]

Than in my situation it would never pick up the episode again as I do a direct call the the temporary folder of the episode, not the root folder.

[Rouzax]

@medariox I'll try to explain a bit more about my download flow.

1. Medusa will snatch an episode and send it to qBittorrent.
2. qBittorrent will start the download in: C:\torrent\download
3. After qBittorrent has finished the download it kicks off a script
4. Only one instance of the script is allowed to run, any other downloads that finish will queue their script to start after the first one finishes
5. The script will extract rar files if they exist and copy files to: C:\torrent\process\episode_name
6. The script will than call SubtitleEdit in the folder C:\torrent\process\episode_name to extract and cleanup subtitles
7. After that the script will call the Medusa API to start post processing from: C:\torrent\process\episode_name
8. When Medusa finishes the script will finish.

If Post Processing of an episode will just exit without doing anything because the correct subtitle are not yet available then the episode in question will just be sitting in C:\torrent\process\episode_name because the API is never called again with that folder.

Hope it makes a bit more sense.

[medariox]

What you need to do is calling C:\torrent\process as post-processing folder instead of C:\torrent\process\episode_name. That will make sure you try to post-process all episodes every time.

[Rouzax]

The problem with that is that it is possible that a copy is starting to that location so a file may be in use.

Can give it a try to see happens.

[Rouzax]

Just tested it out with the settings as described Postpone if no subtitle + Delete unwanted subtitles but I still end up with all the subtitles in the show folder.
Logs: https://pastebin.com/v58bzCpM

Post Process folder

Show folder end result

[Rouzax]

@medariox: if I may bump this one?