Reversed punctuation in RTL subtitles in Kodi 19

[baerlein]

Bug report

Describe the bug

Here is a clear and concise description of what the problem is:

Punctuation is reversed when using Hebrew subtitles in Kodi 19. This is a common issue with RTL (right to left) languages but it used to work properly in Kodi 18.

Expected Behavior

Here is a clear and concise description of what was expected to happen:

Punctuation should be placed at the end of the sentence, see attached screenshot for reference.

Actual Behavior

Punctuation is placed in the beginning of the sentence.

Possible Fix

To Reproduce

Steps to reproduce the behavior:

1. Change language in Player settings to Hebrew, character set to Hebrew (Windows).
2. Place attached subtitle file "Hebrew_subs.srt" in same dir of a video named accordingly (e.g. "Hebrew_subs.mp4").
3. Play the video in Kodi

Debuglog

The debuglog can be found here:
https://paste.kodi.tv/aqunebojuw.kodi

Screenshots

Here are some links or screenshots to help explain the problem:

Reference screenshot:

Additional context or screenshots (if appropriate)

Here is some additional context or explanation that might help:

Sample subtitle file:

Hebrew_subs.zip

Your Environment

Used Operating system:

• Android

• iOS

• Linux

• OSX

• Windows

• Windows UWP

• Operating system version/name: Windows 10 x64, Android

• Kodi version: Kodi 19.0 "Matrix" - Release

note: Once the issue is made we require you to update it with new information or Kodi versions should that be required.
Team Kodi will consider your problem report however, we will not make any promises the problem will be solved.

[baerlein]

This is still an issue with Kodi 19 beta 1.

[nirmv77]

I confirm that it's still in issue in Kodi 19 beta 1.
BTW I'm using kodi on Android.

[EyalRotem]

as of beta 2 this is still an issue. (windows)

[nirmv77]

Hi, this is still an issue in RC1 (Android)

[EyalRotem]

Hi, this is still an issue in RC1 (Android)

Yes, on windows as well

[ntzb]

can someone take a look at this? it's a real problem.

[Obi-Wan-Aloni]

Joining the request to look into this.
Will gladly help on anything I can.

[enen92]

Provide a sample, a screenshot identifying/explaining the problem and explain what should happen instead. I doubt we have any Hebrew, Arabic, etc team members.

[enen92]

nvm, all is in the bug report.

[EyalRotem]

yes, @baerlein actually did a thorough job. so if anything else is needed, let us know and we'll try to help.

it seems that in addition the problem is cross-platform (as @nirmv77 says it's on android as well).
and as mentioned - 18.9 and bellow worked good without the issue that was introduced in 19+

[GuyKh]

It looks the issue here is the RTL alignment.

[Nir-Zigler]

The problem is also in v19 GA. It's a bad regression from v18. hope it will be fixed soon...

[naamapps]

Can also confirm, xiaomi mi box s, kodi 19, Hebrew subtitles have reversed punctuation.

[ztslil]

confirmed - fire tv stick 4k, kodi 19 latest release, subtitles that are working on 18.9, not working anymore.

[LiorAmram]

Confirmed - Nvidia Shield 2019 pro, kodi 19 latest release.

[itrigalo]

confirmed - Xiaomi mi box s, kodi 19, Hebrew subtitles have reversed punctuation

[cred-o]

confirmed - Nvidia shield pro 2019 - Kodi 19 latest release

[EyalRotem]

This is still an issue with Kodi 19 beta 1.

Can you update the issue with the new information?
(Latest Kodi release, Android as well)

[reuvenwittenberg]

Windows 10, reverse punctuation! It's a shame, ther first time KODI got it wrong, and we need it badly!

[EyalRotem]

Windows 10, reverse punctuation! It's a shame, ther first time KODI got it wrong, and we need it badly!

Stuff like this happens, especially when so much has changed (moving to python 3, adding hdr support and more).

I just wish this would be assigned to someone so at least it would be looked over.
No progress ha been made since the bug was reported 🤷🏼‍♂️

[enen92]

From https://forum.videolan.org/viewtopic.php?f=2&t=14063

The punctuation ordering problem is solved by calling fribidi with LTR as the main direction. This is a work-around. Since most players (under Windows) don't perform bidi, but let the system do it, the subtitles are prepared in logical order (RTL), but with main order (LTR), since players don't support setting the main order of the subtitles. This makes authoring subtitles a little harder, but everyone's already doing it.

In case anyone wants to look at this

[EyalRotem]

From https://forum.videolan.org/viewtopic.php?f=2&t=14063

The punctuation ordering problem is solved by calling fribidi with LTR as the main direction. This is a work-around. Since most players (under Windows) don't perform bidi, but let the system do it, the subtitles are prepared in logical order (RTL), but with main order (LTR), since players don't support setting the main order of the subtitles. This makes authoring subtitles a little harder, but everyone's already doing it.

In case anyone wants to look at this

The thing is, enen92, that this is a regression from previous releases. So there was a built in solution before. I wonder what changed..

[enen92]

@EyalRotem most likely a regression from #17790
You fix one, you break another :)

Maybe @AhmedAdelAbouzeid has any idea

[EyalRotem]

@EyalRotem most likely a regression from #17790
You fix one, you break another :)

Maybe @AhmedAdelAbouzeid has any idea

That just might be relevant, in any case- Ahmed probably has more idea about the subject as he already coded in kodi and uses an RTL language.

let’s hope this can be fixed with ease.

another thought- since python 3 is now incorporated- it really changes how strings and Unicode is handled, this might also be a cause.

[AhmedAdelAbouzeid]

@enen92 Thanks for the mention, I'll work on fixing this issue.

[EyalRotem]

Alright!
thank you @AhmedAdelAbouzeid for the quick response !

What version is this going to be on?
I guess now it’s on the nightly, but will 19.1 (for example) incorporate this fix?

thanks again for the fast work 💪🏼

[AhmedAdelAbouzeid]

Happy to help!
The previous PR was against master branch.

I have a PR open against Matrix branch now; after it is merged, the change can be tested in the next Matrix nightly build (tomorrow?)
http://mirrors.kodi.tv/nightlies/windows/win64/Matrix/
I have no idea about release versions though.

[EyalRotem]

In the 12/03/2021 nightly for windows 64, subtitles are fixed!
Waiting for the stable release now!

thanks for the effort @enen92 and @AhmedAdelAbouzeid !

[Shimronfio]

Any updates for a working version for Android TV?
I still got the issue on KODI 19.0 on Nvidia Shield Pro.

[EyalRotem]

Any updates for a working version for Android TV?
I still got the issue on KODI 19.0 on Nvidia Shield Pro.

The fix has been pushed.
So it’s up to the Kodi team- it will be incorporated in the next release.
Or you could maybe install the nightly build if you wish

[ntzb]

Happy to help! The previous PR was against master branch.

I have a PR open against Matrix branch now; after it is merged, the change can be tested in the next Matrix nightly build (tomorrow?) http://mirrors.kodi.tv/nightlies/windows/win64/Matrix/ I have no idea about release versions though.

I see that this code is indeed already inside master branch, but the nightlies of master still show reversed punctuation.
is something else needed to fix this in master?

[scott967]

I once looked at this in regards libass, and after studying it I believe the code is working as designed, and looking at some fansubs it looked like they take it into account when subbing. but I don't remember the details now.