Support for RTL languages

[HamzaAbdullahMoh]

• VSCode Version: 1.5.0
• OS Version: windows 10

I can honestly dare to say that this is the best open source editor for web developers but, the missing of the small things like the Arabic language support is really frustrating!!

@alexandrudima yes I mean to work with source code files containing Arabic or Hebrew.

[iam3yal]

Yeah, I'm in the same boat when it comes to Hebrew., even though I don't use Hebrew much in my daily day job, especially when it comes to development but this can come in handy at times so 👍 from me.

[alexdima]

@HamzaAbdullahMoh @eyalsk Do you mean you'd want VSCode to be translated to Arabic or Hebrew or do you mean that you cannot work with source code files containing Arabic or Hebrew ?

[iam3yal]

@alexandrudima Well, I'll speak for myself, I refer to the capability that allows people to write from right-to-left as opposed to left-to-right.

Translation isn't needed, I don't use the Hebrew translation at all in any program that I use so personally it's less useful for me but maybe some people would want that too.

[elkebirmed]

I think @HamzaAbdullahMoh want to see an RTL vscode editor as an option, and this will be so helpful

This's the needed information i think 😄

[kazemihabib]

👍

[elkebirmed]

I think this should be labeled as a requested feaure

[HamzaAbdullahMoh]

@alexandrudima @elkebirmed @kazemihabib @eyalsk @ramya-rao-a
First of all, a huge thank you for all of you.
What I really meant is that if I can edit the text in the code itself not the interface, because half of my time I develop sites which they include Arabic texts.
I am sure some of you had to write some rtl text and find it hard with most of text editors!!

[atlanteh]

Just to clarify, the problem is that RTL words get messed up:
rtlB rtlA looks like rtlA rtlB
rtlD rtlC engA engB rtlB rtlA looks like rtlA rtlB engA engB rtlC rtlD

[iddan]

To clarify the problem: when typing in an RTL language in VSCode there are two notable problems:

• It should be aligned to the right while it aligns to the left
• The cursor should be in the left edge of the line while it is in the right

Google Docs is a good example for a product that solved the problems listed above.

Unlike Sublime in VSCode there is no problem of chars in the wrong order.

[kookma]

Yes, Persian, Arabic and Hebrew are right-to-left languages. The editor should support BIDIRECTIONAL (BIDI) texts to lets RTL language users develop their code inside VSCode editor.
Most of the time we work with mixed of English -Persian or in better words mixed of RTL and LTR texts.
Microsoft Word, Google Docs support BIDI feature and allow mixed language typing.

This feature is very useful for a crowd users around the world with RTL language.

[Parsoa]

It is really frustrating that many otherwise great tools (Atom, VS Code, Sublime, you name it!) lack bidirectional editing capabilities. I usually end up writing each word in a separate line when working with HTML that includes Persian.

[athamzaabdullah]

It is been almost two years since we discussed and requested this feature, yet, nothing had changed. I still suffer a great deal when I venture to edit Arabic.

Nonetheless, I tried PHPStorm editor and it astonished me in how it supports RTL languages!!.

I wonder is it still that hard to do the same in VSCode or are we trying to solve geopolitical conflict here!!?!?

[iddan]

I feel your pain, but this is an open source project so anyone can open a PR that fixes this. For the great majority of editors RTL hasn't been top priority and I guess that's the same situation with VSCode.

[athamzaabdullah]

Why it is not top priority??!

[iam3yal]

@iddan That's actually far-fetched, I haven't looked into the code but I think that before anyone would even consider sending a PR the core team needs to have a plan for this so people would know what components/things require work.

[iddan]

An undirected PR is better than no PR. The VSCode team would probably have to be involved in the process, not necessarily the first commit.

[urielha]

FYI, I found an earlier version of this issue (containing less data) #4994

[marktani]

I agree @iddan. Can you, or someone else, offer advice on where in the code base to start looking? I'm interested in helping with the implementation for this feature.

[babakfp]

Setting a secondary font does not solve the issue - there are fonts that provide support for both RTL and LTR languages, but are far better on what character set than the other. So a secondary font won't be used.

You need to get the Latin/English only version of that font then.

[sarmadka]

If the RTL font you're looking for is Arabic then you can try Alusus Mono (https://alusus.org/fonts/AlususMono.otf). This font has both Arabic and English characters.

[munael]

@sarmadka Is there some public info about this font (family?)?

And why is it recommended for English/Arabic combined workspaces? I wouldn't have expected a font to help fix VSCode's RTL issues, so I must be missing something 👀

[sarmadka]

@munael This was in response to the messages above about fonts. It has nothing to do with the RTL issue itself.

[dushkin]

Is there a solution allready?

[iAhmadKarmola]

Important: i posted solution 2 in the comments below.

Solution 1:
Hi, based on @YoniChechik answer:

• Install Custom CSS and JS Loader extension.
• Create a CSS file and save it somewhere on your computer.
• Put the following codes in the CSS file:

/* For Farsi/Arabic text (RTL languages) in codes */
[style="unicode-bidi:isolate"] {
direction: rtl;
}

/* For Farsi/Arabic text (RTL languages) in codes and comments */
[style="unicode-bidi:isolate"], [style*="unicode-bidi: isolate"] {
direction: rtl;
}

• In vs code, press ctrl + shift + p and type Open User Settings (JSON) and add the following code to the settings.json file:

"vscode_custom_css.imports": ["file:///D:/vs-code/vs-code-rtl-support.css"]

• In vs code, press ctrl + shift + p and type Reload Custom CSS and JS and restart vs code.
• Done.

Note: If you want to disable RTL, In vs code, press ctrl + shift + p and type Disable Custom CSS and JS, then restart vs code.

[dushkin]

Thank you @iAhmadKarmola .
But still - is there a way to have the file direction from right to left?
The page direction.

[iAhmadKarmola]

Hi @dushkin , add the following codes to the CSS file:

.view-lines { direction: rtl; text-align: right; }

Then in vs code, press ctrl + shift + p and type Reload Custom CSS and JS and restart vs code.

[MasterInQuestion]

    Friendly notice:
    Your CSS is probably not working: unclosed comment marking pairs `/* ... */`.

    And the `[style="unicode-bidi:isolate"]` implementation seems brittle: no guarantee for the whitespace after ":".
    See also: https://developer.mozilla.org/en-US/docs/Web/CSS/Attribute_selectors#syntax

    The more appropriate way would probably be:
[[

/*
	.view-line [dir="ltr"] { direction: rtl; }
	.view-line [style*="unicode-bidi"] { unicode-bidi: normal; }
*/
/* ^ Seemingly wreaking havoc. */
	.view-line [style*="unicode-bidi"] { direction: rtl; }

]]
    After some review, the mentioned extension should also support data URI.
    Thus the above may be simplified as: `"vscode_custom_css.imports": ["data:,.view-line%20[style*='unicode-bidi']{direction:rtl}"]`

    Testing text:
[[

// Testing the RTL text in VS Code.
/* Testing the RTL text in VS Code. */
const text1 = 'Testing the RTL text in VS Code.';

// تست متن RTL در VS Code.
/* تست متن RTL در VS Code. */
const text2 = 'تست متن RTL در VS Code.';

]]

[iAhmadKarmola]

@alexdima

Solution 2:
Hi guys, in the previous solution, there were the following problems:

• Codes and comments were not selected correctly.
• When using the Grave Accent character (`) to format texts, the Grave Accent character was not placed at the beginning and end of the text.
• The text direction of non-RTL languages was reversed.

In this solution, the above problems are solved:

• Install Custom CSS and JS Loader extension.
• Create a CSS file and save it somewhere on your computer.
• Put the following codes in the CSS file:

/* RTL support - For Farsi/Arabic text (RTL languages) in codes and comments */
.view-line span[dir="ltr"] [style*="unicode-bidi:isolate"] {
    direction: rtl;
}

.view-line span[dir="ltr"] [style*="unicode-bidi:isolate"]:before,
.view-line span[dir="ltr"] [style*="unicode-bidi:isolate"]:after {
    content: "\200f";
}

/* change font-style from italic to normal */
.view-line span[dir="ltr"] span:has(+ [style*="unicode-bidi:isolate"]),
.view-line span[dir="ltr"] [style*="unicode-bidi:isolate"]+span,
.view-line span:not([dir="ltr"]) span,
.auto-closed-character {
    font-style: normal;
}

• In vs code, press ctrl + shift + p and type Open User Settings (JSON) and add the following code to the settings.json file:

"vscode_custom_css.imports": ["file:///D:/vs-code/vs-code-rtl-support.css"],
"editor.tokenColorCustomizations": {
        "textMateRules": [
            {
                "scope": ["punctuation.definition.string.template.begin.js", "punctuation.definition.string.template.end.js"],
                "settings": {
                    "fontStyle": "italic"
                }
            },
            {
                "scope": ["punctuation.definition.string.begin.js", "punctuation.definition.string.end.js"],
                "settings": {
                    "fontStyle": "italic"
                }
            }
        ]
    },

• In vs code, press ctrl + shift + p and type Reload Custom CSS and JS and restart vs code.
• Done.

Note: If you want to disable RTL, In vs code, press ctrl + shift + p and type Disable Custom CSS and JS, then restart vs code.

[MasterInQuestion]

    I think however it's implemented it would be broken... one way or another.

    Significant portion of your code appears no-op:
    |1| Right-to-Left Mark insertion seems meaningless. [ See: https://unicode.org/reports/tr9/#Implicit_Directional_Marks ]
    |2| The purpose of meddling "fontStyle"?

    Overall, it appears no essentially different from your previous solution.

    ----

    [ Quote iAhmadKarmola @ CE 2023-11-21 00:30:08 UTC:
https://github.com/microsoft/vscode/issues/11770#issuecomment-1820021849
    I had to change something about the characters `'` and `"` and "`": so that these characters are placed in a separate "<span>" and are not placed together with the RTL texts in the same "<span>". ]
<^>    Based on your description I created below sample HTML:
[[

<meta charset="UTF-8" />
<body style="font-family: monospace">
	<span dir="ltr">t0 = <span dir="rtl" style="unicode-bidi: isolate">`تست متن RTL در VS Code.'</span></span><br/>
	<span dir="ltr">x0 = <span>`</span><span dir="rtl" style="unicode-bidi: isolate">تست متن RTL در VS Code.</span><span>'</span></span><br/>
	<span dir="ltr">x1 = <span>`</span>&rlm;<span dir="rtl" style="unicode-bidi: isolate">تست متن RTL در VS Code.</span><span>'</span></span><br/>
	<span dir="ltr">x2 = <span>`</span><span dir="rtl" style="unicode-bidi: isolate">تست متن RTL در VS Code.</span>&rlm;<span>'</span></span><br/>
	<span dir="ltr">x3 = <span>`</span>&rlm;<span dir="rtl" style="unicode-bidi: isolate">تست متن RTL در VS Code.</span>&rlm;<span>'</span></span><br/>
	--------
<pre style="white-space: pre-wrap; word-wrap: break-word; tab-size: 4">
	t0: Original
	x0: Separated quotes
	x1: Prefix RLM
	x2: Postfix RLM
	x3: Both
</pre>

]]
    And I noted the quote problem: probably you're right..?
    But such BiDi texts' directionality remains undeterministic whatsoever...

    Regardless, I didn't notice the purpose of the Right-to-Left Mark insertion: seemingly no difference.

    ----

    Derived from the sample:
[[

<meta charset="UTF-8" />
<body style="font-family: monospace">
	<span dir="ltr"><span>//</span><span dir="rtl" style="unicode-bidi: isolate"> تست متن Ridiculously Long RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRTL در VS Code.</span></span><br/>
	<span dir="ltr"><span>// </span><span dir="rtl" style="unicode-bidi: isolate">تست متن Ridiculously Long RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRTL در VS Code.</span></span><br/>
	<span dir="ltr">// <span dir="rtl" style="unicode-bidi: isolate">تست متن Ridiculously Long RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRTL در VS Code.</span></span><br/>

]]
    Potentially more acceptable? (if the whitespace characters hinting problem could be solved)
    [ Would tweaking "editor.experimentalWhitespaceRendering" (Editor: Experimental Whitespace Rendering) work? ]
    [ ^ Differs, but... helps not. ]
    But what characters to isolate appears much unsolvable... [ Condensing which to the simplest form, it would be: "a.b" or "b.a"? ]
    .
    For reference: https://www.unicode.org/charts/nameslist/n_0000.html#0021

[iAhmadKarmola]

• This solution works for Farsi/Arabic language, it is enough for us.
• These codes are just CSS and are not dangerous for you.

In the Open User Settings (JSON) file, I had to change something about the characters ' and " and ` so that these characters are placed in a separate span and are not placed together with the RTL texts in the same span, and with CSS i can set style only for the texts RTL.

If you think this solution is no different than solution 1, it's because you haven't noticed the problems I mentioned in solution 2.
You are free to use any solution.
However, those problems are fixed with solution 2 and this solution works like RTL support in PHPStorm.

Note: Still waiting for the VS Code team to fix this simple problem.

Hi @alexdima , in VS Code, you have recognized Farsi/Arabic texts by adding unicode-bidi:isolate to spans. Why don't you give direction to these spans? Why don't you put the characters ' and " and ` in separate spans? so that we don't have problems when selecting the text after applying the direction

[MasterInQuestion]

    Thanks for your effort.

    I think the fundamental problem is Unicode has such RTL characters' implementation de facto broken.
    Effectively rendered these languages unfit for General Information Exchange.

    "These codes are just CSS and are not dangerous for you."
<^>    False, but not much relevant.
    Knowing why it works matters. Lest caught us into shotgun debugging style development...


    The problem is really not simple. (or simply havoc, if must be...)

    “Why don't give direction to these "<span>"s?”
<^>    This is why:

    See also: https://github.com/microsoft/vscode/issues/169459#issuecomment-1362023309
    .
    Well precisely the issue is not caused by the change: it exists without which...

    I'm not quite sure what your selection problems are.
    For me the current implementation involving RTL characters... tricky selection has always been the problems. (and actually just a minor one)

[MasterInQuestion]

    After reading much of [ https://unicode.org/reports/tr9/ ], I worked out a plausible workaround:
    Wrapping all RTL character sequences with "‎" (Left-to-Right Mark) (U+200E, 0xE2808E, &#8206;). [1]

[[

<meta charset="UTF-8" />
<pre style="white-space: pre-wrap; word-wrap: break-word; tab-size: 4">
	`تست متن RTL در VS Code.'
	`‎تست ‎متن RTL ‎در VS Code.'
	`تست‎ متن‎ RTL در‎ VS Code.'
	`‎تست‎ ‎متن‎ RTL ‎در‎ VS Code.'
</pre>

]]
    At least it resolves much of the general interoperability issue. (no more BiDi disorder) [2]
    Though editing is still much tricky.

    And which may look quite ugly in VS Code...

[ [1]
    Pseudo Perl RegEx matching all irrelevant LRM:
    /(?<!(?&RTL))\x{200E}(?!(?&RTL))|(?<=(?&RTL))\x{200E}(?=(?&RTL))(?(DEFINE)
    (?<RTL>[[:Bidi_Class=/^(?:AL|R)$/:]&[:General_Category=/^[^CZ]$/:]]) )/g

    So few implementations actually support such...
    See:
    |1| https://unicode-org.github.io/icu/userguide/strings/unicodeset.html
    |2| https://www.unicode.org/reports/tr44/#GC_Values_Table
    |3| https://util.unicode.org/UnicodeJsps/properties.jsp?a=Bidi_Class
    |4| https://util.unicode.org/UnicodeJsps/list-unicodeset.jsp?a=%5B%5B:Bidi_Class=/^(?:AL|R)%24/:%5D%26%5B:General_Category=/^%5B^CZ%5D%24/:%5D%5D
    ; for reference.

    The so messily defined "Bidi_Class"...
    Right-To-Left doesn't include Arabic Letter..?

    See also [ https://github.com/MasterInQuestion/talk/discussions/2#discussioncomment-7924135 ] for applicable workaround. ]

[ [2]
    Note things are unsalvageably broken in the 2nd last shell case...
    Not only for the lack of support for contextual shaping. And the notoriously wrong rendering due to BiDi disorder. ]

[cben]

The so messily defined "Bidi_Class"... Right-To-Left doesn't include Arabic Letter..?

U+0646 ARABIC LETTER NOON
UTF-8: d9 86 UTF-16BE: 0646 Decimal: ن Octal: \03106
ن
Category: Lo (Letter, Other); East Asian width: N (neutral)
Bidi: AL (Right-to-Left Arabic)

There is a separate bidi category AL (Right-to-Left Arabic) — see Table 4. Bidirectional Character Types
(the name "AL" despite being RTL is confusing, but I guess here it means "Arabic Letter" as opposed to "AN Arabic Number")

See also https://www.unicode.org/Public/UCD/latest/ucd/extracted/DerivedBidiClass.txt file listing ranges and their classes, followed by full lists of characters in each Bidi_Class=...

---

If I understand correctly, the only purpose of separating AL is in [3.3.4 Resolving Weak Types to re-label European Number (e.g. regular ascii digits) occuring near Arabic text as Arabic Number; right after that, it does:

W3. Change all ALs to R.

...so for all later purposes, AL behaves identical to R.

---

And afaict the only purpose of separating AN from EN was to fine-tune the behavior of numbers with separators/terminators per language conventions.
A thousands separator like comma belongs to Common Separator class so "123,456" will be treated as a single number in all languages (though cultures may disagree whether that's a decimal point or group separator);
however these European Separators will not form a single AN number:

# Bidi_Class=European_Separator

002B          ; ES # Sm       PLUS SIGN
002D          ; ES # Pd       HYPHEN-MINUS
207A..207B    ; ES # Sm   [2] SUPERSCRIPT PLUS SIGN..SUPERSCRIPT MINUS
208A..208B    ; ES # Sm   [2] SUBSCRIPT PLUS SIGN..SUBSCRIPT MINUS
2212          ; ES # Sm       MINUS SIGN
FB29          ; ES # Sm       HEBREW LETTER ALTERNATIVE PLUS SIGN
FE62          ; ES # Sm       SMALL PLUS SIGN
FE63          ; ES # Pd       SMALL HYPHEN-MINUS
FF0B          ; ES # Sm       FULLWIDTH PLUS SIGN
FF0D          ; ES # Pd       FULLWIDTH HYPHEN-MINUS

(By "single number" I don't mean semantically one number; clearly addition and subtraction involves separate numbers, and hyphen-minus may express either subtraction or range... I just mean the bidi algorithm will join them to one sequence of EN-class characters.)

=> Together these rules allow e.g. the same ASCII 1-9 and 50% in following 2 examples to render in different order, due to R vs. AL affecting the number type:

• א 1-9 ל 50% ת
• تست 1-9 متن 50% در

If the digits were explicitly AN, then surrounding letters would not matter:

• א ١-٩ ל ٥٠٪ ת
• א ١-٩ ל ٥٠% ת

The case of ٪ U+066A ARABIC PERCENT SIGN is funny as it's also an European_Terminator, same as ASCII % U+0025 PERCENT SIGN!? Ah I see, there is no Arabic terminator class. But IIUC the point is not that these terminators are culturally European — clearly ٪ isn't — but just that they'll "stick" to EN (step W5) but won't stick to AN, in that case they revert to neutral (step W6), hence the percents in the latter three examples stayed after the number in sentence order.

[disclaimers: I don't speak Arabic, TR9 is pretty complex and I may be speaking nonsense :-]

[MasterInQuestion]

    So overly complex (and likely entirely wrong from the very beginning), that's how it had things 3D broken:
    Differently broken in Different implementations in Different manner...

    Several countries have switched the alphabet to workaround the issue.

[sarmadka]

So overly complex (and likely entirely wrong from the very beginning), that's how it had things 3D broken: Differently broken in Different implementations in Different manner...

Several countries have switched the alphabet to workaround the issue.

What?!! Countries switch alphabets for political reasons, not for technical reasons, especially when it's been solved over and over again. Bidi works perfectly fine in Firefox, Chrome, Internet Explorer, Edge, Safari, Office, LibreOffice, GEdit, Kate, Notepad++, Gnome Builder, ......
The list of apps that got bidi correctly is very long. It's not complex if you are a good programmer.

[MasterInQuestion]

    The technical cause contributes.
    Please trace the relevant posts first.

[Hamcker]

It's 2024 and our Farsi scripting language is about to release RC version, and still, there is no support for RTL languages.

[MasterInQuestion]

    I tried to address the issue... But really couldn't figure out how.
    See: https://github.com/MasterInQuestion/talk/discussions/2

[jgmention]

we have also strange behaviour with the editor and arabic language.
Issue: Arabic keywords with hashtags pasted into the query will flip the hashtag symbol to the start of the keyword instead of the end. Spacing is incorrect, and attempting to modify the keywords in the code will not edit where the cursor header is positioned

STEPS TO RECREATE

1. copy and paste the following hashtag into the editor: #اليوم_العالمي_للتصلب_المتعدد the hashtag will move to the opposite side of the keyword.

2. When you copy the keyword, it will add extra spacing.

[sarmadka]

It's 2024 and our Farsi scripting language is about to release RC version, and still, there is no support for RTL languages.

@Hamcker We were also waiting for VS Code RTL support while working on Alusus programming language (which has support for RTL scripting), but eventually we gave up and ended up forking Gnome Builder instead. Reach out if you want to give that a try, it's not hard to add Farsi localization and support for your new scripting language. Not as powerful as VS Code, but better than nothing.

[mohmmadView]

#11770 (comment)

I did this method, there was no problem in js, but there was a problem with jsx format and html Do you know the problem, why it shows the codes like this in the editor?

I would be grateful if anyone could help

)

[MasterInQuestion]

    iAhmadKarmola's design likely considered the JS only.

    The direct cause for your problem is:
    The regular texts and the XML tag delimiters' presentation are not separated per the highlighter's design. (i.e. they are put in the same HTML tag)
    Thus the "direction: rtl" would apparently break things.
    .
    Similar should also occur if you just disable syntax highlighting.

    Eventually the unsolvable part in [ https://github.com/microsoft/vscode/issues/11770#issuecomment-1819265075 ].

[mohmmadView]

@MasterInQuestion Yes, exactly

[hesami]

It would be greatly beneficial if support for right-to-left (RTL) text orientation could be added to VSCode, particularly for languages such as Persian and Arabic. While the inclusion of these languages is not essential, ensuring that text in these languages displays correctly, especially in chatbot interfaces like Copilot AI Chat, would vastly improve the user experience for many developers.
Additionally, I suggest that VSCode should automatically detect the text direction and set it accordingly without manual configuration. This would enhance convenience for users writing in RTL languages.

[MasterInQuestion]

    Bi-Disorder unsolvable:

    "‎این‎ ‎یک‎ ‎متن‎ ‎نمونه‎ ‎به‎ ‎زبان‎ ‎فارسی‎ ‎است‎ - ‎بدی‎-‎راستچین‎ ‎متن‎"