Translation into other languages

[kkapsner]

Which languages should we support?

I think German and English are given - anything else?

Should we include all quote rules we know of? (A quick search offered this)
Or only German and Englisch? I think French is also commonly used.

[theAlinP]

I will translate it into Romanian.

Let's include as many as we can. I noticed that only the beginning quotes used in the German language are identical with the ones used in Romanian, so maybe we could let the user choose the beginning and end quotes.

[kkapsner]

My idea in https://github.com/spodermenpls/smartquotes/blob/kkapsner-dev/compose/rules.js is to have different rules which are determined by the UI language and can also be change manually (currently it's only per composer window and not yet a global setting, but we could change that).

Maybe a "custom" quote rule there where the user can provide the beginning and end quotes in a setting page can be a good idea. For all the folks that are not German, English or Romanian.

[jobisoft]

try to get a free account at crowdin.com - they make it free for open source. The crowdin interface to github makes this sooooo easy.

[kkapsner]

Found this - there are very unfamiliar quotation marks... I have never seen the Japanese ones.

[spodermenpls]

@jobisoft There's no trying needed, I already have one. 😅 I'll set up the project there with its source strings as soon as possible.

Choosing between different writing languages and their quotation systems (typographic, guillemots..) isn't the same as localizing the UI texts though, so those should be handled / work independently from each other.

[spodermenpls]

I was about to fill out the "open-source application form" over there, the conditions however say that one has to have worked on it for at least three months.. with now having barely 1 month of time on this repo (and no publishment on ATN, yet), a request right now may be getting declined, it's probably better to do it the oldschool way (manually / PR) in the time being.

[jobisoft]

Why not have the different quote styles per language in the localization files?

[spodermenpls]

Because one can be native to one language and write in multiple other ones, deviating from the native tongue?

[jobisoft]

ah, there could be more/other quote rules included than actual languages prepared for UI. got it.

[kkapsner]

@jobisoft: It there an API to get the current dictionary language for the composer window? I think it would be neat to have a mode where the quote rules are automatically changed upon the selected spell checking language.

[kkapsner]

with its source strings as soon as possible.

You do not have to set up the source strings. The Github interface can do that automatically (that's what I do for CB).

it's probably better to do it the oldschool way (manually / PR) in the time being.

For CB I created a translate.js script that helped with that. I will copy it to this project.

[spodermenpls]

I also found this site: https://smartquotesforsmartpeople.com/ , implementing support for single openings/closings might be worth consideration too.

[kkapsner]

They are already implemented for English. I think they are not used in German...

[spodermenpls]

The Crowdin open-source license was granted after all. I contacted the support since I still had the "14 day trial" (with deletion of my whole account after that) dangling over my account after creating the project, so I mentioned the insufficent age of the project as the reason of my reluctance to send the request and my decision to delete the whole thing again. The conversationist replied to that with "I guess we can make a little exception in terms [of] 3-months rules 😉".
So I did apply today, after setting it up once again, and after mere minutes the request was approved. Getting Crowdin to do exactly what I want in regards to this repo is still a bit complex for me, but I'll get used to it, I am sure.

I also put the first "release" together, so it's a bit easier for outsiders to install the extension (which already achieved parity with the original "Smart Quotes").

[spodermenpls]

@jobisoft: Is there an API to get the current dictionary language for the composer window?

@jobisoft Is there such an API, or not? I'd like to know, too, automatic detection would be nice.

@kkapsner Did you implement 'them', or were they already shipped by TB? In German, the typographic variant as ‚ ‘ is in use.

[jobisoft]

Currently not, added it to my list

[spodermenpls]

@kkapsner My question above answered itself after I took a look at the rules.js, following your "typographic quotation mark" template, I've added support for the single ‚ ‘ ones. They work well already, but maybe you can take a quick glance at my commit, to make sure I did no mistakes.

Edit: Damn, I've realized that the 'closing' quotation mark is tilted in the wrong direction, when it's supposed to be used as an apostrophe: https://typefacts.com/artikel/apostroph (described there as an "InDesign-Apostrophe"). Is there somehow a possibility to tell "Smart Quotes" it's supposed to inject the "u2019" symbol, when typing directly after or in between letters?

[kkapsner]

My question above answered itself after I took a look at the rules.js, following your "typographic quotation mark" template,

Great. That was part of the reason why I made this a separate file: to have an easy to understand way to add new/change rules. I wonder if I should make it less verbose. At the moment the code for the different rules only differs in the Unicode characters that are outputted (I think en should also have the RegExp-check). Would make it even easier to add "standard" rules.

I've added support for the single ‚ ‘ ones. They work well already, but maybe you can take a quick glance at my commit, to make sure I did no mistakes.

Looks fine - as long as the Unicode characters are correct... ;)

Is there somehow a possibility to tell "Smart Quotes" it's supposed to inject the "u2019" symbol, when typing directly after or in between letters?

Yes there is. It may be a little bit complicated but I think we can define a rule for that. Can you formulate the exact criteria? "typing directly after" is not sufficient as a closing quote has the exact same property.

[spodermenpls]

Great. That was part of the reason why I made this a separate file: to have an easy to understand way to add new/change rules. I wonder if I should make it less verbose. At the moment the code for the different rules only differs in the Unicode characters that are outputted (I think en should also have the RegExp-check). Would make it even easier to add "standard" rules.

It's almost as if you've done stuff like this before. 😄

Determining the best 'minimalist' approach is a bit tricky, with just two languages supported right now. I have the idea to switch from the current "languages in a list" to a language-agnostic "radio buttons" menu, that would ideally consist just of previews of each set's quotation marks in "preview" (since not every language has its own quotation symbols, scrolling through >10 langs is way too much clutter), e.g. like this:

[Apply on Selection]
☑ Smart Quote active
Ruleset in use:
  ◉ „ “ / ‚ ‘
  ◎ " " / ' '
  ◎ » « / › ‹

If there's (some day) an automatic detection of dictionary languages, the countries could be tied to the ruleset primarily used in each country, and get automatically set up for the user (e.g. German and Romanian have the first set activated, English the second, French the third).
I am in the process of determining a list of rulesets that would be worth to implement, considering Thunderbird's supported languages and those countries' keyboard layouts. The "Guillemets" for example are a bit of a mystery, since the '-key isn't really suited for those symbols - also there are inward and outward-tilted ones in use (Austria and Switzerland supposedly prefer them over the " "), maybe the <-key could be used to inject those? (a typed > turns automatically into ›, as an idea).

Is there somehow a possibility to tell "Smart Quotes" it's supposed to inject the "u2019" symbol, when typing directly after or in between letters?

Yes there is. It may be a little bit complicated but I think we can define a rule for that. Can you formulate the exact criteria? "typing directly after" is not sufficient as a closing quote has the exact same property.

The two (non-fringe) usecases in German that I view as worth considering are words with a apostrophe in between (" K'lautern ") or at the end, as a possessive indicator for words ending with "s" sounding at the end (" Firefox' " or " Max' "). Described here https://www.duden.de/sprachwissen/rechtschreibregeln/apostroph under chapters D15 and D16.
"Using the '-key directly after a word" entails that no opening quotation mark was set in front of the word, that'd need a closing partner.

On an independent note, and out of curiosity, how different technically speaking are Firefox text fields and Thunderbird composer windows? 😅 I know both programs share the same codebase (and add-on architecture..), so maybe there's a path one day to go full 360 and cover Firefox as well (as originally requested in the Bugzilla report I linked in the README.MD). Having webmailer, Social Media, platforms like Github, blogs and whatnot with high demand for text input, yet solely browser-based, might profit from this as well, I'd imagine.

[kkapsner]

Radio buttons will work until you have around 5 choices I would say.

Using > and < is possible. I think it would be best to have << be transformed into the double Guillemets.

Please check my branch for the apostrophe implementation.

Thunderbird is an altered Firefox so there is much in common. "Normal" text fields are a little bit different but nothing too complicated.

[spodermenpls]

@kkapsner Thank you very much for your new commits. I've tested it, and everything seems to work as intended.

I think the 'workaround' with the "<>|"-Key is indeed the best solution - using the guillemets like »Word« would be the Austrian and German variant, labeling it as "French" therefore is a kind of half-truth (edit: the French variant is also in use in German-speaking Switzerland), hence my idea of radio buttons and just the symbols as previews, also saving the user of more interactions than just 2 clicks* (opening popup, selecting rule), when switching between rulesets. I didn't have time to set the "definitive" list of supported rulesets yet, but I don't think we'll be ending up with a overly long pop-up panel in the end.

* Thinking about it, is it possible to have "automatic, temporary pop-up opening" by hovering the cursor on the "Smart Quotes" badge? This would speed up the "workflow" considerably, I'd think.

[kkapsner]

Thinking about it, is it possible to have "automatic, temporary pop-up opening" by hovering the cursor on the "Smart Quotes" badge? This would speed up the "workflow" considerably, I'd think.

The badge is controlled by Thunderbird code and as far as I know we cannot create such a behaviour.

But we have two keyboard shortcuts: Alt+Q (open popup) and Alt+Ctrl+Q (toggle smart quotes). We could also add keyboard control in the popup. For example one could hit Alt+Q to open the popup and then hit 3 for the Guillements.

Thinking more about it: a shortcut system in the popup would really help. Here are my suggestions:

• t: toggle smart quote
• a: apply to selection
• 1, 2, 3: switch rule set (will work until we have more than 9...)

Theoretical we could make these shortcuts different in different UI languages but that seems wrong to me.
On the other hand making it customizable to the user might be handy. But that's another issue: #10.

[spodermenpls]

@kkapsner After my little hiatus, I am back to work on this and I am making progress. I've summarized the Wikipedia "quotation mark" table, and I think with these seven double/single quotation mark variants we're globally covered:

(The chevrons/guillemets are being handled as one variant, since they're only different in their usage of the "SHIFT"-key, as already implemented.)

I assume that top-to-down vertical typing (as in some Asian script languages) is not supported by Thunderbird, so I ignored those. There are however some right-to-left typing languages (like Arabic or Hebrew) that should be supported, the compatibility with "Smart Quotes" therefore has to be subject of consideration / testing, in case of technical differences to the usual left-to-right langs.

The Wikipedia table entry for Macedonian has a unique 2nd set (German-like, but with both marks located on top), I did however study some Macedonian newspaper websites and didn't see such (real world) usage for words in single quotation marks, so I'd say there is no need to include a dedicated ruleset just for that - otherwise there's still the possibility for two-stepping it by mixing variant 2 and 4, though a bit more burdensome.
As proposed earlier, the list is language-agnostic now, it would be a nightmare of a scroll list otherwise (to use and to maintain). The definitive layout obviously isn't set in stone yet, however I do think my little sketch isn't too shabby.

I took a closer look into the keyboard shortcut idea of yours, Alt+Q indeed opens up the badge menu already, Alt+Ctrl+Q on the other hand just leads to @ symbols being inserted. I think using Alt+(number) (or Alt+A / Alt+T) would be preferable to just pressing (number)-key to switch the rulesets, to avoid conflicts between shortcut-use and number-writing. The Alt-key is ergonomically easy to reach, also in combination with the keys 1-7, so I think adding a third key like "Ctrl" to the mix is unnecessary.

In general, I think we should prefer arranging the menu items into width rather than height, so that the overlap stays at the (for writing unneeded) address, subject and attachment fields, rather than the message area.

I've assumed that the "automatic menu opening when hover over badge" wouldn't be feasible with TB, however a simple checkbox in the corner, to keep the menu always open, should be, right?
Something like: "☑ Stay open", in case a language/use case requires switching between different variants, and hence repeated interactions with the menu - also in case the user prefers to use mouseclicks to switch between rulesets, instead of keyboard shortcuts.

As I was looking through the rulesets for all these different languages, I've noticed that there are a lot of apostrophe-like characters, that visually resemble them and are often mistakingly used instead (since even native keyboards don't offer those, often times), even though they're separate, distinct symbols. I haven't determined that list fully yet, however candidates I'd like to include so far are:

• single and double Prime
• Geresh and Gershayim
• 'Okina
• capital and small Saltillo

To avoid bloating and feature creep, I'd propose to restrict such "extras" to strictly "apostrophe-resembling characters", keeping things like "haček" or "Kahakō" out of scope, turning the extension into sorta "Smart Quotes and Apostrophes" is aready a reality, since German and English apostrophe converting is already part of the package.
Since those symbols above are much rarer in use than the seven "classic" variants, maybe they could be moved to a "page 2" in the menu or something of that sort (I am having Video DownloadHelper's badge menu in mind), to keep the menu as lean/small as possible.

[kkapsner]

otherwise there's still the possibility for two-stepping it by mixing variant 2 and 4, though a bit more burdensome.

I think the best solution for such an edge case would be to offer a custom ruleset where the user can simply enter four characters to describe their ruleset.

Alt+Ctrl+Q on the other hand just leads to @ symbols being inserted

That's what I get for using Linux... you could still use AltGr + Q to get @
This shortcut is already in the current version - but it may map to Alt+Shift+Q... more testing has to be done on that.

I think using Alt+(number) (or Alt+A / Alt+T) would be preferable to just pressing (number)-key to switch the rulesets, to avoid conflicts between shortcut-use and number-writing.

The number-keys would only act as a shortcut when the popup is open. You can check it out in the popup-shortcuts branch.

Something like: "☑ Stay open", in case a language/use case requires switching between different variants, and hence repeated interactions with the menu - also in case the user prefers to use mouseclicks to switch between rulesets, instead of keyboard shortcuts.

I don't think this is possible.

To avoid bloating and feature creep, I'd propose to restrict such "extras" to strictly "apostrophe-resembling characters", keeping things like "haček" or "Kahakō" out of scope, turning the extension into sorta "Smart Quotes and Apostrophes" is aready a reality, since German and English apostrophe converting is already part of the package.

So your idea is to have two different rules. One for quotes and one for apostrophe?

[spodermenpls]

@kkapsner

That's what I get for using Linux... you could still use AltGr + Q to get @
This shortcut is already in the current version - but it may map to Alt+Shift+Q... more testing has to be done on that.

The default shortcuts are set now to ALT+T(oggle) and ALT+Q(uotes), as it's optimal for Windows. Are there conflicts with Linux, using this combination? In case it is, is there maybe a possibility to define default presets for different platforms (Windows, macOS, Linux)?

The number-keys would only act as a shortcut when the popup is open. You can check it out in the popup-shortcuts branch.

Yes, I got it, my thinking involved the (not possible) possibility of an "always open" SQ popup, in the scenario where the Popup is only opened when one doesn't write, there's obviously no conflict between e.g. typing numbers and selecting presets.

So your idea is to have two different rules. One for quotes and one for apostrophe?

I think the best solution for such an edge case would be to offer a custom ruleset where the user can simply enter four characters to describe their ruleset.

I have thought about this for quite some time and bounced ideas back and forth in my head (also the reason why I didn't answer you back in December right off the bat, since I didn't have a definitive answer yet), how an optimal UI would look like / how all desired features would be optimally wrapped together - which is quite the task, since all major text processing programs deliver "Smart Quotes" converting by default for decades, and there's really no concept to lean on, to my knowledge - however, I think I may have thought up something usable.

The six usual templates, as commandeered by keys 1-6, template 7 as an "add-on", so to speak, since it re-programs the "<>|"-key (which is rarely used while writing), and not the "2"²"- and "#'"-key, as the previous six, so it can get used in conjunction with them. And as option 8, freely choosable programming for the 2-key and the #-key. In both cases ideally with templates to choose from a dropdown list (?) plus custom Unicode symbol code input (?), and for fields 3 and 4 (counted from left to right) additionally templates for Prime, Saltillo, 'Okina etc., representing "apostrophe-like" symbols (in case there are small/capital resp. "dual" versions of the symbol, simple press of "#" for small/single - field 3 [replacing the "#", as long as a "dual versions" symbol template is selected (?)], SHIFT+# for capital/double symbol - field 4).

[kkapsner]

The default shortcuts are set now to ALT+T(oggle) and ALT+Q(uotes), as it's optimal for Windows. Are there conflicts with Linux, using this combination? In case it is, is there maybe a possibility to define default presets for different platforms (Windows, macOS, Linux)?

They should not interfere with Linux (at least I know of nothing). You can specify the shortcuts individually for each platform (https://webextension-api.thunderbird.net/en/78/commands.html). If we want we could even have the user change the shortcut binding.

so it can get used in conjunction with them

I think that's a good idea. But in the UI I would put it at the end to keep the "/' things together.

And as option 8, freely choosable programming for the 2-key and the #-key. In both cases ideally with templates to choose from a dropdown list (?) plus custom Unicode symbol code input (?), and for fields 3 and 4 (counted from left to right) additionally templates for Prime, Saltillo, 'Okina etc., representing "apostrophe-like" symbols (in case there are small/capital /"dual" versions of the symbol, e.g. press on "#" for small/single - field 3 [replacing the "#", as long as a "dual version" symbol template is selected (?)], SHIFT+# for capital/double symbol - field 4).

Not sure if I understood it 100% but I think it's doable and makes sense. For the first draft of the feature I would go without the dropdown lists.

PS:

"<>|"-key (which is rarely used while writing), and not the "2"²"- and "#'"-key

Good that I also have a German keyboard (am not on a Mac)... otherwise I would be confused as hell

[spodermenpls]

@kkapsner

I think that's a good idea. But in the UI I would put it at the end to keep the "/' things together.

By "/"-things you mean the "either or"/radio button templates, I assume? My thought was that the "individual" template section should be at the very end, to emphasize that it's a totally separate entity/"can of worms" than the other preset ones. I also have the experience in mind, that those kind of "we're giving you 10 options to choose from, but if you're not satisfied with them, there's a custom setting option too" - to set a freestyle color, custom CSS, or whatever - menus tend to have their "on your own" buttons/section at the very end.

I don't think that the "checkmark" box before the "custom layout" radio button would induce confusion, considering I'd also add a little "chevrons/guillemets are, if activated, usable in conjunction with the other templates, as they use the "<"-key independently" sentence in the add-on description text.

"<>|"-key (which is rarely used while writing), and not the "2"²"- and "#'"-key

Good that I also have a German keyboard (am not on a Mac)... otherwise I would be confused as hell

Just first, second and third key layouts in one swoop, to demonstrate all choices that a writing person is able to use.