Language-specific OpenType Feature Names for Programming Ligatures

[be5invis]

Given the fact that in different languages, the symbol combination should be combined into a ligature are different, is it possible to provide a language-specific feature name to distinct them? For example, in C++, >>= should not be combined together, and we can assign a feature XCPP to combine >> only. In Haskell, >>= should be combined, and feature XHS_ will be applied. The default calt can be restricted to "most-common" languages only.

cf. tonsky/FiraCode#192, atom/atom#11846, #6918, source-foundry/Hack#211, larsenwork/monoid#155

[be5invis]

for Code, this feature can be impelmented in CSS only: Assign font-feature-settings to token.<type>, like:

token.js{
    --webkit-font-feature-settings: "XJS_" 1;
    font-feature-settings: "XJS_" 1;
}

Perhaps these CSSes can be automatically built from syntax extension list registered.

[be5invis]

A preview:

Code is injected with this CSS:

token.haskell{ font-feature-settings: "XHS_" 1 }

Therefore the ligature for >>= is only enabled in Haskell, not in C++.

[tobia]

Is there a place where one can write some custom CSS to be added to the editor? Maybe a "user_css" file?

I'm asking because there are many tweaks that need custom CSS, for example custom font feature sets (such as -webkit-font-feature-settings: "zero" on; to have slashed zeros in some fonts)

Right now we're adding stuff to workbench.main.css which is not ideal :-)

[be5invis]

@tobia my plugin
https://github.com/be5invis/vscode-custom-css
https://marketplace.visualstudio.com/items?itemName=be5invis.vscode-custom-css

[siegebell]

@tobia @be5invis prettify-symbols-mode might also meet your needs.

[alexdima]

If we standardize on the font feature settings name (language id -> font feature setting name table), this is something we could add to our language configurations (in the API) and then the editor could dynamically create these styles with the right selectors to pass on the setting to fonts.

Is there a limit on these things? Why are they all 4 letters wide?

[be5invis]

@alexandrudima All font “tags” are exactly four ASCII letters. Feature name is a kind of “Tag”. And as purposed in OpenType Spec, all custom feature names should use capital letters only.

[be5invis]

@alexandrudima Spec

Feature List Table

The headers of the GSUB and GPOS tables contain offsets to Feature List tables (FeatureList) that enumerate all the features in a font. Features in a particular FeatureList are not limited to any single script. A FeatureList contains the entire list of either the GSUB or GPOS features that are used to render the glyphs in all the scripts in the font.

The FeatureList table enumerates features in an array of records (FeatureRecord) and specifies the total number of features (FeatureCount). Every feature must have a FeatureRecord, which consists of a FeatureTag that identifies the feature and an offset to a Feature table (described next). The FeatureRecord array is arranged alphabetically by FeatureTag names.

Note: The values stored in the FeatureIndex array of a LangSys table are used to locate records in the FeatureRecord array of a FeatureList table.

FeatureList table

Type	Name	Description
uint16	FeatureCount	Number of FeatureRecords in this table
struct	FeatureRecord[FeatureCount]	Array of FeatureRecords-zero-based (first feature has FeatureIndex = 0)-listed alphabetically by FeatureTag

FeatureRecord

Type	Name	Description
Tag	FeatureTag	4-byte feature identification tag
Offset	Feature	Offset to Feature table-from beginning of FeatureList

Source: https://www.microsoft.com/typography/otspec/chapter2.htm

[be5invis]

@alexandrudima And according to Adobe, a Tag is a four-letter identifier. In feature files (used by MakeOTF) it should follow the glyph name specification (section 2.f). For tags shorter than four characters, it will be extended to four with spaces (so DEU is identical to DEU).

[be5invis]

@alexandrudima My proposals:

Language	File Extensions	Proposal 1	Proposal 2
C	.c, .h	XC__	C *1
C++	.cpp, .cc, .hh	XCPP	CPP
Java	.java	JAVA	JAVA
JavaScript	.js	XJS_	JS
C#	.cs	XCS_	CS
HTML	.htm, .html	XHTM	HTML
CSS	.css	XCSS	CSS
Less	.less	LESS	LESS
Stylus	.styl	STYL	STYL
Haskell	.hs	XHS_	HSKL
Python	.py	XPY_	PYTH
Scala	.scala	SCAL	SCAL
Ruby	.rb	XRB_	RUBY
JSON	.json	JSON	JSON
Swift	.swift	SWFT	SWFT
Objective-C	.m	OBJC *2	OBJC
x86 Assembly (AT&T)	.s	XS86	SX86
x86 Assembly (Intel & NASM)	.asm	XASM	AX86
TypeScript	.ts	XTS_	TS

Notes

1. All tags are exactly four letters. The trailing spaces are not shown, but present in fonts.
2. .m is used for multiple languages, including Matlab and Objective-C. Therefore the feature tag of the corresponded languages is not named using the file extension.

[siegebell]

@alexandrudima @be5invis The feature names being proposed are nonstandard, so I think fixing them is a bad idea because some fonts may use them for other [private] features now or in the future.

[be5invis]

@siegebell I know, there is a better way is to use the OpenType’s script-language system. That is, assign calt to languages like code/DFLT or code/CPP or even latn/CPP. FireFox supports font-language-override to do that.
But Electron (and Chrome) does not support customized OT script/language assignment, so using custom feature tags is necessary. All of them are capital to avoid conflict with existing or future tags.

[siegebell]

@alexandrudima @be5invis It should be up to the user to select the appropriate font and font-features per language.

I propose the following two features be added to settings.json:

1. Add "editor.fontFeatureSettings": string as a setting.
   2a. Allow certain editor settings to be configured per language. E.g. "editor.language-specific" : [{"language": "java", "fontFeatureSettings": "JAVA ordn"}, {"language": ["haskell","**/*.ks"], "fontFeatureSettings": "HSKL", "fontFamily": "'Courier New'"}]
   or optionally
   2b. Support a general mechanism for language-specific configuration. Settings, provided by vscode or extensions, may specify whether they are allowed to be language specific in their schema. Let workspace.getConfiguration take an optional language-id argument (where the default behavior is to return the language-agnostic setting). E.g. "language-specific" : [{"language": "java", "editor.fontFeatureSettings": "JAVA ordn"}, {"language": ["haskell","**/*.ks"], "editor.fontFeatureSettings": "HSKL", "editor.fontFamily": "'Courier New'", "myextension.foo": true}]

[be5invis]

@siegebell For the feature tag, I think exporting an API to extension makers is enough. They can register the tag to the file type and users can actually not focusing on these configurations. Once someone REALLY want to override, use things like this:

{ "editor.languageSpecificFontFeatureSettings": {"source.haskell" : "XHS_"}}

[siegebell]

@be5invis Custom feature tags are specific to the font family. An extension that specifies a [nonstandard] feature tag will break any user-selected font that happens to define a conflicting tag -- this will be highly confusing and unintuitive for the user to diagnose. How are they supposed to figure out that they need to either override the feature settings, choose another font, or else uninstall the extension?

Thus this setting should be provided by the user. If you really want, I believe an extension can contribute arbitrary settings, e.g. "editor.language-specific": [{"language": "haskell", "fontFeatureSettings": "XHS_"}], so the user still does not have to focus on configuring such subtleties. The advantage here is that the user can check the default settings.json to see what the extension did if their font breaks.

[be5invis]

@siegebell I've investigated all existing programming fonts I can find and there is no conflict with the purposed feature tags (actually most of them do not support any ligature). I know your consideration, and it can be simplified into a bool setting which turn on/off the entire L-S thing:

{
    "editor.languageSpecificFontFeatures": true,
    "editor.languageSpecificFontFeatureOverride": {"source.haskell" : "XHS_"}
}

cc. @larsenwork @tonsky

[siegebell]

@be5invis If an override is added to the configuration options and an extension is already able to contribute configuration options, why not just have everything specified in the configuration options instead of providing an extension API that is redundant, OpenType-standards-breaking*, and opaque to the user?

*OpenType allows nonstandard tags, but my interpretation is that they should only be invoked by users that know what they're doing or programs/interfaces that do not use arbitrary fonts. I do not believe that "all existing programming fonts I can find" is a sufficient metric, although I do agree that encountering a conflict is unlikely.

[be5invis]

@siegebell There is an option to turn off the entire thing, and it is off by default. So it is an opt-in, not an opt-out. I do not think that standardizing a “non-standard” tag set is a bad idea.
A better solution is that when Chrome and Electron get support of font-language-override, then move these tags to sub-languages under latn script (or maybe code if they support), and the conflicts will gone.

[siegebell]

@be5invis can you be clear: are you proposing that this feature be off by default by providing "editor.languageSpecificFontFeatures" with default value false, such that the user must explicitly enable the feature? If so, then I have no more technical objections. (I'm only against adopting nonstandard font feature tags when it is done opaquely to the user.)

[be5invis]

@siegebell Yes.
And actually, turning on editor.languageSpecificFontFeatures will replace the calt feature with the language-specific feature (as purposed in the table) and PLIG as a fallback.

[siegebell]

@be5invis @larsenwork @tonsky Can multiple language ligature support be implemented via font fallbacks?

For example, given a base font myfont and a font that only defines a subset of necessary characters and ligatures for a particular language, e.g. myfont Haskell defines only characters - and > with a ligature converting -> into a nice arrow, add this setting:

"editor.fontFamily": "'myfont Haskell', 'myfont', monospace"

(I'm not necessarily advocating this approach, but I do wonder if it is feasible and what its drawback may be.)

[larsenwork]

@siegebell I'd prolly use stylistic alternates within a font for programming-language-specific custom ligatures — so you'd only have the haskell ligature with e.g. ss01 turned on.

I've never used vscode so I don't know how a font stack there behaves

[tobia]

As a potential user of this feature, I would like to add my 2¢, because I've seen some wild ideas in this thread that don't make much practical sense.

First things first: Font features are 4-byte tags that are chosen and/or invented by the font designer and are baked into the font file. They allow the user to choose among stylistic variations of the font, mixing and matching the variations they like the best, or that are most appropriate for the task at hand. These features are usually small things that don't warrant creating an entirely different font file, such as empty vs. slashed zero, or fixed-width vs. proportional decimal digits, or increased character spacing. The name of a feature is a 4-byte ASCII string and the value is a small integer (not just true and false!)

The font standard defines names for common features, such as zero denoting alternative shapes for the zero digit. What particular styles are selected with the various integer values of the features is partly specified in the standard, partly left to the font designer. Tag names with lowercase letters are reserved for standard features, while names in all caps are reserved for the font designer, if they can't find a standard feature name that denotes what they are trying to do.

More and more fonts are supporting features, especially those managed as open source projects, because different people have different ideas on what the best shape for some character is, including which (if any) common character combinations that are usually read together, such as ->, should be rendered as a single symbol (a ligature.)

My original post in this thread was asking whether there is a user setting for font features, because I wanted to turn on the slashed zero feature for the particular font I'm using.

Given what features are, I'm pretty sure the best place for them would be a user setting right among the other font-related ones:

"editor.fontFamily": "My cool font",
"editor.fontSize": 12,
"editor.fontFeatures": {"zero": 1, "salt": 3, "COOL": 1},
"editor.fontLigatures": true,

In particular, I fail to see what the table posted by @be5invis is supposed to represent. Font features are chosen by the font designer, so it makes no sense at all for an editor to come up with its own custom names.

As far as language-specific configuration, there might be value in configuring different editor settings depending on the current language, but it's not limited to font features. I'm pretty sure it should be split into a specific issue.

For example, I might want to use different values for editor.useTabStops, editor.trimAutoWhitespace, and the proposed editor.fontFeatures when I'm editing Python code vs. JSON files. I think this would make a lot of sense and is already provided by most programming editors, including Sublime Text. (Maybe it's already possible in VSCode? I'm a newbie so I might have missed it!)

Hope this helps to clear things up.

[be5invis]

@tobia
My proposal is that, we can standardize a series of tags for programming-language-specific ligatures, so that type designers (like @larsenwork) can instantly create these features to solve the ligature conflict problem.
I do not think that using ss01 to represent programming-language-specific ligatures is a good idea, given that type designers may not have a standard list of style-language mapping, and configuring the editor will be extremely tedious.
The best solution will be using the script/language system of OpenType, so that we can assign a feature to a special language like latn/HSKL, but currently Electron does not support this mechanism.

[abhayghatpande]

@tobia since font feature settings don't seem to be implemented as yet, could you please advise how you added the -webkit-font-feature-settings: "zero" on; to workbench.main.css? What css selector do you use? Thanks!
Or if you are able to use the custom css extension made by @be5invis, please share how. Will appreciate it.

[Lucretia]

See tonsky/FiraCode#628 - I would like something like this for Ada.

[voronoipotato]

Don't control the flags, just let me put in what I need. I should be able to enable stylistic sets (of any kind) for any language. If I want to have haskell ligatures in C that's my own business. I should also be able to turn on SS12 for markdown so that I can use the pragmatapro ligatures.

I don't think we should need extensions to use OpenType standards compliant font configuration.

tonsky/FiraCode#192
fabrizioschiavi/pragmatapro#84
#10435

[alexdima]

I'm happy to let you know that different font features per language now works on Insiders with the work done for #82153

With the following settings:

    "editor.fontFamily": "Fira Code"    
    "editor.fontLigatures": true,
    "[cpp]": {
        "editor.fontLigatures": "'ss06', 'ss19'",
    },
    "[javascript]": {
        "editor.fontLigatures": "'ss06'",
    }, 

The editor is configured to turn on ss19 only for C++ (https://github.com/tonsky/FiraCode/releases):

And here is switching between the files: