[Erlang] Variables, modules, record names and fields are indistinguishable

[eproxus]

• Sublime Version: 4074
• OS Version: macOS 10.15.4

#foo{foo = Foo} = bar:baz()
%^ variable.other.record.erlang
%    ^ variable.other.field.erlang
%          ^ variable.other.erlang
%                 ^ variable.namespace.erlang

Everything is prefixed with variable. which makes color schemes highlight everything in the same color. In Erlang records, record fields and modules are not "variables".

(Monokai color scheme on meck_proc.erl)

[eproxus]

Ping @deathaxe

[deathaxe]

With regards to https://www.sublimetext.com/docs/3/scope_naming.html, what top-level scope would you choose for records, fields and modules?

I know Erlang has the concept of atoms vs. variables. But that's not covered by any specification atm.

The variable top-level scope is generally used for user defined identifiers. Function identfiers use variable.function for instance. A field would normally need to be scoped variable.other.member, but I chose variable.other.field as it felt more suitable for Erlang. Maybe we should change that. A record in general may be compared to C's struct? Thus it's identifier is nothing else than a variable even though I am not sure whether this assumption is perfect with regards to the usage in Erlang.

Some legacy syntaxes use support.class, support.namespace or even support.type.class ... for modules but support is reserved for elements of standard libraries. Thus variable.namespace feels as the best common alternative for user defined modules or those not supplied with OTP (or those we can't match for some reason). Here's the related jet open RFC discussion #1842.

That said, the main goal is to define semantically acceptable/correct consistent general purpose scope names, which can be used by color schemes to provide a consistent UX for as many syntaxes as possible with minimal effort. We do not want to hack scope names to match certain color schemes as it fails with most of them anyway.

I'd be fine to modify scopes which fulfill that requirement.

Otherwise
a) the color scheme might propably need some tweaks to address the new scopes, which does not only apply to variables scopes in Erlang, maybe.
b) the existing scope names should be sufficient enough to define user defined overrides if existing colorization is not enough.

Here's my look with a slightly modified Brackets Dark

Just added

        {
            "scope": "variable.namespace",
            "foreground": "var(purple)",
        },
        {
            "scope": "variable.other.record",
            "foreground": "var(blue)",
        },
        {
            "scope": "meta.atom variable.other",
            "foreground": "var(orange)",
        },

... next to some other tweaks to properly highlight operators and punctuations.

[eproxus]

If the variable. prefix is used for almost everything in most other syntaxes, I would say we should keep it that way. Perhaps there are more general names under than namespace that we can use?

For example, I think using variable.other.member for record fields is more appropriate, because it will most probably result in the most appropriate colouring. That being said, the default Monokai scheme shipping with sublime groups entity.name etc. along with variable.other.member. Not sure that is "correct" (otherwise, what's the point of having two different namespaces?).

The downside with the current scope naming conventions is that they have an object oriented heritage. That can be seen in the variable.function scope for example, which I assume is intended to be used in a ObjectInstance.my_function() fashion.

The problem with Erlang is that ideally the following would be coloured distinctly:

• Atoms: already covered by constant.other
• Functions: already covered by variable.function
• Modules: not sure what would fit here. Python, for example, doesn't even set a scope for modules used in the code... (maybe we should use variable.other.constant?)
• Variables: already covered by variable.other (perhaps should be variable.other.readwrite?)
• Records and their fields: fields can be covered by variable.other.member but what about records?

[deathaxe]

what's the point of having two different namespaces?

With regards to functions:

• the name in a definition statement is scoped entity.name.function
• the name in a function call uses variable.function

The original scope names come from TextMate. When studying its naming guidelines and comparing different syntaxes, I found many missing details and different solutions for compareable statements. Maybe because things moving forward?

It looks like entity.name is used for definitions and variable. for references in some cases. There are still inconsitencies about that in different syntaxes, but maybe that's a strategy, which can help to distinguish definitions and references easily. ST4's indexer seems to rely on entity.name to specify a symbol's kind for instance.

Perhaps there are more general names under than namespace that we can use?

You mean instead of variable.namespace?

What's more general? ST's scope naming guidelines suggest this scope for namespaces, modules, packages, etc. which all do nothing else than create certain lookup areas to avoid duplicated or insanely named functions. OK, the name itself is a bit C-style and may be a compromise for syntaxes which group functions/variables via modules or packages, but is there such a big difference semantically?

I'd keep the variable.namespace for modules. We already moved a couple of syntaxes to it.

Variables: already covered by variable.other (perhaps should be variable.other.readwrite?)

Reasonable.

Records and their fields: fields can be covered by variable.other.member but what about records?

I'll change the field scopes.

Some general notes about records:

In a record definition clause (-record(name, {field=value, ...})), name and field are scoped as entity.name while they are variable. ... in the #record{field=value} statement.

I am not quite sure about the real meaning of #record. Is it a local definition of a record? Then we could propably also use entity.name.record. I read it as some kind of expression to return certain fields of a record to the caller, thus understood it as a reference of an existing record. What do you think?

[eproxus]

I agree about stickting to conventions. 👍

Could a possibility for module name scopes be variable.other.constant.erlang.module and normal variables thus be variable.other.readwrite?

I am not quite sure about the real meaning of #record. Is it a local definition of a record? Then we could propably also use entity.name.record. I read it as some kind of expression to return certain fields of a record to the caller, thus understood it as a reference of an existing record. What do you think?

Records are like named structs. So #foo{} means create a new record of the type foo. Then in patterns it of course means match against the record type foo and any internal field patters, e.g. #foo{bar = 1} = MyRecord. The record is implicitly "created" there, but never "used" so to speak.

Question is what scope to use for record. You don't want the entity. name space if that will make Sublime index every use of the record all throughout the code.

[eproxus]

Actually, looking through the suggestions for various variable. scopes in the documentation you linked, there's simply not enough sub types to cover all of Erlang.

[deathaxe]

The guidelines can't provide details for any kind of syntax as each of them comes with unique features and concepts. It therefore can just be a base to start with. The RFC issues have been created to discuss possible extensions, but following them quickly reveals all the different apsects and oppinions about certain points of syntaxes. Therefore it's hard to find common conclusions.

The basic rules I try to stick to are:

1. Use one of the 1st level scopes
2. Try to use 2nd level scopes as well, otherwise sublimehq should decide whether to add a new one
3. 3rd level scope should propably contain some kind of semantic group name. (see keyword.operator.arithmetic and friends
4. 4th level may contain arbitrary details.

Records

Your description sounds very much like an instantiation of a class in C++/Java.

Hence storage.type.struct or storage.type.record may be appropriate.

Note: C++/Java use support.class as storage.type.class is already used for the class keywords.

Modules

1. The variable.other is used for general purpose variables of no special meaning in most syntaxes.
2. There are some exceptions in some packages, but a common rule is to just place the syntax's main scope to the very end. This enables color schemes or plugins to distinguish between variable.other.erlang and variable.other.c when used in the same file for instance.
3. I don't find modules to be constant variables. The variable.other.constant scope was created as counterpart of variable.other.readwrite, most likely for const or final variables (see C/C++/Java). IMHO it makes only little sense as a static lexer can't distinguish reliably.

The scope naming guileline proposes entity.name.namespace for namespace, package or module definitions. Thus using variable.namespace for any kind of refernce seems most logical with regards to how functions are defined/referenced.

If a further distinction is wanted, we could propably define ...

entity
  name
    namespace
      package
      module
      ...
      other

variable
  namespace
    package
    module
    ...
    other

Java would need that as it distinguishes between packages and modules for instance.

[eproxus]

I like your suggestions a lot. I think using the same scopes as class instantiations for Java makes sense, perhaps also coupled with the same scopes for class definitions when records are defined (-record(...))?

It makes sense to add the language scope at the end, of course. In my observation almost no color schemes actually makes use of these, except for very common languages, so therefore I wanted to make the top level scopes more expressive if possible.

If what you propose under entity and variable would take off, I assume that would mean a large refactoring of many syntax definitions and themes in Sublime?

[deathaxe]

almost no color schemes actually makes use of these

That's the main reason to place the language scope to the end. A color scheme shouldn't need to be forced to use syntax specific rules. In a perfect world it would only need at most 3 levels of subscopes to address all required tokens.

... I assume that would mean a large refactoring of many syntax definitions

I guess many syntaxes could profit from the latest ST4 branching features and therefore would need some attention anyway.

With regards to namespaces etc. there is currently no real common solution. Each syntax seems to use different scopes for nearly the same thing. The hope was just to find one scope which can be used everywhere.

In a more general manner it may result in a bit more work. But I guess the entity vs. variable scheme can't be applied to all syntaxes in all situations anyway. In python for instance, the static lexer can't know about which variable is the declaration (first usage) or which ones are references only.

It more or less suggests to use those when possible or appropriate to help driving the Goto Definition vs. Goto Reference features in a consistent way.