Add "Option" and "Alias" token clusters

[reblim]

One thing I miss in this specification draft (but maybe there is a valid reason for it) is to capture more deeply how token inheritance works, what are the two main token clusters, and how groups play an essential part in token inheritance.

Tokens that inherit their value from other tokens are called "alias" tokens. However, I am missing the name of the tokens which contain the actual value. For a few years, I've been calling these types of tokens "options", but perhaps a better name could be "primitives". Another word I've seen circulating for this concept is "global" but I don't think "global" captures the true intention of this concept.

I believe it's critical to define a name for two types of token clusters, as this will allow tool makers to have a more reliable source when dealing with token inheritance. If we divide "option" tokens from "alias" tokens it will be also easier to understand how tokens interact with each other when groups are formed in a token system.

When seeing tokens this way, "Option" or "primitive" tokens should be those which have raw values exclusively (when talking about single tokens), or objects with raw values exclusively (when talking about composite tokens).

On the other hand, "alias" tokens should never have a raw value, instead, they always refer to an existing "option" token within the token system.

When dividing tokens into these two main clusters, some "best practices" could be introduced for naming tokens. The name of an "option" token, for example, should be completely agnostic. While "alias" tokens, on the other hand, should give more meaning to the token.

Option token's main purpose is to create a bucket of the available options within a system and prevent values from repeating in the main scope.

Therefore, "option" and "alias" tokens should be default clusters in every token group. This will also help to better identify groups from tokens. Alias tokens will always inherit their value from their direct group or any other group within the same scope.

The design token tools should only use option tokens to satisfy the references in alias tokens, but should not render "option" tokens as output during translation. Also, the "options" and "aliases" keywords could be used as keys of the cluster objects. But these should be removed from the token names by trankation tools.

The following is an example of how the concept above could look like:

https://gist.github.com/reblim/2953660b0d256fe0c230f2c5b0e1ab32

And a translation tool should then output something like this (CSS Variables example):

https://gist.github.com/reblim/97b54d27ea2bff18f7cd98da9e6ba6f8

Once we have defined "options" or "primitives", and "aliases", one could argue that there is a third type, which I have been calling "scoped", and seen other people call "component" tokens. These tokens are essentially a sub-type of the "alias" tokens, as they are only able to inherit a value from an option or an alias token within their scope. Because of this, we might not need to have this third token name and the spec should work fine with only the two main clusters; "options" or "primitives", and "aliases".

For a better understanding of the concept, see the following mind-map. Here, you will see how groups can become a determining boundary for token inheritance.

Token system mind-map showing how "options" and "aliases" clusters.

Token system mind-map with inheritance arrows showing how "options" and "aliases" clusters work.

Minimal Token system with "options" and "aliases" clusters.

[c1rrus]

If I've understood correctly, your proposal essentially does 2 things:

1. Forces "option" and "alias" tokens to be explicitly partioned within the file
2. Prevents "option" tokens from being exported to code

I'm not sure how useful the 1. part is. Given that the current spec draft already allows both kinds of tokens (if a token's $value is a string in the form of "{path.to.other.token}" it must be an "alias", otherwise it must be an "option"), asking tools and authors to additionally place tokens into different group-like objects therefore seems redundant. I'm not convinced that makes the files easier to read or understand either.

In fact, I'd argue it makes things more difficult because, at first glance, they look like groups (i.e. JSON objects that have tokens as child objects) but they're not actually groups as they are not part of the path when you reference tokens and they are omitted when generating variable names for CSS output. The current spec draft only has 2 kinds of entities that can exist in token files - groups and tokens. This would introduce a 3rd thing which I feel adds unnecessary complexity.

As per the current spec draft, your example could be rewritten as follows (omitting the descriptions and comments for brevity):

{
  "ds": {
    "color": {
      // "option" in ds.color group
      "50": {
        "$value": "#ffffff",
        "$type": "color"
      },
      // "alias" in ds.color group
      "primary": {
        "$value": "{ds.color.50}",
        "$type": "color"
      },
      "group-1": {
        // "options" in ds.color.group-1 group
        "50": {
          "$value": "#000000",
          "$type": "color"
        },
        "150": {
          "$value": "#ff0000",
          "$type": "color"
        },
        // "aliases" in ds.color.group-1 group
        "primary": {
          "$value": "{ds.color.50}",
          "$type": "color"
        },
        "secondary": {
          "$value": "{ds.group-1.color.50}",
          "$type": "color"
        },
        "tertiary": {
          "$value": "{ds.group-1.color.150}",
          "$type": "color"
        }
      },
      "group-2": {
        // "option" in ds.color.group-2 group
        "50": {
          "$value": "#0000ff",
          "$type": "color"
        },
        // "aliases" in ds.color.group-2 group
        "primary": {
          "$value": "{ds.color.50}",
          "$type": "color"
        },
        "secondary": {
          "$value": "{ds.group-2.color.50}",
          "$type": "color"
        }
      }
    }
  }
}

Admittedly this is subjective, but to me that is no more difficult to understand than your proposal. It's also a bit more succinct.

As for the 2nd point - omitting "option" tokens from the output - that's something the current spec draft does not have a solution for. If my above example was exported to CSS, I'd expect the output to be something like this:

:root {
  --ds-color-50: #ffffff; /* now also gets output */
  --ds-color-primary: #ffffff;
  --ds-color-group-1-50: #000000; /* now also gets output */
  --ds-color-group-1-150: #ff0000; /* now also gets output */
  --ds-color-group-1-primary: #ffffff;
  --ds-color-group-1-secondary: #000000;
  --ds-color-group-1-tertiary: #ff0000;
  --ds-color-group-2-50: #0000ff; /* now also gets output */
  --ds-color-group-2-primary: #ffffff;
  --ds-color-group-2-secondary: #0000ff;
}

However, there are other ways we could prevent some tokens from being exported such as the $private proposal being discussed in issue #110. I believe that proposal is preferable to this one as it's more flexible. It could let you exclude any tokens, not just "option" ones. (In fact, some of the ideas folks have proposed in that thread could allow even more ganualarity about where certain tokens can appear or not).

[reblim]

@c1rrus

Essentially yes, it does that + one more thing:

1. It forces "option" and "alias" tokens to be explicitly grouped within the token system as clusters.
2. It prevents "option" tokens from being exported to code.
3. It allows for specific cluster rules, like for example: "options" should always have a raw/pure value, while "aliases" should always refer to an existing "option" or "alias."

A benefit of point one is that by preventing translation tools from rendering "options", we can allow users to think of meaningful names for all "alias" tokens, and completely forget about naming taxonomies for "options" because these can be auto-generated by the tool as "option" names should be completely agnostic. Moreover, token inheritance would be a lot more reliable since users can trust that no one option can ever repeat. It will be easier for tools to handle repeated "options". This way, designers will be able to experience the value of DRY (Don’t Repeat Yourself) as we developers do in programming.

What you propose could also work, of course, but clustering these two groups would also help increase the understanding of how tokens interact with each other.

To sum up, I believe that the nature of "options" vs "aliases" is very different, and as design tokens evolve, these will become even more noticeable. Therefore, clustering will allow for rules to be applied to each group more intuitively.

[TravisSpomer]

I was actually in a meeting just this week that was caused by our "option" (global) tokens not being exported to code in one case—we needed to surface those in some fashion for third-party extensibility. And I was in a chat a couple of weeks ago about our shadow tokens, which don't fit nearly into either bucket and are treated like "option" tokens but are technically alias tokens since their colors point to other tokens that change with the theme.

Each project using tokens is going to have different technical and design requirements. I think this sort of thing is best left to project-specific rules. Baking it into the format in any way adds unnecessary constraints that would be very burdensome for anyone who had different needs.

[reblim]

@TravisSpomer Exactly these kinds of issues you talk about are those that will be fixed by adding the "options" and "aliases" clusters with specific rules for each.

As mentioned before, if "options" can strictly only have raw values, this will help us to avoid repeated values polluting the system. On the other hand, "aliases" should only refer to existing "options" and other "aliases", but never a raw value. Then it will always be clear what cluster type the token at hand is.

Clustering tokens will also give us more control to decide whether, for a project, you would like to export "options" or not.

Regarding your example about the shadow tokens, I believe you may be confusing them with composite tokens. A shadow token is formed out of various tokens, which could be both "options" and "aliases", depending on how the system is put together.

[delisma]

Hope that this is relevant to the discussion, because I can relate to @reblim point. We call the "option tokens" "base tokens" or "raw tokens" because they hold raw value and their naming convention follow the Category Type Item system. The "aliases" are split into 3 categories to convey the design intent, and these tokens can only inherit from the parent level. This allow us to have a specific naming convention at each level depending on the audience that will use them.

I don't know if @reblim proposal has to be in the specs in order to build an appropriate token architecture, I think it depends on the complexity of the project. For us because we are federated and distributed, we needed a way to convey to 5000 designers and developers how to build their own components while keeping the brand consistent and maintainable by a team of 20 token maintainers.

At different level we have a "cluster" of tokens that can be used by different audiences in their mental model. This approach may be overkill in a team of 1.

{
ds: {
  base: {
    color: {
      orange: {
        200:  {
          $value: rgb(232, 86, 79, 1),
          $type: color
        }
      }
    }
  },
  standard: {
    color: {
      action: {
          $value: {ds.base.color.orange.200},
          $type: color
      }
    }
  },
  component: {
    button: {
      action: {
        background: {
          color: {
             $value: {ds.standard.color.action},
             $type: color
          }
        }
      }
    }
  }
}

For us the second level of the token is documented to convey where we accept raw values or aliases. Do we want to make it explicit or not with a spec? I could see the value of having only one source of truth, but for how many people have that use case? I wonder.

[kevinmpowell]

The format editors discussed this issue in this week's meeting and agreed that it's not a feature we're going to add to the specification.

There are organizational and naming best practices for tokens, but the aim of the specification is to support as many different organizing principles for tokens as possible by remaining unopinionated. This proposal would force an organizational constraint that is antithetical to the goals we want to achieve with the specification.

The problems raised by @TravisSpomer in #145 (comment) surface some of the concerns we have with this approach and the solutions suggested by @c1rrus in #145 (comment) address how to achieve similar outcomes within the existing spec. #110 is still open and may be a better place to provide further feedback on how to mark tokens as "private" in order to alter their behavior within different tools.