token-transformer merges tokens before deciding what set to exclude

[six7]

If you have the following structure:

themes
  Foundations
  Light
  Dark

and the "Dark" theme is setup like this:

Foundations: SOURCE
Light: SOURCE
Dark: ENABLED

And your Light theme contains all tokens that the dark theme uses:

Foundations:
{
  "color": {
    "neutral": {
      "10": {
        "value": "#E6EFFF",
        "type": "color"
      },
      "50": {
        "value": "#2B70E8",
        "type": "color"
      }
    }
  }
}

Light:
{
  "default": {
    "value": "{color.neutral.50}",
    "type": "color"
  },
  "gray": {
    "value": "{default}",
    "type": "color"
  },
  "background": {
    "value": "{color.neutral.10}",
    "type": "color"
  }
}

Dark:
{
  "gray": {
    "value": "{default}",
    "type": "color"
  },
  "background": {
    "value": "{color.neutral.10}",
    "type": "color"
  }
}

And you want to transform this with token-transformer like this:

npx token-transformer themes output.json Foundations,Light,Dark Foundations,Light

Your output file will be empty. Reason for that is that we probably merge tokens before we decide what to exclude. Ideally, something set as SOURCE doesn't overwrite something set as ENABLED, and we don't remove tokens that are part of any set that is ENABLED.

[krizkonte]

I think that I having a relationed issue.

Imagine that I got 3 sets:

• core (source)
• desktop(enabled)
• mobile(enabled)

If I try to select a single set as:
token-transformer tokens.json core.json core
This works is fine, and i got the core tokens only.

But if i try to use 2 sets like:
token-transformer tokens.json desktop.json core,desktop
This already return a blank json, even if I do not use the excludes command.

[alecgeatches]

Just to tag on with a similar problem that may be related from the issue description:

Ideally, something set as SOURCE doesn't overwrite something set as ENABLED

We have a font family in our core set of tokens globals and are overriding it in an enabled set. From globals:

In an enabled token set expressive-type we're overriding the font set with Aktiv Grotesk VF and use a reference to it:

We run token-transformer with globals as a source:

token-transformer tokens.json transformed-tokens.json --throwErrorWhenNotResolved --expandTypography=true globals,expressive-type globals

The resulting output uses the core value of Helvetica instead of expressive-type's override value:

We're able to work around by removing fontFamily from the globals set, but ideally we could override the base font settings per-theme. Is this the same problem, or something else?

Thank you!