Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loosen up the color type #79

Open
sbaechler opened this issue Sep 25, 2021 · 32 comments
Open

Loosen up the color type #79

sbaechler opened this issue Sep 25, 2021 · 32 comments

Comments

@sbaechler
Copy link

The color type is currently very specific, allowing only 8-bit per channel RGB(A) values.

I see two problems with this approach:

  1. Perfectly understandable and valid color definitions like hsla(300, 100%, 50%, 0.5) are not allowed.
  2. It is not future proof. We might eventually go past the sRGB color space when HDR displays become the new normal. In the future there could be color definitions with 16 bits per channel. Those could definitely make sense for gradients or in combination with filter effects.

Allowing all values that CSS understands would make it easier to build the file by hand but it would be more work for the design tools since they need a parser for the color values.

HDR for the web is already being considered and a limit to 8 bits per channel might be a problem in the future.

@c1rrus
Copy link
Member

c1rrus commented Dec 2, 2021

I believe there are 2 separate issues here:

  1. How many different ways should the format allow the same color to be written as?
  2. Should the format support color-spaces other than sRGB and/or greater precision than 8bits per channel. Or, to put it another way, should we increase the range of different colors that can be expressed?

I'll therefore address them separately:

1) Allowing the same colour to be written in different ways
Here are a few ways of expressing the exact same color (if we assume sRBG color space & 8bits per channel):

  • #ff0000
  • #f00
  • rgb(255, 0, 0)
  • { r: 255, g: 0, b: 0 }
  • hsl(0, 100, 50)
  • { h: 0, s: 100, l: 50)

This is just like how 0°C is the exact same temperature as 32°F, or 1 km is the same distance as 0.62 miles. And, as with temperatures and distances, different people will have different preferences. Some folks find HSL a more intuitive way of expressing a color and others prefer RGB. Some like hex numbers, others like decimal numbers.

From a human-friendliness perspective, offering more choices is desirable. But, in this instance, the choice is only being limited for people wanting to create or edit DTCG token files in a text editor. That's an important audience no doubt (and I certainly see myself hand-editing token files in the future! 😜), but I'm not convinced only being able to write color values as hex triplets would be a deal-breaker for many people. (For many years, that was the only way you could write colors in CSS too)

Bear in mind that the spec is only mandating how color values can be written inside DTCG token files. There is absolutely nothing stopping tools converting to or from other formats when presenting the color value or letting users input them. For example, a color picker widget in a design tool could let a user enter HSL values or even provide a choice of HSL, RBG, HSB, etc. All that the spec is mandating is that if that color gets saved out to a DTCG token file, it then needs to be converted to an RGB hex triplet.

Furthermore, from an implementation perspective, this adds complexity and increases the risk of bugs or interoperability issues. If the spec mandated that all of the above formats for writing color values needed to be supported, then every tool that implements our spec would need to have code that can:

  • Look at a color value and figure out which of those formats it is, so that it can...
  • Convert from that format to whatever internal represenation of color values that software uses

Granted, it's not too complex to do and there are plenty of color conversion libraries out there that can help. But, nonetheless it is extra work, more code to test and maintain and more chances for bugs to creep in. It might also be a barrier to entry for small teams or inexperienced programmers wanting to make tools that do things with token files. And the more formats we permit, the more that complexity grows.

I strongly believe that interoperability is an important goal of the DTCG's file format. You should be able to save tokens in one tool and open them in any other tool and it should "just work". So, what happens when one tool has a bug in how it handles the many color formats, or another tool forgets to add support for one? Interoperabilty will suffer. Color tokens exported by tool A might not be able to be imported into tool B anymore.

If our spec allows 1 or only a small number of color formats, we keep that implementation complexity low and, hopefully, ensure a higher degree of interoperability between tools.

As you can tell, I'm very much in favour of keeping the number of permitted color formats in our spec to a minimum. However, I don't have a strong preference as to which one(s) we choose. Our current draft only allows hex triplets, but that could be changed to something else or extended to allow an alternative.

Another angle to consider is that whatever we settle on for version 1 of the spec, can be expanded in future versions of the spec. Obviously, we'd want to do so in a backwards compatible way (i.e. tools that support v2 files should still be able to read v1 files without problems), but adding more formats (within reason) shouldn't be a problem.

2) Allowing a greater range and/or precision of color values
This is something I'm keen on too. Many modern devices have amazing screens than represent more colors than ever before. Folks should be able to take advantage of that in their design tokens.

But, does that necessarily need to be in v1 of the spec? If we stick with just 8bit per channel, sRBG hex triplets for now, there is nothing stopping us broadening that in future versions. Just like CSS once only had hex triplets and now has many more formats and support for different color spaces, we too could add those things in future versions (in fact, we could just adopt the same formats CSS uses - as others have already suggested in comments elsewhere).

As far as I know, popular design tools like Figma don't support these things yet and browser support is still limited as well. So, in the spirit of keeping things lean and simple for now to get a v1 spec sooner rather than later and then also gaining adoption across many tools and some momentum, my preference would be not to incorporate this in the spec right now. But I'd definitely want to revisit this topic in the future.

@NateBaldwinDesign
Copy link

I like where you’re at with this @c1rrus, especially regarding the addition of color spaces or formats in the future if needed.

One item that might be valuable is to have the color space named as a property of the color token. This is especially true for RGB spaces.

For example, if you define colors in a design tool that supports Display P3 but implementation only supports sRGB, your engineering team may want to know that in order to properly convert the values. From a cursory glance, you would not be able to know if an RGB color (whether in string, object, or hex formats) was defined in P3, sRGB, AdobeRGB, etc. Having that data attached to the color token could be very useful for teams that want to ensure proper conversion, since the color space cannot be inferred from the value itself.

@ChucKN0risK
Copy link
Contributor

ChucKN0risK commented Jan 20, 2022

Color format

Thanks for your feedback @sbaechler.

While I understand your points, I tend to agree with @c1rrus regarding interoperability.

The DTCG has 2 main objectives:

  1. Create a standard format for design tokens so that people can freely define and use design tokens whatever the tool they're using.
  2. Engage with the community and promote best practices.

Interoperability is therefore something we really strive for. The more adoption this first specification gets, the better it is. Thus, I also think we should keep the number of permitted color formats in our spec to a minimum.

The current format is an hex triplet/quartet. I don't have any preference on the format itself. I only want this value to include a potential alpha value so that users and tools can generate other formats from it.

Color space

Having that data attached to the color token could be very useful for teams that want to ensure proper conversion, since the color space cannot be inferred from the value itself.

I think this could easily be done by including the color space in the token metadata property.

@kaelig kaelig added dtcg-color All issues related to the color module specification. dtcg-format All issues related to the format specification. Needs Feedback/Review Open for feedback from the community, and may require a review from editors. and removed dtcg-format All issues related to the format specification. labels Mar 9, 2022
@romainmenke
Copy link
Contributor

The current proposal is limited in 3 ways (not 2) :

  • only 3 channels + alpha
  • only 8bit per channel
  • only supports sRGB

This actually makes it less suitable in terms of interoperability.

The CSS color() function has non of these problems.
It is still a single format but it can express :

  • any number of channels + alpha
  • 16bit per channel (and higher)
  • any color space
color(rec2020 0.42053 0.979780 0.00579);

This could also be broken down to a "raw" format :

{
  "colorSpace": "rec2020",
  "channels": [
    0.42053,
    0.979780,
    0.00579
  ],
  "alpha": "1"
}

This will ensure interoperability of color information from the start.
Without a format like this an export to design tokens will be lossy.

The initial specification can still start with only srgb as a supported color space.
But the format doesn't have to change when other colors spaces are added.

With the current format early adopters of the specification would eventually need to support two color formats and/or go through migrations.

Support for more color spaces is part of interop 2022 and large parts are already supported by webkit.

@c1rrus c1rrus mentioned this issue Jun 26, 2022
@kevinmpowell kevinmpowell added this to the Next Draft Priority milestone Oct 3, 2022
@kevinmpowell kevinmpowell removed the Needs Feedback/Review Open for feedback from the community, and may require a review from editors. label Oct 3, 2022
@kevinmpowell kevinmpowell added Color Type Enhancements and removed dtcg-color All issues related to the color module specification. labels Oct 17, 2022
@Shrinks99
Copy link

Shrinks99 commented Nov 25, 2022

Because feedback was requested and digital color is a particular area of interest (and often frustration!) for me... Here are some thoughts!

+1 to alpha as a separate distinct channel. Alpha (according to its creators) represents a percentage of occlusion and has no interaction with the colour encoding model, colourspace, or its values until it's time to composite the image. Some formats (CSS' rgba() specifically) treat alpha as part of the colour data for convenience, but design token values can always compile down to those in the end if people so choose while offering the added benefits of specificity and modularity.

Adding other data channels (depth, normals, motion vectors, deep, etc) can be done much the same way! While alpha and depth can be represented by single values, the others mentioned here have 3 values each. At this point we're encoding full pixels, not just colours... But we were doing that before with the inclusion of "alpha" so are arbitrary channels really out of scope? 🙃

On hex codes...

+1 for using and actually enforcing non-hex values. Hex codes offer an incredibly oversimplified method of moving colour information around and can really only be relied upon to deliver 8-bit sRGB data (sometimes 8-bit P3 data if you use the macOS colour picker! Therefore, the same hex value can be interpreted completely differently by software! Awful!)

Regarding this comment in #137, I argue that hex is widely misunderstood by almost everyone who uses it! Hex codes transfer no colour space information yet many designers consistently expect them to operate as if "this is a colour value" when instead all they are actually transferring is an RGB value which means nothing without a defined colourspace — hence the sRGB assumption. Hex codes themselves obscure the 8-bit integer RGB value reality such that people really have no idea what they're actually transferring, only that when they copy this code the same colour appears in both places. I find this format is not at all empowering for non-technical users as it provides another layer of obfuscation to what people are actually doing when manipulating digital colour values.

RE @romainmenke's comment in the same thread, I would love for design tokens themselves to replace hex codes as a standardized method of colour data transfer, perhaps with the added data we could finally accurately move colour information between programs! Hex as an encoding format inherently achieves none of this and should be abandoned wherever possible.

Why not go with the status quo?

Building on that, please do not restrict this spec based on what is possible in today's design software that you are familiar with. You are writing a spec for people to follow, push the industry forward! Tailoring a spec only to what is possible in current software adds barriers to moving both software and the spec forward in the future (see SVG & digital colour). If the initial reality is that programs only implement part of a properly defined colour spec, they can do the rest later with newly added incentive to actually handle colours correctly. Here are two specific things I'd like to see:

  • Per-channel / component 8, 16, and 32-bit int / float values supported. The design programs of today pail in comparison to the computer graphics industry where 32-bit values are required for accurate photo-real compositing. Programs like Nuke and Blender exclusively support entering RGB values as floats. Given the design industry's adoption of 3D software, I would propose that these value types be supported properly to enable the accurate transport of colour information between programs that require and output this colour data through industry standard formats like OpenEXR (which notably supports every value type I've mentioned here). They haven't signed onto this spec yet and they won't if they can't. More colour aware DCCs aside, > 8 bit colour is already being used in the world for UI design work today! The browser and subsequently Electron aren't the only content delivery methods. ;)

  • Colourspace included as a seperate field. RGB is a colour encoding model, sRGB, P3, and rec2020 are all colour spaces with a defined white point, primaries plotted against the standard observer model, and a transfer function that relates the non-linear encoding of RGB values within the colourspace to the linear nature of light. We can probably skip defining colour encoding models separately as they can be reverse engineered from the colorspace value if need be, but this value is a MUST. Without it you are making assumptions and assumptions (as detailed above) will eventually result in an inaccurate transfer of colour information at the user level. Even with sRGB this colourspace value should be defined every time. Looks like this is already happening and there aren't many arguments about not doing it so uh, thank you! :)

Thank you for reading my spiel <3 Here's one bonus question!

What's in scope regarding colour?

The "design systems world" seems to revolve pretty much entirely around digital work, but as I'm sure we're all aware, component & defined-style-centric design work extends past the screen and the need for sharing colour data extends past RGB. Is the CMYK colour encoding model in scope? Data needs to travel along with that too, and it's all printer specific information which is hard to standardize in specs like this! Perhaps this would be something to leave for a later version of the spec... but something to think about!

@NateBaldwinDesign
Copy link

@Shrinks99 recommendation here is spot on:

  1. Per channel integer floats
  2. Color space as another field

I would go a step further to suggest some consideration around whether the channels should or should NOT be labeled. In other words, channel values as an array. The reason for that is colors are defined as tuples in all color spaces, and the association of what those values represent is done via color space. So an sRGB color may be represented like this:

value: [0, 0.5, 1],
colorspace: 'sRGB'

Regarding floats, you may not need any defined bit depth, or really any "support" for depth— just capture precise decimal values and consuming applications can match closest supported depths on their end.

Finally, regarding the color space field, it is worth starting with ICC profiles— these are common in the industry for standardizing how colors are interpreted and converted between spaces. It doesn't account for all spaces that designers may use, but will ensure standards for common ones like sRGB, cmyk, etc.

@Shrinks99
Copy link

Shrinks99 commented Nov 30, 2022

@NateBaldwinDesign I agree that the "channels" value should not be labeled, storing them in an array makes sense to me and allows for an extensible system of arbitrary channels and data types where we can define the encoding model, the colourspace (how software should interpret these values), and finally the values themselves in the array. I'd also like to ammed my previous comment about skipping the encoding model, it would be useful to specify both in the case of normal map channels where there are two different common encodings (OpenGL & DirectX) but both should be interpreted as data. I'm sure this isn't the only use case where it's useful to seperate the two.

Perhaps something along these lines...

{
"color": {
  "encoding": "rgb",
  "colorspace": "srgb",
  "channels": [0, 0.54321, 1]
},
  
"alpha": {
  "encoding": "data",
  "colorspace": "data",
  "channels": [0.5]
},

"normals": {
  "encoding": "directx",
  "colorspace": "data",
  "channels": [0, 0.54321, 1]
}
}

I don't know why I had it in my mind that different numerical data types should be supported, maybe becuase I wrote that late at night. 😅 Storing all values as floats all the time is a good idea.

As for enforcing integer floats however, it is worth letting users and programs write values in a design token that are open-domain. RGB values "higher than 1" are often required when doing photoreal compositing tasks. Software like Nuke and After Effects allows users to blow their pixel values past their set gamut boundary and will not clip values until they get sent through the EOTF either on-write or in the viewport. For most design software today of course this isn't the case (as much as I'd like it to be) and I don't think it's unreasonable for those open-domain values to be clipped according to their encoded colorspace tag when the token is used (though perhaps with a warning) in less colour aware programs. I would expect software to handle things similarly when shifting out of gamut colours into the destination space in the case of using P3 encoded design token values in an sRGB document for example.

Similairly, if we want design tokens to fully accomidate the intricacies of digital colour, they should be able to store negative values as well. CIE XYZ tristimulus values will plot to negative values if they are outside the destination gamut and the same will occur when reprojecting P3 values onto sRGB. As such, despite the fact that they may be outside of the current working space gamut (which we've already defined as a feature of some image manipulation software) users should be able to encode these as they are not actually "invalid" values at all. I imagine this will rarely be used in practice, but in cases where negative values are present in a design token but once again unsupported by software they should clip to 0 and again, show a warning. In any case, I'd just once again make the case that we shouldn't limit ourselves to what most software can do and instead strive to properly implement the transfer of digital colour values while giving software developers a path to implementing digital colour correctly.

@NateBaldwinDesign
Copy link

NateBaldwinDesign commented Dec 1, 2022

@Shrinks99 Good call out on negative values and values greater than one. This has me rethinking a bit here— mostly the floating point values are a recommendation for RGB colors, as it’s a format that is used but also supports higher color depth than 8 bit channels as we see in browsers (0-255). For RGB, it’s a simple conversion (plus rounding) to get 0-255 equivalent, so it would scale well for folks using RGB. But yes, if defining colors in an opponent color space, negative values are needed and values typically go up to (and above) 100. Cylindrical models would need up to 360 as well, so restricting min/max values for channels is not a great idea.

I’m not familiar with use cases for an encoding value, as an ICC profile name or color space defines that information (with the exception of bit depth in RGB).

I agree there is some flexibility gains by not clamping values to their colorspace. It also ensures the value is resolved on the consumption side, when rendering context is applied (ie how you want to map back into the color gamut, such as perceptual, relative colorimetric, or absolute colorimetric).

One point I would question is using negative values in sRGB to specify a color in the P3 gamut… that sounds like a bad practice, but I don’t know that I would expect the spec to enforce these rules. There are lots of cases where it’s easy to assume the data is wrong, when it’s actually what the user wants. Perhaps at least some degree of flagging in the system to ensure it’s an intended token value rather than a mistake, but still allowing users to keep the value of they want it that way.

@Shrinks99
Copy link

Shrinks99 commented Dec 1, 2022

Good point with the cylindrical models, obvious in hindsight!

I’m not familiar with use cases for an encoding value, as an ICC profile name or color space defines that information (with the exception of bit depth in RGB).

With normal maps the texture is stored in RGB channels but the green channel either signifies +Y in OpenGL or -Y in DirectX. Artists authoring them (in this imaginary use case picking the value of part of a normal map which is actually stored in non-rgb channels to transfer that value between applications) must keep this in mind while creating their textures to ensure they are properly interpreted by the renderer. Therefore, it's potentially useful to specify both the colorspace (in this case interpret these values as data and forego any transformations) and the encoding format (how the green channel should be interpreted specifically).

Maybe that's the only situation where this would be helpful and it's niche enough to not bother including?

that sounds like a bad practice, but I don’t know that I would expect the spec to enforce these rules.

Just an exmple, it wouldn't be what I would consider a good workflow at all! And also I agree.

@romainmenke
Copy link
Contributor

romainmenke commented Dec 1, 2022

I might be missing something but isn't this an over complication of the issue?

It should be sufficient to have these two things :

  • srgb support to accelerate implementations
  • a color space to express all possible colors
  • a format that can ideally express both

Design tokens are never a final product, so there is no need to support all complexities around color. It is only important to be able to accurately express all possible colors.

{
  "srgb-color": {
    "$type": "color",
    "$value": {
      "colorSpace": "srgb",
      "channels": [0.1, 0.2, 0.3],
      "alpha": 0.6
    }
  },
  "wide-gamut-color": {
    "$type": "color",
    "$value": {
      "colorSpace": "xyz-d65",
      "channels": [0.1, 0.2, 0.3],
      "alpha": 0.6
    }
  }
}

When writing tokens to a file, design tools can "gravitate" towards srgb.
If the color is in gamut for srgb it must be stored with "colorSpace": "srgb",.
Only when out of gamut must it be "colorSpace": "xyz-d65",.

xyz-d65 is able to store all possible and impossible colors.

This will create a smoother transition for tools that do not have color space support today.


This format can be extended in the future to allow custom color spaces with arbitrary numbers of channels.

This is mainly useful for print where spot inks, varnishes, ... can be expressed with extra color channels.


It should not be the concern of the design tokens file format how a designer picks a color.
This can be a color picker for hwb, oklch, hsl, rgb, ... but the color value isn't influenced by the preferred picker. srgb|xyz-d65 is sufficient.

It should also not be the concern of the design tokens file format how the color token is used. Translation tools are responsible for generating the correct color format for final use.


But as I said before, maybe I am missing the point here :)

@ilikescience
Copy link

ilikescience commented Dec 2, 2022

I was initially very pro-hex, but seeing the comments I have been convinced that in order to define a color accurately, an author should use the proposed colorspace + channels/components + alpha format. This will allow a color to be accurately translated for any target platform, without requiring a the translator or platform to make any assumptions.

To maintain the maximum amount of compatibility and ease of writing, Maybe we go with the following:


If a token has a $type of color, the $value MUST be:

EITHER
A string containing the six-character sRGB hex value ("#0000ff") or eight-character hex+alpha value ("#0000ffff"). If a six-character hex value is present, it is safe to assume the color is fully opaque (alpha value of "ff" / "100%" / "255").

OR
An object containing the following properties:
- REQUIRED: The color space to be used when interpreting the color, as a string.
- REQUIRED: the non-alpha components1 of the color listed as an array of floating-point numbers or integers].
- OPTIONAL: The alpha component of the color listed as a single floating-point number or integer in the range [0,1]. If the alpha component is omitted, it is safe to assume the color is fully opaque.


✅ So this is valid:

{
  "srgb-color": {
    "$type": "color",
    "$value": {
      "colorSpace": "srgb",
      "components": [
        0.1,
        0.2,
        0.3
      ],
      "alpha": 0.6
    }
  }
}

✅ This is also valid:

{
  "srgb-color-hex": {
    "$type": "color",
    "$value": "ff0000ff"
  }
}

✅ And this:

{
  "srgb-color": {
    "$type": "color",
    "$value": {
      "colorSpace": "srgb",
      "components": [
        0.1,
        0.2,
        0.3
      ]
    }
  }
}

❌ But this is not:

{
  "srgb-color-rgba": {
    "$type": "color",
    "$value": "rgba(255, 255, 255, 1)"
  }
}

As for which color spaces to support - I think it's safe to allow the value of "colorSpace" to be anything. It's best for us to allow translators to choose which color spaces they want to support beyond srgb - including wide gamut like display P3 or fancy futuristic spaces like xyz-d65, or all the color spaces we haven't invented yet.

Footnotes

  1. using "components" instead of "channels" is my preference.

@ilikescience
Copy link

Oh I just read all of the discussion on #137, and it seems like there's a lot of convergence. Generally speaking, I sense a general consensus around: 1. Allowing authors to write colors as hex values, and 2. Allowing authors to be more precise and/or descriptive if they'd like to be (allowing for more color spaces than sRGB).

@romainmenke
Copy link
Contributor

romainmenke commented Dec 2, 2022

As for which color spaces to support - I think it's safe to allow the value of "colorSpace" to be anything. It's best for us to allow translators to choose which color spaces they want to support beyond srgb - including wide gamut like display P3 or fancy futuristic spaces like xyz-d65, or all the color spaces we haven't invented yet.

What is the practical benefit of allowing colorSpace to be anything?

I think this must be strictly defined.
If it can be anything tool creators can not write the code for the conversions.

fancy futuristic spaces like xyz-d65

In what way is xyz-d65 futuristic or fancy?

Generally speaking, I sense a general consensus around: 1. Allowing authors to write colors as hex values,

There definitely isn't a consensus about hex or a dual format.
I am strongly against hex strings. I don't see why I need to complicate my tool's implementation by adding support for this format in the parser logic.

using "components" instead of "channels" is my preference

This is unusual, the most often used technical term is "channels"


I think there is some confusion between certain things:

  • color profiles for screen/print
  • device capabilities
  • color pickers in design tools
  • color data in a platform specific format
  • color data format
  • color spaces

color profiles for screen/print and device capabilities

That a person's display is limited to ±sRGB or ±displayP3 today must not influence how we store color data. Tomorrows screens might be ±rec2020.

Design token files also aren't pdf.
These will not be send to a printer who has a custom ICC profile specific to their equipment.

Design token files will always be processed further.

color pickers in design tools

In a design tool's color picker it makes sense to have multiple color models like hsl, lch, oklch, rgb, hwb,... as these allow designers to use different mental models to pick and tweak colors.

These color models have no benefit in a raw data format.
A computer doesn't have a preference or opinion about these things, it only needs a format that is able to correctly record the color with high enough fidelity.

If a designer prefers to work in oklch but only picks colors that fit within the sRGB color space then the output could be written as sRGB to maximize compatibility with other tools.

color data in a platform specific format

Some platforms (iOS, Android, CSS) might have limitations today and maybe only support sRGB or only displayP3.

With this I mean the API's to instruct the platform to use a specific color, not the device capabilities.

CSS in Firefox can only express sRGB today but this could be on a device capable of rec2020.

This is the responsibility of translation tools to form the bridge between raw data and what can be expressed on a specific platform.

It can be a tool specific feature to allow developers to format color data as a specific color model. For example : output everything in CSS as oklch. But this is unrelated to the token data itself.

color data format

This is purely the data structure, not the contents.

  • hex format
  • structured color format

#fe05c2

{
  "srgb-color": {
    "$type": "color",
    "$value": {
      "colorSpace": "srgb",
      "channels": [0.1, 0.2, 0.3],
      "alpha": 0.6
    }
  }
}

If the expectation is that colors will be mostly manually typed then hex can be useful.
However this format is intended to be written and read by software.

Do we need the extra complication of a dual format?
It adds extra burden on tools to write the parser logic for colors.
It isn't much, but little things add up.

This was asked in detail here : #149

color spaces

These are purely a mapping of color information.
By going for something like xyz-d65 today the specification will be future proof.

Design tools only need limited conversion logic between internal color representation and xyz-d65.

Translation tools can be platform specific and can specialize in converting from xyz-d65 to the largest color space supported on their respective platforms.

If the list of color spaces to support is exhaustive then all tools will need a lot more logic for all the possible combinations.


Only the data format and the list of color spaces are a factor for the design token file format. Everything else is irrelevant.

@ilikescience
Copy link

What is the practical benefit of allowing colorSpace to be anything?

It gives authors and parsers the maximum latitude, and makes it much easier for this format to be forward-compatible. If we define error handling well (eg, what happens when a parser sees a color space it doesn't support), I see the agreement between authors' needs and parsers' feature set as being a sort of "invisible hand" that can guide the ecosystem.

In what way is xyz-d65 futuristic or fancy?

I realize that "fancy futuristic color spaces" might have sounded pejorative. I'm sorry, I should have phrased that differently.

Fancy because:

  1. xyz is unintuitive, eg white is [1,1,1] in rgb but [95.047, 100.000, 108.883] in xyz-d65
  2. converting colors into xyz space requires linear algebra – from srgb needs the additional step of conversion to linear rgb before going to xyz

Futuristic because:

  1. most design tools don't support xyz colors today
  2. css color 4 does support xyz, but it isn't implemented yet in most browsers

I am strongly against hex strings. I don't see why I need to complicate my tool's implementation by adding support for this format in the parser logic.

I think the fact that hex strings are one of the most commonly-used and commonly-supported color formats makes them an extremely useful medium of exchange. It would be good to be clear about how much complication it introduces, compared to the value it represents to token authors.

This is unusual, the most often used technical term is "channels"

A bit of bikeshedding, but a few examples:

Wikipedia uses both interchangeably. From RGB Color Model: "The color is expressed as an RGB triplet (r,g,b), each component of which can vary from zero to a defined maximum value." And "This indirect scheme restricts the number of available colors in an image CLUT—typically 256-cubed (8 bits in three color channels with values of 0–255)"

The CSS color module level 4 spec also uses them interchangeably. "Colors in CSS are represented as a list of color components, also sometimes called 'channels'."

SwiftUI's docs use 'component' exclusively: "Specify component values, like red, green, and blue; hue, saturation, and brightness; or white level."

Kotlin's docs use 'component' exclusively: "A color int always defines a color in the sRGB color space using 4 components packed in a single 32 bit integer value".

I don't feel strongly either way, "components" is just a personal preference.


If the expectation is that colors will be mostly manually typed then hex can be useful.
However this format is intended to be written and read by software.

Do we need the extra complication of a dual format?
It adds extra burden on tools to write the parser logic for colors.
It isn't much, but little things add up.

If token files are meant only as a data interchange format, it follows that a color should be encoded in a single way (to reduce encoder/decoder complexity), in high fidelity (many bits per channel/component), without any loss (widest possible gamut). An array of floats representing a xyz-d65 color plus a single float for alpha would be a great candidate for this.

If token files are meant only as human-written-and-read documentation tools, it follows that an author should have the freedom to write colors in whichever format is most convenient to them, as long as that format is unambiguous to whoever is reading the file later.

In reality, we're trying to land somewhere in the middle. I don't think we'll ever have a perfect measuring rod for the solution, but it's going to involve some compromise between parser complexity and author ergonomics.


Only the data format and the list of color spaces are a factor for the design token file format. Everything else is irrelevant.

100% agreed.

@romainmenke
Copy link
Contributor

romainmenke commented Jan 24, 2023

What is the practical benefit of allowing colorSpace to be anything?

It gives authors and parsers the maximum latitude, and makes it much easier for this format to be forward-compatible. If we define error handling well (eg, what happens when a parser sees a color space it doesn't support), I see the agreement between authors' needs and parsers' feature set as being a sort of "invisible hand" that can guide the ecosystem.

That is not technically accurate.

1 The specification allows colorSpace to be any string value.

  • tool X might have an implementation for a value : alpha.
  • tool Y might have an implementation for a value : beta.

Neither are real color spaces.
Both tools are following the specification.
This would be allowed.

There would be no interop of design tokens between tools.

2 The specification allows colorSpace to be any known color space but it doesn't specify a list of color spaces. It only requires that a color space must have a specification published by orgs/groups X, Y, Z,....

  • tool X might have an implementation for a value : alpha.
  • tool Y might have an implementation for a value : ALPHA.
  • tool Z might have an implementation for a value : beta.

All are real color spaces.
All tools are following the specification.

There would be no interop of design tokens between tools.

  • alpha is a new color space that was released "today".
  • tool X might have an implementation for a value : alpha.
  • tool Y doesn't have support (yet) for alpha.

All tools are following the latest specification and can claim to implement it fully.

There would be no interop of design tokens between tools.


Only when all tools use the same notation for the same list of color spaces can we have interop.

This is why it can not be any value.
It can still be a very long list if people think there is value in that (I don't).
But it must be a list in this specification.


Allowing this to be any value doesn't benefit forwards compat in any way.
Even worse it makes it harder by not triggering an error.

Tools that encounter a color space they do not support would have to invent some magic to still produce an output token but would not be able to correctly process the actual value.


I don't feel strongly either way, "components" is just a personal preference.

Interesting!
Yes I think this is something that can differ greatly based on personal experience.

I also do not have a strong preference.
As long as it is something that makes sense for a lot of people.


In reality, we're trying to land somewhere in the middle. I don't think we'll ever have a perfect measuring rod for the solution, but it's going to involve some compromise between parser complexity and author ergonomics.

I don't mind a more complex parser.
But currently the specification doesn't support this.

There is only $type and that is not sufficient to have both a simple hex string and a future proof color notation as referenced above.

Either the specification needs to be greatly expanded or we will have to make compromises on what can be expressed in the format.

If we have to pick either hex or a future proof color notation then I think it is obvious that we should not pick hex. But that is my personal opinion :)

@romainmenke
Copy link
Contributor

But currently the specification doesn't support this.

Just to clarify, this is a technical issue.

"$type": "color",
"$value": "#fff"
"$type": "color",
"$value": {
  "colorSpace": "srgb",
  "channels": [0.1, 0.2, 0.3],
  "alpha": 0.6
}

In the specification $type is the only signal for how to interpret a value.
So if I create a design token parser I need to be able to write this function :

function parse(token) {
  switch (token.type) {
    case "color":
      return parseColor(token.value);
    // ...
  }
}

I am only supposed to differentiate based on the value of $type.
What does parseColor do?

@romainmenke
Copy link
Contributor

About hex : https://chriscoyier.net/2023/02/01/hex-colors-arent-great-at-anything-except-being-popular/

@kaelig
Copy link
Member

kaelig commented Feb 2, 2023

One factor to consider is that HEX->HSL->HEX conversions don't guarantee that the initial HEX value and the resulting HEX value are equal.

This can lead to awkward situations where design tools could author colors in HEX, and code pulls them in HSL, but then a token conversion resolves those... to a different HEX value.

(I recently came across this issue for the first time... To solve it, we simply started using HEX across the board, and we have predictable outcomes)

@romainmenke
Copy link
Contributor

romainmenke commented Feb 2, 2023

@kaelig Can you elaborate on that?
There is no color technical reason that should be happening.

Was this a rounding error with floating point values, a bug in a tool,... ?
Which hex value?

To solve it, we simply started using HEX across the board

This strategy can work for one team if they control all design and dev tools, but it doesn't scale well to the design tokens format.

Color conversions must be implemented well enough to avoid this issue.
I don't think it is an option to enforce hex everywhere for all eternity because one tool had a bug.

Ideally designers have freedom to use any tool and any color format that works best for them and this just works reliably in whatever output platform. The design token format maybe only supports one format but it shouldn't place limits in my opinion.

@kaelig
Copy link
Member

kaelig commented Feb 2, 2023

I'm not familiar with the mathematical transforms enough to understand how conversions are made between HEX and HSL, but several packages and tools I've tried ended up reproducing this:

#DFFEF0 -> hsl(153, 94%, 94%) -> #E1FEF1

It does look like a rounding issue because tools round up what should perhaps be:

hsl(152.9,93.9%,93.5%)

What do you think? With enough backing we could get tools to fix this!

@romainmenke
Copy link
Contributor

romainmenke commented Feb 2, 2023

Thank you for sharing this!
Can confirm this is happening.

Because hsl and rgb describe different color models and different channels the whole (integer) numbers land on slightly different points in the srgb color space.

If the in-memory color representation is not rounded you will be able to cycle through different representations without altering the value. (I only took this scenario into account in my earlier comment)
When storing the values as string however they will be rounded in both hex an hsl formats.

#DFFEF0 in hsl without rounding : 152.90322580645162 93.93939393939395 93.52941176470588

rounded hsl to hex :

  • #E1FEF1

unrounded hsl to hex :

  • #DFFEF0

The only solution to this is to store colors in a 16bit per channel format, a.k.a. floats.


What do you think? With enough backing we could get tools to fix this!

That would be awesome :)

@NateBaldwinDesign
Copy link

Yes 100% — color data should be stored as floats, per channel. 16 bit per channel should be sufficient. This ensures color conversion precision as demonstrated, but also ensures precision of color when used in technology that supports higher bit depth of color.

Furthermore, sRGB should not be the primary color space for storing color data. Entry and consumption of that data may be in sRGB by majority (web apps), but it does not scale. Many technologies, including recent CSS updates, support wide gamut color.

Gamut-awareness and bit depth, in my opinion, are of equal importance when it comes to a standard for storing color data. Although there's been discussion on color space support elsewhere. Hex is a convenience for manual evaluation and memorization, but that's it.

@romainmenke
Copy link
Contributor

romainmenke commented Feb 7, 2023

It seems there is growing consensus that there is a need for 16 bit per channel, wide gamut support and that srgb hex isn't good enough.

If there are still concerns that hex is required to gain adoption it could be an option to have a single value definition that has optional subfields?

  • hex subfield for a fallback srgb color

  • channels subfield for color information in 16 bit per channel, wide gamut

  • alpha subfield (shared field for both hex and channels)

  • all color tokens must include hex

  • optionally color tokens can include channels + colorSpace

  • colorSpace must be one of several color spaces referenced within this specification

Valid color values :

{
  "foo": {
    "$type": "color",
    "$value": {
      "hex": "#000",
      "colorSpace": "xyz-d65",
      "channels": [0.1, 0.2, 0.3],
      "alpha": 0.6
    }
  }
}
{
  "foo": {
    "$type": "color",
    "$value": {
      "hex": "#000"
    }
  }
}

Invalid color values :

{
  "foo": {
    "$type": "color",
    "$value": {
      "colorSpace": "xyz-d65",
      "channels": [0.1, 0.2, 0.3],
      "alpha": 0.6
    }
  }
}

Tools must follow this logic when reading :

  1. if you don't support 16 bit per channel, wide gamut values
    1.1. use the hex fallback
  2. if the token doesn't have a 16 bit per channel, wide gamut value
    2.1. use the hex fallback
  3. if you don't support the color space
    3.1. use the hex fallback
  4. use the 16 bit per channel, wide gamut value

Tools must follow this logic when writing :

  1. if you don't support 16 bit per channel, wide gamut values
    1.1. emit a token with only a hex field
  2. emit a token with a hex field and a 16 bit per channel, wide gamut value
    2.1 if the value is within the srgb color space
    2.2 do a simple conversion
    2.3 else, apply gamut mapping

2.3. is lossy, this can not be avoided

This gravitates heavily towards hex.
But it also places pressure on tools to support modern formats because the conversion to hex can be lossy.

None of this complicates the design token format as a whole.
The logic around $type and $value is still valid.

@Shrinks99
Copy link

optionally color tokens can include channels + colorSpace

Colourspace information is not an option when defining colours. It is a requirement. The value of RGB = 0.5 is unknowable without this information. Is it in sRGB, Display-P3, linearized sRGB? You can make a guess but there is no way of knowing what colour "0.5" represents without a colourspace value being defined. Even with hex codes!

But it also places pressure on tools to support modern formats because the conversion to hex can be lossy.

There is no pressure placed on anybody if this spec adopts hex codes in any way. The rounding issues you mention are also only present if colours are defined with a higher bit depth than the tools that use the token are capable of. I consider this a feature and not a bug.

  • If users require absolute consistency throughout their pipeline they should pick colours as 8-bit values. Those values should be stored as 16-bit floats, and they should work fine with proper reproduction once converted back to 8-bit integers by programs that can't handle anything else.

  • If users do not care about this and pick values outside of the destination space with 16-bit float percision they will be reasonably transported through their pipeline and converted to the destination space & bit depth with the percision one would expect. In this way the fallback is provided for automatically with the rendering intent set in the destination program be it a design suite or a CSS pre-processor.

In both cases the design token spec would provide a robust container for colour information without carving out fallbacks that will likely just become defacto standards — why bother with 16-bit float values and colourspace conversion if all the other design apps just default to 8-bit sRGB hex values anyways right? It's certianly not "idiot proof" but neither is the hex fallback method, and in both cases values outside of the destination space are going to end up with a fundementally different colour as the fallback anyways.

Forcing developers to put in the legwork to handle the conversion of colourspaces and bit-depths also has its risks... I believe that providing a spec that pushes the industry forward by encoding colours properly and providing documentation on how to handle this would be the best approach. Providing a recommendation for an 8-bit sRGB compatibility mode in colour pickers might be a good path forward? In terms of adoption, I think any barriers created by this approach are incentivised to be solved by the organizational value that design tokens provide.

@romainmenke
Copy link
Contributor

romainmenke commented Feb 7, 2023

@Shrinks99 I fully agree and have been advocating for exactly that for months.
I personally don't see the point in hex support. But the counter argument has been that everyone uses hex today and that there are concerns that tools won't implement the design token file format because it is too much work to handle color values.

That is why I started my previous comment with :

If there are still concerns that hex is required to gain adoption


Colourspace information is not an option when defining colours. It is a requirement. The value of RGB = 0.5 is unknowable without this information. Is it in sRGB, Display-P3, linearized sRGB? You can make a guess but there is no way of knowing what colour "0.5" represents without a colourspace value being defined. Even with hex codes!

This is incorrect.
The specification can define hex rgb as srgb only.
This is how it works in CSS/browsers.

There is no need to add a color space keyword with each hex value if the specification requires that any hex is interpreted as srgb.

This makes this token valid :

{
  "foo": {
    "$type": "color",
    "$value": {
      "hex": "#000"
    }
  }
}

It is a little bit more work to type than :

{
  "foo": {
    "$type": "color",
    "$value": "#000"
  }
}

But it still adheres to the principle that design tokens files should be easy to edit by humans : #149

It is definitely easier than writing 16 bit per channel, wide gamut color values.

I don't personally agree with this principle but I try to follow it when making suggestions.

@Shrinks99
Copy link

@romainmenke I know you have, I've appreciated your thoughts in this thread! To everyone else giving those counter arguments, I appeal to you to not appeal to mediocrity. ;)

There is no need to add a color space keyword with each hex value if the specification requires that any hex is interpreted as srgb.

So it's defined somewhere! I'd give consistency bonus points for adding it at all times — encouraging a complete colour-aware standard would be good? Programs only capable of writing sRGB design tokens can hard code the value. Technically speaking however, this is good enough for me :)

Apple's system colour picker is notably able to deliver P3 hex codes.

Screenshot 2023-02-07 at 11 25 34 AM

I don't personally agree with this principle but I try to follow it when making suggestions.

Fair enough. I'd advocate for correctness over convenience of manually typing things in that are wrong? Add me to the list of people who think bending this rule is okay. 🙃 IMO human readable > easily human writable, especially regarding colour.

@romainmenke
Copy link
Contributor

romainmenke commented Feb 7, 2023

I'd advocate for correctness over convenience of manually typing things in that are wrong? Add me to the list of people who think bending this rule is okay. 🙃 IMO human readable > easily human writable, especially regarding colour.

Give it a few years for good editing tools to be created and no one will be writing token files by hand.
(who enjoys fiddling in a json?)

Being stuck with legacy bits of data in each token value for eternity (backwards compat) is the price we will pay for this.

Long terms gains vs. Short term inconvenience.

And as you say there is the risk that a small number of (critical) tools keep using hex srgb, holding everyone back.


In case anyone reading is doing an implementation of a translation tool in JavaScript:

We do color conversions to generate fallback CSS for modern notations.
We very recently bundled and published the color conversions as a separate package : https://github.com/csstools/postcss-plugins/tree/main/packages/color-helpers

import { conversions } from '@csstools/color-helpers';

console.log(conversions.cie_XYZ_65_to_sRGB([0, 1, 0]))

This also handles gamut mapping.

It is derived from the sample code found in : https://github.com/w3c/csswg-drafts/tree/main/css-color-4

@kaelig
Copy link
Member

kaelig commented May 16, 2024

A group of editors met today and we feel very close to a resolution on this.

We have an important question regarding color spaces that I asked here, and I'd appreciate folks who weighed in on this thread to also take a look: #137 (comment) 🙏🏻

(sorry this is becoming a bit scattered across multiple threads!)

@doozMen
Copy link

doozMen commented Sep 13, 2024

Could I make this interesting discussion even more complicated? The current spec allows for gradients to be part of the color type as there is no gradient type yet. So the discussion should also include how to include gradients under color no? Or would it not be easier to add the gradient type?

@drwpow
Copy link
Contributor

drwpow commented Sep 13, 2024

The current spec allows for gradients to be part of the color type as there is no gradient type yet.

@doozMen Not sure I follow. There is a gradient type defined? Unless you mean “does this also apply to the color within gradients” and the answer is “yes.”

@doozMen
Copy link

doozMen commented Sep 18, 2024

The current spec allows for gradients to be part of the color type as there is no gradient type yet.

@doozMen Not sure I follow. There is a gradient type defined? Unless you mean “does this also apply to the color within gradients” and the answer is “yes.”

hmm it is then wrongfully implemented by token studio that puts colors and gradients under the same type. Wrong report here

@SamIam4Hyma
Copy link

@doozMen you are correct. Tokens Studio has been supporting Gradients for a couple of years now, long before gradient was defined as an independent Token Type in the spec.

We have an item for this on our requests & roadmap tool. If you want to track the progress on that item, feel free to subscribe for free.

https://feedback.tokens.studio/p/add-gradient-token-type-and-expand-functionality-past-linear

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests