Discontinuities of chromaticity diagrams colours in "colour.plotting" package.

[fangjy88]

for example, function visible_spectrum_plot() renders the visible spectrum on sRGB values, there are 5 discontinuities of RGB values on the figure, which locates at around 460nm, 465nm, 520nm, 550nm and 610nm respectively. The discontinuity locates at the intersection of spectrum locus and the extension line of sRGB triangle. I think it belongs to the problem of gamut mapping, that is the gamut of spectrum locus is out of the gamut of sRGB. When XYZ_2_sRGB() is called for the spectrum colors, negative numbers will switch to another channel suddenly at 5 locations.
Another example is CIE_1931_chromaticity_diagram_plot() function, there is a "triangle" in the diagram.

I have tried to employ gamut mapping method on CIELAB color space to render the spectrum colors. The workflow is below: spectrum XYZ -> Lab -> hold the hue angle to gamut mapping -> HSV -> sRGB. For simplity, S and V in HSV is fixed as 1 which need to be imporved. The results are good for long waveleths but not very well for short wavelength.

Anyway maybe it is not a very important problem.

[KelSolaar]

Yes it is definitely related to the fact that sRGB / Rec. 709 colourspaces cannot encode all the visible light possible values. We are for now clipping any values not in [0, 1] domain however it would be definitely interesting to have an option to have a remapping / compression of the colours. #154 is related and I wanted to explore rendering intent: http://dba.med.sc.edu/price/irf/Adobe_tg/manage/renderintent.html

[henczati]

Hi,
I would need the single_spd_plot() in publication quality, and tried to hack my way around the colour-overshoot issue by defining kind of a desaturated variant by temporarily (just for the plot) modifying the used global CMFs' SPD definitions. I just quickly trial&error-ed some smooth(er) but still seemingly colourful looks out of it, without deep considerations.

Is any proper fix planned for the near future?

In the meantime, could I get some "measure of wrongness", or a better (easy&fast) workaround suggestion?

def single_spd_plot_desat(spd, cmfs='CIE 1931 2 Degree Standard Observer', **kwargs):
    """
    Show a single SPD plot with desaturated wavelegth colours underneath.

    The colours are in a smooth transition.

    (Hack by henczati)

    Parameters
    ----------
    spd : colour.SpectralPowerDistribution
        SPD to plot.
    cmfs : string
        Colour matching functions definition name.
    \*\*kwargs : \*\*
        Keyword arguments.
    """

    from colour import SpectralPowerDistribution

    assert type(spd) is SpectralPowerDistribution

    from colour.plotting import get_cmfs, single_spd_plot
    import numpy as np

    # TODO: Slow hack!
    # get cmfs spd
    cmfs_spd = get_cmfs(cmfs)
    # backup original
    cmfs_bkp = cmfs_spd.clone()
    # modify
    maxval = np.amax(cmfs_spd.values)
    cmfs_spd * 0.7 + (0.3 * maxval)
    # plot
    single_spd_plot(spd, cmfs, **kwargs)
    # restore original
    cmfs_spd * 0 + cmfs_bkp

from colour.plotting import get_illuminant
single_spd_plot_desat(get_illuminant('d65'))

[KelSolaar]

Hi,

It is the same issue than @fangjy88 but actually even worse because there isn't any single colour in the spectral locus that can be represented correctly by sRGB colourspace, here is the truncated output of sRGB colour values for each line of this D65 plot:

[ 0.00145703 -0.00120632  0.00836139]
[ 0.00163359 -0.00135431  0.00939296]
[ 0.00183193 -0.00152092  0.0105554 ]
...
[  1.65441405e-03  -1.79738081e-04  -1.11078335e-05]
[  1.54368828e-03  -1.67708631e-04  -1.03644183e-05]
[  1.44042806e-03  -1.56490272e-04  -9.67112441e-06]

Another way to see it: chromaticities outside the sRGB triangle cannot be represented correctly:

What the API does so far is just normalising & clipping the displayed colours such as:

colour.normalise([0.00145703, -0.00120632, 0.00836139])
# array([ 0.17425691,  0.        ,  1.        ])

An alternative could be to use a wider gamut colourspace such as ACES2065-1:

However the colours displayed would be inconsistent with your screen:

[henczati]

Thanks for the nicely aided explanation.

Wouldn't it then be a more "acceptable" solution to uniformly scale the sRGB gamut triangle "out", keeping it's shape, without rotating it and keeping the whitepoint at the same place (or, respectively, "shrink" the colour space leaving the sRGB whitepoint in-place), so it would exactly envelop the whole spectral locus?

If I understand it correctly, it would mean sacrificing colour depth (having a lot of the gamut unused) and also leading to desaturation, but would result in a continuous transition and colours that are closer in chroma to what they (theoretically) should be, wouldn't it?...well, if I'm correct, the pure spectral colours would be radially at the same place seen from the whitepoint (preserving hue), just now inside the gamut.

[KelSolaar]

Yes there is no reason it wouldn't work, as you can see the ACES2065-1 plot is very smooth. I don't think that such a colourspace exists though, however we can certainly compute its primaries.

[henczati]

It would be great if you could add it, even if just through some optional parameter.

[KelSolaar]

I have just computed an optimised set of primaries, I'll give a go at a rendering later today:

[[ 1.91940636  0.33383091]
 [ 0.25031545  1.65928545]
 [-0.48604818 -0.99162364]]

[KelSolaar]

Unsurprisingly (considering the massive gamut size) the rendering is quite dull and desaturated:

[henczati]

Yeah, now I see why nobody uses it. It is truly not as representative as I hoped. I miss the blues. :) Thanks for working it out anyway.

I suspect that if we did not have the whitepoint fixed, and just matched the scaled unrotated triangle to best envelop the locus, all the colours would be shifted towards green, wouldn't they? And if we rotate or "distort" the gamut triangle, the hues of the spectral colours would get "remapped", too (as we've seen for the ACES2065-1 colourspace).

I see that the overshoots/discontinuities in the current (default) plot are basically at wavelengths with colours corresponding to areas around the vertices of the gamut triangle. I wonder how it would look if you did something like only half the scale, when only a not too significant part of the "green lobe" would fall outside the gamut. I would expect a discontinuity in green, mainly at the 2 places where the locus goes outside the gamut, but maybe the difference in tangent of the true line of the locus and the line clipped by the gamut edge would not make that noticeable a break in colour continuity.

Makes me wonder why I do not see the discontinuities on the chromaticity diagram itself, though. Probably my untrained eyes.

[henczati]

I hoped to have something like the spectrum in this image

from the wikipedia article https://en.wikipedia.org/wiki/Visible_spectrum#Color_display_spectrum
Wonder how exactly they produced it. This was the motivation (and visual reference) for my original hack.

[fangjy88]

The four kinds of rendering intents seem to be used in ICC profile, which usually work for gamut mappings from displays to printers. Unfortunately, I have not studied very deep in ICC profile.

I guess that @henczati method of modifying CMFs actually mix 0.7 unit spectrum light and 0.3 unit reference white. Does it exactly reproduce the wiki image?

[henczati]

Does it exactly reproduce the wiki image?

No, it just looked kinda' similar. To me. Just by looking at it. Without an actual comparison. But now that you asked...

I did say trial&error and quick hack, didn't I. :) I tried 50-50, too, but it felt like unnecessarily desaturating it. I tried to find a point where it already looks smooth but still as colourful as possible.

[henczati]

It looks like they did something similar. I wager that by looking at their grey background colour, it would be simple and straightforward to determine a ratio that would get very close to their image.

[KelSolaar]

Wikipedia is usually painful because there is no way (that I'm aware of) to contact an author. However in that case the person pseudo seems to be his real identity: NickSpiker

Googling for him yields the following article: http://irphotogirl.deviantart.com/journal/A-Full-Spectrum-Interview-Nick-Spiker-445834950

Looks like it is him to me, I'll try to contact him in order to know what he did. From the Wikipedia image description it looks like the spectrum has only been averaged with some gray:

The color spectrum rendered into the sRGB color space using a gray background to preserve the actual colors. The numbers are wavelength, in nanometers.

[henczati]

Thanks.
I think it's him, too. The Wikipedia uploader is 'spigget' and https://www.elance.com/s/spigget/ seems to be the same guy.

[henczati]

@fangjy88: The "continuity/smoothness properties" of the plots (in the "non-distorted" wavelength range) seem quite similar.

[KelSolaar]

I don't think that he just mixed the grey in after the spectrum was calculated. Wouldn't he have to have a smooth spectrum already for that to work?

Actually yes I think so.

[KelSolaar]

I got it I think:

The fact you have a reddish background is because I wasn't doing chromatic adaptation from CIE Illuminant E to CIE Illuminant D Series D65 in the XYZ to sRGB conversion.

Here is the code I used for the above figure (need to find out what gives the smoothest spectra while keeping the background to minimum now):

def visible_spectrum_plot_smooth(cmfs='CIE 1931 2 Degree Standard Observer',
                                 **kwargs):
    """
    Plots the visible colours spectrum using given standard observer *CIE XYZ*
    colour matching functions.

    Parameters
    ----------
    cmfs : unicode, optional
        Standard observer colour matching functions used for spectrum creation.
    \*\*kwargs : \*\*
        Keywords arguments.

    Returns
    -------
    bool
        Definition success.

    Examples
    --------
    >>> visible_spectrum_plot()  # doctest: +SKIP
    True
    """

    cmfs = get_cmfs(cmfs)
    cmfs = cmfs.clone().align(colour.DEFAULT_SPECTRAL_SHAPE)
    cmfs += 0.5

    wavelengths = cmfs.shape.range()
    colours = colour.XYZ_to_sRGB(colour.wavelength_to_XYZ(wavelengths, cmfs),
                                 colour.ILLUMINANTS['cie_2_1931']['E'])

    colours = colour.normalise(colours)

    settings = {
        'title': 'The Visible Spectrum - {0}'.format(cmfs.title),
        'x_label': 'Wavelength $\\lambda$ (nm)',
        'x_tighten': True}
    settings.update(kwargs)

    return colour_parameters_plot([colour_parameter(x=x[0], RGB=x[1])
                                   for x in tuple(zip(wavelengths, colours))],
                                  **settings)


visible_spectrum_plot_smooth()

[KelSolaar]

I just found this article: http://www.repairfaq.org/sam/repspec/, I haven't read it entirely but it seems to point to some addition done on the CMFS.

[KelSolaar]

I discussed a bit with Nick, so apparently he is altering the RGB Linear values, by fiddling quickly I came up with that:

and ultra smooth 10th of nanometer interpolated version:

def visible_spectrum_plot_smooth(cmfs='CIE 1931 2 Degree Standard Observer',
                                 **kwargs):
    """
    Plots the visible colours spectrum using given standard observer *CIE XYZ*
    colour matching functions.

    Parameters
    ----------
    cmfs : unicode, optional
        Standard observer colour matching functions used for spectrum creation.
    \*\*kwargs : \*\*
        Keywords arguments.

    Returns
    -------
    bool
        Definition success.

    Examples
    --------
    >>> visible_spectrum_plot()  # doctest: +SKIP
    True
    """

    cmfs = get_cmfs(cmfs)
    cmfs = cmfs.clone().align(colour.DEFAULT_SPECTRAL_SHAPE)

    wavelengths = cmfs.shape.range()
    oecf = colour.RGB_COLOURSPACES['sRGB'].transfer_function
    colours = colour.XYZ_to_sRGB(colour.wavelength_to_XYZ(wavelengths, cmfs),
                                 colour.ILLUMINANTS['cie_2_1931']['E'],
                                 transfer_function=False)

    colours = oecf(colour.normalise(colours + np.abs(np.min(colours))))

    settings = {
        'title': 'The Visible Spectrum - {0}'.format(cmfs.title),
        'x_label': 'Wavelength $\\lambda$ (nm)',
        'x_tighten': True}
    settings.update(kwargs)

    return colour_parameters_plot([colour_parameter(x=x[0], RGB=x[1])
                                   for x in tuple(zip(wavelengths, colours))],
                                  **settings)


visible_spectrum_plot_smooth()

[henczati]

Thanks, it looks promising awesome. :) Will this get into the codebase soon, or should I try to hack it in somewhere myself?

You mentioned earlier that the chromatic adaptation was missing/incorrect, and that lead to a reddish tint in my plots. Do you think a similar issue might affect other computations, too?
E.g. couldn't this also be behind the cause of your note on suspiciously high differences in colour in the Smits (1999) Reflectance Recovery Method notebook?

[KelSolaar]

This last version is not touching the CMFS at all, just acting on the final RGB values, (which I subjectively prefer).

Do you think that similar missing/incorrect chromatic adaptation issues might affect other computations, too?

It might in the case of the Smits (1999) notebook or other complex studies as chromatic adaptation is a very delicate subject and people sometimes don't agree if you need to apply it or not depending the context. I'm quite confident that for the core API it is fine.

[KelSolaar]

Going back to Smits (1999) notebook after rerun it, I think I concluded back then that the differences were coming from the fact I'm interpolating the recovered spectral power distributions and it slightly changes the resulting tristimulus values.

[henczati]

That specifically interests me, as a main use I have for your package is to compare altered versions of SPDs (spectral data) with respect to colorimetric errors (i.e. how much difference it makes using one or the other).

[KelSolaar]

@henczati : Let's continue the Smits (1999) discussion on #196 if you don't mind! :)

[KelSolaar]

In latest develop branch https://github.com/colour-science/colour/:

visible_spectrum_plot(out_of_gamut_clipping=False)

single_spd_plot(colour.ILLUMINANTS_RELATIVE_SPDS['D65'], out_of_gamut_clipping=False)

I will tackle the diagrams later as I need to re-render the images.

[henczati]

Awesome, thanks. Will clone and try.

[fangjy88]

Great! @KelSolaar produced exactly as the Nick's image.

However, in the http://www.repairfaq.org/sam/repspec/ @KelSolaar mentioned, Nick's highly saturated spectrum version is still discontinuous.(2 Degree-sRGB-False-Spectra-Direct-Conversion)

It is not obvious in narrow band, but almost the same as the visible_spectrum_plot() does

[KelSolaar]

I suspect it is less obvious because of the high contrast ratio overtaking your ability to distinguish hue variations properly on his thin rendering, the fact it is presented with a white surround (browser / Github's page background) is not helping either. He has blurred the resulting spectrum too, I'm wondering if it's only a vertical directional blur or a regular square gaussian blur that will also blur the image horizontally. That would help with the discontinuities, although I'm not really into data creative adjustments for that kind of things.

[KelSolaar]

I'm closing that discussion for now. Feel free to comment and I'll reopen it! :)