[Feature] Customize error function in ColorDistanceSensor.color()

[Novakasa]

Is your feature request related to a problem? Please describe.
In my project I want to detect 2x8+ color plates on train tracks on a train with a Color & Distance sensor pointed down at the tracks. The reliability of detecting them correctly is very important. In the past, I've tried to use ColorDistanceSensor.color() along with custom detectable colors for this, but I found that using my own detection algorithm using the following error function worked much more reliably (edit: see below comment for correction):

sdelta = (color.s-test_color.s)
hdelta = ((color.h-test_color.h + 180) % 360) - 180
vdelta = color.v-test_color.v
err = 2*sdelta*sdelta + hdelta*hdelta + vdelta*vdelta

Notice that I doubled the weight for the saturation delta compared to the hue and value.

Describe the solution you'd like
The above solution probably slower and more memory-intensive than just using the builtin color matching, so I propose that we could introduce a custom weighting for hue, saturation and value in ColorDistanceSensor.color() to make it more flexible for such usecases. We could also choose between squared sum of the errors or just the sum of absolutes (or more?).
My proposed API for this would change the parameters of ColorDistanceSensor.detectable_colors(colors) to:

ColorDistanceSensor.detectable_colors(colors, hsv_weights, method)

where hsv_weights might be a tuple of weights and method could be either "squared", "linear" (or an arbitrary integer exponent > 0).

Another method could be to perform the delta in RGB space (though I realize converting to rgb is not trivial depending on the resources we want to spend on this).

Another possibility could be to provide a custom micropython callable for the error function, but then the advantage over matching the color fully in micropython might diminish.

Describe alternatives you've considered
One could implement the matching in micropython as above, which costs performance and memory.

Additional context
This issue is inspired by the discussion in #619 (comment)

[laurensvalk]

The above solution probably slower and more memory-intensive than just using the builtin color matching

Actually, not really: Fun fact, it sounds like we are almost doing exactly the same. Our current cost function is:

return hue_error * hue_error + 5 * saturation_error * saturation_error + 2 * value_error * value_error;

So maybe these weights should be configurable. It would be great if you want to have a look through the referenced code to see if you can find any mistakes.

I'm also open to changing the default weights if we find that yours are better across a range of applications.

Another method could be to perform the delta in RGB space (though I realize converting to rgb is not trivial depending on the resources we want to spend on this).

The original data is RGB so this is possible. In general we seem to be getting better results with HSV though.

[Novakasa]

Actually, not really: Fun fact, it sounds like we are almost doing exactly the same. Our current cost function is:

I was aware that the cost function looks similar, but I assume that implementing it in C is more efficient than running it in micropython.

I had referenced the source code as well when working on this, but was surprised just now to see that it seems to use a large weight for saturation. After looking into it, I might have referenced the wrong version of my implementation, since in a previous version I had used this:

sdelta = 2*(color.s-test_color.s)
vdelta = (color.s-test_color.s)
hdelta = 2*(((color.h-test_color.h + 180) % 360) - 180)
err = 2*sdelta*sdelta + hdelta*hdelta + vdelta*vdelta

Which is equivalent to the weight 8 in saturation, 4 in hue and 1 in value. This approach should have been the one that was more reliable for me than default color matching.

Since I am now using a quite different approach, as outlined in #619 (comment) I don't use an error function like this for development anymore, but since I have dumped my logged data to PC, it might be easy for me to try to get reliable weights for the error function.

[Novakasa]

Probably the main reason for bad performance of the original function is the noise in the hue due to the low saturation of the dark gray tracks and the black floor.

Here is the data for a train driving over [blue, green, red, blue] markers in HSV and RGB:

So maybe the proper way to match colors would be to calculate the euclidean distance in the hsv cone:

One can still use different weights for the "dimensions" of the cone.

At some point I did try this by basically copying from here: https://stackoverflow.com/a/39113477
But it did not immediately work and I did not spend much time trying to debug it.

[dlech]

Is "black" or "no color" in your list of matching colors? I tried a similar train color sensor project a few months ago (I didn't use it extensively, so I don't know how reliable it was long term). I found that if I only put the colors I wanted to match in the list (using the built-in color matching) then there were errors. But if I also included "colors" I didn't want, which included the color of the track and the color of the floor underneath the track, then the matching seemed to work fairly well.

[dlech]

Here is the data for a train driving over [blue, green, red, blue] markers in HSV and RGB:

Can you add a scale to the graphs so that we can see what the threshold is for the noise levels?

[Novakasa]

Is "black" or "no color" in your list of matching colors? I tried a similar train color sensor project a few months ago (I didn't use it extensively, so I don't know how reliable it was long term). I found that if I only put the colors I wanted to match in the list (using the built-in color matching) then there were errors. But if I also included "colors" I didn't want, which included the color of the track and the color of the floor underneath the track, then the matching seemed to work fairly well.

Yes, that was one of the first realizations for me as well. For detectable colors I included the track sleepers, the floor, and each of the markers colors

Here is the data for a train driving over [blue, green, red, blue] markers in HSV and RGB:

Can you add a scale to the graphs so that we can see what the threshold is for the noise levels?

Sorry about that. There are actually scales, but due to transparency they don't show up well in dark mode on github. If I click on the images, I can see the scales though.

[dlech]

they don't show up well in dark mode on github.

Sure enough - if you squint you can see them.

[Novakasa]

Here is the data for the above plots, and the plots without transparency.
Note that in this snippet I multiply the hue data by 2 since I halfed it on the hub for sending it in a bytearray. Also, the data should be "rolled" by the dump_index since that's the index where it last updated the data.

#morning, lights on, other side, with reflection, fixed hue buffer
hbuf = [81, 120, 60, 75, 0, 105, 105, 111, 111, 107, 96, 72, 90, 75, 142, 90, 100, 120, 127, 127, 111, 102, 103, 85, 94, 96, 126, 60, 110, 120, 127, 97, 112, 103, 120, 101, 
90, 90, 71, 132, 60, 90, 120, 120, 112, 120, 90, 108, 101, 107, 78, 90, 90, 90, 30, 112, 105, 130, 93, 114, 95, 105, 90, 90, 90, 120, 120, 130, 105, 105, 111, 100, 97, 75, 73, 67, 66, 66, 67, 68, 66, 66, 65, 67, 68, 65, 65, 67, 66, 66, 67, 70, 65, 66, 65, 67, 67, 67, 67, 69, 86, 90, 135, 50, 60, 90, 105, 120, 105, 90, 94, 90, 90, 
84, 96, 37, 90, 90, 130, 130, 114, 108, 107, 111, 109, 109, 85, 126, 60, 90, 45, 120, 120, 130, 78, 109, 101, 101, 85, 75, 30, 67, 90, 100, 130, 110, 97, 110, 100, 97, 90, 103, 81, 108, 108, 105, 105, 0, 110, 85, 109, 97, 100, 72, 135, 5, 177, 179, 0, 0, 0, 0, 179, 179, 0, 0, 179, 1, 0, 178, 0, 1, 0, 0, 0, 0, 0, 179, 179, 0, 177, 
177, 157, 150, 120, 120, 120, 94, 94, 105, 107, 94, 135, 100, 0, 120, 80, 112, 120, 120, 105, 103, 111, 111, 82, 127, 60, 90, 120, 142, 120, 112, 94, 107, 99, 97, 97, 90, 80, 90, 90, 105, 110, 110, 76, 112, 105, 102, 90, 90, 75, 127, 127, 130, 120, 114, 113, 108, 110, 108, 108, 108, 108, 108, 108, 110, 108, 108, 108, 108, 108, 108, 108, 108, 108, 107, 109, 110, 107, 107, 110, 110, 103, 97, 95, 96, 100, 80, 135, 60, 30, 130, 100, 120, 114, 77, 77, 97, 96, 90, 127, 60, 75, 90, 120, 120, 138, 120, 114, 90, 82, 82, 85, 105, 108, 30, 90, 90, 0, 120, 110, 120, 114, 78, 99, 103, 90, 90, 127, 60, 75, 90, 90, 90, 114, 108, 116, 100, 99, 72, 90, 110, 120, 0, 60, 120, 135, 120, 120, 120, 120, 81, 103, 94, 60, 60, 50, 0, 0, 60, 120, 75, 105, 110, 120, 111, 90, 90, 105, 124, 90, 90, 138, 120, 110, 110, 116, 100, 60, 117, 120, 90, 90, 135, 120, 112, 111, 107, 78, 115, 90, 60, 90, 120, 105, 110, 127, 117, 100, 63, 116, 100, 0, 90, 120, 130, 120, 113, 115, 103, 85, 115, 96, 60, 90, 90, 36, 60, 120, 130, 120, 120, 120, 110, 120, 114, 75, 116, 111, 51, 123, 69, 0, 30, 90, 120, 120, 120, 105, 105, 109, 81, 107, 100, 102, 116, 116, 131, 75, 30, 138, 105, 90, 120, 
105, 120, 90, 100, 94, 85, 96, 84, 145, 90, 0, 90, 120, 120, 110, 110, 127, 110, 105, 116, 101, 101, 85, 97, 125, 30, 90, 0, 130, 120, 135, 105, 120, 116, 97, 103, 80, 112, 90, 67, 114, 30, 67, 100, 120, 120, 105, 105, 102, 110, 110, 82, 99, 84, 82, 47, 140, 90, 0, 135, 120, 120, 120, 111, 120, 96, 80, 84, 100, 112, 0, 120, 0, 105, 90, 120, 105, 108, 101, 120, 120, 90, 90, 128, 75, 60, 120, 60, 75, 0, 120, 120, 120, 108, 90, 104, 107, 97, 110, 90, 85, 90, 100, 60, 75, 120, 120, 120, 110, 110, 
107, 105, 65, 105, 90, 94, 63, 30, 120, 0, 135, 135, 105, 105, 105, 114, 104, 70, 90, 75, 80, 63, 120, 60, 97, 120, 120, 120, 108, 111, 100, 100, 90, 84, 120, 96, 120, 75, 90, 90, 135, 120, 105, 105, 103, 112, 114, 70, 108, 96, 90, 60, 30, 75, 100, 120, 120, 120, 114, 90, 107, 109, 85, 103, 90, 84, 60, 105, 90, 60, 120, 120, 120, 
108, 111, 113, 107, 120, 90, 95, 75, 71, 120, 0, 100, 135, 120, 120, 105, 116, 80, 103, 94, 78, 107, 90, 90, 150, 30, 60, 90, 135, 135, 120, 110, 107, 105, 115, 99, 116, 90, 72, 130, 0, 105, 90, 120, 110, 110, 116, 107, 107, 116, 75, 120, 81, 96, 130, 30, 60, 90, 105, 105, 114, 111, 90, 103, 107, 63, 85, 99, 90, 56, 110, 90, 90, 90, 114, 130, 115, 80, 90, 107, 66, 112, 75, 109, 120, 45, 0, 0, 105, 95, 120, 111, 114, 112, 105, 60, 114, 85, 102, 150, 30, 90, 130, 100, 105, 120, 110, 115, 103, 105, 101, 125, 90, 67, 120, 60, 90, 90, 130, 120, 100, 103, 113, 90, 66, 105, 90, 100, 60, 90, 60, 60, 120, 100, 112, 120, 100, 114, 105, 75, 115, 96, 96, 150, 60, 60, 
60, 100, 75, 110, 100, 117, 102, 105, 90, 90, 130, 49, 90, 26, 80, 80, 120, 110, 114, 65, 105, 102, 82, 105, 100, 78, 135, 60, 90, 120, 120, 120, 111, 84, 103, 102, 96, 113, 105, 60, 127, 120, 75, 120, 120, 112, 111, 86, 113, 120, 75, 120, 96, 101, 120, 60, 75, 110, 120, 120, 114, 111, 107, 103, 78, 95, 107, 97, 90, 60, 60, 75, 120, 110, 120, 130, 82, 110, 90, 90, 109, 78, 72, 120, 0, 0, 120, 120, 105, 108, 100, 110, 103, 103, 114, 90, 90, 35, 75, 0, 90, 165, 120, 120, 115, 75, 105, 85, 101, 103, 78, 90, 128, 75, 90, 120, 130, 116, 122, 101, 97, 95, 93, 112, 60, 60, 60, 120, 0, 110, 120, 116, 116, 113, 109, 112, 109, 110, 110, 110, 110, 109, 110, 110, 109, 110, 110, 110, 111, 109, 110, 111, 111, 109, 109, 112, 110, 110, 111, 110, 107, 93, 81, 105, 60, 90, 100, 112, 105, 100, 107, 95, 108, 94, 85, 90, 90, 60, 90, 0, 120, 105, 108, 101, 114, 97, 120, 90, 96, 85, 105, 105, 100, 110, 112, 110, 90, 110, 99, 107, 84, 100, 82, 120, 60, 60, 120, 90, 97, 114, 104, 101, 101, 103, 90, 45, 75, 90, 0, 120, 120, 94, 105, 103, 107, 86, 90]
hbuf = [2*h for h in hbuf]
sbuf = [37, 29, 24, 29, 0, 29, 27, 57, 57, 36, 34, 26, 24, 24, 43, 15, 43, 43, 51, 43, 52, 27, 32, 29, 36, 29, 31, 24, 40, 32, 55, 51, 43, 60, 36, 48, 22, 22, 40, 36, 13, 29, 43, 27, 45, 37, 36, 43, 37, 32, 24, 31, 31, 15, 15, 48, 48, 57, 61, 42, 31, 43, 37, 20, 24, 31, 31, 43, 27, 55, 52, 31, 43, 57, 77, 80, 89, 88, 87, 91, 94, 94, 92, 
89, 89, 91, 89, 87, 91, 90, 89, 90, 92, 88, 89, 94, 94, 88, 87, 68, 39, 36, 15, 36, 15, 29, 29, 36, 40, 49, 39, 39, 24, 31, 29, 43, 15, 32, 43, 43, 49, 39, 36, 46, 36, 36, 39, 36, 15, 17, 32, 15, 36, 27, 49, 48, 37, 34, 26, 32, 36, 45, 17, 40, 43, 52, 53, 40, 43, 36, 31, 32, 37, 32, 32, 29, 29, 0, 52, 32, 49, 39, 37, 29, 22, 62, 88, 94, 97, 97, 97, 97, 96, 98, 96, 96, 98, 97, 96, 97, 97, 96, 95, 95, 95, 95, 96, 96, 97, 95, 96, 89, 51, 15, 15, 32, 43, 43, 43, 29, 32, 39, 13, 32, 0, 32, 43, 45, 
43, 52, 40, 42, 36, 36, 26, 34, 36, 32, 32, 51, 36, 36, 43, 59, 34, 43, 20, 26, 32, 29, 32, 29, 40, 52, 51, 45, 36, 42, 36, 22, 31, 26, 26, 43, 37, 82, 93, 95, 96, 95, 93, 94, 96, 93, 93, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 94, 93, 96, 96, 95, 95, 95, 57, 40, 31, 26, 32, 36, 17, 24, 15, 43, 43, 36, 43, 37, 37, 37, 29, 26, 26, 
36, 29, 32, 43, 43, 61, 43, 43, 31, 19, 19, 36, 24, 46, 13, 17, 32, 0, 32, 40, 43, 43, 48, 36, 27, 22, 17, 31, 24, 29, 17, 32, 43, 52, 46, 51, 15, 36, 45, 36, 37, 20, 0, 15, 15, 32, 32, 43, 36, 37, 45, 37, 39, 20, 20, 51, 0, 13, 17, 32, 32, 51, 37, 48, 37, 37, 11, 36, 51, 15, 32, 69, 32, 40, 52, 40, 46, 29, 40, 26, 32, 32, 32, 43, 43, 52, 43, 46, 26, 37, 24, 17, 32, 29, 37, 61, 43, 31, 37, 36, 29, 0, 32, 32, 43, 13, 64, 49, 37, 24, 49, 26, 36, 13, 46, 31, 24, 13, 43, 32, 32, 43, 40, 36, 43, 40, 37, 37, 37, 34, 55, 0, 13, 29, 32, 32, 43, 48, 40, 61, 45, 46, 46, 27, 29, 29, 55, 24, 15, 65, 29, 43, 51, 48, 40, 40, 46, 40, 32, 29, 31, 39, 24, 13, 29, 43, 43, 40, 40, 43, 52, 55, 39, 39, 39, 32, 22, 40, 13, 29, 0, 43, 32, 27, 27, 29, 51, 22, 37, 13, 36, 27, 48, 36, 11, 45, 40, 32, 32, 29, 48, 49, 46, 46, 57, 27, 27, 26, 49, 
32, 13, 0, 32, 43, 43, 27, 65, 43, 29, 32, 27, 49, 22, 5, 20, 0, 51, 17, 43, 59, 55, 75, 43, 43, 43, 22, 48, 32, 40, 39, 36, 32, 0, 32, 32, 43, 55, 26, 52, 32, 19, 29, 29, 36, 13, 32, 36, 29, 32, 51, 43, 40, 40, 36, 29, 29, 9, 37, 39, 48, 11, 15, 0, 32, 32, 29, 27, 27, 46, 55, 48, 19, 31, 36, 51, 37, 13, 51, 32, 43, 55, 46, 52, 46, 46, 22, 26, 40, 31, 20, 24, 15, 17, 32, 32, 51, 51, 49, 52, 26, 29, 22, 29, 19, 37, 24, 29, 43, 32, 32, 51, 43, 36, 43, 39, 37, 31, 32, 29, 26, 20, 36, 15, 32, 32, 
36, 43, 49, 43, 36, 39, 22, 31, 22, 49, 46, 0, 43, 32, 32, 32, 40, 56, 36, 37, 39, 20, 32, 22, 20, 20, 24, 17, 17, 32, 32, 46, 48, 36, 40, 32, 37, 36, 24, 31, 37, 13, 29, 32, 43, 40, 57, 52, 48, 43, 37, 19, 31, 37, 29, 43, 24, 15, 32, 29, 29, 52, 57, 13, 37, 32, 36, 27, 36, 39, 40, 29, 24, 29, 29, 61, 36, 48, 36, 37, 32, 39, 36, 22, 46, 13, 22, 0, 0, 51, 70, 40, 51, 45, 39, 40, 24, 42, 31, 31, 13, 24, 15, 43, 40, 64, 46, 46, 32, 37, 32, 40, 27, 22, 26, 37, 13, 32, 32, 43, 36, 52, 49, 43, 37, 24, 26, 22, 17, 20, 36, 15, 15, 32, 40, 45, 37, 43, 46, 39, 19, 27, 29, 29, 15, 24, 15, 15, 43, 45, 52, 45, 43, 26, 32, 24, 24, 19, 49, 24, 27, 43, 43, 13, 36, 43, 37, 51, 49, 17, 37, 36, 32, 42, 24, 17, 32, 43, 36, 57, 36, 39, 49, 27, 42, 24, 26, 37, 13, 29, 32, 32, 45, 57, 53, 43, 32, 19, 31, 29, 43, 7, 24, 29, 46, 32, 32, 49, 52, 43, 42, 46, 26, 36, 26, 7, 24, 15, 32, 32, 64, 32, 66, 26, 46, 32, 27, 39, 27, 36, 52, 0, 0, 32, 32, 48, 46, 46, 32, 37, 37, 22, 17, 26, 37, 24, 13, 17, 51, 32, 32, 48, 37, 40, 31, 37, 42, 40, 20, 48, 64, 32, 32, 43, 52, 52, 39, 43, 27, 49, 22, 13, 24, 15, 32, 0, 40, 32, 52, 52, 85, 91, 95, 95, 94, 94, 96, 93, 94, 95, 96, 95, 94, 96, 96, 95, 95, 94, 94, 95, 95, 94, 94, 95, 93, 93, 94, 89, 42, 37, 11, 24, 32, 43, 55, 27, 32, 52, 31, 43, 32, 31, 24, 24, 24, 29, 0, 32, 27, 46, 56, 24, 43, 36, 43, 29, 36, 26, 26, 43, 40, 45, 52, 29, 52, 32, 40, 24, 32, 24, 34, 24, 24, 15, 43, 56, 42, 48, 36, 37, 31, 40, 11, 24, 15, 0, 32, 32, 43, 43, 40, 31, 40, 40]
vbuf = [24, 17, 9, 7, 5, 7, 9, 13, 13, 24, 19, 24, 20, 20, 11, 7, 7, 7, 9, 11, 15, 24, 26, 26, 24, 22, 20, 9, 9, 7, 7, 9, 11, 17, 27, 26, 27, 27, 24, 17, 9, 7, 7, 9, 9, 13, 
17, 27, 26, 26, 26, 24, 24, 7, 7, 9, 9, 11, 19, 29, 24, 22, 20, 24, 20, 17, 17, 7, 9, 13, 15, 24, 24, 27, 29, 45, 42, 52, 49, 45, 48, 48, 49, 52, 48, 46, 52, 49, 46, 
51, 52, 45, 45, 51, 52, 43, 43, 42, 43, 29, 24, 20, 17, 9, 7, 7, 7, 9, 11, 17, 22, 19, 20, 20, 22, 11, 7, 7, 7, 7, 11, 15, 24, 26, 27, 27, 22, 17, 7, 7, 7, 7, 9, 13, 
19, 26, 26, 29, 29, 24, 9, 9, 7, 9, 7, 13, 17, 27, 26, 27, 24, 26, 24, 19, 19, 7, 7, 9, 13, 24, 26, 24, 20, 22, 22, 32, 48, 64, 73, 73, 73, 73, 78, 75, 78, 79, 75, 77, 77, 79, 75, 77, 79, 79, 79, 79, 78, 79, 72, 64, 40, 20, 9, 7, 7, 11, 13, 19, 19, 26, 26, 22, 19, 11, 7, 7, 7, 9, 11, 15, 24, 26, 24, 24, 20, 15, 9, 7, 7, 9, 9, 13, 
20, 26, 26, 24, 24, 20, 11, 7, 7, 7, 9, 13, 20, 29, 27, 29, 27, 27, 24, 19, 19, 7, 9, 17, 31, 49, 61, 64, 65, 67, 62, 64, 64, 65, 61, 61, 61, 61, 61, 61, 61, 61, 59, 
64, 66, 62, 60, 64, 62, 62, 31, 26, 24, 24, 24, 20, 15, 9, 7, 7, 7, 9, 13, 24, 24, 26, 22, 19, 19, 9, 7, 7, 7, 7, 9, 11, 13, 20, 26, 26, 24, 20, 13, 9, 7, 7, 5, 7, 9, 11, 13, 20, 24, 31, 24, 22, 17, 9, 7, 7, 7, 7, 11, 13, 17, 24, 24, 26, 24, 20, 11, 9, 7, 7, 7, 7, 7, 9, 13, 19, 24, 22, 29, 29, 20, 9, 9, 7, 7, 7, 9, 9, 15, 24, 24, 
20, 20, 17, 7, 7, 7, 7, 9, 13, 24, 24, 26, 26, 9, 7, 7, 7, 7, 11, 15, 24, 26, 29, 24, 9, 7, 7, 7, 9, 15, 24, 24, 26, 27, 13, 7, 7, 7, 7, 9, 15, 26, 24, 29, 26, 24, 27, 27, 24, 20, 9, 9, 7, 7, 7, 7, 9, 9, 13, 19, 24, 24, 32, 29, 20, 9, 9, 7, 7, 7, 7, 9, 11, 15, 19, 24, 24, 22, 29, 29, 17, 9, 7, 7, 7, 7, 9, 9, 11, 19, 24, 24, 24, 22, 20, 19, 9, 9, 7, 7, 7, 9, 9, 11, 13, 17, 24, 26, 26, 24, 22, 19, 9, 7, 7, 7, 7, 9, 9, 13, 17, 22, 24, 27, 27, 24, 20, 17, 9, 9, 9, 7, 7, 7, 9, 11, 15, 15, 24, 31, 24, 20, 22, 11, 9, 7, 7, 7, 7, 9, 11, 13, 22, 24, 24, 22, 22, 20, 11, 9, 9, 7, 7, 7, 9, 11, 13, 13, 24, 27, 27, 24, 20, 15, 9, 7, 5, 7, 7, 7, 9, 20, 24, 26, 27, 26, 22, 20, 19, 11, 9, 7, 7, 9, 7, 9, 9, 24, 26, 26, 27, 24, 22, 20, 9, 7, 7, 7, 7, 7, 9, 9, 26, 24, 26, 26, 24, 20, 20, 13, 9, 9, 7, 7, 7, 13, 15, 24, 24, 22, 24, 24, 20, 
19, 9, 7, 7, 7, 7, 9, 9, 17, 24, 24, 26, 27, 22, 20, 20, 9, 7, 7, 7, 7, 9, 13, 17, 24, 24, 24, 29, 24, 22, 20, 11, 9, 7, 7, 7, 9, 13, 17, 24, 24, 26, 27, 24, 24, 19, 
15, 9, 7, 7, 7, 7, 11, 17, 20, 24, 22, 29, 26, 22, 19, 19, 9, 7, 7, 7, 7, 13, 15, 24, 24, 22, 24, 27, 20, 20, 20, 9, 7, 7, 7, 9, 11, 15, 22, 24, 26, 27, 29, 24, 22, 17, 9, 7, 7, 7, 7, 11, 13, 20, 24, 26, 27, 27, 24, 22, 20, 13, 9, 7, 7, 9, 9, 15, 20, 24, 26, 27, 27, 24, 20, 20, 11, 7, 7, 9, 9, 11, 17, 26, 26, 24, 31, 29, 24, 20, 19, 9, 7, 7, 9, 9, 13, 15, 24, 24, 24, 24, 27, 22, 20, 13, 9, 7, 7, 7, 9, 13, 17, 24, 24, 26, 29, 22, 22, 19, 9, 7, 7, 7, 9, 9, 13, 17, 26, 24, 27, 27, 22, 22, 17, 9, 
7, 7, 7, 9, 13, 17, 24, 24, 24, 26, 20, 20, 19, 9, 9, 7, 7, 9, 9, 13, 20, 24, 24, 29, 26, 20, 19, 17, 9, 7, 7, 7, 9, 13, 17, 22, 24, 22, 26, 20, 20, 13, 9, 7, 7, 7, 9, 13, 17, 24, 26, 27, 27, 22, 22, 17, 9, 7, 7, 7, 7, 11, 15, 24, 26, 26, 29, 24, 20, 17, 9, 7, 7, 7, 9, 11, 15, 20, 24, 24, 27, 26, 24, 17, 15, 7, 7, 7, 7, 9, 13, 15, 22, 24, 24, 27, 24, 20, 20, 9, 9, 7, 9, 7, 7, 15, 20, 24, 24, 26, 26, 24, 19, 17, 9, 7, 7, 7, 15, 26, 26, 24, 27, 24, 22, 20, 9, 7, 7, 7, 9, 11, 15, 15, 36, 57, 65, 
65, 67, 67, 66, 66, 67, 67, 66, 67, 69, 67, 67, 68, 66, 68, 70, 66, 68, 70, 70, 67, 69, 70, 62, 55, 26, 24, 20, 9, 7, 7, 7, 9, 11, 15, 24, 27, 26, 24, 26, 26, 9, 7, 5, 7, 9, 13, 17, 26, 24, 24, 19, 22, 20, 19, 19, 7, 9, 9, 13, 17, 26, 26, 26, 26, 24, 20, 19, 9, 9, 7, 11, 17, 29, 27, 27, 26, 29, 24, 22, 9, 7, 7, 7, 7, 19, 27, 26, 27, 24, 24]
rbuf = [13, 9, 6, 5, 4, 4, 5, 7, 7, 13, 10, 13, 12, 12, 6, 5, 4, 4, 4, 6, 7, 13, 15, 14, 14, 12, 10, 6, 5, 4, 4, 5, 6, 8, 14, 14, 16, 16, 13, 9, 5, 5, 4, 5, 5, 7, 10, 14, 14, 15, 15, 14, 14, 5, 5, 5, 5, 6, 9, 15, 14, 12, 11, 14, 12, 8, 8, 4, 5, 6, 8, 14, 13, 14, 13, 19, 16, 21, 20, 17, 16, 16, 18, 21, 19, 17, 20, 20, 17, 20, 21, 18, 16, 
20, 20, 15, 15, 16, 17, 14, 14, 12, 10, 6, 5, 5, 4, 5, 6, 9, 12, 11, 12, 11, 12, 6, 5, 4, 4, 4, 6, 8, 13, 13, 15, 15, 12, 9, 5, 4, 4, 5, 5, 7, 10, 14, 14, 16, 16, 13, 6, 5, 4, 5, 4, 7, 9, 14, 14, 15, 14, 15, 13, 10, 10, 4, 4, 6, 6, 13, 13, 13, 11, 12, 13, 15, 18, 22, 23, 23, 23, 23, 27, 24, 28, 28, 24, 26, 27, 27, 23, 27, 28, 28, 
28, 28, 27, 28, 24, 21, 12, 7, 5, 5, 5, 6, 7, 10, 10, 14, 14, 12, 11, 6, 5, 4, 4, 5, 6, 7, 13, 14, 13, 13, 11, 8, 5, 4, 4, 5, 5, 8, 11, 13, 14, 13, 14, 11, 6, 5, 4, 4, 5, 7, 11, 15, 15, 15, 15, 16, 14, 10, 10, 4, 5, 7, 10, 18, 23, 25, 27, 28, 24, 27, 27, 25, 24, 24, 24, 24, 24, 24, 24, 24, 22, 26, 27, 23, 23, 26, 24, 24, 16, 15, 13, 14, 13, 12, 8, 6, 5, 4, 4, 5, 7, 12, 12, 15, 12, 11, 10, 5, 4, 4, 4, 4, 4, 5, 7, 11, 15, 15, 13, 12, 6, 5, 4, 4, 4, 4, 5, 5, 7, 10, 13, 17, 13, 13, 9, 6, 4, 4, 4, 
4, 5, 6, 8, 14, 13, 14, 14, 10, 7, 5, 5, 5, 4, 4, 4, 5, 7, 10, 12, 12, 17, 17, 11, 5, 5, 4, 4, 4, 4, 5, 7, 12, 13, 12, 11, 8, 5, 4, 4, 4, 5, 7, 12, 12, 14, 14, 5, 4, 
4, 4, 4, 6, 8, 13, 13, 16, 13, 6, 4, 4, 4, 5, 7, 12, 14, 14, 15, 7, 5, 4, 4, 4, 5, 7, 13, 12, 17, 13, 13, 14, 17, 13, 11, 6, 5, 4, 4, 4, 4, 5, 5, 7, 10, 12, 12, 17, 16, 10, 6, 5, 5, 4, 4, 4, 5, 6, 7, 10, 12, 12, 13, 16, 16, 8, 6, 5, 4, 4, 4, 4, 5, 6, 10, 12, 13, 13, 12, 11, 10, 6, 5, 5, 4, 4, 5, 5, 6, 7, 8, 13, 14, 14, 13, 12, 9, 
5, 5, 4, 4, 4, 5, 5, 7, 8, 13, 13, 16, 15, 13, 10, 9, 6, 5, 5, 4, 4, 4, 5, 6, 8, 8, 12, 17, 13, 11, 11, 6, 5, 4, 4, 4, 4, 5, 5, 7, 12, 13, 13, 11, 12, 12, 7, 5, 4, 4, 4, 4, 5, 5, 7, 7, 14, 16, 14, 13, 11, 8, 5, 4, 4, 4, 4, 4, 5, 11, 13, 14, 16, 14, 12, 12, 11, 6, 5, 4, 4, 4, 4, 5, 5, 13, 14, 14, 16, 13, 12, 10, 6, 5, 4, 4, 4, 4, 5, 5, 13, 12, 14, 16, 13, 12, 10, 7, 5, 5, 4, 4, 4, 6, 8, 12, 12, 13, 14, 12, 11, 10, 6, 5, 4, 4, 4, 4, 4, 9, 12, 14, 14, 15, 12, 12, 10, 6, 5, 4, 4, 4, 4, 7, 10, 13, 
13, 13, 16, 13, 12, 11, 7, 6, 5, 4, 4, 5, 7, 8, 13, 13, 13, 16, 13, 13, 9, 8, 5, 4, 4, 4, 4, 6, 7, 12, 13, 12, 17, 15, 13, 11, 11, 6, 4, 4, 4, 4, 6, 7, 13, 13, 12, 13, 15, 12, 11, 11, 5, 4, 4, 4, 5, 6, 7, 11, 13, 14, 16, 15, 13, 12, 9, 6, 5, 4, 4, 4, 5, 7, 12, 12, 14, 15, 16, 13, 12, 11, 7, 6, 5, 5, 4, 5, 7, 12, 13, 15, 15, 15, 13, 10, 11, 6, 5, 5, 5, 4, 6, 8, 13, 13, 13, 17, 15, 13, 11, 11, 6, 5, 4, 5, 5, 6, 8, 13, 13, 13, 13, 15, 13, 11, 7, 5, 4, 4, 4, 5, 7, 9, 12, 13, 15, 16, 13, 13, 10, 6, 5, 5, 4, 5, 5, 7, 9, 13, 13, 16, 15, 12, 12, 10, 6, 5, 5, 4, 5, 7, 9, 12, 14, 13, 16, 12, 12, 10, 6, 5, 4, 4, 5, 5, 7, 10, 12, 12, 17, 14, 12, 10, 9, 6, 4, 4, 4, 5, 
7, 10, 12, 12, 13, 13, 12, 11, 7, 5, 4, 4, 4, 5, 7, 9, 13, 14, 16, 15, 12, 11, 10, 6, 4, 4, 4, 4, 6, 8, 13, 14, 14, 17, 13, 11, 10, 6, 5, 4, 4, 5, 6, 7, 11, 12, 13, 16, 13, 13, 9, 7, 5, 4, 4, 4, 5, 6, 8, 12, 12, 12, 16, 13, 11, 11, 6, 5, 4, 4, 4, 4, 7, 11, 13, 13, 15, 14, 13, 11, 9, 5, 4, 4, 4, 7, 13, 14, 13, 16, 13, 12, 11, 6, 5, 4, 4, 5, 6, 7, 7, 14, 23, 25, 25, 27, 27, 25, 27, 27, 26, 25, 27, 28, 25, 25, 27, 26, 28, 28, 26, 27, 29, 28, 26, 29, 29, 25, 24, 14, 13, 12, 6, 4, 4, 4, 5, 6, 8, 13, 14, 14, 13, 15, 15, 6, 5, 4, 4, 5, 6, 8, 15, 13, 13, 10, 12, 12, 11, 11, 4, 5, 5, 7, 10, 13, 15, 14, 14, 13, 12, 10, 6, 6, 5, 6, 8, 15, 14, 15, 15, 16, 13, 13, 6, 5, 4, 4, 4, 10, 15, 14, 15, 13, 13]
dump_index = 400
true_colors = ['blue', 'green', 'red', 'blue']

[Novakasa]

For completeness, that's the code for getting to the hsv plot:

from matplotlib import pyplot as plt
lineweight = .4

import numpy as np
hue = np.roll(np.array(hbuf), -dump_index)
sat = np.roll(np.array(sbuf), -dump_index)
val = np.roll(np.array(vbuf), -dump_index)
ref = np.roll(np.array(rbuf), -dump_index)
hue = (hue-20)%360

lineweight = .4
fig = plt.figure(dpi=150,)
fig.patch.set_facecolor('white')

plt.plot(sat, lw=lineweight, label="saturation")
plt.plot(val, lw=lineweight, label="value")
#plt.plot(hue, lw=lineweight, label="hue", ls="", marker="x", markersize=.4)
plt.plot(hue, lw=lineweight, label="hue")
plt.plot(ref, lw=lineweight, label="reflection")
plt.legend()
plt.show()

[Novakasa]

Here is the graph for a slower train. The earlier part has less noise, probably because that area has a white background (I put white paper sheets under the track). At the end, the black background produces the hue noise probably.

The data:

#noon, lights off, fixed hue, more markers, white paper surface below tracks, slow train
hbuf = [111, 111, 110, 109, 109, 108, 107, 108, 108, 108, 108, 108, 108, 108, 108, 107, 108, 108, 108, 108, 110, 110, 109, 109, 108, 108, 108, 108, 108, 108, 108, 108, 109, 
108, 107, 108, 107, 107, 108, 109, 110, 109, 109, 108, 107, 108, 109, 108, 109, 111, 110, 110, 109, 108, 107, 108, 107, 109, 110, 110, 109, 109, 108, 108, 107, 108, 108, 108, 109, 109, 109, 108, 108, 107, 108, 108, 103, 105, 103, 101, 102, 105, 109, 112, 112, 119, 118, 115, 116, 113, 117, 114, 116, 111, 115, 116, 109, 109, 107, 109, 112, 116, 115, 109, 115, 90, 63, 51, 0, 102, 72, 65, 70, 90, 90, 105, 97, 120, 118, 110, 113, 103, 108, 108, 104, 115, 116, 118, 115, 118, 116, 116, 118, 120, 120, 115, 112, 110, 112, 109, 111, 110, 107, 105, 112, 114, 116, 113, 105, 105, 75, 72, 45, 52, 60, 56, 63, 63, 71, 67, 85, 105, 107, 112, 99, 115, 104, 99, 111, 112, 106, 106, 106, 112, 115, 114, 116, 115, 117, 117, 116, 115, 118, 115, 113, 111, 111, 109, 105, 109, 108, 109, 112, 94, 82, 67, 66, 72, 60, 50, 56, 56, 56, 56, 56, 77, 76, 50, 90, 86, 105, 105, 99, 90, 77, 81, 81, 50, 120, 110, 120, 0, 75, 60, 0, 0, 84, 75, 90, 0, 132, 130, 120, 127, 120, 90, 110, 110, 125, 110, 120, 120, 120, 120, 120, 99, 81, 90, 115, 103, 103, 107, 85, 97, 95, 78, 64, 70, 82, 90, 90, 90, 78, 100, 97, 82, 66, 60, 47, 140, 126, 105, 120, 0, 0, 0, 0, 90, 0, 135, 90, 100, 130, 140, 130, 120, 135, 90, 100, 120, 110, 120, 129, 120, 123, 120, 111, 105, 84, 84, 105, 115, 111, 113, 113, 96, 78, 57, 53, 84, 90, 90, 108, 105, 105, 97, 96, 90, 68, 66, 
39, 150, 127, 75, 30, 30, 60, 60, 75, 60, 80, 120, 120, 0, 120, 0, 130, 120, 120, 110, 120, 128, 110, 116, 103, 75, 108, 105, 108, 103, 90, 77, 84, 90, 82, 105, 103, 
82, 90, 90, 75, 45, 0, 126, 75, 120, 60, 90, 90, 90, 0, 0, 1, 0, 0, 0, 179, 178, 178, 0, 0, 1, 0, 0, 0, 179, 178, 0, 0, 0, 1, 177, 176, 1, 154, 124, 110, 108, 108, 115, 119, 120, 118, 116, 114, 116, 118, 115, 113, 113, 113, 113, 109, 114, 117, 109, 0, 102, 75, 53, 80, 68, 54, 90, 116, 112, 103, 101, 95, 90, 101, 109, 112, 106, 106, 107, 119, 116, 116, 116, 116, 116, 116, 116, 116, 116, 116, 110, 108, 112, 114, 110, 112, 72, 60, 80, 68, 57, 74, 72, 75, 78, 72, 65, 64, 70, 70, 67, 66, 65, 66, 66, 67, 68, 70, 69, 67, 68, 67, 66, 66, 66, 66, 66, 67, 67, 65, 65, 66, 65, 66, 67, 69, 69, 67, 67, 66, 66, 66, 66, 66, 66, 67, 67, 65, 66, 66, 66, 67, 67, 69, 69, 69, 
64, 65, 65, 66, 66, 68, 70, 67, 67, 66, 66, 65, 66, 65, 67, 71, 68, 68, 68, 68, 71, 79, 73, 79, 97, 99, 102, 101, 110, 119, 115, 115, 117, 115, 116, 114, 117, 116, 115, 112, 110, 108, 110, 110, 113, 113, 107, 100, 82, 57, 60, 52, 67, 72, 77, 95, 114, 108, 107, 90, 96, 110, 107, 107, 106, 106, 108, 113, 122, 115, 115, 116, 116, 116, 116, 112, 110, 109, 108, 111, 107, 116, 107, 116, 110, 102, 77, 68, 85, 78, 77, 82, 90, 117, 115, 108, 93, 86, 103, 115, 111, 109, 108, 111, 120, 116, 114, 114, 116, 115, 115, 116, 115, 114, 115, 112, 112, 112, 110, 111, 110, 116, 107, 95, 60, 44, 66, 90, 78, 72, 76, 76, 81, 106, 90, 94, 106, 104, 108, 100, 97, 102, 105, 115, 119, 115, 114, 115, 116, 116, 117, 114, 113, 109, 109, 107, 111, 109, 112, 113, 103, 116, 70, 60, 60, 66, 78, 75, 78, 76, 86, 101, 113, 100, 90, 90, 111, 106, 106, 106, 104, 113, 119, 117, 115, 117, 117, 115, 115, 115, 116, 117, 109, 111, 102, 109, 110, 114, 111, 120, 111, 99, 63, 57, 57, 64, 70, 82, 105, 114, 109, 111, 97, 80, 48, 
30, 0, 170, 177, 0, 179, 178, 0, 178, 0, 179, 0, 179, 0, 0, 0, 0, 0, 0, 178, 179, 179, 0, 0, 0, 179, 0, 0, 1, 179, 178, 178, 179, 0, 179, 179, 1, 1, 0, 178, 178, 179, 1, 1, 179, 179, 179, 0, 178, 178, 179, 179, 0, 1, 179, 179, 0, 0, 179, 179, 177, 179, 179, 179, 179, 179, 179, 179, 178, 0, 179, 0, 0, 0, 0, 0, 179, 179, 179, 179, 179, 177, 173, 170, 166, 161, 154, 144, 134, 130, 127, 123, 123, 114, 108, 111, 112, 115, 114, 111, 120, 120, 116, 110, 78, 51, 60, 66, 49, 78, 82, 97, 109, 107, 100, 
90, 99, 104, 111, 106, 104, 104, 116, 119, 120, 116, 113, 116, 113, 117, 117, 113, 113, 111, 111, 107, 107, 112, 114, 113, 120, 111, 70, 66, 60, 60, 80, 66, 68, 72, 72, 110, 99, 105, 97, 95, 109, 111, 111, 109, 105, 108, 109, 119, 118, 117, 117, 117, 117, 114, 117, 117, 112, 111, 111, 111, 113, 106, 116, 114, 114, 112, 63, 51, 50, 0, 70, 66, 77, 80, 90, 103, 111, 105, 109, 112, 112, 111, 110, 105, 106, 113, 120, 119, 114, 115, 117, 115, 117, 115, 118, 112, 113, 110, 112, 112, 112, 111, 114, 112, 116, 101, 65, 46, 78, 68, 66, 80, 90, 90, 101, 100, 90, 107, 112, 102, 102, 104, 102, 105, 116, 118, 114, 113, 115, 114, 115, 117, 117, 115, 116, 109, 117, 107, 113, 111, 117, 116, 109, 120, 90, 71, 75, 90, 90, 85, 102, 105, 105, 112, 109, 109]
hbuf = [2*h for h in hbuf]
sbuf = [92, 95, 96, 95, 95, 95, 95, 92, 93, 93, 93, 93, 93, 93, 96, 94, 93, 93, 92, 92, 93, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 94, 92, 93, 92, 94, 92, 96, 96, 94, 96, 94, 92, 93, 91, 92, 93, 95, 96, 97, 96, 93, 92, 93, 93, 93, 95, 96, 96, 96, 95, 93, 92, 92, 93, 92, 92, 96, 94, 92, 91, 86, 84, 78, 67, 52, 52, 48, 43, 43, 45, 45, 42, 45, 42, 40, 42, 39, 36, 32, 29, 26, 26, 24, 20, 19, 19, 19, 15, 11, 17, 13, 9, 24, 26, 1, 17, 26, 39, 39, 32, 22, 34, 34, 31, 46, 45, 34, 45, 46, 40, 39, 37, 42, 45, 40, 39, 37, 34, 26, 27, 29, 27, 26, 26, 26, 24, 24, 22, 22, 19, 19, 19, 15, 19, 13, 9, 9, 19, 17, 13, 13, 29, 31, 31, 37, 36, 31, 39, 29, 46, 48, 49, 40, 36, 43, 46, 43, 36, 36, 43, 39, 40, 37, 34, 32, 29, 29, 27, 24, 24, 24, 24, 20, 17, 20, 19, 19, 15, 15, 15, 9, 9, 13, 15, 19, 20, 26, 26, 26, 26, 26, 32, 37, 26, 34, 34, 26, 36, 36, 27, 37, 53, 53, 36, 13, 24, 9, 0, 24, 24, 0, 0, 61, 27, 29, 15, 65, 43, 43, 51, 43, 51, 40, 40, 66, 36, 43, 40, 37, 43, 48, 49, 43, 17, 49, 37, 37, 36, 32, 22, 31, 39, 31, 26, 32, 22, 22, 32, 27, 32, 26, 27, 51, 45, 40, 22, 36, 20, 11, 0, 0, 0, 0, 15, 0, 32, 17, 43, 43, 43, 43, 43, 32, 29, 40, 26, 36, 40, 76, 37, 
64, 24, 52, 31, 36, 31, 22, 49, 37, 43, 43, 31, 49, 40, 37, 22, 26, 22, 22, 37, 32, 22, 29, 39, 43, 53, 40, 13, 36, 24, 24, 13, 45, 13, 29, 15, 43, 32, 32, 0, 32, 15, 43, 13, 13, 40, 36, 70, 52, 52, 49, 37, 29, 43, 29, 37, 24, 37, 27, 32, 19, 42, 37, 24, 36, 13, 36, 24, 29, 55, 24, 13, 36, 15, 29, 29, 95, 94, 94, 93, 92, 92, 93, 95, 95, 94, 94, 93, 93, 92, 92, 94, 95, 94, 93, 91, 87, 88, 87, 73, 27, 22, 40, 37, 39, 39, 43, 48, 40, 37, 37, 34, 29, 29, 26, 26, 26, 26, 19, 17, 17, 15, 5, 11, 15, 
24, 9, 24, 19, 26, 37, 37, 37, 39, 29, 19, 29, 49, 36, 40, 43, 45, 42, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 19, 17, 19, 19, 17, 17, 26, 20, 19, 24, 32, 53, 52, 57, 61, 78, 84, 79, 81, 85, 93, 92, 88, 89, 88, 89, 87, 89, 92, 94, 96, 94, 91, 88, 89, 89, 89, 93, 94, 93, 91, 89, 87, 88, 87, 88, 91, 92, 93, 93, 93, 93, 93, 93, 93, 93, 93, 91, 88, 88, 88, 88, 87, 88, 91, 91, 90, 87, 87, 90, 89, 87, 89, 91, 94, 93, 94, 89, 88, 87, 86, 89, 89, 89, 87, 81, 73, 72, 71, 52, 42, 51, 42, 45, 37, 43, 42, 42, 42, 37, 37, 36, 29, 20, 26, 24, 22, 22, 24, 24, 17, 17, 15, 15, 20, 26, 19, 11, 13, 26, 42, 26, 42, 49, 43, 36, 20, 31, 43, 40, 40, 40, 39, 42, 48, 43, 40, 37, 37, 37, 37, 22, 22, 17, 20, 20, 20, 15, 17, 13, 13, 11, 19, 19, 19, 17, 26, 31, 31, 37, 40, 42, 39, 37, 26, 37, 39, 40, 39, 39, 43, 48, 42, 39, 40, 37, 36, 31, 26, 27, 17, 24, 24, 24, 20, 17, 17, 15, 15, 15, 24, 27, 13, 13, 17, 27, 36, 36, 27, 49, 34, 31, 39, 43, 46, 43, 43, 42, 40, 37, 46, 40, 39, 36, 31, 31, 27, 26, 24, 24, 20, 20, 20, 19, 13, 15, 15, 17, 22, 27, 19, 15, 17, 22, 36, 34, 31, 31, 36, 27, 31, 31, 40, 43, 45, 39, 43, 40, 42, 48, 43, 42, 40, 39, 36, 27, 26, 15, 20, 20, 19, 19, 20, 19, 19, 7, 15, 17, 24, 27, 27, 24, 24, 32, 26, 42, 49, 37, 22, 32, 24, 13, 9, 26, 40, 48, 82, 87, 91, 95, 95, 96, 97, 97, 97, 97, 96, 96, 95, 96, 98, 98, 98, 97, 97, 96, 97, 96, 96, 96, 96, 97, 98, 97, 96, 96, 96, 96, 96, 96, 97, 98, 97, 98, 97, 96, 96, 96, 96, 97, 98, 98, 98, 97, 96, 96, 96, 95, 95, 96, 97, 99, 98, 98, 98, 98, 98, 98, 98, 98, 96, 98, 97, 96, 96, 96, 96, 96, 96, 96, 96, 96, 87, 84, 80, 67, 59, 45, 37, 36, 32, 32, 27, 27, 22, 26, 20, 19, 19, 17, 22, 13, 13, 15, 13, 24, 26, 15, 17, 29, 39, 32, 36, 31, 36, 26, 20, 27, 40, 49, 39, 40, 40, 40, 45, 39, 40, 39, 37, 37, 31, 29, 26, 26, 24, 20, 20, 20, 19, 19, 17, 7, 15, 22, 26, 20, 11, 9, 17, 26, 27, 27, 43, 37, 39, 42, 27, 31, 45, 43, 40, 40, 42, 42, 40, 43, 46, 46, 46, 46, 31, 27, 27, 22, 24, 20, 20, 20, 19, 11, 19, 19, 9, 22, 19, 17, 5, 20, 19, 27, 26, 27, 29, 39, 34, 39, 40, 36, 40, 40, 45, 42, 40, 42, 45, 45, 40, 36, 40, 32, 26, 26, 17, 19, 22, 17, 17, 19, 17, 19, 15, 15, 19, 29, 27, 17, 26, 36, 24, 26, 27, 34, 37, 31, 43, 31, 56, 43, 34, 43, 37, 40, 46, 42, 42, 37, 37, 36, 31, 29, 26, 20, 24, 27, 22, 20, 17, 19, 15, 17, 17, 7, 20, 17, 13, 17, 26, 46, 59, 59, 87, 91, 91]
vbuf = [61, 62, 62, 61, 61, 61, 62, 64, 65, 65, 64, 64, 64, 64, 62, 62, 64, 65, 65, 65, 66, 66, 64, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 62, 64, 65, 66, 66, 66, 64, 60, 60, 60, 62, 65, 65, 66, 66, 66, 66, 62, 60, 60, 65, 65, 66, 66, 66, 65, 64, 61, 60, 61, 62, 64, 64, 64, 64, 64, 59, 55, 52, 49, 45, 40, 36, 36, 36, 36, 36, 39, 42, 45, 45, 59, 61, 64, 66, 68, 70, 71, 72, 72, 71, 71, 70, 70, 67, 66, 65, 61, 57, 55, 49, 45, 45, 45, 40, 36, 34, 32, 32, 26, 27, 29, 29, 31, 32, 31, 32, 36, 37, 42, 45, 51, 57, 59, 67, 68, 70, 71, 72, 72, 72, 72, 73, 72, 72, 72, 71, 70, 68, 67, 65, 62, 59, 56, 52, 49, 45, 45, 42, 39, 36, 34, 32, 32, 26, 27, 29, 31, 29, 31, 31, 31, 32, 34, 42, 45, 45, 49, 49, 61, 65, 67, 69, 70, 71, 72, 72, 72, 72, 72, 72, 71, 69, 67, 66, 65, 62, 60, 57, 52, 49, 48, 45, 42, 40, 36, 37, 37, 37, 37, 37, 26, 29, 29, 29, 29, 29, 27, 24, 24, 24, 22, 22, 20, 19, 17, 11, 9, 9, 9, 7, 7, 9, 9, 7, 7, 7, 7, 7, 9, 7, 9, 9, 9, 9, 9, 11, 11, 13, 13, 15, 17, 20, 22, 26, 24, 24, 24, 24, 22, 24, 26, 27, 29, 29, 27, 27, 24, 24, 22, 20, 19, 20, 20, 20, 17, 17, 13, 11, 9, 9, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 9, 9, 9, 11, 13, 13, 13, 15, 15, 17, 17, 20, 22, 26, 24, 24, 24, 20, 24, 26, 29, 27, 29, 27, 27, 26, 24, 22, 22, 19, 20, 22, 20, 19, 13, 9, 9, 9, 9, 9, 7, 7, 7, 7, 7, 5, 7, 7, 7, 9, 9, 9, 9, 9, 13, 15, 17, 20, 22, 22, 22, 24, 20, 24, 24, 24, 26, 29, 24, 20, 20, 20, 20, 20, 22, 19, 9, 9, 9, 7, 7, 7, 93, 92, 93, 94, 95, 94, 95, 94, 92, 92, 92, 94, 93, 94, 93, 92, 89, 87, 83, 79, 75, 70, 60, 49, 36, 39, 42, 46, 51, 56, 61, 62, 66, 68, 70, 71, 72, 72, 71, 71, 71, 71, 66, 64, 60, 57, 52, 48, 45, 45, 40, 36, 32, 29, 26, 26, 24, 26, 26, 27, 34, 37, 39, 43, 49, 56, 59, 62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 67, 66, 64, 61, 59, 55, 46, 45, 40, 36, 34, 32, 31, 31, 29, 32, 40, 45, 45, 43, 45, 49, 51, 52, 52, 52, 48, 46, 45, 45, 45, 51, 49, 51, 52, 52, 52, 45, 45, 49, 52, 52, 53, 52, 51, 48, 45, 45, 46, 48, 48, 48, 48, 48, 48, 45, 45, 51, 52, 52, 52, 52, 51, 48, 45, 45, 52, 52, 49, 51, 52, 49, 46, 46, 45, 46, 49, 52, 51, 52, 49, 43, 43, 43, 45, 45, 40, 31, 32, 31, 34, 36, 37, 40, 53, 60, 61, 62, 66, 68, 70, 71, 71, 70, 71, 70, 70, 70, 68, 68, 61, 59, 53, 49, 45, 46, 45, 40, 36, 32, 34, 29, 29, 24, 24, 24, 29, 36, 39, 40, 43, 48, 52, 57, 60, 62, 65, 68, 68, 68, 68, 71, 70, 70, 69, 67, 66, 62, 60, 57, 53, 49, 46, 45, 39, 36, 34, 32, 31, 32, 32, 29, 27, 29, 34, 37, 39, 45, 52, 56, 60, 62, 65, 67, 70, 70, 72, 73, 72, 73, 71, 72, 72, 72, 67, 64, 62, 57, 52, 48, 49, 46, 45, 39, 36, 32, 31, 31, 31, 31, 29, 31, 34, 36, 37, 40, 45, 49, 55, 60, 64, 67, 71, 72, 73, 73, 73, 75, 73, 72, 71, 70, 69, 67, 66, 62, 52, 48, 46, 46, 43, 39, 36, 34, 32, 32, 31, 31, 31, 26, 27, 27, 37, 40, 42, 45, 49, 57, 60, 62, 64, 66, 68, 70, 72, 72, 71, 70, 70, 69, 67, 66, 65, 62, 60, 56, 53, 49, 48, 46, 46, 36, 32, 29, 29, 29, 26, 24, 22, 24, 26, 26, 27, 29, 29, 32, 48, 52, 57, 62, 68, 69, 71, 73, 75, 76, 78, 79, 80, 79, 77, 76, 75, 75, 77, 78, 78, 79, 78, 80, 80, 78, 76, 75, 79, 79, 77, 78, 79, 78, 77, 75, 73, 75, 75, 79, 79, 79, 79, 80, 78, 76, 76, 78, 80, 81, 81, 81, 81, 81, 80, 77, 76, 76, 76, 76, 76, 76, 76, 78, 76, 75, 77, 79, 77, 79, 80, 80, 80, 80, 80, 80, 72, 70, 68, 66, 66, 66, 68, 71, 72, 73, 75, 75, 73, 73, 72, 71, 69, 67, 66, 64, 59, 55, 51, 49, 46, 39, 36, 34, 31, 29, 31, 31, 31, 29, 29, 32, 37, 37, 40, 42, 42, 60, 62, 64, 66, 69, 70, 72, 72, 72, 72, 72, 71, 70, 70, 70, 65, 62, 59, 55, 52, 46, 46, 45, 42, 37, 36, 34, 31, 31, 26, 24, 24, 26, 27, 32, 36, 37, 40, 45, 49, 55, 59, 61, 62, 62, 62, 62, 71, 72, 72, 70, 71, 70, 69, 68, 67, 64, 62, 62, 49, 46, 45, 42, 37, 36, 32, 31, 29, 27, 29, 31, 29, 32, 36, 36, 40, 42, 45, 51, 57, 60, 62, 64, 68, 69, 71, 72, 73, 72, 71, 70, 70, 68, 67, 66, 64, 61, 57, 55, 48, 48, 46, 36, 34, 34, 29, 29, 27, 29, 29, 29, 32, 32, 34, 36, 39, 48, 53, 59, 62, 64, 66, 68, 70, 72, 72, 72, 72, 71, 72, 71, 69, 67, 64, 62, 59, 55, 51, 46, 45, 42, 39, 36, 34, 32, 34, 34, 49, 52, 56]
rbuf = [24, 23, 23, 24, 24, 24, 25, 26, 27, 27, 26, 26, 26, 26, 24, 26, 26, 27, 28, 28, 27, 25, 24, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 25, 27, 27, 29, 27, 28, 24, 23, 23, 24, 26, 28, 27, 29, 28, 27, 25, 23, 22, 23, 27, 28, 28, 27, 27, 25, 24, 23, 23, 24, 26, 27, 27, 27, 27, 27, 22, 21, 21, 20, 19, 17, 17, 17, 18, 19, 19, 21, 23, 25, 25, 33, 34, 37, 40, 41, 44, 46, 46, 48, 47, 47, 47, 47, 44, 43, 42, 40, 37, 34, 31, 29, 27, 26, 25, 22, 19, 18, 18, 15, 16, 16, 16, 17, 16, 16, 17, 19, 20, 23, 26, 29, 32, 33, 40, 42, 44, 45, 48, 47, 47, 48, 49, 49, 48, 49, 48, 48, 46, 45, 43, 40, 38, 35, 33, 31, 28, 27, 25, 23, 21, 19, 18, 18, 14, 15, 16, 17, 16, 16, 17, 15, 17, 19, 22, 24, 25, 29, 29, 36, 39, 41, 43, 45, 46, 47, 48, 48, 48, 48, 49, 48, 47, 45, 44, 42, 41, 39, 37, 33, 32, 29, 27, 25, 24, 21, 21, 21, 21, 21, 21, 15, 15, 16, 16, 16, 16, 15, 14, 13, 12, 11, 11, 11, 10, 9, 7, 6, 6, 6, 5, 5, 5, 5, 5, 4, 4, 4, 4, 4, 4, 5, 5, 5, 4, 6, 6, 6, 7, 7, 7, 9, 11, 13, 13, 12, 13, 13, 13, 13, 13, 14, 15, 16, 17, 16, 16, 13, 13, 12, 11, 10, 10, 10, 11, 10, 9, 7, 6, 6, 6, 5, 5, 5, 5, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 6, 6, 5, 7, 6, 8, 8, 9, 10, 11, 13, 13, 12, 13, 
13, 12, 13, 14, 16, 16, 16, 16, 16, 14, 13, 12, 12, 10, 10, 11, 11, 11, 7, 6, 6, 5, 5, 5, 4, 5, 4, 4, 4, 4, 4, 5, 4, 5, 5, 5, 5, 4, 7, 7, 9, 11, 12, 12, 12, 13, 12, 12, 13, 13, 15, 15, 12, 12, 12, 12, 12, 12, 12, 9, 6, 5, 5, 5, 5, 5, 38, 38, 40, 43, 44, 43, 43, 41, 38, 38, 38, 42, 42, 42, 42, 40, 36, 35, 33, 32, 31, 29, 24, 22, 21, 22, 23, 26, 29, 32, 35, 35, 39, 42, 44, 45, 47, 47, 48, 48, 48, 48, 43, 41, 39, 36, 33, 30, 28, 27, 25, 21, 18, 16, 14, 14, 13, 14, 15, 16, 19, 20, 22, 24, 28, 33, 
33, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 45, 44, 41, 40, 37, 34, 28, 26, 24, 20, 18, 16, 16, 16, 15, 14, 16, 19, 19, 18, 16, 18, 20, 21, 21, 21, 20, 18, 16, 16, 15, 18, 18, 20, 20, 20, 20, 16, 16, 17, 19, 21, 21, 21, 20, 19, 17, 17, 17, 17, 17, 17, 17, 17, 17, 16, 16, 19, 21, 21, 21, 21, 21, 19, 18, 18, 19, 20, 19, 20, 20, 20, 
18, 17, 16, 16, 18, 20, 20, 21, 20, 17, 16, 17, 18, 19, 18, 15, 15, 16, 19, 20, 20, 23, 31, 34, 35, 37, 39, 43, 44, 45, 46, 47, 48, 47, 47, 47, 45, 45, 39, 37, 34, 31, 28, 27, 27, 25, 21, 19, 18, 16, 15, 12, 13, 14, 17, 20, 21, 22, 24, 27, 30, 33, 33, 36, 39, 42, 42, 42, 42, 48, 47, 48, 46, 44, 44, 41, 39, 37, 34, 31, 29, 27, 23, 
21, 19, 18, 18, 17, 17, 15, 16, 16, 19, 20, 21, 25, 30, 33, 33, 35, 38, 41, 43, 45, 47, 48, 49, 49, 49, 49, 49, 49, 44, 42, 40, 36, 33, 30, 29, 28, 27, 23, 21, 18, 17, 17, 17, 16, 16, 18, 19, 19, 20, 23, 26, 29, 32, 35, 36, 40, 45, 46, 48, 48, 49, 50, 50, 49, 49, 48, 46, 45, 44, 41, 33, 29, 28, 27, 25, 23, 21, 20, 18, 18, 18, 18, 
16, 15, 16, 16, 20, 22, 23, 26, 28, 33, 34, 35, 37, 39, 41, 44, 46, 47, 47, 48, 47, 46, 45, 44, 42, 40, 38, 36, 33, 31, 28, 27, 27, 21, 18, 17, 16, 15, 13, 12, 12, 13, 14, 16, 17, 16, 16, 16, 19, 19, 21, 20, 23, 22, 23, 23, 24, 25, 27, 28, 29, 27, 26, 24, 24, 24, 26, 27, 27, 27, 27, 28, 28, 27, 25, 24, 28, 28, 27, 27, 28, 27, 26, 
24, 24, 23, 25, 28, 28, 27, 27, 27, 26, 25, 24, 26, 28, 29, 29, 30, 29, 29, 28, 24, 24, 24, 24, 24, 24, 24, 24, 26, 26, 24, 26, 27, 27, 28, 28, 28, 28, 28, 28, 28, 31, 32, 33, 36, 38, 42, 45, 46, 48, 48, 50, 50, 50, 49, 49, 49, 47, 45, 43, 41, 38, 35, 32, 30, 27, 22, 20, 19, 17, 17, 17, 17, 17, 16, 17, 18, 21, 20, 22, 24, 24, 34, 
35, 37, 39, 43, 44, 46, 47, 48, 49, 48, 48, 48, 47, 47, 43, 40, 38, 35, 33, 28, 27, 26, 25, 23, 20, 19, 17, 17, 13, 13, 13, 14, 16, 18, 19, 20, 22, 25, 28, 32, 33, 35, 35, 35, 35, 35, 47, 48, 48, 48, 48, 47, 46, 45, 44, 42, 40, 40, 31, 27, 26, 25, 23, 20, 18, 17, 16, 16, 16, 17, 16, 18, 19, 20, 22, 23, 25, 29, 33, 34, 35, 37, 41, 
43, 45, 47, 49, 48, 49, 48, 47, 46, 45, 44, 41, 39, 37, 34, 30, 28, 28, 21, 19, 18, 17, 17, 16, 16, 16, 16, 17, 18, 17, 19, 22, 27, 31, 34, 35, 37, 40, 42, 45, 46, 47, 48, 48, 48, 49, 47, 46, 44, 42, 40, 37, 35, 31, 30, 27, 26, 24, 21, 20, 18, 17, 17, 20, 21, 23]
dump_index = 369
true_colors = ['red', 'green', 'red', 'blue']

[dlech]

After just a quick glance at the code, it looks like we might need to increase our thresholds for low value and low saturation quite a bit. We may also need to make these dependent on the sensor type.

I also wonder if it would be possible to do some sort of discrete time analysis of the hue and saturation to detect if these signal are currently "noisy" or not instead of the current method of "if X is below threshold then Y is noisy".

[laurensvalk]

Yes, that was one of the first realizations for me as well. For detectable colors I included the track sleepers, the floor, and each of the markers colors

Did you use the standard colors like Color.RED or did you provide your own?

[laurensvalk]

I found that if I only put the colors I wanted to match in the list [...] then there were errors. But if I also included "colors" I didn't want, which included the color of the track and the color of the floor underneath the track, then the matching seemed to work fairly well.

This is probably where the documentation should be clearer. The intention is indeed to include all colors that it can detect in your application.

Maybe distinguishable is a better word than detectable (but it is harder to spell 😄)

[Novakasa]

Did you use the standard colors like Color.RED or did you provide your own?

I used my own, measured using sensor.get_hsv(). To make it a bit more consistent I used for one color multiple measurements to cover the variability of individual measurements.

[Novakasa]

But back then I didn't know that the hue was so erratic, I probably could have improved the scheme a lot if I put in colors for the floor and sleepers with the maximum range of hues.

But to reiterate, I think doing the color error like outlined here seems like the most sensible thing to me, as it doesn't care for the undefined hue for dark or unsaturated colors.

[laurensvalk]

That sounds good. Are you interested in giving it a try with a pull request?

(And since this happens under the hood, there’s no rush.)

[Novakasa]

Sure, I'll try! Never compiled the firmware myself, but I had been meaning to try anyways.

[Novakasa]

Here are some plots for the difference between the error estimations.
The HSV data:

Mapping the data into HSV Cone (notice how this is insensitive to hue noise):

Error compared to color at index 600 using squared cartesian distance between cone coordinates:

Error as currently implemented in pybricks (hopefully I treated the value threshold and low saturation penalty right):

[Novakasa]

And here are the errors compared to color at index 0

Squared cartesian distance between cone coordinates:

As implemented in pybricks:

[Novakasa]

One can choose a different weight for the z-scale of the cone (=value), increasing this gives increased sensitivity between the dark colors in the cartesian error estimate:

[Novakasa]

I have now managed to set the environment up (using windows subsystem for linux) and can compile the firmware, so I will work on a pull request for this. Since the mapping requires trigonometric functions, is there a implementation for fixed point sine/cosine in the codebase that I can use? I am guessing that we want to avoid floating point functions because Move Hub doesn't have floats.

[dlech]

is there a implementation for fixed point sine/cosine in the codebase that I can use?

We are using https://github.com/PetteriAimonen/libfixmath already. lib/libfixmath/ in the pybricks-micropython repository.