- Sponsor
-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] Calibrate HSV measurements #116
Comments
I'd like to tag @tomalphin who looks to be quite the expert on LEGO colors. The colors in your diagrams seem to be slightly different than the Bricklink colors. If not Bricklink, which reference would you recommend as the standard to pick the right hue for LEGO colors? EDIT: Ah, I see this comment on your blog:
In case you are interested to be involved, it would be awesome to have accurate reference values for the primary/secondary LEGO colors. It would be pretty cool to calibrate the sensor as a super accurate color scanner which goes at just $15 on the official shop. |
A patch for this is applied here. More accurate HSV representation would be nice, but it is less critical now that the user can do their own calibration. |
This is apparently not good enough, so let's reopen it. Related: pybricks/pybricks-micropython#93. @Novakasa |
I could take a look at this, at least I can extract some raw RGB data and look at the deviation for some LEGO colors. Should we then incorporate a mapping already in |
In theory, there could be a distance-dependent mapping to attempt to compensate for a given distance of the surface. This probably should be provided by the user. |
Instead of fixing/equalizing the RGB readings, we could perhaps have some mapping, e. g. like the inverse of https://github.com/pybricks/pybricks-micropython/blob/f69bd694ba481563c980e09fae41b60c0ecea4fc/lib/pbio/src/color/conversion.c#L85 And use sensor type specific parameters if needed to ensure that the rgb-to-hsv conversion produces consistent hsv values. I don't think we need to match LEGO colors at this level per se, but they are still useful as a reference since all of us can reproduce the same results then. |
Would only v be distance dependent in theory? If we can get H to be consistent across sensors, I think that's a good start. |
That was my impression when I last did a very superficial test, but I'll look at it more systematically. |
Today I played around with this setup: It's still early, but here is a result that kind of shows that in large part only the value changes with the distance, with the sweetspot being 2 plates distance. In my case the effect of ambient light conditions is rather negligible. I've pushed the related files to this repo: https://github.com/Novakasa/pybricks-color |
Also, the largest RGB value I encountered is 473, exceeding the 440 that is used when mapping to 0-255 in |
Nice, that is good to know. Which mapping was used to produce this? The default or your PR? If I recall correctly, this also hold qualitatively for the SPIKE sensor, but the value for saturation and value were a bit different under the same conditions. Perhaps the sensor-type specific mapping from rgb-to-hsv can take care of making them more similar.
This might well be the case. I think there was an (unfortuate) hard cap of 440 on the SPIKE sensor RGB values coming out of the sensor. As a result, hue calculations break when the brick is too close.
Yes, these are the RGB values we use. |
I only recorded the RGB values from
So the ColorDistanceSensor uses the same mapping to make its RGB values consistent with the SPIKE sensor? If I see it correctly, we get an int overflow in |
Except for the hack that skews some of the values. :) So when posting graphs (which are awesome, keep making them!), maybe add one line directly underneath to say how it was made.
It uses the same mapping right now, but that does not result in consistent HSV values. The idea is to make one rgb-to-hsv mapping for the SPIKE sensor, and another for the Boost Color Distance Sensor, such that the end-user HSV values are comparable. |
Oh yeah, kind of essential for this topic :P
Hmm, I think I still don't quite get it. If I look at the mapping from raw to rgb both seem to use different approaches. SPIKE vs Color-Distance-Sensor. The ColorDistanceSensor maps from 0-440 to 0-255, while SPIKE maps from 0-1023 to 0-255. As I encountered raw values above 440 using the ColorDistanceSensor, shouldn't we adapt the threshold there? BTW, In my plot above the hue is in radians, so it is kind of hard to see it really, but the hue didn't really change much with distance. |
You're right 👍
We should probably drop every scaling and skewing operation we have, including the one you found (and I forgot about 😄), and replace it with a sensor-type-dependent raw-rgb-to-hsv conversion. |
I tried a linear mapping in RGB space (a 3x3 matrix) and used We can see that it improves in some cases, but can't fit everything simultaneously. I think this shows that at least in RGB, the mapping has to be nonlinear. The resulting matrix looks like this: array([[ 1.39618714, -0.04407636, -0.18775358],
[ 0.02527175, 1.59905428, -0.37466398],
[-0.21313028, 0.23361669, 1.20627808]] Since I haven't put any absolute normalization on the data, this matrix can be multiplied by an arbitrary scalar factor to compensate for RGB normalization ( = value calibration) |
In pybricks/pybricks-micropython#104, I mentioned this:
It just came to me that that is probably due to what I talked about in the above comment #116 (comment). White at distance of 2 plates probably clips the range of 0-440, so the mapping produces values above 255, leading to an int overflow. |
Is your feature request related to a problem? Please describe.
The hues reported by the SPIKE Prime Color sensor, as well as the ones we calculate for the BOOST
ColorDistanceSensor
aren't really great in the 0--120 degrees region.For example, yellow is reported at a hue of about 30, while in reality it should be about 50--60.
This is currently not a problem since the matching value is set to 30 in the color map. However, it would be a problem for displaying hsv colors on the status light, because then it would look orange.
Describe the solution you'd like
Pass the RGB and/or HSV values through a mapping that corrects this. This way, measuring a color (e.g. yellow) and showing it on the status light should produce the correct result.
In the case of SPIKE Prime, it means we should no longer use its HSV mode, but compute HSV ourselves from the RGB values. This seems to be what the sensor is doing anyway, so we don't seem to lose any information with this approach.
First steps towards solution
I've been starting to collect some measurents of scanning LEGO bricks. Each object is a stack of at least 3 bricks of the same color, and I was scanning the flat side of the stack. I placed them at such a distance from the sensor such that the V value was constant across all measurements.
This table reports the values we currently read, compared to what they should be according to the Bricklink color guide.
Notes
The text was updated successfully, but these errors were encountered: