Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve conversions and support ImageCore #4

Merged
merged 12 commits into from
May 11, 2022
Merged

Conversation

timholy
Copy link
Member

@timholy timholy commented May 10, 2022

These are required for compatibility with ImageView.

Two important notes:

  • because I had prototypes of this already developed, I wanted to get this merged before tackling generalize the ColorMixture concept #2.
  • the requirement for Julia 1.7+ is worth noting as a potential source of trouble. At the moment I don't envision this package becoming "core" to the JuliaImage ecosystem so I think that dependency is OK. At present the only packages that I can imagine might want to use this are OMETIFF.jl and ZeissMicroscopyFormat.jl.

timholy added 2 commits May 10, 2022 08:17
These are required for compatibility with ImageView
@codecov
Copy link

codecov bot commented May 10, 2022

Codecov Report

Merging #4 (2676268) into main (f22ebe3) will increase coverage by 4.73%.
The diff coverage is 75.00%.

@@            Coverage Diff             @@
##             main       #4      +/-   ##
==========================================
+ Coverage   62.33%   67.07%   +4.73%     
==========================================
  Files           5        6       +1     
  Lines          77       82       +5     
==========================================
+ Hits           48       55       +7     
+ Misses         29       27       -2     
Impacted Files Coverage Δ
src/types.jl 84.84% <70.58%> (+7.42%) ⬆️
src/FluorophoreColors.jl 100.00% <100.00%> (ø)
src/imagecore.jl 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f22ebe3...2676268. Read the comment docs.

@timholy
Copy link
Member Author

timholy commented May 10, 2022

While it seems simple, I ended up changing my mind about how things work here several times. For that reason I'll leave this open a bit in case anyone feels like reviewing it. The current behavior is to throw errors upon conversion to RGB if overflow occurs; you can circumvent this by using convert(RGB{T}, c) for some type T which can support the full range of values (e.g., Float32).

Comment on lines 5 to 6
[compat]
Documenter = "0.27"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't need fancy features, then it's perhaps a good idea to just drop the compat. I'm okay either way.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly it's to avoid breakage if Documenter 0.28 or higher breaks some things we use, but that hasn't happened in quite some time so...:shrug:

I'll change it.

Copy link
Member

@johnnychen94 johnnychen94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current behavior is to throw errors upon conversion to RGB if overflow occurs; you can circumvent this by using convert(RGB{T}, c) for some type T which can support the full range of values (e.g., Float32).

It seems quite clearly documented in README, I believe it's also more stable and safer to be more restrictive for the first few versions.

README.md Outdated Show resolved Hide resolved
src/types.jl Outdated
@@ -69,8 +69,8 @@ ColorMixture{T}(Cs::NTuple{N,AbstractRGB}, channels::Vararg{Real,N}) where {T,N}
@inline promote_typeof(obj1, obj2, objs...) = _promote_type(typeof(obj1), typeof(obj2), objs...)

computeT(Cs::NTuple{N,AbstractRGB}, channels::NTuple{N,Real}) where {N} = eltype(promote_typeof(map(*, Cs, channels)...))
ColorMixture(Cs::NTuple{N,AbstractRGB}, channels::NTuple{N,Real}) where {N} = ColorMixture{computeT(Cs, channels)}(Cs, channels)
ColorMixture(Cs::NTuple{N,AbstractRGB}, channels::Vararg{Real,N}) where {N} = ColorMixture{computeT(Cs, channels)}(Cs, channels)
Base.@constprop :aggressive ColorMixture(Cs::NTuple{N,AbstractRGB}, channels::NTuple{N,Real}) where {N} = ColorMixture{computeT(Cs, channels)}(Cs, channels)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the optimization that reduces the generalization overhead you found in #2 (comment)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, in this case the performance hit was about 100x. But that penalty was clearly dependent on specifics of the implementation since as stated in the previous comment it seems to have gone away.

The issue basically is that the RGB values are part of the type, and if we change the values in some way (e.g., RGB{N0f8} -> RGB{Float32}) then unless the values are aggressively const-propped then inference may not be able to calculate them. But now all operations return an element of the same type as the original, by default, so that seems to fix things.

Copy link
Member Author

@timholy timholy May 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In other words, sometimes inference needs to be able to anticipate this change:

julia> C1 = typeof(ColorMixture(channels))
ColorMixture{N0f8, 2, (RGB{N0f8}(0.0, 0.925, 0.365), RGB{N0f8}(1.0, 0.859, 0.0))}

julia> C2 = typeof(ColorMixture(float.(channels)))
ColorMixture{Float32, 2, (RGB{Float32}(0.0, 0.9254902, 0.3647059), RGB{Float32}(1.0, 0.85882354, 0.0))}

and inference is not capable of that on 1.6. I added this point to the README.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And the bigger story is that I'm now supporting 1.6. I think the caution is still important, though.

@timholy
Copy link
Member Author

timholy commented May 11, 2022

Here's an interesting thought: what if the RGB colors are always N0f8 regardless of T? These constant-propagation inference issues will never arise. And if the "weights" are Float32, the product w[i]::Float32 * c[i]::RGB{N0f8} will promote to Float32 regardless. So the only thing lost is a bit of flexibility with respect to the freedom to specify the RGB colors. But very fine differences among the color channels will not be resolvable anyway, so perhaps it doesn't matter?

@johnnychen94
Copy link
Member

johnnychen94 commented May 11, 2022

Here's an interesting thought: what if the RGB colors are always N0f8 regardless of T?

I thought by parameterizing on values Cs instead of types typeof(Cs), we already assume the Cs choices are limited (N0f8, N0f8, ..., N0f8); otherwise the type dispatch can be quite unstable and wasteful?

@timholy
Copy link
Member Author

timholy commented May 11, 2022

Currently the type declaration is

struct ColorMixture{T,N,Cs} <: Color{T,N}
    channels::NTuple{N,T}

    Compat.@constprop :aggressive function ColorMixture{T,N,Cs}(channels::NTuple{N}) where {T,N,Cs}
        Cs isa NTuple{N,RGB{T}} || throw(TypeError(:ColorMixture, "incompatible color types", NTuple{N,RGB{T}}, typeof(Cs)))
        return new{T,N,Cs}(channels)
    end
end

so there's an inner-constructor check that Cs::NTuple{N,RGB{T}} (i.e., it uses the same eltype for channel weights and channel colors).

@johnnychen94
Copy link
Member

Right, I was assuming that Cs should always be backed by integers and not by float point numbers.

@timholy
Copy link
Member Author

timholy commented May 11, 2022

OK, I'll make that change and then OK to merge?

@timholy timholy merged commit 21e7a6b into main May 11, 2022
@timholy timholy deleted the teh/ct_conversions branch May 11, 2022 11:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants