Countdown to the new Images.jl #542
Comments
|
Absolutely! Right now I do the operation chaining for my image augmentation package unnecessary inefficient by creating temporary copies after each operation (input -> cropped image -> rotated image -> distorted image .... etc). After watching your talk I am really sold on the idea of lazy views, which should - if I am not mistaken - allow for just nesting views instead; which would avoid my predicament. That said, I am not an image processing or computer vision person, so bare with my newbie mistakes I am bound to make, but I am catching on. To be a little less vague about my true intentions, I essentially want to create a bridge between julias nice Image functionality and machine learning needs, which diverge in some strange places. For example the "z" or "time" dimension does for us store observations with no intrinsic order/relation. Even stranger: in some of our usecases the color dimension of an image is actually farther apart in memory than the spatial!. so a 3x20x20x1000 image would often be represented in 20x20x3x1000 (this is just a consequence of how some of our algorithms work) Anyway, thank you for taking the time to explain this! |
|
With regards to your 20x20x3x1000 array, using ImagesCore you can already do this: julia> using ImagesCore, Colors
julia> a = rand(UInt8, 20, 20, 3, 1000);
julia> summary(a)
"20×20×3×1000 Array{UInt8,4}"
julia> p = permuteddimsview(a, (3,1,2,4));
julia> c = colorview(RGB, ufixedview(p));
julia> summary(c)
"20×20×1000 ImagesCore.ColorView{ColorTypes.RGB{FixedPointNumbers.UFixed{UInt8,8}},3,MappedArrays.MappedArray{FixedPointNumbers.UFixed{UInt8,8},4,Base.PermutedDimsArrays.PermutedDimsArray{UInt8,4,(3,1,2,4),(2,3,1,4),Array{UInt8,4}},ImagesCore.##15#17{FixedPointNumbers.UFixed{UInt8,8}},ImagesCore.##16#18}}"
julia> col = c[5, 3, 80]
RGB{U8}(0.953,0.914,0.208)
julia> red(col).i
0xf3
julia> a[5,3,1,80]
0xf3
julia> green(col).i
0xe9
julia> a[5,3,2,80]
0xe9
julia> blue(col).i
0x35
julia> a[5,3,3,80]
0x35
julia> a[5,3,2,80] = 0
0
julia> c[5,3,80]
RGB{U8}(0.953,0.0,0.208)That's the thing about these views...it takes a while to realize how something that seems so "boring" can be so insanely useful. In this case we put a "permuteddimsview" inside of a "ufixedview" inside of a "colorview" and should get darn good performance despite all those shennanigans. Now the only thing left is to wrap it in an |
|
Maybe we could include 👍 for 100% test coverage ! Love to see the bright green coverage badge 😄 Regarding the documentation, I was thinking of having one |
|
Should TestImages.jl be moved to JuliaImages? We could also have some other package (ImageSets.jl ?) which will have the famous datasets in a easily importable format. |
|
I think it would make sense to keep the documentations separate and focused, but with a link section somewhere which point to each other. |
|
Maybe we could have stuff like tutorials etc on the main website and if someone wants to dig into the function reference then they would be pointed to the individual websites. |
|
Before I read any of the above, let me say thanks for filling us in! I was wondering "what exactly is happening???" |
|
All of the above sounds great, I can't wait to kick the tires. Here's one thing for my personal wishlist, a pretty printer for image types. When I see something like: I have to sit there and count curly braces to figure out where I am! A few spaces in there would help as well. |
I was wondering the same thing. I am in the middle of putting the gradient kernels into ImagesFiltering, but the edge-detection stuff doesn't quite fit. One option is to leave them in Images (src/algorithms won't entirely go away), but I tend to agree that ImageFeatures might be a better choice. CC @kmsquire to see if he has a preference.
Yes, that's important. The new repos are generally closer to that goal. ImagesAxes is much better than it looks, because some bug (probably in julia) incorrectly misses test coverage of traits-functions. See mauro3/SimpleTraits.jl#6.
I've had the same thought. I think it will be good to have the docs for Images.jl take much more of a tutorial style, and they can link out to the docs for the individual packages for details. But it would be good to also have one giant function reference. I think that will be fairly straightforward with Documenter, but I haven't tested yet.
Yes, good idea.
Yes, I agree. Ideally that would be something fixed in base julia, but perhaps we could pioneer something here. Jeff has been violently opposed to abbreviating the output of |
|
I think putting edge detection in ImageFeatures would be fine. |
|
When we move an algorithm say A, how do we do so without affecting either of the packages? If I just directly copy, it gives a redefinition of A error. What is the correct way to do this so that A is ported and none of the packages depending on the older A are affected? |
@timholy What do you have in mind here? I wrote a beginner's tutorial on julia's approach to handling image/pixel data for a project of mine over here, and I'd happily contribute it if you want it (it is somewhat WIP still). But since it is mainly aimed at ML students (possibly new to Julia), and non-image-experts, I am guessing it might be a little too beginner oriented to be useful to you? (also, feel free to criticize me when I made a mistake or over/understated something) |
I think we have to do merges to master/tagging in synchrony. In other words, start a new branch in both packages that make the same change.
I haven't read it in detail, but this looks like exactly the kind of thing I was envisioning. Most will probably display images in the docs, too. I haven't looked in detail, but http://scikit-image.org/docs/dev/user_guide.html seems like a promising model to follow. |
|
where would you want such tutorials to be located at? |
|
I think Images.jl should be the home of all the "major" documentation. (Images.jl will move to JuliaImages at the time of the transition.) |
|
We could have the docs in a separate repository -> https://github.com/JuliaImages/juliaimages.github.io so that juliaimages.github.io becomes the website. |
|
One thing to keep in mind is that if the docs should contain lots of images, then they have to be stored somewhere. That can be a hassle depending on where the docs are (since one wants to keep the julia package size small and the git history clean) |
|
We faced this problem in TestImages.jl for the documentation. Turns out you can use the |
|
All good suggestions. 👍 to putting it in the org. |
|
Is |
|
We should certainly do a blog post on julialang.org once this is done. |
|
Will the new |
|
Yes, it will. First, they will all be REQUIREd packages, so they'll be installed automatically. Second, their functions will all be This split-up is partly designed to make them more accessible to people who want a subset of the functionality of Images...for example JuliaLang/julia#18384. There is no way to satisfy the desire to "reuse code across different types of applications" unless we split things into relatively focused units of functionality. Reexporting is the key that allows you to have this without sacrificing on other equally-worthy goals, like "provide a comprehensive Images package." |
|
Thanks for the clarification. That sounds promising, I haven't heard of
the @reexport feature yet. Is there a framework for running integration
tests in the meta package?
|
|
You can run any tests you want...but it sounds like you have something specific in mind with "framework", what do you have in mind? |
|
So currently most Julia packages have a test/ folder with unit tests On Wed, Sep 7, 2016, at 15:06, Tim Holy wrote:
Links: |
|
We could define the |
|
Is there some documentation how to combine |
|
Sure, they're completely orthogonal: just use an |
|
So, a typical usage of Images.jl was What is your advice to handle this in the future. I will try playing around with this myself, but maybe it would be good to have this particular case discussed in the documentation somewhere. And whats about the other image loader, are the already ported to the new Image infrastructure? |
|
In case you haven't seen it, https://juliaimages.github.io/ImageAxes.jl/latest/ might help. Simple (ignoring pixel spacing): There isn't quite as simple an approach for handling the pixel spacing (this is exactly the kind of feedback I'm looking for), but it is more flexible: ImageMeta(AxisArray(colorview(Gray, ni.raw), Axis{:x}(rngx), Axis{:y}(rngy), Axis{:z}(rngz), Axis{:time}(rngt)))where Do you need an easier way to specify pixel spacing? The default |
|
Note you don't really need the |
|
Hm, this is of course flexible but also pretty complicated. |
|
So if I see it correctly I will have to but the |
|
Well, A = AxisImage(data, (:x, :y, :z, :time), (dx, dy, dz, dt), [(x0, y0, z0, t0)])? As for rotation goes...that would either have to be in the metadata, or possibly you could use a |
|
Yes, such a syntax would be definitely a nice convenience constructor, so thumbs up. |
|
Oh, and to answer an earlier question, yes, all the IO packages have been updated too if you check things out as described in #542 (comment). That gist has been updated several times, so you may want a fresh copy. |
|
Ok thanks. Its quite interesting to see how this restructuring will change the workflow. Before, all parameters such as |
|
Yes, the idea is you just pass an Could you open an issue on ImageAxes with #542 (comment), so I don't forget? |
|
Added to |
|
Closed by #577 |
There have been justifiable questions about "what exactly is happening?" so let me post my TODO list here (including completed items, with an explanation of purpose):
Range(even one with physical units) for each axis, which thus incorporates thepixelspacing. All this information is exposed to the type system (rather than through fields of the properties dict). What that means is that indexing operations become type-predictable: for example, if you slice at a particular spot in time, the new array doesn't have a time axis anymore. In the old Images.jl we had to check for this and set thetimedimproperty to 0 at runtime, and because that's slow we were constantly throwing away the dictionary for performance and therefore losing track of orientation. Now everything should Just Work.imfilterimplementations. Everything will be type-stable, efficient algorithms will be chosen automatically based on the kernel size, and a platform will be in place for more efficient operations with separable kernels. Also options for performing the computation on the GPU, etc. This package will also likely be the first place to test another experimental direction: the ability (at the user's choice) to dispatch to algorithms in other libraries like OpenCV. (Currently this package is well underway but still needs work, and has been the occasion for me making changes to julia and a half dozen other packages. It won't be runnable by anyone else until julia-0.5-rc3 at the earliest.)MapInfomachinery with something based on functions. This will probably go in ImageCore.?pkggive an API summary@rexport. Get the old tests passing:Things that could be done before or after:
as well as thesrc/map.jlfunctionality. (For now I don't really plan on touching that, so that file could probably be copied verbatim.)Longer-term future: implement lots of missing functions. Other BSD-licensed projects should be used as important inspiration.
The text was updated successfully, but these errors were encountered: