[OSPP]Adding image segmentation algorithms

[JKay0327]

Hi

I' m Yifan Chen, a graduate student major in mathematics from China. My daily study and work are related to Image processing, so I'm interested in the project 'Add performant image processing algorithms'. I have some programming experience on Python and Julia. I have used Julia to solve some machine learning task.

For this OSPP project, I currently plan to implement the following two algorithms:

• L0-smooth: Xu, Li, et al. "Image smoothing via L 0 gradient minimization." Proceedings of the 2011 SIGGRAPH Asia Conference. 2011. [paper]
• Chan-Vese segmentation: An Active Contour Model without Edges, Tony Chan and Luminita Vese, Scale-Space Theories in Computer Vision, 1999, DOI:10.1007/3-540-48236-9_13

L0-smooth is only available in Matlab, Chan-vese is available in scikit-image.

I have just finished a demo which implements the algorithm in paper, I have put this demo in the repo, please see https://github.com/JKay0327/Image-Smoothing-via-L0-Gradient-Minimization, here's a preview of it:

If there are some suggestions, I'm glad to consider these problems and discuss with you. Thanks~

[johnnychen94]

Tim has written this freqkernel for over two years on my request in #755 but I haven't really used it in practice. There are no codes in JuliaImages built on top of this. I'm impressed that you just added the first one!

---

Given that you've shown your coding ability by basically putting l0-smooth together, I'm greedily asking for more algorithms to implement for this OSPP event:

• Jinshan's L0 deblur is very similar to L0-smooth.
• reimplement imROF #755 is perhaps just a piece of cake for you since it uses the same split bregman solver. Maybe you can help closing it by adding it to https://github.com/JuliaImages/ImageNoise.jl?
• dark channel prior proposed by Kaiming is quite famous. These are not available in JuliaImages yet. Given your solid math background, maybe you'll be interested in dark channel dehaze or dark channel deblur.
• We don't have many image inpainting algorithms in JuliaImages, so it might be worth adding more algorithms to https://github.com/JuliaImages/ImageInpainting.jl. I know little about this field but maybe you'll be interested in the MCA inpainting.

For image segmentation, I've also heard of these two algorithms (but never read the details)

• Vese, Luminita A., and Tony F. Chan. "A multiphase level set framework for image segmentation using the Mumford and Shah model." International journal of computer vision 50.3 (2002): 271-293.
• Arbelaez, Pablo, et al. "Contour detection and hierarchical image segmentation." IEEE transactions on pattern analysis and machine intelligence 33.5 (2010): 898-916.

All these are classical image processing algorithms that have a lot of math under the hood so I listed them here in case you're interested. Of course, this is not a hard requirement at all. Two algorithms would be really good enough to me, and three/four/five/... would make me excited for the entire summer 😆

Let me know if you're interested in any of these algorithms I listed or anything that I didn't mention here. I encourage you to challenge yourself by proposing more 😆.

---

Given that you've never participated in open source projects before, I have a few words about the workflow/timeline: having a working demo is still quite far from it being used by others. You need to write tests, to tweak the performance and default parameters, you need to support different inputs. My suggestion is that we keep it slower at first by polishing one algorithm. After you become familiar with how to write high-performance Julian codes and learning more about the JuliaImages ecosystem, you can move faster by adding more algorithms.

[JKay0327]

Thanks for your suggestions! 😄

Given that you've shown your coding ability by basically putting l0-smooth together, I'm greedily asking for more algorithms to implement for this OSPP event:

• Jinshan's L0 deblur is very similar to L0-smooth.
• We don't have many image inpainting algorithms in JuliaImages, so it might be worth adding more algorithms to https://github.com/JuliaImages/ImageInpainting.jl. I know little about this field but maybe you'll be interested in the MCA inpainting.

I'm pretty interested in these two algorithms. Considering that I'm a beginner of open source projects, I cannot make sure that how much time I'll need to finish an entire implement work including writing the testfiles, documents and adjusting the parameters. If I can complete the implement tasks of L0-smooth and Chan-Vese Segmentation successfully, I'll try my best to implement as many algorithms as I could. 😀

Given that you've never participated in open source projects before, I have a few words about the workflow/timeline: having a working demo is still quite far from it being used by others. You need to write tests, to tweak the performance and default parameters, you need to support different inputs. My suggestion is that we keep it slower at first by polishing one algorithm. After you become familiar with how to write high-performance Julian codes and learning more about the JuliaImages ecosystem, you can move faster by adding more algorithms.

The truth is that this is the first time I participate in an open source project, so these words are very helpful for a beginner like me. And I think my first step will focus on the futher complete of L0-smoothing. Although I've already finish the demo, I still need to test whether it can always work, think of how to make my code run with high efficiency and it also need a document. As your saying, the process of polishing my first work is really important, I still need to lots of work on my first task. After finishing the first one, I have confidence to move faster. I'm looking forward to make my codes a part of JuliaImages. 😆

[johnnychen94]

Sorry for the late response.

The OSPP application submission due is on June 13, I hope I can get an early draft of your proposal so that I can also provide early feedback to it. This is not a hard requirement.

The API design we use among JuliaImages is the functor design, in plain words, users are expected to use your codes in the following way:

img = testimage("cameraman")

alg = L0GradientMinimization(λ, κ)
out = smooth(img, alg)

# and also an in-place version `smooth!`
smooth!(img, alg)

An example of this design can be found in https://github.com/JuliaImages/ImageContrastAdjustment.jl. You can also check the reasoning behind it at https://nextjournal.com/johnnychen94/the-principles-of-imagesjl-part-i You also need to learn how to properly write test cases and documentation, ImageContrastAdjustment is also a very good example to learn from.