How to implement an image backgound eraser?(Like PS)

[BigHeadDev]

Hello everyone,

I am currently developing an image editing software using WPF. The primary functionality involves sending an image to a server via an API, where the server processes the image (i.e., performs background removal) and returns a transparent PNG along with a black and white mask image. My task is to display this PNG image in a WPF Image control, which I have successfully implemented in the first version of my application.

---

Issue at Hand:

However, I've encountered a limitation with the server's AI processing—it does not always accurately recognize and process the elements in the images. In some instances, it either removes additional elements that should have remained or fails to remove some that should have been deleted.

---

Planned Enhancements for Version 2:

To mitigate this issue, I aim to develop a second version that enhances user interaction, allowing for manual correction or deletion of elements within the images. The requirements for this version are as follows:

1. Mouse-Driven Image Corrections: Enable users to interact directly with the displayed image using the mouse, making it possible to manually repair or remove data on the image.
2. Adjustable Tool Settings: Introduce a slider control to modify the size and hardness of the tool used for image corrections.

These enhancements are inspired by similar functionalities observed in the Windows Photos app and some web applications. I will attach a screenshot demonstrating the desired functionality for clearer reference.

Image app on Windows 10

An Cutout Website

---

Challenge:

Despite numerous attempts, I have struggled to implement these functionalities using WPF and am now reaching out to the community for technical guidance and support.

---

Assistance Sought:

Recommendations for integrating mouse-driven image interactions in WPF.
Suggestions on how to effectively implement a slider to adjust tool settings (size and hardness) in the context of image editing.

Finally, I am attaching three images for testing purposes:

The Original Image

The Transparent Image returned by the API

The Black and White Mask Image returned by the API

These images should help in providing a clear context for the functionalities I aim to implement.

---

Any guidance, examples, or references to similar projects would be deeply appreciated. Thank you in advance for your support!

[lindexi]

You can utilize the Clip feature to achieve this function. In WPF, clipping can be accomplished by assigning a value to the Clip property. Additionally, you can push the clipping range in the DrawingContext for more granular control. Below is a screenshot of a WPF application that my team has developed. In this application, we used the calculated eraser path geometry in conjunction with assigning a value to the Clip property to achieve the erasing effect.

[lindexi]

Feel Free to let me know what you don't understand

[BigHeadDev]

Thank you, I am currently trying to write some test code

[BigHeadDev]

Hello, in your demonstration project, how to achieve 【real-time】 mouse turning points into Clip's PathGeometry? Can you provide an idea? Thanks

[lindexi]

@BigHeadDev Continuously calculate the geometric shape constructed between the current mouse drop point and the previous one, then overlay it with the current Clip using the Geometry.Combine method in the GeometryCombineMode.Exclude mode. This will allow the clipping part to continuously subtract the current mouse trajectory. The mouse trajectory, as shown in the figure below, is a combination of two circles and a rectangle.

不断的计算当前鼠标落点和上个鼠标落点之间构建的几何形状，然后通过 Geometry.Combine 方式与当前 Clip 采用 GeometryCombineMode.Exclude 方式叠加，如果即可让裁剪部分源源不断减去当前鼠标的轨迹。鼠标轨迹如下图，就是两个圆形和矩形的拼接

[BigHeadDev]

private void Grid_MouseMove(object sender, MouseEventArgs e) {
    if (isMouseDown) {
        var end = e.GetPosition(gridRoot);
        var currentLineGeometry = new LineGeometry(start, end);
       //自定义画刷宽度
        var pen = new Pen(Brushes.Black, penWiidthSlider.Value) { StartLineCap = PenLineCap.Round, EndLineCap = PenLineCap.Round };
        gridRoot.Clip = new CombinedGeometry(GeometryCombineMode.Exclude, 
            gridRoot.Clip, currentLineGeometry.GetWidenedPathGeometry(pen));
        start = end;
    }
}

Based on your suggestion, I made some modifications in MouseMove.
Two points connected to form a line, and I implemented it through LineGeometry.
I used Pen.StartLineCap and Pen.EndLineCap and GetWidenePathGeometry have implemented graphics that connect two circles and rectangles
At the beginning of the program, it met my requirements, but as the traces increased, it showed a noticeable discount rather than a smooth curve
May I ask if I did something wrong? Resulting in lower and lower drawing performance during the MouseMove process,Thanks

大佬，感谢您的建议，我在MouseMove方法中做了一些修改。
两点连线，我使用了LineGeometry实现，根据你画图的参考，我用了Pen.StartLineCap和Pen.EndLineCap，以及GetWidenePathGeometry去扩展一个画笔宽度来实现，两个圆和一个矩形的连接图形。
运行程序之后，刚开始还挺流畅，但是随着画图痕迹越来越多，线条也从刚开始圆滑的曲线变成了卡顿的折现，请问是否我的方案有什么出入导致了绘制性能越来越低？期待您的再次解答，感谢~

[lindexi]

@BigHeadDev Try replace the gridRoot.Clip = new CombinedGeometry(GeometryCombineMode.Exclude, gridRoot.Clip, currentLineGeometry.GetWidenedPathGeometry(pen)); to Geometry.Combine. Because the CombinedGeometry will store a lot of geometry, performance degrades.

试试换成 Geometry.Combine 方法，因为 CombinedGeometry 将会存储大量的几何，导致你越移动所需的计算量越大而卡顿

[BigHeadDev]

Great! According to your prompt, the current mouse connection is very smooth 💕
But there is a new problem that I don't know where it is. During my mouse move process, if I move too fast, the real-time effect will briefly disappear. You can watch a video demonstration

太赞了！根据你的提示，目前的鼠标连线很圆滑了💕
但是有一个新的问题不知道是否你也遇到过：在鼠标滑动过程中，如果移动太快，实时效果会短暂消失（卡顿），可以看gif演示

[BigHeadDev]

Hi~
Are there any solutions to improve performance？

大佬~
有没有方案可以解决Geometry越来越复杂之后的合并问题 方案？

[miloush]

You could also just use WriteableBitmap and do the blending yourself.

[BigHeadDev]

Thanks for your replay.
Can writeablebitmap achieve erasing images with wide lines?

[miloush]

You need to modify the pixels yourself, so you can have any brush shape you want, even bitmap based...

Example:
DELwpfFX.zip

[BigHeadDev]

Thanks for your reply and code~ ❤️
Wow~ Besides some jagged edges, this effect looks very good!
But it seems that there are some algorithms here that I need to study carefully, which are a bit patterned and complex for me

[miloush]

Yeah you probably want the brush to fall off a bit more softly on the periphery. Or as I suggested a palette of bitmap brushes that you will copy over to the cursor location.

This might be helpful: https://ciechanow.ski/alpha-compositing/

Note the code relies on the bitmap being Bgra32 pixel format or similar (i.e. the pixel being 4 bytes where alpha is the last one). You can however convert pixel formats using FormatConvertedBitmap.

[BigHeadDev]

Thanks for your supplementary explanation,I will consider your plan~💕