after manually crop result not correct.

[sokphengcheang]

No description provided.

[algorr]

Manually crop is not working correctly especially with gallery picked image.

[criistian14]

@algorr Could you please provide a step-by-step list of how to reproduce this issue, along with the type of device you're using (Android or iOS)?

Additionally, could you share the image that is causing the problem?

[nilotpalkapri]

@algorr Could you please provide a step-by-step list of how to reproduce this issue, along with the type of device you're using (Android or iOS)?

Additionally, could you share the image that is causing the problem?

I simply installed your example app. Crop is not working with captured image, it is giving wrong output. Gallery image cropping is correct. I think it is because you are using video for capturing and detecting. Also flash option is not showing, it is a highly required feature.
Android 9
API 28

[mrasityilmaz]

I've been dealing with this problem for about two days, finally I found it!

The problem stems from this;

When reading the image bytes coming from the camera, it is read in a different format, which causes the size of the image to change. (ex: normally my image is 1280x720, but what is read is like 720x1280)

I found the solution as follows:

These lines in OpenCVPlugin file => val src = Imgcodecs.mdecode(MatOfByte(*byteData), Imgcodecs.IMREAD_UNCHANGED)

I replaced it with this => val src = Imgcodecs.mdecode(MatOfByte(*byteData), Imgcodecs.IMREAD_COLOR)

But I don't know how healthy a change it is. Because not all Android phones may capture images in the same format.

[criistian14]

Thanks @mrasityilmaz for your contribution. I will review it, but I am also adjusting the detection of the image orientation to properly arrange the points for cropping.

[kaushik072]

@mrasityilmaz I used your work around but I am still facing the same issue so do you have any other idea about it?

@criistian14 Thanks for this package! It's a very important package but this issue we should fix asap.

[mrasityilmaz]

@mrasityilmaz I used your work around but I am still facing the same issue so do you have any other idea about it?

@criistian14 Thanks for this package! It's a very important package but this issue we should fix asap.

You downloaded the package codes, added them locally to your project and made changes, right?

[kaushik072]

@mrasityilmaz I used your work around but I am still facing the same issue so do you have any other idea about it?
@criistian14 Thanks for this package! It's a very important package but this issue we should fix asap.

You downloaded the package codes, added them locally to your project and made changes, right?

Here I replaced your line.

Yes, Right Man!

[mrasityilmaz]

@mrasityilmaz I used your work around but I am still facing the same issue so do you have any other idea about it?
@criistian14 Thanks for this package! It's a very important package but this issue we should fix asap.

You downloaded the package codes, added them locally to your project and made changes, right?

Here I replaced your line.

Yes, Right Man!

Imgcodecs.IMREAD_UNCHANGED is used 3 times in OpenCvPlugin.kt, did you change them all?

[mrasityilmaz]

package com.christian.flutterDocumentScanner

import io.flutter.plugin.common.MethodChannel
import org.opencv.core.*
import org.opencv.imgcodecs.Imgcodecs
import org.opencv.imgproc.Imgproc


class OpenCVPlugin {
    companion object {
        fun findContourPhoto(
            result: MethodChannel.Result,
            byteData: ByteArray,
            minContourArea: Double
        ) {
            try {
                val src = Imgcodecs.imdecode(MatOfByte(*byteData), Imgcodecs.IMREAD_COLOR)

                val documentContour = findBiggestContour(src, minContourArea)

                // TODO: Use for when to use real time transmission
                // Scalar -> RGB(235, 228, 44)
                // Imgproc.drawContours(src, listOf(documentContour), -1, Scalar(44.0, 228.0, 235.0), 10)

                // Instantiating an empty MatOfByte class
                val matOfByte = MatOfByte()

                // Converting the Mat object to MatOfByte
                Imgcodecs.imencode(".jpg", src, matOfByte)
                val byteArray: ByteArray = matOfByte.toArray()


                val points = mutableListOf<Map<String, Any>>()

                if (documentContour != null) {
                    points.add(
                        mapOf(
                            "x" to documentContour.toList()[0].x,
                            "y" to documentContour.toList()[0].y
                        )
                    )
                    points.add(
                        mapOf(
                            "x" to documentContour.toList()[3].x,
                            "y" to documentContour.toList()[3].y
                        )
                    )
                    points.add(
                        mapOf(
                            "x" to documentContour.toList()[2].x,
                            "y" to documentContour.toList()[2].y
                        )
                    )
                    points.add(
                        mapOf(
                            "x" to documentContour.toList()[1].x,
                            "y" to documentContour.toList()[1].y
                        )
                    )
                }

                val resultEnd = mapOf(
                    "height" to src.height(),
                    "width" to src.width(),
                    "points" to points,
                    "image" to byteArray
                )

                result.success(resultEnd)

            } catch (e: java.lang.Exception) {
                result.error("FlutterDocumentScanner-Error", "Android: " + e.message, e)
            }
        }

        private fun findBiggestContour(src: Mat, minContourArea: Double): MatOfPoint? {
            // Converting to RGB from BGR
            val dstColor = Mat()
            Imgproc.cvtColor(src, dstColor, Imgproc.COLOR_BGR2RGB)

            // Converting to gray
            Imgproc.cvtColor(dstColor, dstColor, Imgproc.COLOR_BGR2GRAY)

            // Applying blur and threshold
            val dstBilateral = Mat()
            Imgproc.bilateralFilter(dstColor, dstBilateral, 9, 75.0, 75.0, Core.BORDER_DEFAULT)
            Imgproc.adaptiveThreshold(
                dstBilateral,
                dstBilateral,
                255.0,
                Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C,
                Imgproc.THRESH_BINARY,
                115,
                4.0
            )

            // Median blur replace center pixel by median of pixels under kernel
            val dstBlur = Mat()
            Imgproc.GaussianBlur(dstBilateral, dstBlur, Size(5.0, 5.0), 0.0)
            Imgproc.medianBlur(dstBlur, dstBlur, 11)


            val dstBorder = Mat()
            Core.copyMakeBorder(dstBlur, dstBorder, 5, 5, 5, 5, Core.BORDER_CONSTANT)


            val dstCanny = Mat()
            Imgproc.Canny(dstBorder, dstCanny, 75.0, 200.0)

            // Close gaps between edges (double page clouse => rectangle kernel)
            val dstEnd = Mat()
            Imgproc.morphologyEx(
                dstCanny,
                dstEnd,
                Imgproc.MORPH_CLOSE,
                Mat.ones(intArrayOf(5, 11), CvType.CV_32F)
            )


            // Getting contours
            val contours = mutableListOf<MatOfPoint>()
            val hierarchy = Mat()
            Imgproc.findContours(
                dstEnd,
                contours,
                hierarchy,
                Imgproc.RETR_TREE,
                Imgproc.CHAIN_APPROX_SIMPLE
            )
            hierarchy.release()

            // Finding the biggest rectangle otherwise return original corners
            val height = dstEnd.height()
            val width = dstEnd.width()
            val maxContourArea = (width - 10) * (height - 10)

            var maxArea = 0.0
            var documentContour = MatOfPoint()

            for (contour in contours) {
                val contour2f = MatOfPoint2f()
                contour.convertTo(contour2f, CvType.CV_32FC2)
                val perimeter = Imgproc.arcLength(contour2f, true)

                val approx2f = MatOfPoint2f()
                Imgproc.approxPolyDP(contour2f, approx2f, 0.03 * perimeter, true)

                // Page has 4 corners and it is convex
                val approx = MatOfPoint()
                approx2f.convertTo(approx, CvType.CV_32S)
                val isContour = Imgproc.isContourConvex(approx)
                val isLessCurrentArea = Imgproc.contourArea(approx) > maxArea
                val isLessMaxArea = maxArea < maxContourArea

                if (approx.total()
                        .toInt() == 4 && isContour && isLessCurrentArea && isLessMaxArea
                ) {
                    maxArea = Imgproc.contourArea(approx)
                    documentContour = approx
                }
            }

            if (Imgproc.contourArea(documentContour) < minContourArea) {
                return null
            }

            return documentContour
        }



            fun adjustingPerspective(
                byteData: ByteArray,
                points: List<Map<String, Any>>,
                result: MethodChannel.Result
            ) {
                try {

                    println(byteData.size);
                    // Decode the input byte array to a Mat object
                    val src = Imgcodecs.imdecode(MatOfByte(*byteData), Imgcodecs.IMREAD_COLOR)

                    println(src);
                    if (src.empty()) {
                        throw Exception("Input image is empty or invalid")
                    }

                    // Create a MatOfPoint2f for the source document contour
                    val documentContour = MatOfPoint2f(
                        Point(points[0]["x"].toString().toDouble(), points[0]["y"].toString().toDouble()),
                        Point(points[1]["x"].toString().toDouble(), points[1]["y"].toString().toDouble()),
                        Point(points[2]["x"].toString().toDouble(), points[2]["y"].toString().toDouble()),
                        Point(points[3]["x"].toString().toDouble(), points[3]["y"].toString().toDouble())
                    )

                    // Perform the perspective warp
                    val imgWithPerspective = warpPerspective(src, documentContour)

                    // Encode the result to a byte array
                    val matOfByte = MatOfByte()
                    Imgcodecs.imencode(".jpg", imgWithPerspective, matOfByte)
                    val byteArray: ByteArray = matOfByte.toArray()

                    // Return the result
                    result.success(byteArray)
                } catch (e: Exception) {
                    // Return an error if something goes wrong
                    result.error("FlutterDocumentScanner-Error", "Android: " + e.message, e)
                }
            }

            private fun warpPerspective(src: Mat, documentContour: MatOfPoint2f): Mat {
                // Define the destination points based on the size of the source image
                val srcSize = Size(src.width().toDouble(), src.height().toDouble())
                val dstContour = MatOfPoint2f(
                    Point(0.0, 0.0),
                    Point(srcSize.width, 0.0),
                    Point(srcSize.width, srcSize.height),
                    Point(0.0, srcSize.height)
                )

                // Get the perspective transformation matrix
                val warpMat = Imgproc.getPerspectiveTransform(documentContour, dstContour)

                // Apply the perspective warp
                val dstWarpedPerspective = Mat()
                Imgproc.warpPerspective(src, dstWarpedPerspective, warpMat, srcSize)

                return dstWarpedPerspective
            }



        fun applyFilter(result: MethodChannel.Result, byteData: ByteArray, filter: Int) {
            try {
                val filterType: FilterType = when (filter) {
                    1 -> FilterType.Natural
                    2 -> FilterType.Gray
                    3 -> FilterType.Eco

                    else -> FilterType.Natural
                }
                val src = Imgcodecs.imdecode(MatOfByte(*byteData), Imgcodecs.IMREAD_COLOR)

                var dstEnd = Mat()

                when (filterType) {
                    FilterType.Natural -> dstEnd = src

                    FilterType.Gray -> Imgproc.cvtColor(src, dstEnd, Imgproc.COLOR_BGR2GRAY)

                    FilterType.Eco -> {
                        val dstColor = Mat()
                        Imgproc.cvtColor(src, dstColor, Imgproc.COLOR_BGR2GRAY)

                        val dstGaussian = Mat()
                        Imgproc.GaussianBlur(dstColor, dstGaussian, Size(3.0, 3.0), 0.0)

                        val dstThreshold = Mat()
                        Imgproc.adaptiveThreshold(
                            dstGaussian,
                            dstThreshold,
                            255.0,
                            Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C,
                            Imgproc.THRESH_BINARY,
                            7,
                            2.0
                        )

                        Imgproc.medianBlur(dstThreshold, dstEnd, 3)
                    }
                }


                // Instantiating an empty MatOfByte class
                val matOfByte = MatOfByte()

                // Converting the Mat object to MatOfByte
                Imgcodecs.imencode(".jpg", dstEnd, matOfByte)
                val byteArray: ByteArray = matOfByte.toArray()

                result.success(byteArray)

            } catch (e: java.lang.Exception) {
                result.error("FlutterDocumentScanner-Error", "Android: " + e.message, e)
            }
        }
    }

    private enum class FilterType {
        Natural,
        Gray,
        Eco,
    }
}

This is my version

[kaushik072]

Yeah, Now It's working @mrasityilmaz Thank you so much for saving my time.

@criistian14 Please update this and release a new version as soon as possible.