Using SetImage(buffer, width, height, ...) result worse than when using SetImage(Pix*)

[tailsu]

Environment

• Tesseract Version: tesseract 4.0.0
  leptonica-1.76.0
  libjpeg 9c : libpng 1.6.35 : libtiff 4.0.9 : zlib 1.2.11
  Found AVX512BW
  Found AVX512F
  Found AVX2
  Found AVX
  Found SSE
• Platform: Darwin 18.2.0 Darwin Kernel Version 18.2.0: Fri Oct 5 19:41:49 PDT 2018; root:xnu-4903.221.2~2/RELEASE_X86_64 x86_64

Current Behavior:

Behavior reproduced with the following image:

API initialized with "eng+deu" languages with language data from the tessdata_fast repository, mode is PSM_AUTO, everything else is default.

Text recognition is very accurate when using the following API:

Pix* pix = pixRead(imageFilePath.c_str());
tesseractApi->SetImage(pix);
tesseractApi->GetHOCRText(0);

In particular, the words "Geschäftsführer", "Gesellschaft", "Registergericht" and "Charlottenburg" are recognized completely and accurately.

I also get the following message in stderr:

Warning: Invalid resolution 0 dpi. Using 70 instead.
Estimating resolution as 287

If I use the OpenCV imgcodecs API to load the image (it shouldn't matter how the image was read), and use the SetImage(buffer) overload:

cv::Mat image = cv::imread(imageFilePath);
tesseractApi->SetImage(
    image.data,
    image.cols,
    image.rows,
    image.channels(),
    (int) image.step1()
);

Then the quality is much worse. All of the above-mentioned words are completely absent from the result.

Expected Behavior:

There should be zero difference in behavior when using either SetImage() overload when given the exact same source data.

Suggested Fix:

I've tracked the problem to this line:

tesseract/src/ccmain/thresholder.cpp

Line 115 in dba7f45

pixSetYRes(pix, 300);

If I add the same line to my original working example:

        Pix* pix = pixRead(imageFilePath.c_str());
        pixSetYRes(pix, 300);
        tesseractApi->SetImage(pix);

then I get the same crappy result. I also don't get the messages in stderr about the resolution getting estimated.

Suggested fix is to remove that line.

[amitdo]

Warning: Invalid resolution 0 dpi. Using 70 instead.
Estimating resolution as 287

These two messages are confusing. 70 or 287?

[tailsu]

These two messages are confusing. 70 or 287?

Well, both messages are printed by Tesseract, so how should I answer that?

[amitdo]

My comment was targeted to other tesseract devs.

[zdenop]

@tailsu : thanks for report.
It seems that using 300 DPI was forgotten when estimation of DPI was implemented. Maybe we should extend SetImage dpi setting... Also message related to DPI should re-written

[zdenop]

There are 2 checks for credible DPI (>kMinCredibleResolution):

tesseract/src/api/baseapi.cpp

Line 2353 in 6d06d39

tprintf("Warning: Invalid resolution %d dpi. Using %d instead.\n",

and (>kMinCredibleResolution and <kMaxCredibleResolution):

tesseract/src/ccmain/osdetect.cpp

Line 169 in 6d06d39

tprintf("Warning. Invalid resolution %d dpi. Using %d instead.\n",

In both cases: if credible resolution is not found, resolution is set to kMinCredibleResolution (70).

Additionally if resolution is equal to kMinCredibleResolution then there (SetupPageSegAndDetectOrientation) is algorithm (line_size * 10) for better estimation of DPI:

tesseract/src/ccmain/pagesegmain.cpp

Line 313 in 6d06d39

int res = IntCastRounded(to_block->line_size * kResolutionEstimationFactor);