Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSVC optimization /Ot (included in /O2) breaks tesseract release build #1223

Closed
MarkusGH opened this issue Nov 27, 2017 · 30 comments
Closed

Comments

@MarkusGH
Copy link

MarkusGH commented Nov 27, 2017

Environment

  • Tesseract Version: 4.00.00dev with current training data (tessdata_best).
  • Commit Number: eba0ae3
  • Platform: Windows 64-bit build, Visual Studio 2017 15.4.4

Current Behavior:

Test.zip

  1. MinSizeRel build

Command:

tesseract.exe C:\Test.tif C:\Test -l deu+eng --psm 1 --oem 1 pdf

Error messages:

None

Result:

Correct PDF created, OCR results OK

  1. Release build

Command:

tesseract.exe C:\Test.tif C:\Test -l deu+eng --psm 1 --oem 0 pdf

Error messages:

None

Result:

Correct PDF created, OCR results OK

  1. Release build

Command:

tesseract.exe C:\Test.tif C:\Test -l deu+eng --psm 1 --oem 1 pdf

Error messages:

Multiple times:

Error in pixCreateHeader: height must be > 0
Error in pixCreateNoInit: pixd not made
Error in pixCreate: pixd not made
Error in pixClipRectangle: pixd not made

Result:

Broken PDF created.

Conclusion:

Release build shows erratic behavior when LSTM is used, errormessages and/or wrong output.
Debug And MinSizeRel builds work slowly but correctly.
As soon as the optimization /Ot is switched to /Os (or from /O2 to /O1) or LSTM is not used all problems go away.

Expected Behavior:

Release build behaves like MinSizeRel/Debug built.

Suggested Fix:

Problem is located in Tesseract::GetRectImage

As soon as speed optimization for this function is switched off with #pragma optimize( "t", off ) the problem goes away.

@zdenop
Copy link
Contributor

zdenop commented Nov 27, 2017

How did you build tesseract?
Error messages you posted comes from leptonica, so...

@MarkusGH
Copy link
Author

As I made some progress I updated the issue to better describe what is happening.
I built tesseract exactly like described here:
https://github.com/tesseract-ocr/tesseract/wiki/Compiling#windows
Develop Tesseract -> Building for x64 platform

@zdenop
Copy link
Contributor

zdenop commented Nov 27, 2017

I would suggest you make sure that the all dependencies (at least leptonica) are build as 64bit and with the same version of compiler...

@MarkusGH
Copy link
Author

It is a 64 bit version of leptonica - otherwise it would not work at all.
Why should the compiler used to build leptonica make any difference?
Due to my findings the problem is located in Tesseract::GetRectImage, and goes away as soon as that very function is not optimized for speed anymore.
I would not expect that the way tesseract interfaces with external libraries is changed by the optimization option.

@egorpugin
Copy link
Contributor

Hi,

Not sure if this is your post https://groups.google.com/forum/#!topic/tesseract-ocr/lufafrAAEnk
It points to /Ob2 option while you state /Ot.

Did you try RelWithDebInfo configuration?

@MarkusGH
Copy link
Author

Not my post and it does not look like the problem has anything to do with /Ob2.
RelWithDebugInfo shows the same behaviour as Release as both use /O2.

@MarkusGH
Copy link
Author

Found similar problem in ColPartitionGrid::ComputeTotalOverlap with some input files.
A Release build hits the following assertion:
contains_unichar_id(unichar_id):Error:Assert failed:in file C:\Users\mw\Desktop\OCR\tesseract-master\ccutil\unicharset.h, line 513
As soon as I add #pragma optimize( "t", off ) directly in front of the function ColPartitionGrid::ComputeTotalOverlap the problem goes away.

@stweil
Copy link
Member

stweil commented Nov 28, 2017

@MarkusGH, did you also restore optimization after ColPartitionGrid::ComputeTotalOverlap and Tesseract::GetRectImage, so we can be sure that only those functions used code without optimization in your test?

@MarkusGH
Copy link
Author

Yes - I put #pragma optimize( "", on ) directly after the functions.
It looks like the following have a strong effect on the problem:
Training files used, size of input file (pixel count), engine.
Strange enough I found at least one file which works with --oem 0 and --oem 1 but not with --oem 3.

@egorpugin
Copy link
Contributor

egorpugin commented Nov 28, 2017

To check if this is VS2017 issue, it's necessary to build on VS2015.
As I understand if we haven't any troubles on gcc/clang, it's problem of MSVC, not tesseract.

@MarkusGH
Copy link
Author

I think it is too early to blame MSVC for this issue.
Could be a compiler error or could be sloppy programming which leads to unexpected behaviour when that kind of optimization is active. Again - to me it feels a lot like it has to do with signed integer overflow handling.
Signed integer overflow behaviour is undefined per C++11 standard.
Without optimizations signed integer overflow will simply wrap around (this is simply what the processor does). With optimizations anything can happen...
You actually do not really know, if the issue is not also present in gcc builds, because the effects are not necessarily obvious - could be there are only slight variations in OCR results.

As you use gcc I would kindly suggest to create a build with the gcc option -ftrapv (and maybe other options necessary to bring the debugger to break on integer overflows) and try the test case I provided to see if it actuall produces integer overflows. If so this makes a case to investigate further in that direction.

@stweil
Copy link
Member

stweil commented Nov 29, 2017

A Tesseract executable which was compiled with -ftrapv does not raise an overflow exception. What about debugging the buggy Windows executable and setting a conditional breakpoint on pixCreateHeader (waiting for height <= 0)?

@MarkusGH
Copy link
Author

MarkusGH commented Nov 29, 2017

Oh - so it looks like the integer overflow hypothesis is dead.
About pixCreateHeader: I searched for that already. Unfortunately I did not build leptonica from source and it does not look like this function is called from anywhere outside leptonica.

@stweil
Copy link
Member

stweil commented Nov 29, 2017

I tested the latest binary from AppVeyor. It shows the same problem when running tesseract --oem 1 Test.tif stdout with LSTM.

It looks like pixCreateHeader is called with height == 0.

@stweil
Copy link
Member

stweil commented Dec 2, 2017

@egorpugin, I tried a local build on Windows 7 to debug the problem, but failed: cppan.exe crashed very often with SIGSEGV, and when it finally passed, I got a cmake failure similar to https://github.com/cppan/cppan/issues/36. Is there an alternative way to build Tesseract with VS 2017 community edition?

@egorpugin
Copy link
Contributor

Could you try the latest build?

cppan.zip

@FaisalHussain95
Copy link

FaisalHussain95 commented Dec 2, 2017

@stweil I have the same issue here with Visual 15 2017 Win64 Enterprise & Community editions

Thanks @egorpugin i will try that

Edit: not working... i will try overiding .cppan/helpers.cmake
Related post: cppan/cppan#36

@egorpugin
Copy link
Contributor

@Salfai77210, @stweil
Please, try another one client.

cppan.zip

@FaisalHussain95
Copy link

FaisalHussain95 commented Dec 2, 2017

@egorpugin okay i will try this tkx. Have you read my edited comment on cppan/cppan#36 ?

I have added my fix for this issue as well FIX

@egorpugin
Copy link
Contributor

New client is a possible fix for your issue too. Try it.

@stweil
Copy link
Member

stweil commented Dec 4, 2017

Even with the latest cppan it takes about 20 iterations until cppan has fetched all required packages and passes without failure. It looks like these iterations don't get everything correctly set up, because the following cmake still complains about missing files and fails.

@egorpugin
Copy link
Contributor

Are you trying this on windows?

@stweil
Copy link
Member

stweil commented Dec 4, 2017

Sure. I tried it on Windows 7 / 64 bit from the Visual Studio command line. PATH was fixed to include the CMake binary and cppan.

@egorpugin
Copy link
Contributor

Maybe this one?
Sorry, I can't repro the issue locally, so we have this kinda "game".
Did you see any messages about crash dumps? Check c:\users\u\appdata\local\temp\cppan\dump folder.

cppan.zip

@stweil
Copy link
Member

stweil commented Dec 5, 2017

I got messages about crash dumps (minidumps), but did not find the dump files. I'll have a look at the indicated location later.

@stweil
Copy link
Member

stweil commented Dec 5, 2017

The cppan crash seems to occur while trying to free memory, maybe related to setlocale. Could you please create an executable with debug information (even better: a debug build)? Then I'll get more information about the exact code location.

@egorpugin
Copy link
Contributor

@zdenop
Copy link
Contributor

zdenop commented Sep 12, 2018

@MarkusGH @stweil @FaisalHussain95 Is this issue solved?

@stweil
Copy link
Member

stweil commented Sep 12, 2018

I just repeated the previous test with the latest binary, and it looks like the issue is fixed.

@FaisalHussain95
Copy link

@zdenop Yes it works !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants