Undefined references with gomobile and abi-armeabi-v7a

[ogero]

Hi, i'm trying to use gocv with gomobile and Android, but in the last step (gomobile binding), I get some undefined references on cv::DescriptorMatcher::knnMatch and others..
Is there some configuration on CMake step I could be missing?
Please read below for explanation of all the steps I followed up to the linker errors.
Thanks!

Steps followed to build from sources

I followed gocv build from sources instructions with some tweaks to obtain abi-armeabi-v7a shared libs.
The CMake configuration used is here.
After compilation I obtain all the needed .so files with the correct abi.

Hello world gomobile project

The gomobile project is nothing fancy, just a hello world:

package mobiletest

import (
    "log"
    "gocv.io/x/gocv"
)

func Version(){
   //This line should output on Android Logcat like "GoLog: opencv lib version: 3.4.1"
   log.Printf("opencv lib version: %s\n", gocv.OpenCVVersion())
}

Android java bindings with gomobile

To generate mobiletest package Java bindings for Android, I follow gomobile steps and call gomobile like this:

@echo OFF
set "HERE=%~dp0"
set "HERE=%HERE:\=/%"

set GO_PACKAGE="mobiletest"
set AAR_OUTPUT_PATH="../android/app/libs"
set AAR_OUTPUT_FILENAME="mobiletest.aar"
set TARGET=android/arm
set CGO_CXXFLAGS=--std=c++11
set CGO_CPPFLAGS=-I%HERE%../opencv/include
set CGO_LDFLAGS=-L%HERE%../opencv/libs/armeabi-v7a  -lopencv_core -lopencv_face -lopencv_videoio -lopencv_imgproc -lopencv_highgui -lopencv_imgcodecs -lopencv_objdetect -lopencv_features2d -lopencv_video -lopencv_dnn -lopencv_xfeatures2d -lopencv_plot -lopencv_tracking -lopencv_img_hash

gomobile.exe bind -target %TARGET% -tags customenv -v -o "%AAR_OUTPUT_PATH%/%AAR_OUTPUT_FILENAME%" %GO_PACKAGE%

Linker errors

But I get some undefined references from the linker:

runtime/cgo
golang.org/x/mobile/internal/mobileinit
gocv.io/x/gocv
golang.org/x/mobile/bind/java
# gocv.io/x/gocv
C:\Users\Gero\AppData\Local\Temp\go-build199540534\b032\_x014.o:features2d.cpp:function BFMatcher_KnnMatch: error: undefined reference to 'cv::DescriptorMatcher::knnMatch(cv::_InputArray const&, cv::_InputArray const&, std::__ndk1::vector<std::__ndk1::vector<cv::DMatch, std::__ndk1::allocator<cv::DMatch> >, std::__ndk1::allocator<std::__ndk1::vector<cv::DMatch, std::__ndk1::allocator<cv::DMatch> > > >&, int, cv::_InputArray const&, bool) const'
C:\Users\Gero\AppData\Local\Temp\go-build199540534\b032\_x014.o:features2d.cpp:function BFMatcher_KnnMatchWithParams(cv::Ptr<cv::BFMatcher>*, cv::Mat*, cv::Mat*, int, cv::Mat*, bool): error: undefined reference to 'cv::DescriptorMatcher::knnMatch(cv::_InputArray const&, cv::_InputArray const&, std::__ndk1::vector<std::__ndk1::vector<cv::DMatch, std::__ndk1::allocator<cv::DMatch> >, std::__ndk1::allocator<std::__ndk1::vector<cv::DMatch, std::__ndk1::allocator<cv::DMatch> > > >&, int, cv::_InputArray const&, bool) const'
C:\Users\Gero\AppData\Local\Temp\go-build199540534\b032\_x015.o:highgui.cpp:function Window_SelectROIs: error: undefined reference to 'cv::selectROIs(cv::String const&, cv::_InputArray const&, std::__ndk1::vector<cv::Rect_<int>, std::__ndk1::allocator<cv::Rect_<int> > >&, bool, bool)'
C:\Users\Gero\AppData\Local\Temp\go-build199540534\b032\_x016.o:imgcodecs.cpp:function Image_IMWrite: error: undefined reference to 'cv::imwrite(cv::String const&, cv::_InputArray const&, std::__ndk1::vector<int, std::__ndk1::allocator<int> > const&)'
C:\Users\Gero\AppData\Local\Temp\go-build199540534\b032\_x016.o:imgcodecs.cpp:function Image_IMWrite_WithParams: error: undefined reference to 'cv::imwrite(cv::String const&, cv::_InputArray const&, std::__ndk1::vector<int, std::__ndk1::allocator<int> > const&)'
C:\Users\Gero\AppData\Local\Temp\go-build199540534\b032\_x016.o:imgcodecs.cpp:function Image_IMEncode: error: undefined reference to 'cv::imencode(cv::String const&, cv::_InputArray const&, std::__ndk1::vector<unsigned char, std::__ndk1::allocator<unsigned char> >&, std::__ndk1::vector<int, std::__ndk1::allocator<int> > const&)'
C:\Users\Gero\AppData\Local\Temp\go-build199540534\b032\_x018.o:objdetect.cpp:function CascadeClassifier_DetectMultiScale: error: undefined reference to 'cv::CascadeClassifier::detectMultiScale(cv::_InputArray const&, std::__ndk1::vector<cv::Rect_<int>, std::__ndk1::allocator<cv::Rect_<int> > >&, double, int, int, cv::Size_<int>, cv::Size_<int>)'
C:\Users\Gero\AppData\Local\Temp\go-build199540534\b032\_x018.o:objdetect.cpp:function CascadeClassifier_DetectMultiScaleWithParams: error: undefined reference to 'cv::CascadeClassifier::detectMultiScale(cv::_InputArray const&, std::__ndk1::vector<cv::Rect_<int>, std::__ndk1::allocator<cv::Rect_<int> > >&, double, int, int, cv::Size_<int>, cv::Size_<int>)'
C:\Users\Gero\AppData\Local\Temp\go-build199540534\b032\_x018.o:objdetect.cpp:function GroupRectangles: error: undefined reference to 'cv::groupRectangles(std::__ndk1::vector<cv::Rect_<int>, std::__ndk1::allocator<cv::Rect_<int> > >&, int, double)'
clang60++.exe: error: linker command failed with exit code 1 (use -v to see invocation)
gomobile.exe: go build -tags customenv -v -buildmode=c-shared -o=C:\Users\Gero\AppData\Local\Temp\gomobile-work-903591099\android\src\main\jniLibs\armeabi-v7a\libgojni.so gobind failed: exit status 2

Press any key to continue . . .

My Environment

• Operating System and version: Microsoft Windows 10 Enterprise 2016 LTSB (Version 10.0.14393 Build 14393)
• OpenCV version used: opencv-3.4.1 / opencv_contrib-3.4.1
• GoCV version used: 0.13.1
• Go version: go version go1.10.3 windows/amd64

CMake output

General configuration for OpenCV 3.4.1 =====================================
  Version control:               unknown

  Extra modules:
    Location (extra):            D:/SDK/opencv/opencv_contrib-3.4.1/modules
    Version control (extra):     unknown

  Platform:
    Timestamp:                   2018-06-16T05:01:37Z
    Host:                        Windows 10.0.14393 AMD64
    Target:                      Android 1 armv7-a
    CMake:                       3.11.3
    CMake generator:             MinGW Makefiles
    CMake build tool:            D:/SDK/mingw-w64/x86_64-7.3.0-posix-seh-rt_v5-rev0/mingw64/bin/mingw32-make.exe
    Configuration:               Release

  CPU/HW features:
    Baseline:
      requested:                 DETECT
      disabled:                  VFPV3 NEON

  C/C++:
    Built as dynamic libs?:      YES
    C++11:                       YES
    C++ Compiler:                D:/SDK/Android/sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/windows-x86_64/bin/arm-linux-androideabi-g++.exe  (ver 4.9.0)
    C++ flags (Release):         -fexceptions -frtti -Wno-psabi --sysroot=D:/SDK/Android/sdk/ndk-bundle/platforms/android-16/arch-arm -funwind-tables -finline-limit=64 -fsigned-char -no-canonical-prefixes -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -fdata-sections -ffunction-sections -Wa,--noexecstack  -DANDROID -D__ANDROID_API__=16   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -fdiagnostics-show-option -fomit-frame-pointer -ffunction-sections -fdata-sections  -mfp16-format=ieee -fvisibility=hidden -fvisibility-inlines-hidden -mthumb -fomit-frame-pointer -fno-strict-aliasing -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fexceptions -frtti -Wno-psabi --sysroot=D:/SDK/Android/sdk/ndk-bundle/platforms/android-16/arch-arm -funwind-tables -finline-limit=64 -fsigned-char -no-canonical-prefixes -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -fdata-sections -ffunction-sections -Wa,--noexecstack  -DANDROID -D__ANDROID_API__=16   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -fdiagnostics-show-option -fomit-frame-pointer -ffunction-sections -fdata-sections  -mfp16-format=ieee -fvisibility=hidden -fvisibility-inlines-hidden -marm -fno-omit-frame-pointer -fno-strict-aliasing -O0 -g -DDEBUG -D_DEBUG  -DDEBUG -D_DEBUG
    C Compiler:                  D:/SDK/Android/sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/windows-x86_64/bin/arm-linux-androideabi-gcc.exe
    C flags (Release):           -fexceptions -Wno-psabi --sysroot=D:/SDK/Android/sdk/ndk-bundle/platforms/android-16/arch-arm -funwind-tables -finline-limit=64 -fsigned-char -no-canonical-prefixes -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -fdata-sections -ffunction-sections -Wa,--noexecstack  -DANDROID -D__ANDROID_API__=16   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-narrowing -Wno-comment -fdiagnostics-show-option -fomit-frame-pointer -ffunction-sections -fdata-sections  -mfp16-format=ieee -fvisibility=hidden -mthumb -fomit-frame-pointer -fno-strict-aliasing -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fexceptions -Wno-psabi --sysroot=D:/SDK/Android/sdk/ndk-bundle/platforms/android-16/arch-arm -funwind-tables -finline-limit=64 -fsigned-char -no-canonical-prefixes -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -fdata-sections -ffunction-sections -Wa,--noexecstack  -DANDROID -D__ANDROID_API__=16   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-narrowing -Wno-comment -fdiagnostics-show-option -fomit-frame-pointer -ffunction-sections -fdata-sections  -mfp16-format=ieee -fvisibility=hidden -marm -fno-omit-frame-pointer -fno-strict-aliasing -O0 -g -DDEBUG -D_DEBUG  -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,--fix-cortex-a8 -Wl,--no-undefined -Wl,-allow-shlib-undefined -Wl,--gc-sections -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now    
    Linker flags (Debug):        -Wl,--fix-cortex-a8 -Wl,--no-undefined -Wl,-allow-shlib-undefined -Wl,--gc-sections -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now    
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          dl m log
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 aruco bgsegm bioinspired calib3d ccalib core datasets dnn dnn_objdetect dpm face features2d flann fuzzy hfs highgui img_hash imgcodecs imgproc java_bindings_generator line_descriptor ml objdetect optflow phase_unwrapping photo plot python_bindings_generator reg rgbd shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    js saliency world
    Disabled by dependency:      -
    Unavailable:                 cnn_3dobj cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev cvv dnn_modern freetype hdf java matlab ovis python2 python3 sfm viz
    Applications:                -
    Documentation:               NO
    Non-free algorithms:         NO

  Android: 
    Android ABI:                 armeabi-v7a
    STL type:                    gnustl_static
    Native API level:            android-16
    SDK target:                  android_sdk_target_status-NOTFOUND
    Android NDK:                 D:/SDK/Android/sdk/ndk-bundle (toolchain: arm-linux-androideabi-4.9)
    android tool:                NO

  GUI: 

  Media I/O: 
    ZLib:                        build (ver 1.2.11)
    JPEG:                        build (ver 90)
    WEBP:                        build (ver encoder: 0x020e)
    PNG:                         build (ver 1.6.34)
    TIFF:                        build (ver 42 - 4.0.9)
    JPEG 2000:                   build (ver 1.900.1)
    OpenEXR:                     build (ver 1.7.1)

  Video I/O:

  Parallel framework:            pthreads

  Trace:                         YES (built-in)

  Other third-party libraries:
    Custom HAL:                  NO
    Protobuf:                    build (3.5.1)

  NVIDIA CUDA:                   NO

  Python (for build):            C:/Python27/python.exe

  Java:                          
    ant:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    D:/SDK/opencv/build_android_arm/install
-----------------------------------------------------------------

[deadprogram]

Hello @ogero I have not done anything at all with gomobile, but this looks very interesting.

From the error message, it makes me wonder if the opencv/build_android_arm/install/sdk/native/libs/armeabi-v7a dir contains all of the .so files you expect. If the build did not build those libs for some reason, for example? What is the contents of that dir?

[ogero]

Hi @deadprogram, unfortunately I described my environment wrong, I've updated the compiler used -hope you didn't read that part too thoroughly-, and added a large chunk of info displayed by CMake before generating the makefile.

About the libs dir, yes, it has all the needed files and more (44 files to be exact): libopencv_aruco.so libopencv_bgsegm.so libopencv_bioinspired.so libopencv_calib3d.so libopencv_ccalib.so libopencv_core.so libopencv_datasets.so libopencv_dnn.so libopencv_dnn_objdetect.so libopencv_dpm.so libopencv_face.so libopencv_features2d.so libopencv_flann.so libopencv_fuzzy.so libopencv_hfs.so libopencv_highgui.so libopencv_imgcodecs.so libopencv_imgproc.so libopencv_img_hash.so libopencv_line_descriptor.so libopencv_ml.so libopencv_objdetect.so libopencv_optflow.so libopencv_phase_unwrapping.so libopencv_photo.so libopencv_plot.so libopencv_reg.so libopencv_rgbd.so libopencv_saliency.so libopencv_shape.so libopencv_stereo.so libopencv_stitching.so libopencv_structured_light.so libopencv_superres.so libopencv_surface_matching.so libopencv_text.so libopencv_tracking.so libopencv_video.so libopencv_videoio.so libopencv_videostab.so libopencv_xfeatures2d.so libopencv_ximgproc.so libopencv_xobjdetect.so libopencv_xphoto.so.
Of course, I copied this files to the path I issue on CGO_LDFLAGS=-L.
And between the copied files, there are the ones used on CGO_LDFLAGS.

Coincidentally all undefined referenced functions have as parameters some std::__ndk1 types.
So I kept digging this issue and used nm command to check the symbols inside libopencv_features2d.so.
I was suposed to find between all symbols, this one:
cv::DescriptorMatcher::knnMatch(cv::_InputArray const&, cv::_InputArray const&, std::__ndk1::vector<std::__ndk1::vector<cv::DMatch, std::__ndk1::allocator<cv::DMatch> >, std::__ndk1::allocator<std::__ndk1::vector<cv::DMatch, std::__ndk1::allocator<cv::DMatch> > > >&, int, cv::_InputArray const&, bool) const.
As this symbol is one of the undefined references.

But instead, I found:
cv::DescriptorMatcher::knnMatch(cv::_InputArray const&, cv::_InputArray const&, std::vector<std::vector<cv::DMatch, std::allocator<cv::DMatch> >, std::allocator<std::vector<cv::DMatch, std::allocator<cv::DMatch> > > >&, int, cv::_InputArray const&, bool) const

I have very little knowledge about STL, compilers and linkers, but it seems that somehow, the gomobile bind step, is mixing up the std libs.
As inside gocv features2d.cpp I clearly read std::vector references.. but nope, the compiler switches that references into std::__ndk1::vector references, and that results in undefined references when linking.

[deadprogram]

Looks like that might be related this this https://stackoverflow.com/questions/37370259/undefined-reference-to-cvcascadeclassifierdetectmultiscale-but-other-lib

[ogero]

Yes, it seems to be related to STL. Thanks for that link, I don't know how I haven't seen it before!
I have tried to change ANDROID_STL to other values like gnustl_shared instead of the default one gnustl_static, but after doing a configure on CMake-gui, the var returns to the default value.

The cmake file opencv/opencv-3.4.1/platforms/android/android.toolchain.cmake specifies:

#    ANDROID_STL=gnustl_static - specify the runtime to use.
#
#      Possible values are:
#        none           -> Do not configure the runtime.
#        system         -> Use the default minimal system C++ runtime library.
#                          Implies -fno-rtti -fno-exceptions.
#                          Is not available for standalone toolchain.
#        system_re      -> Use the default minimal system C++ runtime library.
#                          Implies -frtti -fexceptions.
#                          Is not available for standalone toolchain.
#        gabi++_static  -> Use the GAbi++ runtime as a static library.
#                          Implies -frtti -fno-exceptions.
#                          Available for NDK r7 and newer.
#                          Is not available for standalone toolchain.
#        gabi++_shared  -> Use the GAbi++ runtime as a shared library.
#                          Implies -frtti -fno-exceptions.
#                          Available for NDK r7 and newer.
#                          Is not available for standalone toolchain.
#        stlport_static -> Use the STLport runtime as a static library.
#                          Implies -fno-rtti -fno-exceptions for NDK before r7.
#                          Implies -frtti -fno-exceptions for NDK r7 and newer.
#                          Is not available for standalone toolchain.
#        stlport_shared -> Use the STLport runtime as a shared library.
#                          Implies -fno-rtti -fno-exceptions for NDK before r7.
#                          Implies -frtti -fno-exceptions for NDK r7 and newer.
#                          Is not available for standalone toolchain.
#        gnustl_static  -> Use the GNU STL as a static library.
#                          Implies -frtti -fexceptions.
#        gnustl_shared  -> Use the GNU STL as a shared library.
#                          Implies -frtti -fno-exceptions.
#                          Available for NDK r7b and newer.
#                          Silently degrades to gnustl_static if not available.
#        c++_static     -> Use the LLVM libc++ runtime as a static library.
#                          Implies -frtti -fexceptions.
#        c++_shared     -> Use the LLVM libc++ runtime as a static library.
#                          Implies -frtti -fno-exceptions.

So it seems -because of reasons- (Is not available for standalone toolchain./Silently degrades to gnustl_static if not available.) i'm not able to use other than gnustl_static...
This seems very odd, as the libs built with my configuration are shared libs.

I'll give it another try, overriding the cmake var ANDROID_STL=gnustl_shared inside CMakeVars.txt and telling CMake-gui to generate the makefiles.
In hope it's some bug in opencv/opencv-3.4.1/platforms/android/android.toolchain.cmake.

[deadprogram]

Just wondering if this is also related? opencv/opencv#8742

It would probably be worthwhile for you to look thru the various issues on the OpenCV repo itself @ogero to see if any clues there.

[ogero]

Last attempt: ANDROID_STL override in CMakeVars.txt, then configure, compiled successfully, but undefined references still there on linking with gomobile.

My next compile of opencv shared libs, will be using android ndk toolchain ${ANDROID_NDK}/build/cmake/android.toolchain.cmake, as suggested by @alalek in opencv/opencv#8742 (comment).
Up to now, I was using the one provided by opencv opencv/opencv-3.4.1/platforms/android/android.toolchain.cmake.
I can read several differences between the two toolchains.

Thanks again for that discovery @deadprogram .

[ogero]

Ok, hold my chair!
Using the android ndk toolchain actually fixed the linker errors!
I had to make a minor tweak on opencv makefiles, so I updated the gist with that information.

Gomobile now correctly linked the opencv shared libs and generated an AAR for android
After placing the tons of .so files on jniFolder, and also libc++_shared.so, I can happily say that this seems to work!

I/GoLog: opencv lib version: 3.4.1

Now its time for me to try out some opencv apis, and try to generate a fat single shared lib while leaving out unused modules.

@deadprogram thank you so much for all the help provided.