Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed jni test and compile warnings #940

Closed
wants to merge 5 commits into from

Conversation

luyuncheng
Copy link
Collaborator

@luyuncheng luyuncheng commented Jun 20, 2023

Description

when i try to compile and test jni with main branch. i see the following error:

release/libopensearchknn_faiss.so: undefined reference to `_gfortran_compare_string'
release/libopensearchknn_faiss.so: undefined reference to `_gfortran_concat_string'
jni/tests/test_util.h:111:48: error: ‘idx_t’ is not a member of ‘faiss::Index’

i think this because faiss update to new branch,
so i pr this for fix the compile error:
faiss::Index::idx_t ==> faiss::idx_t
cmake add GFORTRAN_LIBRARY

ISSUE: #941

@navneet1v
Copy link
Collaborator

@luyuncheng Thanks for fixing the tests. This: GFORTRAN_LIBRARY, I am not sure what it is doing. Can you add some details why this is needed and what it is doing?

@navneet1v
Copy link
Collaborator

@luyuncheng please look at this: https://github.com/opensearch-project/k-NN/blob/main/CONTRIBUTING.md, I see few things missing like Developer Certificate, and add your change in CHANGELOG.md file too.

@codecov
Copy link

codecov bot commented Jun 20, 2023

Codecov Report

Merging #940 (f3d0aa2) into main (a2ee834) will not change coverage.
The diff coverage is n/a.

❗ Current head f3d0aa2 differs from pull request most recent head c353126. Consider uploading reports for the commit c353126 to get more accurate results

@@            Coverage Diff            @@
##               main     #940   +/-   ##
=========================================
  Coverage     85.08%   85.08%           
  Complexity     1112     1112           
=========================================
  Files           152      152           
  Lines          4519     4519           
  Branches        405      405           
=========================================
  Hits           3845     3845           
  Misses          489      489           
  Partials        185      185           

@luyuncheng
Copy link
Collaborator Author

luyuncheng commented Jun 20, 2023

@luyuncheng Thanks for fixing the tests. This: GFORTRAN_LIBRARY, I am not sure what it is doing. Can you add some details why this is needed and what it is doing?

@navneet1v i think this because OpenBLAS using gfortrian with dynamic link, and in different platform, different version of gcc, would cause compiler throw exception without -lgfortran shows as followings:

release/libopensearchknn_faiss.so: undefined reference to `_gfortran_compare_string'
release/libopensearchknn_faiss.so: undefined reference to `_gfortran_concat_string'

@luyuncheng luyuncheng force-pushed the fix_type branch 2 times, most recently from c93c3df to f3d0aa2 Compare June 20, 2023 06:18
@navneet1v
Copy link
Collaborator

@luyuncheng Thanks for fixing the tests. This: GFORTRAN_LIBRARY, I am not sure what it is doing. Can you add some details why this is needed and what it is doing?

@navneet1v i think this because OpenBLAS using gfortrian with dynamic link, and in different platform, different version of gcc, would cause compiler throw exception without -lgfortran shows as followings:

release/libopensearchknn_faiss.so: undefined reference to `_gfortran_compare_string'
release/libopensearchknn_faiss.so: undefined reference to `_gfortran_concat_string'

@luyuncheng is there any performance benefit of adding this? or this is just to resolve the compilation warnings?

Signed-off-by: luyuncheng <[email protected]>
Signed-off-by: luyuncheng <[email protected]>
Signed-off-by: luyuncheng <[email protected]>
Signed-off-by: luyuncheng <[email protected]>
@luyuncheng
Copy link
Collaborator Author

luyuncheng commented Jun 20, 2023

@luyuncheng is there any performance benefit of adding this? or this is just to resolve the compilation warnings?

@navneet1v Resolve compilation errors

it is because we install OpenBLAS with all package with brew install , and without 'make NO_LAPACK=1' ,
like cmake shows:

Found BLAS: /usr/lib/libopenblas.a;/usr/lib/gcc/x86_64-linux-gnu/6/libgomp.so;/usr/lib/x86_64-linux-gnu/libpthread.so;-lpthread

so we need link to lgfortran3. like this OpenBLAS isses 2287

CHANGELOG.md Outdated
@@ -7,6 +7,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
### Features
### Enhancements
### Bug Fixes
* Fixed jni test and compile error ([#940](https://github.com/opensearch-project/k-NN/pull/940))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we put this like Fixed jni test and compile warnings, as if it would have been error the build should have faied.

Copy link
Collaborator Author

@luyuncheng luyuncheng Jun 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++ LGTM

@luyuncheng luyuncheng changed the title Fixed jni test and compile error Fixed jni test and compile warnings Jun 20, 2023
Signed-off-by: luyuncheng <[email protected]>
@navneet1v
Copy link
Collaborator

@jmazanec15 can you take a look?

@@ -132,6 +132,8 @@ if (${CONFIG_FAISS} STREQUAL ON OR ${CONFIG_ALL} STREQUAL ON OR ${CONFIG_TEST} S
find_package(BLAS REQUIRED)
enable_language(Fortran)
find_package(LAPACK REQUIRED)
find_library(GFORTRAN_LIBRARY NAMES libgfortran.so.3)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need the version number?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libgfortran has different version number for different GCC version.
like gcc 4.5.2 maps /usr/lib64/libgfortran.so.1
like gcc 6.2.0 gcc 5.x maps /usr/lib64/libgfortran.so.3
AND there is no soft link from libgfortran.so to libgfortran.so.3.0.0

So how about enum some version of libgfortran in find_library?

@luyuncheng luyuncheng closed this Aug 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants