Add integration JDBC tests for cursor/fetch_size feature

[Yury-Fridlyand]

• Add integration JDBC tests for cursor/fetch_size feature.

Signed-off-by: Yury-Fridlyand [email protected]

See team review and discussion on Bit-Quill#208

Description

Add new IT suite which uses JDBC to issue queries with and without fetch_size set.
fetch_size parameter invokes pagination feature (cursor) which is currently supported by legacy engine only.

New :integ-test tasks:

integJdbcTest

Run ITs from **/jdbc/*.java. Existing tests in CursorIT create and use DB connection using JDBC driver from maven artifact.

Optional parameters:

• -Dtest.debug
  Enabled test debug.
• --tests '<test_filter>
  Filter test using wildcard, could be specified multiple times.
• -DjdbcDriverVersion=<version>
  Maven artifact version for JDBC driver to use; optional, defaulted to 1.2.0.0.
• -DjdbcFile=<file>
  Relative or absolute path to JDBC driver compiled binary (*.jar); if given, test platform uses it.
• -DjdbcRepo=<repo_path>
  Relative or absolute path to JDBC driver Maven repo to get the driver from. Note: it is highly recommended to have non-default version of the driver in this repo to avoid conflicts.

Test cluster log located in integ-test/build/testclusters/integJdbcTest-0/logs/integJdbcTest.log
Test report saved in integ-test/build/reports/tests/integJdbcTest

Examples:

./gradlew :integ-test:integJdbcTest

# go to SQL JDBC repo
cd sql-jdbc
# build the driver
./gradlew build
# go to SQL plugin repo
cd ../sql
# run tests
./gradlew :integ-test:integJdbcTest '-DjdbcFile=../sql-jdbc/build/libs/opensearch-sql-jdbc-2.0.0.0.jar'

# go to SQL JDBC repo
cd sql-jdbc
# build the driver and publish to local repo
./gradlew publishShadowPublicationToLocalRepoRepository
# go to SQL plugin repo
cd ../sql
# run tests
./gradlew :integ-test:integJdbcTest '-DjdbcRepo=../sql-jdbc/build/repository/' -DjdbcDriverVersion=2.0.0.0

# run tests with non-default (e.g. non-latest) released JDBC version
./gradlew :integ-test:integJdbcTest -DjdbcDriverVersion=1.0.0.0

Output sample:

> Task :integ-test:integJdbcTest
org.opensearch.sql.jdbc.CursorIT.select_count_all_no_cursor(): SUCCESS 19.634s

CursorIT > select count all no cursor PASSED
org.opensearch.sql.jdbc.CursorIT.select_all_big_table_small_cursor(): SUCCESS 8.738s

CursorIT > select all big table small cursor PASSED
org.opensearch.sql.jdbc.CursorIT.select_all_small_table_small_cursor(): SUCCESS 0.238s

CursorIT > select all small table small cursor PASSED
org.opensearch.sql.jdbc.CursorIT.select_all_no_cursor(): SUCCESS 4.163s

CursorIT > select all no cursor PASSED
org.opensearch.sql.jdbc.CursorIT.select_all_small_table_big_cursor(): SUCCESS 0.132s

CursorIT > select all small table big cursor PASSED
org.opensearch.sql.jdbc.CursorIT.select_all_big_table_big_cursor(): SUCCESS 3.398s

CursorIT > select all big table big cursor PASSED
org.opensearch.sql.jdbc.CursorIT.dev_select_all_small_table_small_cursor(): SUCCESS 20.008s

CursorIT > dev_select_all_small_table_small_cursor() PASSED

Notes

Test filter in gradle
https://github.com/Bit-Quill/opensearch-project-sql/blob/24de6ff9eae35a824a664a865ad349788b744288/integ-test/build.gradle#L204-L206
requires
https://github.com/Bit-Quill/opensearch-project-sql/blob/24de6ff9eae35a824a664a865ad349788b744288/integ-test/build.gradle#L177
and doesn't work with JUnit 4. All IT framework uses Junit 4.
So, init() method of test classed isn't being called from superclasses automatically.
@BeforeAll/@BeforeClass require a static method, so I can't call initClient() there. I have to move initClient() to @BeforeEach with initialized flag.

New gradle tasks are not included into CI workflow.

Issues Resolved

N/A

Check List

• New functionality includes testing.
  • All tests pass, including unit test, integration test and doctest
• New functionality has been documented.
  • New functionality has javadoc added
  • New functionality has user manual doc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[MaxKsyunz]

Being able to set JDBC driver version is sufficient to run these tests against a development version of the driver.

I'd keep jdbc integration test suite and remove jdbcDev, cloneJdbcDriverRepo, and integDevJdbcTest .

[MaxKsyunz]

Looks like the cloneJdbcDriverRepo task is breaking both SQL Java CI and CodeQL-Scan.

[codecov-commenter]

Codecov Report

Merging #1315 (891b18f) into main (ef38389) will not change coverage.
The diff coverage is n/a.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff            @@
##               main    #1315   +/-   ##
=========================================
  Coverage     98.38%   98.38%           
  Complexity     3698     3698           
=========================================
  Files           343      343           
  Lines          9121     9121           
  Branches        586      586           
=========================================
  Hits           8974     8974           
  Misses          142      142           
  Partials          5        5           

Flag	Coverage Δ
sql-engine	98.38% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[Yury-Fridlyand]

Rebased + added a fix for ITs. Shouldn't affect BWC.

[dai-chen]

Thanks for adding this test!