Upgrade oracle test version #24505

chenjian2664 · 2024-12-17T08:59:57Z

Description

Additional context and related issues

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

## Section
* Fix some things. ({issue}`issuenumber`)

ebyhr · 2024-12-17T10:47:37Z

Oracle test failed:


System.IO.IOException: No space left on device : '/home/runner/runners/2.321.0/_diag/Worker_20241217-102814-utc.log'
   at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.Diagnostics.TextWriterTraceListener.Flush()
   at GitHub.Runner.Common.HostTraceListener.WriteHeader(String source, TraceEventType eventType, Int32 id)
   at GitHub.Runner.Common.HostTraceListener.TraceEvent(TraceEventCache eventCache, String source, TraceEventType eventType, Int32 id, String message)
   at System.Diagnostics.TraceSource.TraceEvent(TraceEventType eventType, Int32 id, String message)
   at GitHub.Runner.Worker.Worker.RunAsync(String pipeIn, String pipeOut)
   at GitHub.Runner.Worker.Program.MainAsync(IHostContext context, String[] args)
System.IO.IOException: No space left on device : '/home/runner/runners/2.321.0/_diag/Worker_20241217-102814-utc.log'
   at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.Diagnostics.TextWriterTraceListener.Flush()
   at GitHub.Runner.Common.HostTraceListener.WriteHeader(String source, TraceEventType eventType, Int32 id)
   at GitHub.Runner.Common.HostTraceListener.TraceEvent(TraceEventCache eventCache, String source, TraceEventType eventType, Int32 id, String message)
   at System.Diagnostics.TraceSource.TraceEvent(TraceEventType eventType, Int32 id, String message)
   at GitHub.Runner.Common.Tracing.Error(Exception exception)
   at GitHub.Runner.Worker.Program.MainAsync(IHostContext context, String[] args)
Unhandled exception. System.IO.IOException: No space left on device : '/home/runner/runners/2.321.0/_diag/Worker_20241217-102814-utc.log'
   at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.Diagnostics.TextWriterTraceListener.Flush()
   at System.Diagnostics.TraceSource.Flush()
   at GitHub.Runner.Common.Tracing.Dispose(Boolean disposing)
   at GitHub.Runner.Common.Tracing.Dispose()
   at GitHub.Runner.Common.TraceManager.Dispose(Boolean disposing)
   at GitHub.Runner.Common.TraceManager.Dispose()
   at GitHub.Runner.Common.HostContext.Dispose(Boolean disposing)
   at GitHub.Runner.Common.HostContext.Dispose()
   at GitHub.Runner.Worker.Program.Main(String[] args)

plugin/trino-oracle/src/test/java/io/trino/plugin/oracle/TestingOracleServer.java

plugin/trino-oracle/src/test/resources/init.sql

hashhar

LGTM % comment

chenjian2664 · 2024-12-20T02:24:36Z

System.IO.IOException: No space left on device : '/home/runner/runners/2.321.0/_diag/Worker_20241220-020607-utc.log'
   at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
   at System.IO.Strategies.OSFileStreamStrategy.Write(ReadOnlySpan`1 buffer)
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.Diagnostics.TextWriterTraceListener.Flush()
   at GitHub.Runner.Common.HostTraceListener.WriteHeader(String source, TraceEventType eventType, Int32 id)
   at System.Diagnostics.TraceSource.TraceEvent(TraceEventType eventType, Int32 id, String message)
   at GitHub.Runner.Common.Tracing.Error(String message)
   at GitHub.Runner.Worker.JobRunner.ShutdownQueue(Boolean throwOnFailure)
   at GitHub.Runner.Worker.JobRunner.RunAsync(AgentJobRequestMessage message, CancellationToken jobRequestCancellationToken)
   at GitHub.Runner.Worker.Worker.RunAsync(String pipeIn, String pipeOut)
   at GitHub.Runner.Worker.Program.MainAsync(IHostContext context, String[] args)

.

ebyhr · 2024-12-20T02:45:27Z

Please run stress tests by removing unnecessary jobs and duplicating Oracle job in ci.yml.

chenjian2664 · 2024-12-20T07:52:38Z

Failed request: (409) Conflict: an artifact with this name already exists on the workflow run

.github/workflows/ci.yml

ebyhr · 2024-12-24T06:10:19Z

Could you rebase on master to resolve conflicts?

chenjian2664 · 2025-01-15T07:54:24Z

@ebyhr @hashhar , I’d like your suggestions on how to minimize the logs during testing.

To clarify, since I run multiple Oracle tests, only one test is expected to succeed, while the others encounter ...artifact with this name already exists on the workflow run (this steps is encountered on the upload test result phrase). I treat this situation as a success, (and you can see in the result), and I treat No space left as a failure.

Currently, I’m using a Linux pipe to filter logs and only capture ERROR messages. This approach works and can report the correct error message for debug. But, I’m wondering if there’s a more natural or elegant way to handle this.

chenjian2664 · 2025-01-15T08:05:18Z

.github/workflows/ci.yml

+        run: |
+          if [ "${{ matrix.modules }}" == "plugin/trino-oracle" ]; then
+            echo "Running tests for oracle..."
+            $MAVEN test ${MAVEN_TEST} -pl ${{ matrix.modules }} ${{ matrix.profile != '' && format('-P {0}', matrix.profile) || '' }} 2>&1 | grep "ERROR" || true


For reference, Here is the logic to filter only ERROR logs

github-actions · 2025-02-06T17:02:55Z

This pull request has gone a while without any activity. Tagging for triage help: @mosabua

wendigo · 2025-02-06T21:40:37Z

We should get this in @ebyhr @hashhar

mosabua · 2025-02-06T22:24:08Z

If 18 is the highest version we test with .. we have to update the requirements section in the docs to 18 ..

wendigo · 2025-02-07T07:33:19Z

@mosabua agreed

wendigo · 2025-02-07T07:40:26Z

(rebased & solved conflict)

wendigo · 2025-02-07T07:41:07Z

@mosabua:

To connect to Oracle, you need:

Oracle 19 or higher.

mosabua

Good to go from my perspective.. any concerns with merging @wendigo and @ebyhr ?

ebyhr

@chenjian2664 Is this PR ready for merge? How was the stress test result?

ebyhr · 2025-02-07T22:01:04Z

.github/workflows/ci.yml

+          else
+            echo "Running tests for ${{ matrix.modules }}..."
+            $MAVEN test ${MAVEN_TEST} -pl ${{ matrix.modules }} ${{ matrix.profile != '' && format('-P {0}', matrix.profile) || '' }}
+          fi


Can we revert this change?

@ebyhr Let me do more round tests

@ebyhr this change is for reduce the logs of the oracle tests, which is required of this pr.

Could you please see the stress test results? I just triggle tests of oracle, there is one successful and others failed at uploading the test result phrase.

this change is for reduce the logs of the oracle tests, which is required of this pr.

Oh, I thought it's just for stress tests. Is it difficult to reduce logs within Oralce tests instead of ci.yml?

This is a question actually I also want to ask...
cc @wendigo @hashhar do you happen to have any idea how to let the whole test only log of error level

Which loggers are chatty?

I take 2 log in current oracle(version 11) logs, it's suspicious:

2025-02-11T21:35:00.494-0600 DEBUG SplitRunner-20250212_033459_00205_dxdmg.1.1.0-2-4910 oracle.jdbc {0} bytes written to the Socket. parameters=[8192] Its frequency

2025-02-11T21:35:00.499-0600 INFO SplitRunner-20250212_033459_00205_dxdmg.1.1.0-2-4910 oracle.jdbc properties={0}. parameters=[{LOCALE=en_US, DriverVersion=23.3.0.23.09, java.library.path: =/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib, java.class.path: =/home/runner/work/trino/trino/plugin/trino-oracle/target/test-classes:/home/runner/work/trino/trino/plugin/trino-oracle/target/classes:/home/runner/.m2/repository/com/google/guava/guava/33.4.0-jre/guava-33.4.0-jre.jar:/home/runner/.m2/repository/com/google/guava/failureaccess/1.0.2/failureaccess-1.0.2.jar:/home/runner/.m2/repository/com/google/code/findbugs/jsr305/3.0.2/jsr305-3.0.2.jar:/home/runner/.m2/repository/org/checkerframework/checker-qual/3.49.0/checker-qual-3.49.0.jar:/home/runner/.m2/repository/com/google/j2objc/j2objc-annotations/3.0.0/j2objc-annotations-3.0.0.jar:/home/runner/.m2/repository/com/google/inject/guice/7.0.0/guice-7.0.0.jar:/home/runner/.m2/repository/jakarta/inject/jakarta.inject-api/2.0.1/jakarta.inject-api-2.0.1.jar:/home/runner/.m2/repository/aopalliance/aopalliance/... It's not that frequency like first one, but it's body more than 30k characters.

18 image is much larger than 11, thus is there is no enough remain space for logging so much content

@wendigo Just in case you wanna see the failure about No space left, may run multi oracle tests together, like stress test do, single run may not hit the failure sometimes.

chenjian2664 · 2025-02-14T09:33:12Z

I see there still lots of logs like 2025-02-13T07:42:10.607-0600 DEBUG SplitRunner-20250213_134210_00058_2kpwq.2.0.0-1-1021 oracle.jdbc {0} bytes parameters=[8111]
@wendigo Are we ready to fire a stress test now?

cla-bot bot added the cla-signed label Dec 17, 2024

chenjian2664 requested review from ebyhr and hashhar December 17, 2024 10:15

chenjian2664 force-pushed the up_oracle branch from 95c9ddd to 578a6c5 Compare December 17, 2024 10:26

chenjian2664 marked this pull request as draft December 17, 2024 10:48

chenjian2664 force-pushed the up_oracle branch from 578a6c5 to 1a90bd2 Compare December 18, 2024 01:59

chenjian2664 commented Dec 18, 2024

View reviewed changes

plugin/trino-oracle/src/test/java/io/trino/plugin/oracle/TestingOracleServer.java Show resolved Hide resolved

chenjian2664 force-pushed the up_oracle branch 5 times, most recently from d4b697e to 2051b06 Compare December 18, 2024 06:34

chenjian2664 marked this pull request as ready for review December 18, 2024 06:57

hashhar reviewed Dec 19, 2024

View reviewed changes

plugin/trino-oracle/src/test/resources/init.sql Outdated Show resolved Hide resolved

hashhar approved these changes Dec 19, 2024

View reviewed changes

ebyhr previously approved these changes Dec 20, 2024

View reviewed changes

chenjian2664 force-pushed the up_oracle branch 2 times, most recently from 4cdd6b5 to db92378 Compare December 20, 2024 02:04

chenjian2664 force-pushed the up_oracle branch from db92378 to 3e91b15 Compare December 20, 2024 02:43

chenjian2664 force-pushed the up_oracle branch from 1088b34 to fbfc083 Compare December 20, 2024 06:15

chenjian2664 force-pushed the up_oracle branch from 8d0d4ec to 65a9f1c Compare December 20, 2024 09:20

chenjian2664 commented Dec 24, 2024

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

chenjian2664 force-pushed the up_oracle branch from 053d648 to 1a4cd8a Compare December 24, 2024 06:09

chenjian2664 force-pushed the up_oracle branch from 1a4cd8a to 3dab010 Compare December 24, 2024 06:19

chenjian2664 force-pushed the up_oracle branch 2 times, most recently from 7543512 to 2d3f891 Compare January 15, 2025 06:39

chenjian2664 commented Jan 15, 2025

View reviewed changes

chenjian2664 force-pushed the up_oracle branch from 72482f2 to fa40ac8 Compare January 15, 2025 08:25

github-actions bot removed the stale label Jan 15, 2025

github-actions bot added the stale label Feb 6, 2025

wendigo added stale-ignore Use this label on PRs that should be ignored by the stale bot so they are not flagged or closed. and removed stale labels Feb 6, 2025

wendigo force-pushed the up_oracle branch from fa40ac8 to de585cb Compare February 7, 2025 07:40

github-actions bot added the oracle Oracle connector label Feb 7, 2025

mosabua approved these changes Feb 7, 2025

View reviewed changes

wendigo approved these changes Feb 7, 2025

View reviewed changes

ebyhr reviewed Feb 7, 2025

View reviewed changes

chenjian2664 mentioned this pull request Feb 12, 2025

Support MERGE for oracle connector #24482

Draft

chenjian2664 force-pushed the up_oracle branch 3 times, most recently from ad9681d to cf16f4e Compare February 13, 2025 08:00

wendigo force-pushed the up_oracle branch 2 times, most recently from f3076ea to 55a8dce Compare February 13, 2025 11:30

Update oracle test container version to 18.0.4-slim

ace940d

wendigo force-pushed the up_oracle branch from 5c1d228 to ace940d Compare February 13, 2025 13:33

ebyhr removed the stale-ignore Use this label on PRs that should be ignored by the stale bot so they are not flagged or closed. label Feb 26, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade oracle test version #24505

Upgrade oracle test version #24505

chenjian2664 commented Dec 17, 2024

ebyhr commented Dec 17, 2024

hashhar left a comment

chenjian2664 commented Dec 20, 2024

ebyhr commented Dec 20, 2024

chenjian2664 commented Dec 20, 2024

ebyhr commented Dec 24, 2024

chenjian2664 commented Jan 15, 2025 •

edited

Loading

chenjian2664 Jan 15, 2025

github-actions bot commented Feb 6, 2025

wendigo commented Feb 6, 2025

mosabua commented Feb 6, 2025

wendigo commented Feb 7, 2025

wendigo commented Feb 7, 2025

wendigo commented Feb 7, 2025

mosabua left a comment

ebyhr left a comment •

edited

Loading

ebyhr Feb 7, 2025

chenjian2664 Feb 12, 2025

chenjian2664 Feb 12, 2025

ebyhr Feb 12, 2025

chenjian2664 Feb 13, 2025

wendigo Feb 13, 2025

chenjian2664 Feb 13, 2025

chenjian2664 Feb 13, 2025 •

edited

Loading

chenjian2664 commented Feb 14, 2025

Upgrade oracle test version #24505

Are you sure you want to change the base?

Upgrade oracle test version #24505

Conversation

chenjian2664 commented Dec 17, 2024

Description

Additional context and related issues

Release notes

ebyhr commented Dec 17, 2024

hashhar left a comment

Choose a reason for hiding this comment

chenjian2664 commented Dec 20, 2024

ebyhr commented Dec 20, 2024

chenjian2664 commented Dec 20, 2024

ebyhr commented Dec 24, 2024

chenjian2664 commented Jan 15, 2025 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Feb 6, 2025

wendigo commented Feb 6, 2025

mosabua commented Feb 6, 2025

wendigo commented Feb 7, 2025

wendigo commented Feb 7, 2025

wendigo commented Feb 7, 2025

mosabua left a comment

Choose a reason for hiding this comment

ebyhr left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chenjian2664 Feb 13, 2025 • edited Loading

Choose a reason for hiding this comment

chenjian2664 commented Feb 14, 2025

chenjian2664 commented Jan 15, 2025 •

edited

Loading

ebyhr left a comment •

edited

Loading

chenjian2664 Feb 13, 2025 •

edited

Loading