Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky TestIgniteConnectorTest tests: Schema change operation failed: Thread got interrupted while trying to acquire table lock. #16671

Open
findepi opened this issue Mar 22, 2023 · 7 comments · Fixed by #17102
Open

Flaky TestIgniteConnectorTest tests: Schema change operation failed: Thread got interrupted while trying to acquire table lock. #16671

findepi opened this issue Mar 22, 2023 · 7 comments · Fixed by #17102
Assignees
@chenjian2664
Labels
bug Something isn't working

Comments

@findepi
Copy link
Member

findepi commented Mar 22, 2023

https://github.com/trinodb/trino/actions/runs/4488966780/jobs/7895758471?pr=16644

Error:  io.trino.plugin.ignite.TestIgniteConnectorTest.testDropColumn  Time elapsed: 0.067 s  <<< FAILURE!
io.trino.testing.QueryFailedException: Schema change operation failed: Thread got interrupted while trying to acquire table lock.
	at io.trino.testing.AbstractTestingTrinoClient.execute(AbstractTestingTrinoClient.java:122)
	at io.trino.testing.DistributedQueryRunner.executeWithQueryId(DistributedQueryRunner.java:494)
	at io.trino.testing.QueryAssertions.assertDistributedUpdate(QueryAssertions.java:107)
	at io.trino.testing.QueryAssertions.assertUpdate(QueryAssertions.java:63)
	at io.trino.testing.AbstractTestQueryFramework.assertUpdate(AbstractTestQueryFramework.java:401)
	at io.trino.testing.AbstractTestQueryFramework.assertUpdate(AbstractTestQueryFramework.java:396)
	at io.trino.testing.BaseConnectorTest.testDropColumn(BaseConnectorTest.java:2446)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
	Suppressed: java.lang.Exception: SQL: ALTER TABLE test_drop_column_gvj2hlglfk DROP COLUMN IF EXISTS y
		at io.trino.testing.DistributedQueryRunner.executeWithQueryId(DistributedQueryRunner.java:498)
		... 18 more
Caused by: io.trino.spi.TrinoException: Schema change operation failed: Thread got interrupted while trying to acquire table lock.
	at io.trino.plugin.jdbc.BaseJdbcClient.dropColumn(BaseJdbcClient.java:965)
	at io.trino.plugin.jdbc.ForwardingJdbcClient.dropColumn(ForwardingJdbcClient.java:314)
	at io.trino.plugin.jdbc.jmx.StatisticsAwareJdbcClient.lambda$dropColumn$21(StatisticsAwareJdbcClient.java:235)
	at io.trino.plugin.jdbc.jmx.JdbcApiStats.wrap(JdbcApiStats.java:47)
	at io.trino.plugin.jdbc.jmx.StatisticsAwareJdbcClient.dropColumn(StatisticsAwareJdbcClient.java:235)
	at io.trino.plugin.jdbc.CachingJdbcClient.dropColumn(CachingJdbcClient.java:463)
	at io.trino.plugin.jdbc.CachingJdbcClient.dropColumn(CachingJdbcClient.java:463)
	at io.trino.plugin.jdbc.DefaultJdbcMetadata.dropColumn(DefaultJdbcMetadata.java:884)
	at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.dropColumn(ClassLoaderSafeConnectorMetadata.java:416)
	at io.trino.metadata.MetadataManager.dropColumn(MetadataManager.java:758)
	at io.trino.execution.DropColumnTask.execute(DropColumnTask.java:110)
	at io.trino.execution.DropColumnTask.execute(DropColumnTask.java:47)
	at io.trino.execution.DataDefinitionExecution.start(DataDefinitionExecution.java:145)
	at io.trino.execution.SqlQueryManager.createQuery(SqlQueryManager.java:249)
	at io.trino.dispatcher.LocalDispatchQuery.lambda$startExecution$7(LocalDispatchQuery.java:143)
	at io.trino.$gen.Trino_testversion____20230322_133500_71.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.sql.SQLException: Schema change operation failed: Thread got interrupted while trying to acquire table lock.
	at org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:1010)
	at org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute0(JdbcThinStatement.java:234)
	at org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute(JdbcThinStatement.java:560)
	at io.trino.plugin.jdbc.BaseJdbcClient.execute(BaseJdbcClient.java:1163)
	at io.trino.plugin.jdbc.BaseJdbcClient.dropColumn(BaseJdbcClient.java:962)
	... 18 more
	Suppressed: java.lang.RuntimeException: Query: ALTER TABLE `PUBLIC`.`TEST_DROP_COLUMN_GVJ2HLGLFK` DROP COLUMN `Y`
		at io.trino.plugin.jdbc.BaseJdbcClient.execute(BaseJdbcClient.java:1166)
		... 19 more
@findepi findepi added the bug Something isn't working label Mar 22, 2023
@findepi findepi mentioned this issue Mar 22, 2023
Merged
@ebyhr
Copy link
Member

ebyhr commented Mar 22, 2023

https://issues.apache.org/jira/browse/IGNITE-13020 seems the relevant issue. There's no progress though.

@chenjian2664 chenjian2664 mentioned this issue Apr 3, 2023
Merged
@ebyhr
Copy link
Member

ebyhr commented Apr 6, 2023

TestIgniteConnectorTest.testAddColumnConcurrently hit
https://github.com/trinodb/trino/actions/runs/4625409839/jobs/8181211343?pr=16905

@ebyhr
Copy link
Member

ebyhr commented Apr 6, 2023

https://github.com/trinodb/trino/actions/runs/4627973790/jobs/8187975285?pr=16905

@ebyhr
Copy link
Member

ebyhr commented Apr 7, 2023

https://github.com/trinodb/trino/actions/runs/4633655567/jobs/8199707750

This was referenced Apr 11, 2023
Merged
Merged
@findepi
Copy link
Member Author

findepi commented Apr 12, 2023

TestIgniteConnectorTest.testAddColumnConcurrently hit https://github.com/trinodb/trino/actions/runs/4625409839/jobs/8181211343?pr=16905

this looks like a separate problem, tracked in #16980

@findepi findepi changed the title Flaky TestIgniteConnectorTest.testDropColumn: Schema change operation failed: Thread got interrupted while trying to acquire table lock. Flaky TestIgniteConnectorTest tests: Schema change operation failed: Thread got interrupted while trying to acquire table lock. Apr 18, 2023
@findepi
Copy link
Member Author

findepi commented Apr 18, 2023

https://github.com/trinodb/trino/actions/runs/4725414489/jobs/8384163780?pr=17074

Error:  Tests run: 528, Failures: 1, Errors: 0, Skipped: 222, Time elapsed: 944.642 s <<< FAILURE! - in TestSuite
Error:  io.trino.plugin.ignite.TestIgniteConnectorTest.testAddAndDropColumnName[an'apostrophe](14)  Time elapsed: 0.024 s  <<< FAILURE!
io.trino.testing.QueryFailedException: Schema change operation failed: Thread got interrupted while trying to acquire table lock.
	at io.trino.testing.AbstractTestingTrinoClient.execute(AbstractTestingTrinoClient.java:122)
	at io.trino.testing.DistributedQueryRunner.executeWithQueryId(DistributedQueryRunner.java:495)
	at io.trino.testing.QueryAssertions.assertDistributedUpdate(QueryAssertions.java:107)
	at io.trino.testing.QueryAssertions.assertUpdate(QueryAssertions.java:63)
	at io.trino.testing.AbstractTestQueryFramework.assertUpdate(AbstractTestQueryFramework.java:400)
	at io.trino.testing.AbstractTestQueryFramework.assertUpdate(AbstractTestQueryFramework.java:395)
	at io.trino.testing.BaseConnectorTest.testAddAndDropColumnName(BaseConnectorTest.java:4879)
	at io.trino.testing.BaseConnectorTest.testAddAndDropColumnName(BaseConnectorTest.java:4856)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
	Suppressed: java.lang.Exception: SQL: ALTER TABLE tcn_anapostrophe0zlaxldwpy ADD COLUMN "an'apostrophe" varchar(50)
		at io.trino.testing.DistributedQueryRunner.executeWithQueryId(DistributedQueryRunner.java:499)
		... 19 more
Caused by: io.trino.spi.TrinoException: Schema change operation failed: Thread got interrupted while trying to acquire table lock.
	at io.trino.plugin.jdbc.BaseJdbcClient.addColumn(BaseJdbcClient.java:955)
	at io.trino.plugin.jdbc.BaseJdbcClient.addColumn(BaseJdbcClient.java:941)
	at io.trino.plugin.jdbc.ForwardingJdbcClient.addColumn(ForwardingJdbcClient.java:336)
	at io.trino.plugin.jdbc.jmx.StatisticsAwareJdbcClient.lambda$addColumn$24(StatisticsAwareJdbcClient.java:258)
	at io.trino.plugin.jdbc.jmx.JdbcApiStats.wrap(JdbcApiStats.java:47)
	at io.trino.plugin.jdbc.jmx.StatisticsAwareJdbcClient.addColumn(StatisticsAwareJdbcClient.java:258)
	at io.trino.plugin.jdbc.CachingJdbcClient.addColumn(CachingJdbcClient.java:487)
	at io.trino.plugin.jdbc.CachingJdbcClient.addColumn(CachingJdbcClient.java:487)
	at io.trino.plugin.ignite.IgniteMetadata.addColumn(IgniteMetadata.java:183)
	at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.addColumn(ClassLoaderSafeConnectorMetadata.java:328)
	at io.trino.metadata.MetadataManager.addColumn(MetadataManager.java:756)
	at io.trino.execution.AddColumnTask.execute(AddColumnTask.java:137)
	at io.trino.execution.AddColumnTask.execute(AddColumnTask.java:54)
	at io.trino.execution.DataDefinitionExecution.start(DataDefinitionExecution.java:146)
	at io.trino.execution.SqlQueryManager.createQuery(SqlQueryManager.java:251)
	at io.trino.dispatcher.LocalDispatchQuery.lambda$startExecution$7(LocalDispatchQuery.java:143)
	at io.trino.$gen.Trino_testversion____20230417_212434_75.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.sql.SQLException: Schema change operation failed: Thread got interrupted while trying to acquire table lock.
	at org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:1010)
	at org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute0(JdbcThinStatement.java:234)
	at org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute(JdbcThinStatement.java:560)
	at io.trino.plugin.jdbc.BaseJdbcClient.execute(BaseJdbcClient.java:1209)
	at io.trino.plugin.jdbc.BaseJdbcClient.addColumn(BaseJdbcClient.java:969)
	at io.trino.plugin.jdbc.BaseJdbcClient.addColumn(BaseJdbcClient.java:952)
	... 19 more
	Suppressed: java.lang.RuntimeException: Query: ALTER TABLE `PUBLIC`.`TCN_ANAPOSTROPHE0ZLAXLDWPY` ADD `AN'APOSTROPHE` varchar(50)
		at io.trino.plugin.jdbc.BaseJdbcClient.execute(BaseJdbcClient.java:1212)
		... 21 more

[INFO] 
[INFO] Results:
[INFO] 
Error:  Failures: 
Error:    TestIgniteConnectorTest>BaseConnectorTest.testAddAndDropColumnName:4856->BaseConnectorTest.testAddAndDropColumnName:4879->AbstractTestQueryFramework.assertUpdate:395->AbstractTestQueryFramework.assertUpdate:400 » QueryFailed
[INFO] 
Error:  Tests run: 528, Failures: 1, Errors: 0, Skipped: 222
[INFO]

@findepi findepi mentioned this issue Apr 18, 2023
Merged
@chenjian2664 chenjian2664 mentioned this issue Apr 18, 2023
Merged
@krvikash
Copy link
Contributor

krvikash commented May 9, 2023

https://github.com/trinodb/trino/actions/runs/4926424212/jobs/8805991158?pr=17409

@kokosing kokosing reopened this May 16, 2023
@ebyhr ebyhr reopened this Mar 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@chenjian2664 chenjian2664

Labels
bug Something isn't working
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix Ignite schema change operation tests failure
5 participants
@findepi @ebyhr @kokosing @krvikash @chenjian2664