-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Install JDBC drivers for Delta product tests at runtime and upgrade docker-images
to version 81
#17667
Conversation
5e155d3
to
1663c8a
Compare
testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/DeltaQueryExecutor.java
Outdated
Show resolved
Hide resolved
docker-images
to version 81
testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/DeltaQueryExecutor.java
Outdated
Show resolved
Hide resolved
testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/DeltaQueryExecutor.java
Outdated
Show resolved
Hide resolved
testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/DeltaQueryExecutor.java
Outdated
Show resolved
Hide resolved
...duct-tests/src/main/java/io/trino/tests/product/deltalake/TestDeltaLakeDatabricksDelete.java
Show resolved
Hide resolved
...sts/src/main/java/io/trino/tests/product/deltalake/TestDeltaLakeAlterTableCompatibility.java
Show resolved
Hide resolved
d804d9a
to
1ea96c3
Compare
testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/DeltaQueryExecutor.java
Outdated
Show resolved
Hide resolved
testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/DeltaQueryExecutor.java
Outdated
Show resolved
Hide resolved
testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/DeltaQueryExecutor.java
Outdated
Show resolved
Hide resolved
testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/DeltaQueryExecutor.java
Outdated
Show resolved
Hide resolved
testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/QueryExecutors.java
Outdated
Show resolved
Hide resolved
cb71c89
to
c4abb2c
Compare
c4abb2c
to
bd3522c
Compare
Addressed comments. |
testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/DeltaQueryExecutor.java
Outdated
Show resolved
Hide resolved
testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/DeltaQueryExecutor.java
Outdated
Show resolved
Hide resolved
testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/QueryExecutors.java
Outdated
Show resolved
Hide resolved
bd3522c
to
09dc558
Compare
|
09dc558
to
7fd5966
Compare
6b5492d
to
1183780
Compare
testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/DeltaQueryExecutors.java
Outdated
Show resolved
Hide resolved
CI hit #18266 |
CI hit #14239 |
@findepi Could you take another look? |
testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/DeltaQueryExecutors.java
Outdated
Show resolved
Hide resolved
testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/DeltaQueryExecutors.java
Outdated
Show resolved
Hide resolved
I added some code sample how we can avoid regressions and shared state. However, why not simply configure |
I already tried |
let's understand why. It looks like the "correct" approach to do what we're trying to do. |
The path was correct. It looks classloader issue. Changing private static ClassLoader getDriverClassLoader(JdbcConnectivityParamsState jdbcParamsState)
{
try {
if (!jdbcParamsState.jar.isPresent()) {
return JdbcUtils.class.getClassLoader();
}
URL jarURL = new File(jdbcParamsState.jar.get()).toURL();
return URLClassLoader.newInstance(new URL[] {jarURL}, ClassLoader.getPlatformClassLoader());
}
catch (MalformedURLException e) {
throw new RuntimeException("could not create classloader for file" + jdbcParamsState.jar.get(), e);
}
} |
bd0fd16
to
0536cad
Compare
This upgrades Delta Lake version to 2.4.0 Change delta.minWriterVersion to 7. Otherwise, Spark throws delta.minWriterVersion needs to be one of 1, 2, 3, 4, 5, 7 error.
0536cad
to
bec8b7d
Compare
@GuardedBy("DRIVERS") | ||
private static final Map<String, String> DRIVERS = new HashMap<>(); | ||
@GuardedBy("JDBC_EXECUTORS") | ||
private static final Map<JdbcConnectivityParamsState, JdbcQueryExecutor> JDBC_EXECUTORS = new HashMap<>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if we use tempto driver jar
feature, why is the state maintained explicitly?
please use tempto-configuration.yaml to configure drivers
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sent #18464
Description
This upgrades Delta Lake version to 2.4.0.
https://github.com/delta-io/delta/releases/tag/v2.4.0
Release notes
(x) This is not user-visible or docs only and no release notes are required.