Test Iceberg cost-based plans on TPC-H and TPC-DS #14489
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
findepi
added
test
performance
no-release-notes
findepi
force-pushed
the
findepi/iceberg-plans
branch
4 times, most recently
from
763aed1 to
6949624
Compare
findepi
force-pushed
the
findepi/iceberg-plans
branch
from
6949624 to
5928b3f
Compare
|
rebased after #14497 merged |
Migrate convenience methods from HiveMinioDataLake to Minio itself, so that it's viable to use MinIO alone without Hadoop container.
findepi
force-pushed
the
findepi/iceberg-plans
branch
2 times, most recently
from
cf24555 to
0d81a3c
Compare
findepi
requested review from
losipiuk,
sopel39,
ebyhr,
raunaqmorarka,
przemekak,
gaurav8297 and
alexjo2144
findepi
force-pushed
the
findepi/iceberg-plans
branch
from
0d81a3c to
53bbeac
Compare
findepi
changed the title
|
For completeness, I've pulled TPC-H dataset metadata as well. |
losipiuk
reviewed
Oct 7, 2022
| @@ -142,13 +151,20 @@ protected void generate() | |||
| getQueryPlanResourcePath(queryResourcePath)); | |||
| createParentDirs(queryPlanWritePath.toFile()); | |||
| write(generateQueryPlan(readQuery(queryResourcePath)).getBytes(UTF_8), queryPlanWritePath.toFile()); | |||
| System.out.println("Generated expected plan for query: " + queryResourcePath); | |||
| log.info("Generated expected plan for query: %s", queryResourcePath); | |||
There was a problem hiding this comment.
nit: could be separate commit :P
losipiuk
approved these changes
Oct 7, 2022
raunaqmorarka
approved these changes
Oct 7, 2022
| @@ -16,8 +16,8 @@ | |||
| import java.util.stream.Stream; | |||
|
|
|||
| /** | |||
| * This class tests cost-based optimization rules related to joins. It contains unmodified TPCH queries. | |||
There was a problem hiding this comment.
nit: comment improvements in existing code could be separate commit
System.out goes to logs anyway, just the context is stripped.
This uses TPC-H sf1000 and TPC-DS sf1000 Iceberg ORC data sets' metadata files generated on Starburst's benchmark infrastructure. The tables have no history and were created using single CTAS.
|
CI #14519 |
findepi
force-pushed
the
findepi/iceberg-plans
branch
from
53bbeac to
ede2e27
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.