Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid extra TrinoInputFile#lastModified calls in hive filesystem cache #23327

Merged
merged 2 commits into from
Sep 11, 2024

Conversation

raunaqmorarka
Copy link
Member

@raunaqmorarka raunaqmorarka commented Sep 7, 2024

Description

Avoid extra TrinoInputFile#lastModified calls in hive filesystem cache

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

# Hive
* Reduce filesystem accesses made when reading tables with filesystem caching enabled. ({issue}`23327`)

@cla-bot cla-bot bot added the cla-signed label Sep 7, 2024
@github-actions github-actions bot added iceberg Iceberg connector delta-lake Delta Lake connector hive Hive connector labels Sep 7, 2024
@wendigo
Copy link
Contributor

wendigo commented Sep 7, 2024

Typo in the first commit title

@raunaqmorarka raunaqmorarka force-pushed the hive-cache-key branch 2 times, most recently from 9312459 to 6f77c25 Compare September 9, 2024 07:26
@raunaqmorarka
Copy link
Member Author

/test-with-secrets sha=6f77c254df428a126486fdf73b928b83759597b9

Copy link

github-actions bot commented Sep 9, 2024

The CI workflow run with tests that require additional secrets finished as failure: https://github.com/trinodb/trino/actions/runs/10768734494

@@ -112,11 +108,6 @@ public void testCacheFileOperations()
.add(new CacheOperation("Alluxio.writeCache", "key=p3/"))
.add(new CacheOperation("Alluxio.writeCache", "key=p4/"))
.add(new CacheOperation("Alluxio.writeCache", "key=p5/"))
.add(new CacheOperation("InputFile.lastModified", "key=p1/"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

niceeeee

@wendigo
Copy link
Contributor

wendigo commented Sep 10, 2024

LGTM but I'd like @electrum to chime in in regard to the new API

@wendigo
Copy link
Contributor

wendigo commented Sep 11, 2024

@raunaqmorarka can you rebase? My changes conflicted with yours

@raunaqmorarka raunaqmorarka merged commit 527dd0d into trinodb:master Sep 11, 2024
63 of 64 checks passed
@raunaqmorarka raunaqmorarka deleted the hive-cache-key branch September 11, 2024 08:59
@github-actions github-actions bot added this to the 458 milestone Sep 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed delta-lake Delta Lake connector hive Hive connector iceberg Iceberg connector performance
Development

Successfully merging this pull request may close these issues.

3 participants