Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor: allow to build RuntimeEnv from RuntimeConfig #12151

Merged
merged 2 commits into from
Aug 26, 2024

Conversation

theirix
Copy link
Contributor

@theirix theirix commented Aug 24, 2024

Which issue does this PR close?

Closes #12137

Rationale for this change

Allow to create RuntimeEnv directly from the config instance.

What changes are included in this PR?

  • New method
  • Changed test to cover this change

Are these changes tested?

To test this change, I've changed one existing test sort_fuzz to use a new API.

Are there any user-facing changes?

No

@github-actions github-actions bot added core Core DataFusion crate execution Related to the execution crate labels Aug 24, 2024
let runtime_config = RuntimeConfig::new()
.with_memory_pool(Arc::new(GreedyMemoryPool::new(pool_size)));
let runtime = Arc::new(RuntimeEnv::new(runtime_config).unwrap());
let runtime_env = RuntimeConfig::new()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure why it is env, config is more appropriate imho

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's because a newly introduced method build returns env. Please check the rest of this expression in diff below.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @comphead this now looks a bit strange -- I filed #12156 with an idea of how to improve it

let runtime = Arc::new(RuntimeEnv::new(runtime_config).unwrap());
let runtime_env = RuntimeConfig::new()
.with_memory_pool(Arc::new(GreedyMemoryPool::new(pool_size))).build();
let runtime = Arc::new(runtime_env.unwrap());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should avoid calling .unwrap() although its not related to the PR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Even for tests, if we have complex chains of unwraps, we can use expect with a more detailed failure explanation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(note this is a test so I think it is ok to call unwrap() here)

@comphead
Copy link
Contributor

Thanks @theirix for you contribution, I started the pipeline

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @theirix and @comphead -- I think this looks good to me

@theirix -- any chance you can update other uses of RuntimeEnv::new() in the codebase to use the new pattern. There appear to be a few more:

https://github.com/search?q=repo%3Aapache%2Fdatafusion%20RuntimeEnv%3A%3Anew&type=code

let runtime = Arc::new(RuntimeEnv::new(runtime_config).unwrap());
let runtime_env = RuntimeConfig::new()
.with_memory_pool(Arc::new(GreedyMemoryPool::new(pool_size))).build();
let runtime = Arc::new(runtime_env.unwrap());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(note this is a test so I think it is ok to call unwrap() here)

let runtime_config = RuntimeConfig::new()
.with_memory_pool(Arc::new(GreedyMemoryPool::new(pool_size)));
let runtime = Arc::new(RuntimeEnv::new(runtime_config).unwrap());
let runtime_env = RuntimeConfig::new()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @comphead this now looks a bit strange -- I filed #12156 with an idea of how to improve it

@theirix
Copy link
Contributor Author

theirix commented Aug 25, 2024

Thank you! I think tests would benefit from this change, and I see @devanbenz is up to improving tests.

@alamb alamb merged commit ed12f11 into apache:main Aug 26, 2024
24 checks passed
@alamb
Copy link
Contributor

alamb commented Aug 26, 2024

Thanks again @theirix and @comphead

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate execution Related to the execution crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Builder style API for creating RuntimeEnv
3 participants