Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add link to advanced runtime_params configuration #4341

Merged
merged 5 commits into from
Nov 27, 2024

Conversation

DimedS
Copy link
Contributor

@DimedS DimedS commented Nov 19, 2024

Description

Closes #3562

Developer Certificate of Origin

We need all contributions to comply with the Developer Certificate of Origin (DCO). All commits must be signed off by including a Signed-off-by line in the commit message. See our wiki for guidance.

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

Checklist

  • Read the contributing guidelines
  • Signed off each commit with a Developer Certificate of Origin (DCO)
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added a description of this change in the RELEASE.md file
  • Added tests to cover my changes
  • Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team

@DimedS DimedS linked an issue Nov 19, 2024 that may be closed by this pull request
@DimedS DimedS marked this pull request as ready for review November 19, 2024 17:11
@ankatiyar
Copy link
Contributor

I'm wondering if we should update the above "How to specify parameters at runtime" section to use the runtime_params: resolver primarily and then mention that one can directly overwrite params too but it isn't recommended?

@@ -169,3 +169,6 @@ kedro run --params="key1=value with spaces,key2=value"
```

Since key-value pairs are split on the first equals sign, values can contain equals signs, but keys cannot.


> **Note:** If you want to override values of certain keys in your configuration with runtime parameters provided through the CLI option, you can use the [OmegaConfigLoader `runtime_params` resolver](advanced_configuration.md#how-to-override-configuration-with-runtime-parameters-with-the-omegaconfigloader).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use the {note} notation instead, to maintain consistency. Eg -

```{note}
You can use `add_feed_dict()` to inject any other entries into your `DataCatalog` as per your use case.
```

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ankatiyar, thanks! I agree, but I tried that syntax, and it doesn’t support links. Since I have one link there, my suggestion is to remove the word "Note" and keep the current syntax. What do you think?

Signed-off-by: Dmitry Sorokin <[email protected]>
@DimedS
Copy link
Contributor Author

DimedS commented Nov 20, 2024

I'm wondering if we should update the above "How to specify parameters at runtime" section to use the runtime_params: resolver primarily and then mention that one can directly overwrite params too but it isn't recommended?

Do you mean we should merge the two articles into the first one and remove the advanced article?

@ankatiyar
Copy link
Contributor

Do you mean we should merge the two articles into the first one and remove the advanced article?

I mean, keep the section in Advanced Configuration as it is but also mention the runtime_params: here in this section. Would like to see what others think.

@noklam noklam self-requested a review November 21, 2024 10:33
Copy link
Contributor

@noklam noklam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It took me a while to understand the issue here. --params is something that Kedro supports since the beginning. It supports parameters ONLY with arbitary nested level, so I find the explanation here does not explains why user need to use $runtime_params.

$runtime_params is added later with OmegaConfigLoader to support overriding ANY config (i.e. catalog, a common use case is pointing to a different s3 bucket). So maybe the focus here should be, if user need to override configuration other than parameters, they should use $runtime_params?

@astrojuanlu
Copy link
Member

+1 to what @noklam said 👍🏼

Signed-off-by: Dmitry Sorokin <[email protected]>
@DimedS
Copy link
Contributor Author

DimedS commented Nov 25, 2024

So maybe the focus here should be, if user need to override configuration other than parameters, they should use $runtime_params?

Thanks for the clarification, @noklam. I was also confused about why we added additional functionality for doing the same thing. I've made the updates and hope this works well. Requesting your re-reviews.

Co-authored-by: Merel Theisen <[email protected]>
Signed-off-by: Dmitry Sorokin <[email protected]>
Copy link
Member

@merelcht merelcht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DimedS DimedS enabled auto-merge (squash) November 27, 2024 11:48
@DimedS DimedS merged commit b53d365 into main Nov 27, 2024
10 checks passed
@DimedS DimedS deleted the 3562-improve-runtime-params-documentation branch November 27, 2024 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve runtime params documentation
5 participants