Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parametrization: fails for non-string interpolations #5006

Open
skshetry opened this issue Dec 1, 2020 · 2 comments
Open

parametrization: fails for non-string interpolations #5006

skshetry opened this issue Dec 1, 2020 · 2 comments
Labels
A: templating Related to the templating feature bug Did we break something?

Comments

@skshetry
Copy link
Member

skshetry commented Dec 1, 2020

See: https://github.com/iterative/dvc/wiki/Parametrization#known-issues

If the interpolated value is not a string in the params file, DVC will fail with a validation error.
Let's say, you have params with the following values:

# params.yaml
msg: true

# dvc.yaml
stages:
  echo:
    cmd: ${msg}

It might fail with the error similar to the following:

Running stage 'echo' with command:
        True
ERROR: failed to reproduce 'dvc.yaml': expected str, bytes or os.PathLike object, not bool
@skshetry skshetry added bug Did we break something? A: templating Related to the templating feature labels Dec 1, 2020
@skshetry skshetry self-assigned this Dec 1, 2020
@skshetry
Copy link
Member Author

skshetry commented Dec 1, 2020

This happens because DVC does not know about the schema during the parsing time, and is only used for validating when loading the yaml data.

We could use a whitelisting to convert data to an appropriate format, but it might not scale. It'd be better to have some way to "convert" data automatically through the schema itself.

Similar issue in #4996 was fixed with a custom solution for the boolean.

@jorgeorpinel
Copy link
Contributor

jorgeorpinel commented Jan 5, 2021

Great find. I say we could improve the error message for now as a first step to something like "ERROR: ambiguous value type found for msg in params.yaml (and prevent Running stage...).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A: templating Related to the templating feature bug Did we break something?
Projects
None yet
Development

No branches or pull requests

2 participants