Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Any keyword halts on type exception #399

Open
rawrgulmuffins opened this issue Jul 15, 2019 · 3 comments
Open

Any keyword halts on type exception #399

rawrgulmuffins opened this issue Jul 15, 2019 · 3 comments

Comments

@rawrgulmuffins
Copy link

rawrgulmuffins commented Jul 15, 2019

I have a data structure that has a Time field that can either be ISO-6801 timestamp or an epoch time. Unfortunately the Any keyword isn't progressing to the second validation checks on type failure.

Repro

from voluptuous import All, Any, Schema
from voluptuous.humanize import validate_with_humanized_errors

# We won't get to the custom validators in the repro
from custom_validators import IsUnixTimestamp, TimezoneSTRToUTCDatetime

test_data = "2019-01-30 13:04:41.000"

test_schema = Schema(Any(All(int, IsUnixTimestamp), All(str, TimezoneSTRToUTCDatetime()))
validate_with_humanized_errors(test_data, test_schema)

Expectation

Any should notice the failure but not report the validation failures until the all of the validators have failed.

Exact Stack Trace

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 267, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.7/site-packages/voluptuous/validators.py", line 204, in _run
    return self._exec(self._compiled, value, path)
  File "/usr/local/lib/python3.7/site-packages/voluptuous/validators.py", line 256, in _exec
    self.msg, path=path)
  File "/usr/local/lib/python3.7/site-packages/voluptuous/validators.py", line 249, in _exec
    return func(path, v)
  File "/usr/local/lib/python3.7/site-packages/voluptuous/validators.py", line 204, in _run
    return self._exec(self._compiled, value, path)
  File "/usr/local/lib/python3.7/site-packages/voluptuous/validators.py", line 286, in _exec
    raise e if self.msg is None else AllInvalid(self.msg, path=path)
  File "/usr/local/lib/python3.7/site-packages/voluptuous/validators.py", line 284, in _exec
    v = func(path, v)
  File "/usr/local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 804, in validate_instance
    raise er.TypeInvalid(msg, path)
voluptuous.error.TypeInvalid: expected int

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/voluptuous/humanize.py", line 38, in validate_with_humanized_errors
    return schema(data)
  File "/usr/local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 271, in __call__
    raise er.MultipleInvalid([e])
voluptuous.error.MultipleInvalid: expected int

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "repro.py", line 10, in <module>
    validate_with_humanized_errors(test_data, test_schema)
  File "/usr/local/lib/python3.7/site-packages/voluptuous/humanize.py", line 40, in validate_with_humanized_errors
    raise Error(humanize_error(data, e, max_sub_error_length))
voluptuous.error.Error: expected int. Got '2019-01-30 13:04:41.000'

Versions

PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
Python==3.7.2
voluptuous==0.11.5
@rawrgulmuffins
Copy link
Author

rawrgulmuffins commented Jul 15, 2019

After further testing I was able to get this schema to work as expected. This was pretty unclear from the documentation as given and I would love to either submit a patch to improve the keywords or a documentation modification to make things more clear.

from voluptuous import All, Any, Schema
from voluptuous.humanize import validate_with_humanized_errors

from crs_normalizer.validators import IsUnixTimestamp, TimezoneSTRToUTCDatetime

test_data = "2019-01-30 13:04:41.000"

test_schema = Schema(
    All(Any(int, str), Any(IsUnixTimestamp, TimezoneSTRToUTCDatetime())))
validate_with_humanized_errors(test_data, test_schema)

Any preferences?

@svisser
Copy link
Collaborator

svisser commented Jul 16, 2019

I haven't looked into the Any / All validators in details but if you're writing custom validators, it's possibly more clear to follow the structure here: https://github.com/alecthomas/voluptuous#validation-functions (i.e., move the responsibility of checking the type into the custom validator and let it raise ValueError or an exception class from voluptuous):

from voluptuous import *

def IsUnixTimestamp(v):
    if not isinstance(v, int):
        raise Invalid(f"Provided value is not a Unix timestamp: {v}")
    # ...
    return v

which gives:

>>> IsUnixTimestamp('abc')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in IsUnixTimestamp
voluptuous.error.Invalid: Provided value is not a Unix timestamp: abc

and then wrap both validators in Any(... , ...).

@rawrgulmuffins
Copy link
Author

@svisser I originally was moving in that direction but the intent is to both validate and have an easy to read description of the data itself. I ended up keeping the type checks in the schema because it makes a significant difference when explaining the data structures to people.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants