-
Notifications
You must be signed in to change notification settings - Fork 121
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal: Schema Validation #331
Conversation
c53cb45
to
3c3ebb5
Compare
09c7750
to
67fdd73
Compare
d7876e5
to
8b9479e
Compare
8b9479e
to
e8d30b1
Compare
- exclusiveMaximum (is this useful?) | ||
- exclusiveMinimum (is this useful?) | ||
- [multipleOf](https://datatracker.ietf.org/doc/html/draft-wright-json-schema-validation-00#section-5.1) | ||
- pattern (This string SHOULD be a valid regular expression, according to the ECMA 262 regular - expression dialect) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cppforlife noted:
ECMA 262 != RE2
Q: what approach have others who have implemented this feature (specifically JSON Schema validation keyword pattern
) in Go done?
Following discussions with [gcheadle-vmware](carvel-dev/ytt#604 (comment)) and @cppforlife, it became clear that @assert/validate ought to always be consumed in a step after overlays are applied for the current pipeline. When writing schema, the author intends not for the schema document _itself_ to be validated, but to validate the _produced_ document (of which the schema is a basis for). Authors need a way to declare the intention for a validation to be attached to the type composite: @schema/validation.
The former is a predicate (expected to return True or False), the later is an assertion.
744ed61
to
2536c27
Compare
It seems quite likely that we'll need such a facility for more advanced cases. Placed in the "Other approaches considered" means that the ideas are found without sufficient merit; not the case here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I read through the proposal and it looks very well thought out, AWESOME JOB @pivotaljohn
Added some comments and questions in some of the points.
```python | ||
lib.eval(output_disable_validation=True) | ||
``` | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@vrabbi asks:
Any thought on allowing to skip a specific violation?
Not sure what the syntax would be but i can see cases where when consuming third party packages i may want to disable a specific validation that isnt valid in my case.
I don't think its a very common case but it would allow for a break glass solution that is still more secure then disabling all validations.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At first blush, this starts to make more compelling the need for the ability for overlays to modify metadata in addition to structure (i.e. edit annotations, not just the YAML structure).... 🤔
Curious if others have thoughts about this:
- can you name specific use cases where this would occur (or speculate as to how frequent this would be)?
- ideas on a syntax for doing this via the command-line?
(from conversation with @vrabbi; that you Scott!)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the proposal, and looking forward to be able to play with an initial implementation to see if all the use-cases we have can be satisfied. It's difficult to think whether all could work by just reading this proposal. But so happy to see this coming through.
#@ end | ||
|
||
#@data/values-schema | ||
#@schema/validation ("There should be exactly one (1) registry replica if Helm Charts are stored on the filesystem.", one_registry_if_pvc_is_filesystem) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think there's going to be a common pattern for this "top-node" functions where sub-nodes on the schema will be validated (or take the value from/if) a top level node is defined. An example is what CNR/TAP is defining as TLK (Top Level Keys).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting...
We should talk more about what you see, here... cooking up some examples might be useful/insightful.
```yaml | ||
#@data/values-schema | ||
--- | ||
#@schema/validation ("", valid_service_config), when=lambda v: v.enabled |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the lambda v:
part required? Could also this be supported?
#@schema/validation ("", valid_service_config), when=this.enabled
service:
enabled: true
Use of keyword this
to denote this node (or similar), or fully reference a value key:
#@schema/validation ("", valid_service_config), when=service.enabled
service:
enabled: true
I understand the constraint defined above that data.values are not available at pre-process, but definitely syntax is not very user friendly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need some kind of "callable" that can be invoked in the validation stage: yes.
I like the intention, here.
I think this involves detecting when the argument is not a "callable" and then inject the lambda
wrapper...
#@schema/validation ("", valid_service_config), when=this.enabled
is compiled to:
#@schema/validation ("", valid_service_config), when=lambda this: this.enabled
We'll need to make sure that this "auto-wrapping" doesn't accidentally cover-up or make harder-to-debug issues when the expression is invalid for some reason... 🤔
No description provided.