Orthogonal defaulting and new pruning algorithm

[sttts]

This PR allows to implement different algorithms orthogonally to the main
validation recursion. The Result object tracks the validating schemata for
each node in the input JSON. This means that these algorithms do not
have to care about quantors or the different ways to specify properties.

This means:

• defaulting becomes a trival 12 liner:

// ApplyDefaults applies defaults to data.
func ApplyDefaults(r *validate.Result) {
	fieldSchemata := r.FieldSchemata()
	for key, schemata := range fieldSchemata {
	LookForDefaultingScheme:
		for _, s := range schemata {
			if s.Default != nil {
				key.Object()[key.Field()] = s.Default
				break LookForDefaultingScheme
			}
		}
	}
}

• pruning of unspecified fields is implementable in a similar way. It's added as another example.

Running defaulting benchmarks 30000 leads to:

Before PR:
30000	     44573 ns/op	   54581 B/op	     499 allocs/op
30000	     43208 ns/op	   54581 B/op	     499 allocs/op
30000	     39850 ns/op	   54582 B/op	     499 allocs/op
30000	     39427 ns/op	   54582 B/op	     499 allocs/op

After PR:

30000	     48023 ns/op	   66088 B/op	     497 allocs/op
30000	     46331 ns/op	   66086 B/op	     497 allocs/op

In other words: runtime is 15% higher, memory is 20% higher. If this is a problem, we could add a second ValidateWithSchemaTracking() func which does the extra processing and returns a result that has the necessary getters.

TODO:

• benchmark cpu and memory/allocations:

[sttts]

/cc @nikhita

[fredbi]

I like the idea of metadata in the result structure.
That is something I find useful to spew more accurate messages on where the validation error did occur.
But I am not sure this is what you had in mind.
Your PR is lagging master for several months I think.

[sttts]

Your PR is lagging master for several months I think.

Rebased already.

Will push the second algorithm soon to show what this is all about.

[sttts]

@fredbi compare the last commit. This is the kind of trivial algorithm I have in mind, without complicating the main validation recursion.

[sttts]

@s-urbaniak this is what I talked about in.

[codecov]

Codecov Report

Merging #81 into master will decrease coverage by 2.38%.
The diff coverage is 65.6%.

@@            Coverage Diff             @@
##           master      #81      +/-   ##
==========================================
- Coverage   99.14%   96.75%   -2.39%     
==========================================
  Files          19       18       -1     
  Lines        2099     2222     +123     
==========================================
+ Hits         2081     2150      +69     
- Misses         12       64      +52     
- Partials        6        8       +2

Impacted Files	Coverage Δ
helpers.go	95.32% <ø> (ø)	⬆️
schema.go	100% <100%> (ø)	⬆️
object_validator.go	98.09% <100%> (-0.02%)	⬇️
slice_validator.go	100% <100%> (ø)	⬆️
schema_props.go	100% <100%> (ø)	⬆️
result.go	73.78% <59.7%> (-26.22%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 180bba5...563c6e5. Read the comment docs.

[sttts]

@fredbi please take another look. This is ready for review.

[casualjim]

this is causing issues downstream, I'm going to revert this PR.. Will try to add the failing cases to this CI build so we discover this earlier.

[nikhita]

this is causing issues downstream

@casualjim Can you elaborate on this?

[casualjim]

After applying this PR, the types in slice properties are no longer []interface{}

Here's a job that fails:
https://travis-ci.org/go-openapi/runtime/jobs/366714797

just running the tests in go-openapi/runtime shows the issues.

[sttts]

@casualjim will look into that tomorrow.

Some integration CI would be nice. How do you notice those problems now? Just merging and then seeing regression on the next downstream runs?

[casualjim]

yeah basically.
I would like to add the following to .travis.yml

- go get -u github.com/go-openapi/runtime
- cd $GOPATH/src/github.com/go-openapi/runtime
- go test ./...

to run after the tests of the validate repo have finished

[casualjim]

I can let this be on master while you resolve the downstream issue, because I know what this PR is used for and I'm sure it gets a lot more usage than the breaking code.
But the failure should get resolved because it's blocking another PR around authentication

[sttts]

I took a quick look at the issue: my change in the schemaSliceValidator adds the requirement that the input data is a JSON like []interface{} slice, but here it is not: it's a []middleware.friend. We used reflection before my change to work around this.

I noticed that the objectValidator in contrast has a hard requirement on map[string]interface{} which sounds correct.

Am I right that the data used as validator input must be JSON? I.e. not an arbitrary Go type? If yes, the runtime middleware test broken it seems.