Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement strict decoding for JetStream API requests #5858

Merged
merged 7 commits into from
Oct 30, 2024

Conversation

caspervonb
Copy link
Contributor

This implements optional strict JSON decoding for JetStream.

The intent of this is to minimize accidental misalignments between server and clients, we've had numerous of these across the various clients, especially for rarely used fields in the request payloads.

Signed-off-by: Casper Beyer [email protected]

@derekcollison
Copy link
Member

https://app.travis-ci.com/github/nats-io/nats-server/jobs/625767923

Needs a fix

@caspervonb caspervonb force-pushed the optional-strict-json-decoding branch 3 times, most recently from 9beb4f6 to 6b84a9b Compare September 4, 2024 18:37
server/jetstream_api.go Outdated Show resolved Hide resolved
@caspervonb caspervonb changed the title [WIP] Implement strict decoding for JetStream API requests Implement strict decoding for JetStream API requests Sep 6, 2024
@caspervonb caspervonb force-pushed the optional-strict-json-decoding branch 2 times, most recently from a9b2549 to c4418f4 Compare September 6, 2024 14:26
@caspervonb caspervonb marked this pull request as ready for review September 6, 2024 14:27
@caspervonb caspervonb requested a review from a team as a code owner September 6, 2024 14:27
@caspervonb
Copy link
Contributor Author

There's one failing test, but this is failing on main as-well so I don't think that's related.

server/jetstream.go Outdated Show resolved Hide resolved
server/jetstream_api.go Outdated Show resolved Hide resolved
@caspervonb
Copy link
Contributor Author

Ran the meta benchmarks on this as they're pretty request heavy:

JetStreamCreate/N=1,R=1,storage=Memory,C=12/resource=Stream-14        593.6µ ± ∞ ¹   618.6µ ± ∞ ¹       ~ (p=1.000 n=1) ²
JetStreamCreate/N=1,R=1,storage=Memory,C=12/resource=KVBucket-14      1.065m ± ∞ ¹   1.074m ± ∞ ¹       ~ (p=1.000 n=1) ²
JetStreamCreate/N=1,R=1,storage=Memory,C=12/resource=ObjStore-14      2.889m ± ∞ ¹   2.831m ± ∞ ¹       ~ (p=1.000 n=1) ²
JetStreamCreate/N=3,R=3,storage=Memory,C=12/resource=Stream-14        51.54m ± ∞ ¹   56.28m ± ∞ ¹       ~ (p=1.000 n=1) ²
JetStreamCreate/N=3,R=3,storage=Memory,C=12/resource=KVBucket-14      63.51m ± ∞ ¹   58.32m ± ∞ ¹       ~ (p=1.000 n=1) ²
JetStreamCreate/N=3,R=3,storage=Memory,C=12/resource=ObjStore-14      62.60m ± ∞ ¹   64.33m ± ∞ ¹       ~ (p=1.000 n=1) ²
JetStreamCreate/N=3,R=3,storage=File,C=12/resource=Stream-14          60.38m ± ∞ ¹   56.13m ± ∞ ¹       ~ (p=1.000 n=1) ²
JetStreamCreate/N=3,R=3,storage=File,C=12/resource=KVBucket-14        66.11m ± ∞ ¹   56.62m ± ∞ ¹       ~ (p=1.000 n=1) ²
JetStreamCreate/N=3,R=3,storage=File,C=12/resource=ObjStore-14        58.56m ± ∞ ¹   59.47m ± ∞ ¹       ~ (p=1.000 n=1) ²

@neilalexander neilalexander self-requested a review September 10, 2024 10:21
server/opts.go Outdated Show resolved Hide resolved
derekcollison added a commit that referenced this pull request Sep 11, 2024
Currently failures to unmarshal only give a vague error of "invalid
JSON", this adds the original error to it as context.

Before:

````
invalid JSON
````

After:

```
invalid JSON: invalid character '\"' after object key
```

Related to but **not** dependent on
#5858

Signed-off-by: Casper Beyer <[email protected]>
@neilalexander
Copy link
Member

Mind rebasing this & resolving conflicts please?

@caspervonb caspervonb force-pushed the optional-strict-json-decoding branch from 0f7aa53 to b0144fc Compare September 16, 2024 11:11
@caspervonb caspervonb force-pushed the optional-strict-json-decoding branch from b0144fc to 50f27e4 Compare September 23, 2024 11:47
@ripienaar
Copy link
Contributor

Please make sure the strict option value is shown in jsz output

server/jetstream.go Outdated Show resolved Hide resolved
server/jetstream_test.go Outdated Show resolved Hide resolved
Copy link
Member

@neilalexander neilalexander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@derekcollison derekcollison self-requested a review October 25, 2024 19:42
server/jetstream_api.go Outdated Show resolved Hide resolved
server/jetstream_api.go Outdated Show resolved Hide resolved
server/jetstream_api.go Outdated Show resolved Hide resolved
Copy link
Member

@derekcollison derekcollison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@derekcollison derekcollison merged commit ea1df00 into main Oct 30, 2024
5 checks passed
@derekcollison derekcollison deleted the optional-strict-json-decoding branch October 30, 2024 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants