Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

protojson: Add MarshalOption to write int64 as a number and not as a string #1414

Open
corgrath opened this issue Feb 17, 2022 · 4 comments
Open

Comments

@corgrath
Copy link

corgrath commented Feb 17, 2022

Is your feature request related to a problem? Please describe.
When using the protonjson library int64 numbers are outputted as strings in the JSON due to this code:

https://github.com/protocolbuffers/protobuf-go/blob/b92717ecb630d4a4824b372bf98c729d87311a4d/encoding/protojson/encode.go#L275-L278

From what I understand, it is written as a string since JavaScript cannot handle that precision.

However, from a JSON stand point (not JavaScript), it should handle any integers (JSON should not care about precision, just as XML shouldn't care about precision).

We are only interested in marshaling a message to a JSON, not to JavaScript, so would like the protonjson to write int64 as an integer/number and not as a string.

Describe the solution you'd like
Add a new to option to MarshalOptions :

https://github.com/protocolbuffers/protobuf-go/blob/b92717ecb630d4a4824b372bf98c729d87311a4d/encoding/protojson/encode.go#L43

Something like:

	// Write64KindsAsInteger will write any 64 bit kinds as integers instead of a string.
	Write64KindsAsInteger

And then in marshalSingular use this option:

	case pref.Int64Kind, pref.Sint64Kind, pref.Sfixed64Kind:
		if e.opts.Write64KindsAsInteger {
			e.WriteInt64(val.Int())
		} else {
			// 64-bit integers are written out as JSON string.
			e.WriteString(val.String())
		}

	case pref.Uint64Kind, pref.Fixed64Kind:
		if e.opts.Write64KindsAsInteger {
			e.WriteUInt64(val.Int())
		} else {
			// 64-bit integers are written out as JSON string.
			e.WriteString(val.String())
		}

Describe alternatives you've considered
AFAK, there are no other feasible solutions to write int64s as integers in the JSON output.

Additional context
If you use the default JSON marshaller (encoding/json), int64 is written as a number.

However, since we need to output default values, we are forced to use this library due to the EmitUnpopulated option.

So right now our organization is between a hard rock and a stone; Either omit default values (such as 0 and false) or write int64 as strings (and breaking other tools that reads in JSON data).

@neild
Copy link
Contributor

neild commented Feb 17, 2022

The protojson package follows the standard JSON mapping defined here, which states that 64-bit ints are encoded as a JSON string:
https://developers.google.com/protocol-buffers/docs/proto3#json

Changing the standard mapping is out of scope for the Go implementation, as it is not specific to any particular language. If you would like to propose changes to it, you could file an issue on the general protobuf issue tracker: https://github.com/protocolbuffers/protobuf/issues

@corgrath
Copy link
Author

Adding an option to bypass this is out of the question as well?

@dsnet
Copy link
Member

dsnet commented Feb 17, 2022

Go only adds options that the other major language implementations add. There's no such option in C++: https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.util.json_util#JsonPrintOptions

@neild
Copy link
Contributor

neild commented Feb 17, 2022

See also:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#new-marshal-option

Can I add an option to Marshal or Unmarshal to customize it?

Only if that option exists in other implementations (e.g., C++, Java). The encoding of protocol buffers (binary, JSON, and text) must be consistent across implementations, so a program written in one language is able to read messages written by another one.

We will not add any options to the Go implementation that affect the data output by Marshal functions or read by Unmarshal functions unless an equivalent option exist in at least one other supported implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants