✨ Proposal: format registry

[gregsdennis]

Describe the inspiration for your proposal

Open API already has defined several formats that are arguably useful outside of the context of their spec, but I don't think we should add them to our spec.

Additionally, we receive a lot of requests to add new formats to the spec. A registry could be a good middle ground.

Describe the proposal

We should support a registry for different formats.

We don't have any precedent for creating a registry, so I'm open to ideas on exactly how to do that. I think a file in GH that ends up being push to the website that could then be referenced by the spec would be an okay idea.

Describe alternatives you've considered

No response

Additional context

There is a concern about adding additional burden for implementations around having to support all of the formats. I'm open to ideas here as well.

Really just opening the floor for exploration.

[jviotti]

I like the idea, but my main concern is that if most of these formats are optional, very few will actually implement them, rendering them mostly unusable for any interoperable use case.

I would almost prefer a more limited set of formats defined by us and mandatory for everybody. I feel just forcing everybody to do the same thing, even if imperfect, can be better than suggesting everybody to do a lot of things.

[mwadams]

Personally, I think a combination of the two.

I think we should enforce some "near-primitives" that makes interoperability hard if we don't (the date and time related formats specifically. They are ubiquitous and notoriously hard to interop without a solid spec)

Then there are the "arguables" that I think should be required because we use them in the metaschema - the URI/IRI family.

Then there are the things I think should be in a registry. (u)int32,64,128, uuid, email etc. etc.

[jdesrosiers]

Open API already has a format registry. Ideally we should be involved with that registry, but I don't want to have a competing registry. In the past, they've expressed willingness to donate ownership of the registry to us if we want it. Co-ownership of the registry would be a good outcome as well.

[gregsdennis]

The discussions have been to transfer that registry here to be in a centralized location closer to the source. I should have been more explicit.

[jdesrosiers]

Great. Then I'm 100% on board.

[karenetheridge]

100% in favour.

We could include data on what implementations (of JSON Schema or OpenAPI, etc) are known to implement the formats, as well as information about their intended use and their origin, when known/relevant.

For example, for the formats transferred from OpenAPI, there is a specific list of formats that are defined by their specification (see https://spec.openapis.org/oas/v3.1.1#data-type-format), so we can identify that document as the location of the canonical definitions for these formats.

I would also suggest we set out in advance what the criteria are for a format being added to this registry. I don't want people sending PRs for all the formats they can think of from the top of their heads; it should be formats that are already in widespread use, preferrably by more than one application/implementation. We also will want explicit descriptions of the json data type(s) that the format applies to, references to any underlying standards (e.g. RFC documents), or other clearly-defined descriptions of the syntax.

We should also create a parallel set of directories containing tests -- whose syntax we can bikeshed, but something similar to our JSON Schema Test Suite as a starting point would be good (instead of "schema", just use the format name, along with a list of data/description/valid tuples). Any submission to the format registry should be accompanied by a decent corpus of passing and failing tests in this format.

[torsknod-the-caridian]

Then there are the things I think should be in a registry. (u)int32,64,128, uuid, email etc. etc.

The int ones I would not handle in an explicit way, because there will always be one needing something more and tools have to adapt.
Better in my opinion would be to allow to specify the number of total bits, mantisse bits and fraction bits, if one really needs this.
For "float" types, it would have exponent bits instead of fraction bits and perhaps additionally the base of the exponent, defaulting to 10.
However, JSON is transmitting numbers as characters and this is just about an internal representation.
So having the range and multipleOf is already sufficient for integers.
Only for floating point values would need resolution information.

[gregsdennis]

@torsknod-the-caridian thanks for the thoughts. Can you edit that so it doesn't all look like a block-quote, please?

[gregsdennis]

Sounds like we're mostly in favor. Let's discuss what we need to get it done.

At a high level, I expect we'll need:

• A location to publish the list of registered formats
  • I do like @karenetheridge's suggestions for meta-data for each entry (defining authority, etc)
• A source file in this repo that generates the publication page
• A section in the Validation spec defining the registry, its purpose, and implementation requirements

Anything else?