cache loaded remote schemas

[bonnyr]

when resolving components for loaded schemas and when external resolution is enabled,
kin-openapi will read the remote schema, parse it and resolve it in order to satisfy the reference attribute in the calling schema, but it does so every time a reference needs resolving leading
to unnecessary network traffic request for the same remote schema.

For example:

components:
  schemas:
    AnotherTestSchema:
      type: object
      properties:
        ref1:
          "$ref": http://localhost:7965/components.openapi.yml#/components/schemas/CustomTestSchema
        ref2:
          "$ref": http://localhost:7965/components.openapi.yml#/components/schemas/Name

In the above case, external resolution for components.openapi.yml is performed when parsing the
fragment above only to be done again for ref2. This is recursive and if components.openapi.yml
includes other references they are resolved the same way.

This very quickly leads to major network traffic which at the very least is unnecessary.

This pull request caches the loaded schemas in the swagger loader and returns resolved schemas from the cache once they are loaded.

Added tests to ensure the schemas are loaded only once.

[danielgtaylor]

Note that since this is only initialized/cleared once at creation there is the possibility that you could introduce unwanted behavior for existing users. For example, if you re-use the loader instance to reload an OpenAPI doc and expect to pull in the updated references as well this will fail after the proposed change.

I just checked and this is not a problem for API Sprout since it always creates a new loader. Just calling out that it might be problematic for others.

[fenollp]

Could you mention this in the docs (for example as the documentation for this very function?). It's better than buried in a PR comments :)

Also, could you not initialize the map here but instead check for nil in resolveComponent below? This would save allocation in the case the user does not rely on remote schemas.

[fenollp]

Ah well @danielgtaylor you're not the author... Thanks for pointing this out!

[fenollp]

I understand this can be a major PITA and am very curious how this is supposed to be handled by OpenAPI / JSON Schema specs?
Surely caching of HTTP content should at least honor the HTTP cache headers?
Are remote schemas expected to be assumed to never change?

Introducing remote refs immutability may break things, introduce races... so I'm leaning towards "these just have to be immutable". I just can't find the part of the documentation mentioning this?

[fenollp]

@bonnyr Hi! I think this should get in. Do you mind rebasing? Thanks

Actually there now is a swaggerLoader.LoadSwaggerFromURIFunc so just adding an implementation of LoadSwaggerFromURIFunc that relies on an outside cache var (maybe behind a mutex) sounds like an appropriate solution and would make for a great example that others can derive from for their own cache semantics needs.

[fenollp]

See https://pkg.go.dev/github.com/getkin/kin-openapi/openapi3#ReadFromURIFunc and specifically https://pkg.go.dev/github.com/getkin/kin-openapi/openapi3#URIMapCache which is used by default:

var DefaultReadFromURI = URIMapCache(ReadFromURIs(ReadFromHTTP(http.DefaultClient), ReadFromFile))