3.0 refs not resolving when the second route points to any ref that is same as first

[travis5491811]

Hello, I have a strange issue i just ran into and I'm hoping you can help.
I'm using swagger-parser 6.0.1

Below is a full example. In my global.yml i have the following:

openapi: 3.0.0
info:
  title: Swagger API
  version: 48.0.0
servers:
  - url: '{scheme}://localhost:4444'
    description: Development server (uses dev data for SQL)
    variables:
      scheme:
        description: Dev API scheme.
        enum:
          - https
          - http
        default: http
  - url: '{scheme}://localhost:3333'
    description: Sandbox server (uses test data)
    variables:
      scheme:
        description: Test API scheme.
        enum:
          - https
          - http
        default: http
paths:
  /errors/item/new:
    $ref: 'paths/ops/errors/item/item_new.yml'
  /errors/item/edit:
    $ref: 'paths/ops/errors/item/item_edit.yml'

Everything for ref for /errors/item/new resolves fine. Here is a example of that file

post:
  summary: Create new item error...
  tags:
    - OPS API
      - Errors API 
        - item
  description: 'something'
  requestBody:
    content:
      application/json:
        schema:
          type: object
          required:
            $ref: 'schema_required_new.yml#/required'
          properties:
            $ref: 'schema_required_new.yml#/properties'
          example:
            $ref: 'schema_example_new.yml'
  responses:
    '200':
      description: Returns success confirmation message...
    '400':
      description: Bad request....
      content:
        application/json:
          schema:
            $ref: '../../../../models/error.yml'
    '401':
      description: Authorization information is missing or invalid...
      content:
        application/json:
          schema:
            $ref: '../../../../models/error.yml'

The problem is the next file does not resolve any refs properly that were being used by the first. In my test, the second file (/errors/item/edit:) won't resolve any files that were referenced in the first file.

Assume that /errors/item/edit: file reference is a duplicate of the first, here is example output when i run swagger-cli bundle global.yml -o openapi_full.yml -t yaml

openapi: 3.0.1
info:
  title: Swagger API
  version: 48.0.0
servers:
  - url: '{scheme}://localhost:4444'
    description: Development server (uses dev data for SQL)
    variables:
      scheme:
        description: Dev API scheme.
        enum:
          - https
          - http
        default: http
  - url: '{scheme}://localhost:3333'
    description: Sandbox server (uses test data)
    variables:
      scheme:
        description: Test API scheme.
        enum:
          - https
          - http
        default: http
paths:
  /errors/item/new:
    post:
      summary: Create new item error...
      tags:
        - OPS API - Errors API - item
      description: something
      requestBody:
        content:
          application/json:
            schema:
              type: object
              required:
                - defectType
                - date
                - itemSku
                - defectCause
                - numDefective
                - costPerOccurrence
                - itemManufacturer
                - manufacturerCredit
                - returned
                - marketplace
                - note
                - recordStatus
              properties:
                id:
                  type: string
                  description: >-
                    This should be the associated object ID of the id of the
                    error to edit when editing.
                defectType:
                  type: string
                  description: >-
                    This should be the associated object ID of the defectType
                    (same as error type).
                date:
                  type: string
                itemSku:
                  type: string
                defectCause:
                  type: string
                numDefective:
                  type: integer
                costPerOccurrence:
                  type: number
                  description: >-
                    Should autopopulate after user enter itemSku (Do lookup with
                    item-by-itemnum/:item_number)
                itemManufacturer:
                  type: string
                  description: >-
                    Should autopopulate after user enter itemSku (Do lookup with
                    item-by-itemnum/:item_number)
                manufacturerCredit:
                  type: boolean
                returned:
                  type: boolean
                marketplace:
                  type: integer
                refOrderNum:
                  type: string
                refRmaNum:
                  type: string
                note:
                  type: string
                recordStatus:
                  type: integer
              example:
                defectType: asdfasdfasdfasdfasfd
                date: '2008-09-15T15:53:00'
                itemSku: 10326
                defectCause: something test on word tyt
                numDefective: 1
                costPerOccurrence: 8
                itemManufacturer: Ningbo
                manufacturerCredit: true
                returned: false
                marketplace: Other
                refOrderNum: none
                refRmaNum: none
                note: A test note
                recordStatus: 1
      responses:
        '200':
          description: Returns success confirmation message...
        '400':
          description: Bad request....
        '401':
          description: Authorization information is missing or invalid...
          content:
            application/json:
              schema:
                required:
                  - error
                properties:
                  error:
                    type: string
                  message:
                    type: string
  /errors/item/edit:
    post:
      summary: Edit item error...
      tags:
        - OPS API - Errors API - item
      description: something
      requestBody:
        content:
          application/json:
            schema:
              type: object
              required:
                $ref: >-
                  #/paths/~1errors~1item~1new/post/requestBody/content/application~1json/schema/required
              properties:
                $ref: >-
                  #/paths/~1errors~1item~1new/post/requestBody/content/application~1json/schema/properties
              example:
                $ref: >-
                  #/paths/~1errors~1item~1new/post/requestBody/content/application~1json/schema/example
      responses:
        '200':
          description: Returns success confirmation message...
        '400':
          description: Bad request....
        '401':
          description: Authorization information is missing or invalid...

All the references in the second route are resolving improperly. They end up like 1errors~1item~1new/post/requestBody/content/application~1json/schema/required instead of resolving the reference.

I don't need them to use the same ref for anything but schema properties. If i change the reference file for schema required and schema example to be a uniq file that is not used anywhere else, the problem goes away for those, but i need to reuse the schema properties to get rid of duplicate code. I have the same problem when i moved it into model and tried to reference it there. Am I doing something wrong?

[JamesMessinger]

It sounds like you want to dereference rather than bundle your files.

Bundling creates a single file that contains only one copy of each referenced file. All other $ref pointers are rewritten to point to the same copy. This is useful because it produces the smallest possible file size without any duplication.

Dereferencing creates a single file that contains no $ref at all. Every $ref pointer is replaced with a copy of the thing that it points to. This produces a larger file with lots of duplication, but it's useful for many tools that don't understand $ref pointers.

In Swagger CLI, just pass the --dereference option. Or in Swagger Parser, call the dereference() method rather than the bundle() method.

[travis5491811]

Thanks @JamesMessinger for the update.

--dereference is a work around but like you said, the file size is much larger (5K lines and growing). I should have described in the ticket that the main reason I was concerned about this was because swagger-ui throws errors with this syntax. I'm closing this now since the issue isn't really with swagger-parser. I think the real issue is that swagger-ui/editor is not parsing valid refs.

[JamesMessinger]

Yeah, it was a known bug in Swagger Editor:
swagger-api/swagger-editor#1430

According to this, it's been fixed. Are you sure you're using the latest version?

[travis5491811]

Hey @JamesMessinger thanks for finding that. I'm going to link to it in the issue I opened in swagger-ui here.

The problem can be duplicated by pasting the YAML def in swagger-editor online but I'm not sure what version that app is running. When i test this locally using swagger-ui 3.19.0 I have the same problem....swagger does not resolve refs.

[travis5491811]

@JamesMessinger, so the issue is that I was using ref in a place that OpenAPI spec does not say it can be used. swagger-ui only supports refs in the places specified by OpenAPI.

With that said, your tool lets users use refs just about anywhere (which is awesome!). This is great because I can break up my files in may different ways (and use refs in places like required, properties, example, etc). Using dereference (via swagger-cli) makes things compliant.

Is allowing the use of refs anywhere (or at least the way im using it) and swagger-parser bundle or swagger-cli bundle --dereference resolving all the refs intentional? I just want to make sure I'm not digging myself a future whole with the way I'm building out my documentation.

[JamesMessinger]

Yes, all of my Swagger/OpenAPI libraries allow $ref pointers anywhere, and this is very much intentional. There are two reasons for this:

1. Like you, I need to be able to split my API definitions up in many different ways, and the $ref locations that are allowed by the OpenAPI Spec are simply not flexible enough

2. All of my libraries use json-schema-ref-parser under the hood, which is a general purpose JSON Schema library and is not limited to Swagger/OpenAPI files. The JSON Schema spec allows $ref pointers to be used anywhere.

As you've noticed, however, this flexibility can lead to compatibility issues with other tools. Some tools only support $ref pointers in the places that are explicitly defined in the spec. Other tools don't support $ref pointers at all. There are two solutions for this:

1. Use bundle --dereference to remove all $ref pointers. This ensures that your API definition can be used by any tool.

2. Use bundle, but be sure to build your API definition in such a way that the resulting bundle only has $ref pointers in places that are allowed by the spec. This approach is more complicated. It relies on the deterministic behavior of the bundle algorithm. The algorithm will always choose the shortest possible $ref as the "master", and all other $refs to the same value will be rewritten to point to the master $ref.