Feature suggestion: "Backlinks" - Links defined the inverse of the way they are currently

[anentropic]

I would like the be able to define Links, but in the inverse of the way Links are currently implemented.

Take the example from the docs:

paths:
  /users:
    post:
      responses:
        '201':
          links:
            GetUserByUserId:   # <---- arbitrary name for the link
              operationId: getUser
              # or
              # operationRef: '#/paths/~1users~1{userId}/get'
              parameters:
                userId: '$response.body#/id'
              description: >
                The `id` value returned in the response can be used as
                the `userId` parameter in `GET /users/{userId}`.

If you imagine a system of microservices with many interlinked API schemas... There are going to be dozens of other endpoints in different services which can make use of the userId that results from the POST /users operation. Some of these services will be maintained by different teams, and their API schemas will be in a separate repos etc.

In that context it feels to me quite "back to front" to collect all of these links under the POST /users operation responses, and that instead it would make more sense to define the link on the other side of the relation.

For example on the GET /users/{userId} operation, it would be awesome to have a way to specify that {userId} can be obtained from #/paths/~1users~1/post/responses/201/content/id.

(Possibly as an extension to the Parameters? it would also be useful to be able to set fields in the request body this way, as suggested here ).

In these cases it would be much easier to maintain if the "consumer" API specifies where to get its prerequisites from, rather than having the "producer" API try to list all of the places its data can potentially be used, which is what the Links feature provides currently.

possibly related: #2122 #1594 #1593

[anentropic]

It would also be better for tooling (e.g. automated testing etc)

If you want to make a GET /users/{userId} request a tool can just start with the spec where that endpoint is defined and navigate upwards (through the proposed Parameter value links) to determine which prior requests have to be made as prerequisites, and how to do so.

The other way around, as currently, it's not possible. Because, potentially, the tool does not know the location of the doc where the POST /users endpoint is defined. So it can't find the Links which specify how to make a GET /users/{userId} request using the userId from the other request.

[tomgreen98]

This is exactly what Im looking for. We need to start at an api and work our way back through a list of dependencies setting up everything so we can then run the api we started with.

[mkistler]

+1 to this enhancement request.

We have actually already implemented this capability in our tooling with a spec extension, x-linkback, which we describe as:

An annotation on a parameter or request body property to reference a link object in /components/links that represents the source of the value for this parameter/property.

This is a much more compact way of expressing links, particularly when coupled with referenced parameters -- a single link component and a single x-linkback on a referenced parameter can efficiently express a link to any number of dependent operations.

[anentropic]

Since posting the original issue I have also developed an implementation of this idea for use in some tooling I'm building

I will document it and post back here with a concrete proposal

[anentropic]

For context I am in early stages of building tooling around automation of integration tests for app backends which span multiple services.

One important building block, for this tool to be able to cut out manual test-definition boilerplate, is to be able to derive a dependency graph of the endpoints which are exposed across multiple services. OpenAPI's ability to have URI references which span across independent API docs is very useful for this.

However I found three areas where vanilla OpenAPI 3.0 is insufficient:

1. can't link values from a response into fields of a downstream request body... as described in this separate issue Using Links to set a field in the target operation's request body #1594
2. the current "forward-pointing" Link feature means that upstream operations need to specify all their downstream dependents. I think it would be far more maintainable to invert this relationship and have downstream operations define their upstream prerequisites. That is the current thread discussed here.

I think these first two would be generally useful for many use-cases. The last one is maybe a bit more specific to my particular tool use-case:

3. where there are multiple independent upstream operations that can lead to a single downstream operation (i.e. upstream operations which are not required together, an OR rather than an AND), I would like a way to uniquely identify the required-together groups of them.

I have documented here the extension properties I defined in order to get this library working:
https://apigraph.readthedocs.io/en/latest/reference/openapi-extensions.html

(it is still WIP but I can now successfully derive a dependency graph at least)

The parts that are relevant to this thread are x-apigraph-backlinks properties on both Operation and Components objects, and the new Backlink object.

You may ignore the parts about requestBodyParameters and chainId and treat those as separate issues (although for my purpose they are all related and necessary).

[anentropic]

And a basic example:

openapi: 3.0.0
paths: 
   /2.0/users/{username}:
     get:
       operationId: getUserByName
       parameters:
       - name: username
         in: path
         required: true
         schema:
           type: string
       responses:
         '200':
           description: The User
           content:
             application/json:
               schema:
                 $ref: '#/components/schemas/user'
   /repositories/{username}:
     get:
       operationId: getRepositoriesByOwner
       parameters:
         - name: username
           in: path
           required: true
           schema:
             type: string
       x-backlinks:
         Get User by Username:
           operationId: getUserByName
           response: "200"
           parameters:
             username: $response.body#/username

This shows a "backlink" from the getRepositoriesByOwner operation to its prerequisite, the "200" response of the getUserByName operation.

We use a runtime expression (as found elsewhere in OpenAPI) to get a value from the getUserByName response body ($response.body#/username) and use it as the value of the getRepositoriesByOwner operation's username parameter.

[luckybroman5]

After applying OAS extensively towards many different applications, like AWS API Gateway, client generation, documentation, design, and many other things, I MUST request that this be brought to the forefront of discussion for the TSC.

Of the many proposals around this topic, any would do. I don't think the problem is the how, it's the why. Therefore rather than being noise amongst the other great proposals on this thread, I will advocate for why this is a paramount need for OAS.

Documentation

Every set of backend services, whether micro or monolithic, has operations dependent upon responses from the others. It's almost safe to say, that for documentation purposes, if there are no fragments of a request body that aren't from another operation's response body, the two requests don't belong in the same spec because they'd have nothing to do with each other.

It's a common process for any type of modeling tool to granularly define relationships. Whether it be an ERD or a Function Call Graph, there needs to be a way for description.

Further, it's often ambiguous whether identically named parameters are in fact the same or sourced from other responses without explicit declaration. For one to make assumptions about parameters based off name in a large OAS inevitably leads to bugs.

Common, Practical Use Cases

1. Refresh tokens
   • Typically are returned along with an authentication token and expiration date. The refresh token is then used along with other parameters in the request body to facilitate the "refresh". Examples: The Github API (Login) -> The Github API (Refresh), at the time of this writing, it's impossible to represent this common pattern in OAS
2. Bookmarking Video Content
   • Imagine you provide a video streaming platform and want to allow users to partially watch content, then resume playing later. To achieve this at a basic level, you'll need to provide some form of identifier for the video, the user's authentication information, and the playhead. A reasonable place to put the playhead and video Id would be the request body. You'd need to get the video Id from some other meta-data based service.
3. Compare and Swap
   • If you need to update a resource that could potentially be getting updated by multiple users at the same time, a simple method would be to "Compare and Swap". If at first you must read from an endpoint to get it's "compare" value, then how would you include it along with it's "swap" value in the request body?

Design/Architecture

Hardly ever do entire request bodies comprise of a singular response attribute. In fact, if this were the case, one could argue that it's almost certainly flawed API Design. If an entire request body is to be sourced from a response body, why wouldn't the original operation that returned said response not just make the request on the client's behalf? The current implementation of having: responseBody encourages an anti pattern, and goes against modern paradigms. Should a developer who doesn't know better try to design an API completely using OAS, they might be mislead.

The Future and Beyond

The OAS and what it aims to do are truly something great. Using OAS it's possible to document web request for both the human and machine. One of the last things missing is the ability to further describe the relationships each operation might play into another. Should this "last mile" problem be solved, developers integrating with API's might not ever need to even write a line of code nor converse with it's maintainers.

@darrelmiller, @whitlockjc, @earth2marsh, @MikeRalphson, @webron, @usarid, Please do consider/reconsider this enhancement during your next TSC.

[chatelao]

Absolutlly agree with the idea, as process are incremental:
a) Base Services are built first (returing IDs, e.g.: a BankAccounId)
b) Extended Services are build later (e.b.: a Payment using the BankAccount)

=> Forward referencing adds unnecessary complexity to common feature.
=> Backward linking solves this problem in a very elegant way.

Nice to have:

• If implicit referencing with unique parameter names could be a lean solurtion, if the operaton->operatation flow is easily defined.

[ochatelain]

Would be great if Links on Objects would be possible forward and backwar:

• Allow Links to Schema objects (Primitives/Objects/...) #3228

[handrews]

This scale of change is best discussed in Moonwalk (and can be backported if relevant and we decide to keep the 3.x line going). Closing in favor of:

• Improvements for links in v4 sig-moonwalk#58