Skip to content

Updating SCD API

Jump to bottom
Benjamin Pelletier edited this page Sep 14, 2021 · 1 revision

Introduction

The DSS implementation targeted strategic deconfliction API version 0.3.5 circa Q2 2020 to support multiple demonstrations and has (as of Aug 2021) not been updated to the version sent to first ballot (0.3.15). This page tracks the work necessary to complete the upgrade to support for the latest API.

The three major buckets of work to upgrade the DSS to support the latest strategic deconfliction API are

  1. Updating the API/interface of the DSS itself
  2. Making core logic changes in the DSS and database
  3. Updating prober tests to determine when we've succeeded

Interface

The DSS backend logic resides in the "grpc-backend" binary. It exposes gRPC endpoints which are called by the "http-gateway" binary based on HTTP requests that http-gateway translates into gRPC requests. The Protocol buffer definitions ("protos") used by both grpc-backend and http-gateway are generated automatically from the OpenAPI specification (yaml) using the protos target in the root Makefile. The source OpenAPI specification is located in interfaces/astm-utm/Protocol as a submodule reference to the canonical repository at a certain commit.

Update submodule

The first step to updating the DSS interface is to update the git submodule pointing to the strategic deconfliction API so that it references the latest API version rather than 0.3.5. After this change is made, changes to this repository should be merged to a scd_api_update branch rather than master because this is a breaking change until the upgrade to 0.3.15 is complete.

Update adjuster

In the process of converting utm.yaml into protos, the Makefile calls the adjuster tool which makes custom changes to the YAML before generating the protos from it. These custom adjustments will probably need to be changed for the new API.

Update business objects

After appropriate protos and generated Go code are updated successfully, the business object definitions must be updated (especially the API-object conversion routines), as well as the API conversion routines for converting in the other direction.

Update handler signatures

Finally, the handler signatures must be updated to accommodate the updated objects. The logic does not need to be updated at this point; when logic would need changes based on different fields available on the business objects, instead just add a TODO comment. This bucket of updates will be considered complete when the endpoints can be called successfully, regardless of whether they behave correctly.

Core logic

Some changes to the API necessitate changes in the core logic of the DSS handlers; this is what constitutes the work in this bucket. For instance, the DSS now needs to track the state of operational intents to determine whether some actions may be performed, and this will require an addition to the data stored in the database. This bucket of updates will be considered complete when all the updated tests pass.

Prober tests

The prober test suite makes calls to the DSS under many different scenarios and these tests must be updated to support the latest API. Unlike the DSS itself which will no longer support the 0.3.5 API after this update, we would like to retain the ability to perform prober tests on 0.3.5 DSS instances after this update. Therefore, we have added the ability to indicate which API version(s) to which a test applies. This bucket of work should eventually update all existing tests to support the latest API, but the tests which check whether the API interface is functional (apart from the handler logic) should be prioritized first.

Updating a test

If a test is identical under the latest API as under 0.3.5, the test can be updated by adding the new API version to the list of API versions the test supports; e.g. @for_api_versions(scd.API_0_3_5) would change to @for_api_versions(scd.API_0_3_5, scd.API_0_3_15). If most of the test would remain the same under the latest API as 0.3.5, then the change above can be augmented with a condition based on scd_api; e.g.:

@for_api_versions(scd.API_0_3_5, scd.API_0_3_15)
def test_ensure_clean_workspace(scd_api, scd_session):
    if scd_api == scd.API_0_3_5:
        # Do old thing
    elif scd_api == scd.API_0_3_15:
        # Do new thing
    else:
        raise ValueError('Unsupported SCD API version: {}'.format(scd_api))

If the test is now completely different in the latest API but still fits within the sequence it is a part of, a new test can be introduced. E.g.:

@for_api_versions(scd.API_0_3_5)
def test_ensure_clean_workspace_0_3_5(scd_api, scd_session):
    # Do old thing

@for_api_versions(scd.API_0_3_15)
def test_ensure_clean_workspace_0_3_15(scd_api, scd_session):
    # Do new thing

In any of the cases above, make sure that dependent tests are updated as well; for instance, do not update a test that produces a mutation (new operation, etc) without updating the test that ensures a clean workspace before starting and the test that cleans up the mutation in the same PR.

If the entire test sequence is now different in the latest API, rename the Python file with the sequence according to API version (e.g., test_operation_simple.py -> test_operation_simple_0_3_5.py) and duplicate that file with a 0_3_15 suffix to implement the new test sequence for 0.3.15. Make sure all the @for_api_versions decorators in the new file are accurate.

The choice of which option above to use will be a judgement choice. We want to maintain maximum maintainability (suggesting re-use whenever possible and avoiding copy/paste repeating the same code in two places), but also readability (suggesting that a bunch of very different things should not happen in the same test definition depending on API version). When in doubt, we should err on the side of not repeating copy/paste code.

Clone this wiki locally