Refactor hub-js, add value mirroring for specified backend

[mszostok]

Description

Changes proposed in this pull request:

• Refactor hub-js,
  • add prettier and eslint (also enable check on CI)
  • split code to dedicated pkgs/files
  • code hardening
• Add value mirroring for specified backend. With proper rollback in case of any error during transaction.

  NOTE: Because get query was not changed, I still need to save each value into built-in storage. In the follow-up PR I will add support for get/update/delete/lock/unlock and also remove storing value in built-in storage when not needed.

• Add gRPC client service in Local Hub
• Remove duplicated code for createTypeInstance and instead delegate it to createTypeInstances mutation. Because of that I had to change the return type from TypeInstance to { id, alias}
• Update Local Hub GraphQL mutations and queries with backend.context property. I updated Go Client, GraphQL, Local Hub, examples to use new syntax:
  • Mutation

    mutation CreateTypeInstances {
      createTypeInstances(
        in: {
         typeInstances: [
          {
            alias: "helm-release"
            typeRef: { path: "cap.type.helm.chart.release", revision: "0.1.0" }
            value: {
              key: "test" # same as it was
            }
            backend: {
              id: "123" # it's like that already.
              context: { # new property of type `Any!`
                name: "release-name",
                namespace: "release-namespace",
              }
            }
          }
        ]
      }
      ) {
        id
        alias
      }
    }

  • Query

    query GetTypeInstance($typeInstanceID: ID!) {
      typeInstance(id: $typeInstanceID) {
        id
        backend { # current backend entry
          id
          abstract
        }    
        latestResourceVersion {
            resourceVersion
            spec {
              value
            }
            backend {
              context { # new property of type `Any!`
                name
                namespace
              }
            }
        }
      }
    }

Testing

The test that I wrote, proves that it works e2e (Go client -> GraphQL -> Local Hub -> External Storage).

1. Run neo4j:

   docker run -d \
     -p 7687:7687 -p 7474:7474 \
     -e "NEO4J_AUTH=neo4j/okon" \
     -e "NEO4JLABS_PLUGINS=[\"apoc\"]" \
     --name hub-neo4j-instance \
     ghcr.io/capactio/neo4j:4.2.13-apoc

2. Run modified storage server:

   APP_LOGGER_LEVEL="debug" APP_LOGGER_DEV_MODE=true APP_SUPPORTED_PROVIDERS="dotenv" go run ./cmd/secret-storage-backend/main.go

3. Run Local Hub:

   cd hub-js; APP_NEO4J_ENDPOINT=bolt://localhost:7687 APP_NEO4J_PASSWORD=okon APP_HUB_MODE=local npm run dev; cd ..

4. Run test:

   GRPC_SECRET_STORAGE_BACKEND_ADDR="0.0.0.0:50051" go test ./pkg/hub/client/local/ -v -count 1

   You should get similar output:

   === RUN   TestThatShowcaseExternalStorage
               TYPE INSTANCE ID                ALIAS                TYPE                  DATA FROM GQL                       BACKEND                           BACKEND CONTEXT            DATA IN EXTERNAL BACKEND
   ---------------------------------------+--------------+------------------------+---------------------------+--------------------------------------+-----------------------------------+---------------------------
     84d521b9-415e-47cd-b927-c8d97daaac62   second-child   cap.type.child:0.2.0     {"name":"Leia Organa"}      5f7505bc-ba83-4a83-a8fe-16127944dc8d   {"modified":"by-backend-storage"}   {"name":"Leia Organa"}
   ---------------------------------------+--------------+------------------------+---------------------------+--------------------------------------+-----------------------------------+---------------------------
     47897035-150b-4d5b-8d95-26c279aed72d   child          cap.type.child:0.1.0     {"name":"Luke Skywalker"}   318b99bd-9b26-4bc1-8259-0a7ff5dae61c   null
                                                                                                                (abstract)
   ---------------------------------------+--------------+------------------------+---------------------------+--------------------------------------+-----------------------------------+---------------------------
     e9eec931-63b8-48c4-80a8-ff1dde970368   parent         cap.type.complex:0.1.0   {"name":"Darth Vader"}      5f7505bc-ba83-4a83-a8fe-16127944dc8d   {"provider":"dotenv"}               {"name":"Darth Vader"}
   ---------------------------------------+--------------+------------------------+---------------------------+--------------------------------------+-----------------------------------+---------------------------
   --- PASS: TestThatShowcaseExternalStorage (3.26s)
   PASS
   ok  	capact.io/capact/pkg/hub/client/local	3.544s

5. Manually remove dotenv storage data: rm -rf /tmp/capact

6. Modify the relations in test /pkg/hub/client/local/client_test.go from:

   	    UsesRelations: []*gqllocalapi.TypeInstanceUsesRelationInput{
   		    {From: "parent", To: "child"},
   		    {From: "parent", To: "second-child"},
   	    },

   to

   	    UsesRelations: []*gqllocalapi.TypeInstanceUsesRelationInput{
   		    {From: "parent", To: "child"},
   		    {From: "parent", To: "raise-an-error-child"},
   	    },

7. Run tests:

   GRPC_SECRET_STORAGE_BACKEND_ADDR="0.0.0.0:50051" go test ./pkg/hub/client/local/ -v -count 1

   You should observe an error in transaction after storing data in external system, but the related data should be automatically deleted in this case, leaving the system in the expected state.

8. Check that no data is stored: ls /tmp/capact

Related issue(s)

• Implement gRPC delegation in Local Hub #645

[pkosiec]

👍 Good work! I have a few comments regarding app behavior and project structure.

[pkosiec]

LGTM 🚀 Before merge, please just address one last comment from my side 🙂