Merge pull request #582 from axone-protocol/feat/connectors

docs/architecture/governance/rule-of-form.md

@@ -4,7 +4,7 @@ sidebar_position: 4
 
 # The rule of form
 
-The rule of form refers to the principle that governance must adhere to specific formalities to be valid, compliant, and recognized. Within the Axone protocol, governance follows particular formalism which is described by an [ontology](/architecture/ontology/okp4-ontology).
+The rule of form refers to the principle that governance must adhere to specific formalities to be valid, compliant, and recognized. Within the Axone protocol, governance follows particular formalism which is described by an [ontology](/architecture/ontology/axone-ontology).
 
 One essential formality is that Axone governance is recorded directly on the blockchain, ensuring transparent access for all. Apart from security, this formality guarantees predictability and determinism, ensuring that processes and regulations are unambiguous for all participating parties.
 

docs/connectors/_category_.json

@@ -0,0 +1,4 @@
+{
+  "label": "Connectors",
+  "position": 3
+}

docs/connectors/connectors.mdx

@@ -0,0 +1,35 @@
+---
+sidebar_position: 1
+---
+
+# Connectors
+
+This section aims to describe how to connect external resources (e.g. storage service, computation resources) with the Axone protocol.
+
+## Overview
+
+The Axone protocol is designed to be a general-purpose protocol that can be used to connect various resources, we talk about XaaS (i.e. Anything as a Service).
+
+Each resource shall be referenced and described in the protocol through verifiable credential carrying claims aligned with the [Axone Ontology](/architecture/ontology/axone-ontology), along with and their [Governance](/architecture/governance/introduction) rules expressed as Prolog programs.
+
+When multiple resources must interact with each other, they shall be exposed and communicate in a way that allows them to authorize access after verifying:
+
+- The authenticity of their claimed identity;
+- The rules under which they're governed allows the requested interactions;
+
+This authentication & authorization is the responsibility of `Connectors`, that will act as authentication & authorization gateways for the resources.
+
+## Axone SDK
+
+To ease the development of connectors, and more generally interact with the Axone protocol in a programmatic way, we provide an [SDK](https://github.com/axone-protocol/axone-sdk) in Golang.
+
+The SDK provides a set of tools covering different aspects of the Axone Architecture:
+
+- `Verifiable Credentials`: issuance, parsing and verification;
+- `Dataverse`: Submit claims & retrieve information on resources;
+- `Authentication`: Authenticate identities against the Axone protocol;
+- `Authorization`: Query resources governance for authorization purposes by evaluating on-chain Prolog rules;
+- `Keys`: Helper primitives to manage cryptographic keys, used to sign/verify claims.
+- `Proxy`: Provide connectors logic to wire them with the Axone protocol, along with means to expose them through APIs.
+
+To give you an exhaustive idea of the features it offers here is its [documentation](https://pkg.go.dev/github.com/axone-protocol/axone-sdk).

docs/connectors/storage.mdx

@@ -0,0 +1,174 @@
+---
+sidebar_position: 2
+---
+
+# Storage Connector
+
+This section aims to describe how you can build a storage provider connector for the Axone protocol. We will implement here a simple storage proxy using the Axone SDK and expose it over HTTP.
+
+## Functional Specification
+
+The storage proxy will allow the following features:
+
+- `Authentication`: Given an authentication verifiable credential, the proxy will authenticate the identity;
+- `Read`: Allow or not read access to a stored resource based on the governance rules of both the storage service & the requested resource;
+- `Store`: Allow or not to store a resource based on the governance rules of the storage service, issuing a verifiable credential expressing the publication of the stored resource;
+
+## Proxy implementation
+
+To instantiate the proxy we'll need first some element to provide.
+
+In order to issue credentials verifiable on-chain, the storage service must have a set of keys to provide cryptographic proofs, let's create them:
+
+```go
+key, err := keys.NewKeyFromMnemonic(mnemonic)
+```
+
+It'll need also to establish a connection with the Axone blockchain to be able to interact with it, let's create a client for that purpose:
+
+```go
+dataverseClient, err := dataverse.NewQueryClient(
+    context.Background(),
+    nodeGrpcAddr, // The gRPC address of an Axone node
+    dataverseAddr, // The address of the dataverse smart contract
+    grpc.WithTransportCredentials(insecure.NewCredentials()), // Use insecure credentials for demo purposes
+)
+```
+
+For the management of verifiable credentials the proxy will need to resolve JSON-LD contexts, let's create a resolver for that purpose:
+
+```go
+documentLoader := ld.NewCachingDocumentLoader( // Let's cache the resolved documents
+    ld.NewDefaultDocumentLoader(nil), // The default document loader will fetch documents using an HTTP client
+)
+```
+
+We now have all the elements to instantiate the proxy:
+
+```go
+storageProxy, err := storage.NewProxy(
+    ctx,
+    key,
+    baseURL, // the base URL on which the proxy will be exposed, it'll be used to issue publication credentials
+    dataverseClient,
+    documentLoader,
+    func(ctx context.Context, resourceID string) (io.Reader, error) {
+        // Here should lie the logic to retrieve a resource from the storage service.
+        // At this point, the proxy has already authenticated and authorize the identity and checked the governance rules.
+        // You can implement the read logic with a database, a file system or whatever suits your needs.
+    },
+    func(ctx context.Context, resourceID string, data io.Reader) error {
+        // Here should lie the logic to store a resource in the storage service.
+        // At this point, the proxy has already authenticated and authorize the identity and checked the governance rules.
+        // You can implement the store logic with a database, a file system or whatever suits your needs.
+        // The issuance of the publication credential is handled by the proxy. You do not need to manage that.
+    },
+)
+```
+
+## Expose over HTTP
+
+At this point we have a fully functional storage proxy, but it is not available through any communication protocol, let's expose it over HTTP.
+
+We'll expose an API with three endpoints:
+
+- `POST /authenticate`: Authenticate an identity given a verifiable credential and return a JWT token that must be used to access other endpoints;
+- `GET /{resourceID}`: Read a resource given its DID, the JWT token must be provided in the `Authorization` header as a bearer token;
+- `POST /{resourceID}`: Store a resource given its DID, the JWT token must be provided in the `Authorization` header as a bearer token;
+
+```go
+err := http.NewServer( // The sdk provide a configurable HTTP server
+    listenAddr, // The address on which the server will listen (e.g. "0.0.0.0:8080")
+    storageProxy.HTTPConfigurator( // the proxy will provide the necessary options to properly configure the server, you can combine them with your own
+        jwtSecretKey, // The secret key used to sign the JWT tokens
+        jwtDuration, // The duration of the JWT token validity
+    ),
+).Listen() // Start the server, returning an error if something went wrong
+```
+
+## Full source
+
+Until now we saw only separate snippets for pedagogic purpose, let's see what it looks like when we put everything together in a single `main.go`:
+
+```go
+package main
+
+import (
+	"context"
+	"fmt"
+	"io"
+	"os"
+	"time"
+
+	"github.com/axone-protocol/axone-sdk/dataverse"
+	"github.com/axone-protocol/axone-sdk/http"
+	"github.com/axone-protocol/axone-sdk/keys"
+	"github.com/axone-protocol/axone-sdk/provider/storage"
+	"github.com/piprate/json-gold/ld"
+	"google.golang.org/grpc"
+	"google.golang.org/grpc/credentials/insecure"
+)
+
+func main() {
+	mnemonic := os.Args[1]
+	nodeGrpcAddr := os.Args[2]
+	dataverseAddr := os.Args[3]
+	baseURL := os.Args[4]
+	listenAddr := os.Args[5]
+	jwtSecretKey := []byte(os.Args[6])
+	jwtDuration, err := time.ParseDuration(os.Args[7])
+	if err != nil {
+		panic(err)
+	}
+
+	key, err := keys.NewKeyFromMnemonic(mnemonic)
+	if err != nil {
+		panic(err)
+	}
+
+	dataverseClient, err := dataverse.NewQueryClient(
+		context.Background(),
+		nodeGrpcAddr,
+		dataverseAddr,
+		grpc.WithTransportCredentials(insecure.NewCredentials()),
+	)
+	if err != nil {
+		panic(err)
+	}
+
+	storageProxy, err := storage.NewProxy(
+		context.Background(),
+		key,
+		baseURL,
+		dataverseClient,
+		ld.NewCachingDocumentLoader(ld.NewDefaultDocumentLoader(nil)),
+		func(ctx context.Context, id string) (io.Reader, error) {
+			return os.Open("/data/" + id)
+		},
+		func(ctx context.Context, s string, reader io.Reader) error {
+			content, err := io.ReadAll(reader) // Don't do that...
+			if err != nil {
+				return err
+			}
+
+			return os.WriteFile("/data/"+s, content, 0o644)
+		},
+	)
+	if err != nil {
+		panic(err)
+	}
+
+	err = http.NewServer(
+		listenAddr,
+		storageProxy.HTTPConfigurator(jwtSecretKey, jwtDuration),
+	).Listen()
+
+	fmt.Println("Server stopped:", err)
+}
+```
+
+Don't take this implementation as production-grade, it suffers from many issues. It is only a starting point to build your own storage connector.
+
+## Real-world example
+
+As an example, and reference implementation of a complete storage connector, we provide a simple implementation of a connector for an S3 storage service: https://github.com/axone-protocol/s3-auth-proxy.