Skip to content

Commit

Permalink
Endorser doc updates and some bug fixes
Browse files Browse the repository at this point in the history
Signed-off-by: Ian Costanzo <[email protected]>
  • Loading branch information
ianco committed Sep 2, 2022
1 parent 3783ee8 commit 2d508e0
Show file tree
Hide file tree
Showing 16 changed files with 324 additions and 12 deletions.
43 changes: 41 additions & 2 deletions Endorser.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Transaction Endorser Support

Note that the ACA-Py transaction support is in the process of code refactor and cleanup. The following documents the current state, but is subject to change.

ACA-Py supports an [Endorser Protocol](https://github.com/hyperledger/aries-rfcs/pull/586), that allows an un-privileged agent (an "Author") to request another agent (the "Endorser") to sign their transactions so they can write these transactions to the ledger. This is required on Indy ledgers, where new agents will typically be granted only "Author" privileges.

Transaction Endorsement is built into the protocols for Schema, Credential Definition and Revocation, and endorsements can be explicitely requested, or ACA-Py can be configured to automate the endorsement workflow.
Expand Down Expand Up @@ -61,3 +59,44 @@ Endorsement:
For Authors, specify whether to automatically promote a DID to the wallet public DID after writing to the ledger.
```

## How Aca-py Handles Endorsements

Internally, the Endorsement functionality is implemented as a protocol, and is implemented consistently with other protocols:

- a [routes.py](https://github.com/hyperledger/aries-cloudagent-python/blob/main/aries_cloudagent/protocols/endorse_transaction/v1_0/routes.py) file exposes the admin endpoints
- [handler files](https://github.com/hyperledger/aries-cloudagent-python/tree/main/aries_cloudagent/protocols/endorse_transaction/v1_0/handlers) implement responses to any received Endorse protocol messages
- a [manager.py](https://github.com/hyperledger/aries-cloudagent-python/blob/main/aries_cloudagent/protocols/endorse_transaction/v1_0/manager.py) file implements common functionality that is called from both the routes.py and handler classes (as well as from other classes that need to interact with Endorser functionality)

The Endorser makes use of the [Event Bus](https://github.com/hyperledger/aries-cloudagent-python/blob/main/CHANGELOG.md#july-14-2021) (links to the PR which links to a hackmd doc) to notify other protocols of any Endorser events of interest. For example, after a Credential Definition endorsement is received, the TransactionManager writes the endorsed transaction to the ledger and uses teh Event Bus to notify the Credential Defintition manager that it can do any required post-processing (such as writing the cred def record to the wallet, initiating the revocation registry, etc.).

The overall architecture can be illustrated as:

![Class Diagram](./docs/assets/endorser-design.png)

### Create Credential Definition and Revocation Registry

An example of an Endorser flow is as follows, showing how a credential definition endorsement is received and processed, and optionally kicks off the revocation registry process:

![Sequence Diagram](./docs/assets/endorse-cred-def.png)

You can see that there is a standard endorser flow happening each time there is a ledger write (illustrated in the "Endorser" process).

At the end of each endorse sequence, the TransactionManager sends a notification via the EventBus so that any dependant processing can continue. Each Router is responsibiel for listening and responding to these notifications if necessary.

For example:

- Once the credential definition is created, a revocation registry must be created (for revocable cred defs)
- Once the revocation registry is created, a revocation entry must be created
- Potentially, the cred def status could be updated once the revocation entry is completed

Using the EventBus decouples the event sequence. Any functions triggered by an event notification are typically also available directly via Admin endpoints.

### Create DID and Promote to Public

... and an example of creating a DID and promoting it to public (and creating an ATTRIB for the endpoint:

![Sequence Diagram](./docs/assets/endorse-public-did.png)

You can see the same endorsement processes in this sequence.

Once the DID is written, the DID can (optionally) be promoted to the public DID, which will also invoke an ATTRIB transaction to write the endpoint.
16 changes: 15 additions & 1 deletion aries_cloudagent/ledger/indy.py
Original file line number Diff line number Diff line change
Expand Up @@ -770,6 +770,8 @@ async def update_endpoint_for_did(
else None
)

LOGGER.info(f">>> exist_endpoint_of_type = {exist_endpoint_of_type}")
LOGGER.info(f">>> endpoint = {endpoint}")
if exist_endpoint_of_type != endpoint:
if await self.is_ledger_read_only():
raise LedgerError(
Expand All @@ -783,6 +785,9 @@ async def update_endpoint_for_did(
)

with IndyErrorHandler("Exception building attribute request", LedgerError):
LOGGER.info(
f">>> calling build_attrib_request() with {nym} {attr_json}"
)
request_json = await indy.ledger.build_attrib_request(
nym, nym, None, attr_json, None
)
Expand All @@ -791,17 +796,26 @@ async def update_endpoint_for_did(
request_json = await indy.ledger.append_request_endorser(
request_json, endorser_did
)
LOGGER.info(
f">>> calling _submit() with {request_json}, True, etc ..."
)
resp = await self._submit(
request_json,
True,
sign=True,
sign_did=public_info,
write_ledger=write_ledger,
)
if not write_ledger:
LOGGER.info(f">>> Returning ... signed_txn {resp}")
return {"signed_txn": resp}

LOGGER.info(f">>> calling _submit() with {request_json}")
await self._submit(request_json, True, True)
return True

else:
LOGGER.info(">>> NOT updating ledger endpoint!")

return False

async def register_nym(
Expand Down
10 changes: 9 additions & 1 deletion aries_cloudagent/protocols/endorse_transaction/v1_0/manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,10 @@
from ....storage.error import StorageError, StorageNotFoundError
from ....transport.inbound.receipt import MessageReceipt
from ....wallet.base import BaseWallet
from ....wallet.util import notify_endorse_did_event
from ....wallet.util import (
notify_endorse_did_event,
notify_endorse_did_attrib_event,
)

from .messages.cancel_transaction import CancelTransaction
from .messages.endorsed_transaction_response import EndorsedTransactionResponse
Expand Down Expand Up @@ -794,6 +797,11 @@ async def endorsed_txn_post_processing(
did = ledger_response["result"]["txn"]["data"]["dest"]
await notify_endorse_did_event(self._profile, did, meta_data)

elif ledger_response["result"]["txn"]["type"] == "100":
# write DID ATTRIB to ledger
did = ledger_response["result"]["txn"]["data"]["dest"]
await notify_endorse_did_attrib_event(self._profile, did, meta_data)

else:
# TODO unknown ledger transaction type, just ignore for now ...
pass
7 changes: 7 additions & 0 deletions aries_cloudagent/wallet/indy.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""Indy implementation of BaseWallet interface."""

import json
import logging

from typing import List, Sequence, Tuple, Union

Expand Down Expand Up @@ -36,6 +37,8 @@
from .util import b58_to_bytes, bytes_to_b58, bytes_to_b64


LOGGER = logging.getLogger(__name__)

RECORD_TYPE_CONFIG = "config"
RECORD_NAME_PUBLIC_DID = "default_public_did"

Expand Down Expand Up @@ -720,6 +723,7 @@ async def set_did_endpoint(
endpoint_type: the type of the endpoint/service. Only endpoint_type
'endpoint' affects local wallet
"""
LOGGER.info(f">>> in set_did_endpoint() with {did} and {endpoint}")
did_info = await self.get_local_did(did)
if did_info.method != DIDMethod.SOV:
raise WalletError("Setting DID endpoint is only allowed for did:sov DIDs")
Expand All @@ -731,6 +735,7 @@ async def set_did_endpoint(
metadata[endpoint_type.indy] = endpoint

wallet_public_didinfo = await self.get_public_did()
LOGGER.info(f">>> wallet_public_didinfo = {wallet_public_didinfo}")
if (
wallet_public_didinfo and wallet_public_didinfo.did == did
) or did_info.metadata.get("posted"):
Expand All @@ -741,6 +746,7 @@ async def set_did_endpoint(
)
if not ledger.read_only:
async with ledger:
LOGGER.info(">>> calling update_endpoint_for_did() ...")
attrib_def = await ledger.update_endpoint_for_did(
did,
endpoint,
Expand All @@ -750,6 +756,7 @@ async def set_did_endpoint(
routing_keys=routing_keys,
)
if not write_ledger:
LOGGER.info(f">>> returning attrib_def {attrib_def}")
return attrib_def

await self.replace_local_did_metadata(did, metadata)
Expand Down
78 changes: 71 additions & 7 deletions aries_cloudagent/wallet/routes.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from ..ledger.error import LedgerConfigError, LedgerError
from ..messaging.models.base import BaseModelError
from ..messaging.models.openapi import OpenAPISchema
from ..messaging.responder import BaseResponder
from ..messaging.valid import (
DID_POSTURE,
ENDPOINT,
Expand Down Expand Up @@ -442,6 +443,17 @@ async def wallet_set_public_did(request: web.BaseRequest):
connection_id = request.query.get("conn_id")
attrib_def = None

# check if we need to endorse
if is_author_role(context.profile):
# authors cannot write to the ledger
write_ledger = False
create_transaction_for_endorser = True
if not connection_id:
# author has not provided a connection id, so determine which to use
connection_id = await get_endorser_connection_id(context.profile)
if not connection_id:
raise web.HTTPBadRequest(reason="No endorser connection found")

wallet = session.inject_or(BaseWallet)
if not wallet:
raise web.HTTPForbidden(reason="No wallet available")
Expand All @@ -462,6 +474,7 @@ async def wallet_set_public_did(request: web.BaseRequest):
routing_keys = mediation_record.routing_keys

try:
LOGGER.info(">>> calling promote_wallet_public_did() from route ...")
info, attrib_def = await promote_wallet_public_did(
context.profile,
context,
Expand Down Expand Up @@ -534,6 +547,7 @@ async def promote_wallet_public_did(

# check if we need to endorse
if is_author_role(context.profile):
LOGGER.info(">>> IS author ...")
# authors cannot write to the ledger
write_ledger = False

Expand All @@ -542,6 +556,8 @@ async def promote_wallet_public_did(
connection_id = await get_endorser_connection_id(context.profile)
if not connection_id:
raise web.HTTPBadRequest(reason="No endorser connection found")
else:
LOGGER.info(">>> IS NOT author ...")

if not write_ledger:
try:
Expand Down Expand Up @@ -577,14 +593,19 @@ async def promote_wallet_public_did(
did_info = await wallet.get_local_did(did)
info = await wallet.set_public_did(did_info)

LOGGER.info(f">>> did_info = {did_info}")
LOGGER.info(f">>> info = {info}")

if info:
# Publish endpoint if necessary
endpoint = did_info.metadata.get("endpoint")
LOGGER.info(f">>> endpoint = {endpoint}")

if not endpoint:
async with session_fn() as session:
wallet = session.inject_or(BaseWallet)
endpoint = context.settings.get("default_endpoint")
LOGGER.info(f">>> calling wallet.set_did_endpoint() with {endpoint}")
attrib_def = await wallet.set_did_endpoint(
info.did,
endpoint,
Expand All @@ -594,11 +615,6 @@ async def promote_wallet_public_did(
routing_keys=routing_keys,
)

# Commented the below lines as the function set_did_endpoint
# was calling update_endpoint_for_did of ledger
# async with ledger:
# await ledger.update_endpoint_for_did(info.did, endpoint)

# Route the public DID
route_manager = profile.inject(RouteManager)
await route_manager.route_public_did(profile, info.verkey)
Expand Down Expand Up @@ -808,20 +824,68 @@ async def on_register_nym_event(profile: Profile, event: Event):
"""Handle any events we need to support."""

# after the nym record is written, promote to wallet public DID
LOGGER.info(f">>> got a NYM event ... {event}")
if is_author_role(profile) and profile.context.settings.get_value(
"endorser.auto_promote_author_did"
):
LOGGER.info(">>> calling promote_wallet_public_did() ...")
did = event.payload["did"]
connection_id = event.payload.get("connection_id")
try:
await promote_wallet_public_did(
info, attrib_def = await promote_wallet_public_did(
profile, profile.context, profile.session, did, connection_id
)
except Exception:
except Exception as err:
# log the error, but continue
LOGGER.exception(
"Error promoting to public DID: %s",
err,
)
return

transaction_mgr = TransactionManager(profile)
try:
transaction = await transaction_mgr.create_record(
messages_attach=attrib_def["signed_txn"], connection_id=connection_id
)
except StorageError as err:
# log the error, but continue
LOGGER.exception(
"Error accepting endorser invitation/configuring endorser connection: %s",
err,
)
return

# if auto-request, send the request to the endorser
if profile.settings.get_value("endorser.auto_request"):
try:
transaction, transaction_request = await transaction_mgr.create_request(
transaction=transaction,
# TODO see if we need to parameterize these params
# expires_time=expires_time,
# endorser_write_txn=endorser_write_txn,
)
except (StorageError, TransactionManagerError) as err:
# log the error, but continue
LOGGER.exception(
"Error creating endorser transaction request: %s",
err,
)

# TODO not sure how to get outbound_handler in an event ...
# await outbound_handler(transaction_request, connection_id=connection_id)
responder = profile.inject_or(BaseResponder)
if responder:
await responder.send(
transaction_request,
connection_id=connection_id,
)
else:
LOGGER.warning(
"Configuration has no BaseResponder: cannot update "
"ATTRIB record on DID: %s",
did,
)


async def register(app: web.Application):
Expand Down
10 changes: 10 additions & 0 deletions aries_cloudagent/wallet/util.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,9 @@ def abbr_verkey(full_verkey: str, did: str = None) -> str:


DID_EVENT_PREFIX = "acapy::ENDORSE_DID::"
DID_ATTRIB_EVENT_PREFIX = "acapy::ENDORSE_DID_ATTRIB::"
EVENT_LISTENER_PATTERN = re.compile(f"^{DID_EVENT_PREFIX}(.*)?$")
ATTRIB_EVENT_LISTENER_PATTERN = re.compile(f"^{DID_ATTRIB_EVENT_PREFIX}(.*)?$")


async def notify_endorse_did_event(profile: Profile, did: str, meta_data: dict):
Expand All @@ -111,3 +113,11 @@ async def notify_endorse_did_event(profile: Profile, did: str, meta_data: dict):
DID_EVENT_PREFIX + did,
meta_data,
)


async def notify_endorse_did_attrib_event(profile: Profile, did: str, meta_data: dict):
"""Send notification for a DID ATTRIB post-process event."""
await profile.notify(
DID_ATTRIB_EVENT_PREFIX + did,
meta_data,
)
1 change: 1 addition & 0 deletions demo/features/0453-issue-credential.feature
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
@RFC0453
Feature: RFC 0453 Aries agent issue credential

@T003-RFC0453 @GHA
Expand Down
1 change: 1 addition & 0 deletions demo/features/0586-sign-transaction.feature
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
@RFC0586
Feature: RFC 0586 Aries sign (endorse) transactions functions

@T001-RFC0586
Expand Down
9 changes: 8 additions & 1 deletion demo/features/steps/0586-sign-transaction.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,18 @@ def step_impl(context, agent_name, did_role):
)

# make the new did the wallet's public did
created_did = agent_container_POST(
published_did = agent_container_POST(
agent["agent"],
"/wallet/did/public",
params={"did": created_did["result"]["did"]},
)
if "result" in published_did:
# published right away!
pass
elif "txn" in published_did:
# we are an author and need to go through the endorser process
# assume everything works!
async_sleep(3.0)

if not "public_dids" in context:
context.public_dids = {}
Expand Down
2 changes: 2 additions & 0 deletions demo/runners/support/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -600,8 +600,10 @@ async def register_or_switch_wallet(
if self.endorser_role and self.endorser_role == "author":
if endorser_agent:
await self.admin_POST("/wallet/did/public?did=" + self.did)
await asyncio.sleep(3.0)
else:
await self.admin_POST("/wallet/did/public?did=" + self.did)
await asyncio.sleep(3.0)
elif cred_type == CRED_FORMAT_JSON_LD:
# create did of appropriate type
data = {"method": DID_METHOD_KEY, "options": {"key_type": KEY_TYPE_BLS}}
Expand Down
Binary file added docs/assets/endorse-cred-def.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 2d508e0

Please sign in to comment.