diff --git a/Endorser.md b/Endorser.md index 54f545fce3..bf3ec5da8c 100644 --- a/Endorser.md +++ b/Endorser.md @@ -1,7 +1,5 @@ # Transaction Endorser Support -Note that the ACA-Py transaction support is in the process of code refactor and cleanup. The following documents the current state, but is subject to change. - ACA-Py supports an [Endorser Protocol](https://github.com/hyperledger/aries-rfcs/pull/586), that allows an un-privileged agent (an "Author") to request another agent (the "Endorser") to sign their transactions so they can write these transactions to the ledger. This is required on Indy ledgers, where new agents will typically be granted only "Author" privileges. Transaction Endorsement is built into the protocols for Schema, Credential Definition and Revocation, and endorsements can be explicitely requested, or ACA-Py can be configured to automate the endorsement workflow. @@ -61,3 +59,44 @@ Endorsement: For Authors, specify whether to automatically promote a DID to the wallet public DID after writing to the ledger. ``` +## How Aca-py Handles Endorsements + +Internally, the Endorsement functionality is implemented as a protocol, and is implemented consistently with other protocols: + +- a [routes.py](https://github.com/hyperledger/aries-cloudagent-python/blob/main/aries_cloudagent/protocols/endorse_transaction/v1_0/routes.py) file exposes the admin endpoints +- [handler files](https://github.com/hyperledger/aries-cloudagent-python/tree/main/aries_cloudagent/protocols/endorse_transaction/v1_0/handlers) implement responses to any received Endorse protocol messages +- a [manager.py](https://github.com/hyperledger/aries-cloudagent-python/blob/main/aries_cloudagent/protocols/endorse_transaction/v1_0/manager.py) file implements common functionality that is called from both the routes.py and handler classes (as well as from other classes that need to interact with Endorser functionality) + +The Endorser makes use of the [Event Bus](https://github.com/hyperledger/aries-cloudagent-python/blob/main/CHANGELOG.md#july-14-2021) (links to the PR which links to a hackmd doc) to notify other protocols of any Endorser events of interest. For example, after a Credential Definition endorsement is received, the TransactionManager writes the endorsed transaction to the ledger and uses the Event Bus to notify the Credential Defintition manager that it can do any required post-processing (such as writing the cred def record to the wallet, initiating the revocation registry, etc.). + +The overall architecture can be illustrated as: + +![Class Diagram](./docs/assets/endorser-design.png) + +### Create Credential Definition and Revocation Registry + +An example of an Endorser flow is as follows, showing how a credential definition endorsement is received and processed, and optionally kicks off the revocation registry process: + +![Sequence Diagram](./docs/assets/endorse-cred-def.png) + +You can see that there is a standard endorser flow happening each time there is a ledger write (illustrated in the "Endorser" process). + +At the end of each endorse sequence, the TransactionManager sends a notification via the EventBus so that any dependant processing can continue. Each Router is responsible for listening and responding to these notifications if necessary. + +For example: + +- Once the credential definition is created, a revocation registry must be created (for revocable cred defs) +- Once the revocation registry is created, a revocation entry must be created +- Potentially, the cred def status could be updated once the revocation entry is completed + +Using the EventBus decouples the event sequence. Any functions triggered by an event notification are typically also available directly via Admin endpoints. + +### Create DID and Promote to Public + +... and an example of creating a DID and promoting it to public (and creating an ATTRIB for the endpoint: + +![Sequence Diagram](./docs/assets/endorse-public-did.png) + +You can see the same endorsement processes in this sequence. + +Once the DID is written, the DID can (optionally) be promoted to the public DID, which will also invoke an ATTRIB transaction to write the endpoint. diff --git a/aries_cloudagent/ledger/indy.py b/aries_cloudagent/ledger/indy.py index 6f1c06e42f..de3e8d40fe 100644 --- a/aries_cloudagent/ledger/indy.py +++ b/aries_cloudagent/ledger/indy.py @@ -793,7 +793,7 @@ async def update_endpoint_for_did( ) resp = await self._submit( request_json, - True, + sign=True, sign_did=public_info, write_ledger=write_ledger, ) @@ -802,6 +802,7 @@ async def update_endpoint_for_did( await self._submit(request_json, True, True) return True + return False async def register_nym( diff --git a/aries_cloudagent/protocols/endorse_transaction/v1_0/manager.py b/aries_cloudagent/protocols/endorse_transaction/v1_0/manager.py index 8ce93c45a8..3669a16eeb 100644 --- a/aries_cloudagent/protocols/endorse_transaction/v1_0/manager.py +++ b/aries_cloudagent/protocols/endorse_transaction/v1_0/manager.py @@ -21,7 +21,10 @@ from ....storage.error import StorageError, StorageNotFoundError from ....transport.inbound.receipt import MessageReceipt from ....wallet.base import BaseWallet -from ....wallet.util import notify_endorse_did_event +from ....wallet.util import ( + notify_endorse_did_event, + notify_endorse_did_attrib_event, +) from .messages.cancel_transaction import CancelTransaction from .messages.endorsed_transaction_response import EndorsedTransactionResponse @@ -794,6 +797,11 @@ async def endorsed_txn_post_processing( did = ledger_response["result"]["txn"]["data"]["dest"] await notify_endorse_did_event(self._profile, did, meta_data) + elif ledger_response["result"]["txn"]["type"] == "100": + # write DID ATTRIB to ledger + did = ledger_response["result"]["txn"]["data"]["dest"] + await notify_endorse_did_attrib_event(self._profile, did, meta_data) + else: # TODO unknown ledger transaction type, just ignore for now ... pass diff --git a/aries_cloudagent/wallet/indy.py b/aries_cloudagent/wallet/indy.py index bba9f6ed49..4e2406d04f 100644 --- a/aries_cloudagent/wallet/indy.py +++ b/aries_cloudagent/wallet/indy.py @@ -1,6 +1,7 @@ """Indy implementation of BaseWallet interface.""" import json +import logging from typing import List, Sequence, Tuple, Union @@ -36,6 +37,8 @@ from .util import b58_to_bytes, bytes_to_b58, bytes_to_b64 +LOGGER = logging.getLogger(__name__) + RECORD_TYPE_CONFIG = "config" RECORD_NAME_PUBLIC_DID = "default_public_did" diff --git a/aries_cloudagent/wallet/routes.py b/aries_cloudagent/wallet/routes.py index ea039df1aa..eb18163fcb 100644 --- a/aries_cloudagent/wallet/routes.py +++ b/aries_cloudagent/wallet/routes.py @@ -17,6 +17,7 @@ from ..ledger.error import LedgerConfigError, LedgerError from ..messaging.models.base import BaseModelError from ..messaging.models.openapi import OpenAPISchema +from ..messaging.responder import BaseResponder from ..messaging.valid import ( DID_POSTURE, ENDPOINT, @@ -442,6 +443,17 @@ async def wallet_set_public_did(request: web.BaseRequest): connection_id = request.query.get("conn_id") attrib_def = None + # check if we need to endorse + if is_author_role(context.profile): + # authors cannot write to the ledger + write_ledger = False + create_transaction_for_endorser = True + if not connection_id: + # author has not provided a connection id, so determine which to use + connection_id = await get_endorser_connection_id(context.profile) + if not connection_id: + raise web.HTTPBadRequest(reason="No endorser connection found") + wallet = session.inject_or(BaseWallet) if not wallet: raise web.HTTPForbidden(reason="No wallet available") @@ -546,7 +558,6 @@ async def promote_wallet_public_did( connection_id = await get_endorser_connection_id(context.profile) if not connection_id: raise web.HTTPBadRequest(reason="No endorser connection found") - if not write_ledger: try: async with profile.session() as session: @@ -598,11 +609,6 @@ async def promote_wallet_public_did( routing_keys=routing_keys, ) - # Commented the below lines as the function set_did_endpoint - # was calling update_endpoint_for_did of ledger - # async with ledger: - # await ledger.update_endpoint_for_did(info.did, endpoint) - # Route the public DID route_manager = profile.inject(RouteManager) await route_manager.route_public_did(profile, info.verkey) @@ -818,14 +824,60 @@ async def on_register_nym_event(profile: Profile, event: Event): did = event.payload["did"] connection_id = event.payload.get("connection_id") try: - await promote_wallet_public_did( + info, attrib_def = await promote_wallet_public_did( profile, profile.context, profile.session, did, connection_id ) - except Exception: + except Exception as err: + # log the error, but continue + LOGGER.exception( + "Error promoting to public DID: %s", + err, + ) + return + + transaction_mgr = TransactionManager(profile) + try: + transaction = await transaction_mgr.create_record( + messages_attach=attrib_def["signed_txn"], connection_id=connection_id + ) + except StorageError as err: # log the error, but continue LOGGER.exception( "Error accepting endorser invitation/configuring endorser connection: %s", + err, ) + return + + # if auto-request, send the request to the endorser + if profile.settings.get_value("endorser.auto_request"): + try: + transaction, transaction_request = await transaction_mgr.create_request( + transaction=transaction, + # TODO see if we need to parameterize these params + # expires_time=expires_time, + # endorser_write_txn=endorser_write_txn, + ) + except (StorageError, TransactionManagerError) as err: + # log the error, but continue + LOGGER.exception( + "Error creating endorser transaction request: %s", + err, + ) + + # TODO not sure how to get outbound_handler in an event ... + # await outbound_handler(transaction_request, connection_id=connection_id) + responder = profile.inject_or(BaseResponder) + if responder: + await responder.send( + transaction_request, + connection_id=connection_id, + ) + else: + LOGGER.warning( + "Configuration has no BaseResponder: cannot update " + "ATTRIB record on DID: %s", + did, + ) async def register(app: web.Application): diff --git a/aries_cloudagent/wallet/util.py b/aries_cloudagent/wallet/util.py index f87e6a53da..942744bca8 100644 --- a/aries_cloudagent/wallet/util.py +++ b/aries_cloudagent/wallet/util.py @@ -102,7 +102,9 @@ def abbr_verkey(full_verkey: str, did: str = None) -> str: DID_EVENT_PREFIX = "acapy::ENDORSE_DID::" +DID_ATTRIB_EVENT_PREFIX = "acapy::ENDORSE_DID_ATTRIB::" EVENT_LISTENER_PATTERN = re.compile(f"^{DID_EVENT_PREFIX}(.*)?$") +ATTRIB_EVENT_LISTENER_PATTERN = re.compile(f"^{DID_ATTRIB_EVENT_PREFIX}(.*)?$") async def notify_endorse_did_event(profile: Profile, did: str, meta_data: dict): @@ -111,3 +113,11 @@ async def notify_endorse_did_event(profile: Profile, did: str, meta_data: dict): DID_EVENT_PREFIX + did, meta_data, ) + + +async def notify_endorse_did_attrib_event(profile: Profile, did: str, meta_data: dict): + """Send notification for a DID ATTRIB post-process event.""" + await profile.notify( + DID_ATTRIB_EVENT_PREFIX + did, + meta_data, + ) diff --git a/demo/features/0453-issue-credential.feature b/demo/features/0453-issue-credential.feature index 0c74f53645..d8b0188604 100644 --- a/demo/features/0453-issue-credential.feature +++ b/demo/features/0453-issue-credential.feature @@ -1,3 +1,4 @@ +@RFC0453 Feature: RFC 0453 Aries agent issue credential @T003-RFC0453 @GHA diff --git a/demo/features/0586-sign-transaction.feature b/demo/features/0586-sign-transaction.feature index e8c85d1e0b..1a66ef69ab 100644 --- a/demo/features/0586-sign-transaction.feature +++ b/demo/features/0586-sign-transaction.feature @@ -1,3 +1,4 @@ +@RFC0586 Feature: RFC 0586 Aries sign (endorse) transactions functions @T001-RFC0586 diff --git a/demo/features/steps/0586-sign-transaction.py b/demo/features/steps/0586-sign-transaction.py index 04b6249598..61702e84a8 100644 --- a/demo/features/steps/0586-sign-transaction.py +++ b/demo/features/steps/0586-sign-transaction.py @@ -34,11 +34,18 @@ def step_impl(context, agent_name, did_role): ) # make the new did the wallet's public did - created_did = agent_container_POST( + published_did = agent_container_POST( agent["agent"], "/wallet/did/public", params={"did": created_did["result"]["did"]}, ) + if "result" in published_did: + # published right away! + pass + elif "txn" in published_did: + # we are an author and need to go through the endorser process + # assume everything works! + async_sleep(3.0) if not "public_dids" in context: context.public_dids = {} diff --git a/demo/runners/support/agent.py b/demo/runners/support/agent.py index 07da797d0a..bd82314937 100644 --- a/demo/runners/support/agent.py +++ b/demo/runners/support/agent.py @@ -600,8 +600,10 @@ async def register_or_switch_wallet( if self.endorser_role and self.endorser_role == "author": if endorser_agent: await self.admin_POST("/wallet/did/public?did=" + self.did) + await asyncio.sleep(3.0) else: await self.admin_POST("/wallet/did/public?did=" + self.did) + await asyncio.sleep(3.0) elif cred_type == CRED_FORMAT_JSON_LD: # create did of appropriate type data = {"method": DID_METHOD_KEY, "options": {"key_type": KEY_TYPE_BLS}} diff --git a/docs/assets/endorse-cred-def.png b/docs/assets/endorse-cred-def.png new file mode 100644 index 0000000000..ceb3d2fbb1 Binary files /dev/null and b/docs/assets/endorse-cred-def.png differ diff --git a/docs/assets/endorse-cred-def.puml b/docs/assets/endorse-cred-def.puml new file mode 100644 index 0000000000..a1a78c7772 --- /dev/null +++ b/docs/assets/endorse-cred-def.puml @@ -0,0 +1,75 @@ +@startuml +' List of actors for our use case +actor Admin +participant CredDefRoutes +participant RevocationRoutes +participant IndyRevocation +participant Ledger +participant TransactionManager +participant EventBus +participant OutboundHandler +participant EndorsedTxnHandler +boundary OtherAgent + +' Sequence for writing a new credential definition +Admin --> CredDefRoutes: POST /credential-definitions +group Endorse transaction process +CredDefRoutes --> Ledger: create_and_send_credential_definition() +CredDefRoutes --> TransactionManager: create_record() +CredDefRoutes --> TransactionManager: create_request() +CredDefRoutes --> OutboundHandler: send_outbound_msg() +OutboundHandler --> OtherAgent: send_msg() +OtherAgent --> OtherAgent: endorse_msg() +EndorsedTxnHandler <-- OtherAgent: send_msg() +TransactionManager <-- EndorsedTxnHandler: receive_endorse_response() +TransactionManager <-- EndorsedTxnHandler: complete_transaction() +Ledger <-- TransactionManager: txn_submit() +TransactionManager --> TransactionManager: endorsed_txn_post_processing() +TransactionManager --> EventBus: notify_cred_def_event() +end + +' Create the revocation registry once the credential definition is written +CredDefRoutes <-- EventBus: on_cred_def_event() +CredDefRoutes --> IndyRevocation: init_issuer_registry() +IndyRevocation --> EventBus: notify_revocation_reg_init_event() +RevocationRoutes <-- EventBus: on_revocation_registry_init_event() +RevocationRoutes --> RevocationRoutes: generate_tails() +group Endorse transaction process +RevocationRoutes --> Ledger:send_revoc_reg_def() +RevocationRoutes --> TransactionManager: create_record() +RevocationRoutes --> TransactionManager: create_request() +RevocationRoutes --> OutboundHandler: send_outbound_msg() +OutboundHandler --> OtherAgent: send_msg() +OtherAgent --> OtherAgent: endorse_msg() +EndorsedTxnHandler <-- OtherAgent: send_msg() +TransactionManager <-- EndorsedTxnHandler: receive_endorse_response() +TransactionManager <-- EndorsedTxnHandler: complete_transaction() +Ledger <-- TransactionManager: txn_submit() +TransactionManager --> TransactionManager: endorsed_txn_post_processing() +TransactionManager --> EventBus: notify_revocation_reg_endorsed_event() +end + +' Now create the revocation entry (accumulator) +RevocationRoutes <-- EventBus: on_revocation_registry_endorsed_event() +RevocationRoutes --> RevocationRoutes: upload_tails() +RevocationRoutes --> EventBus: notify_revocation_entry_event() +RevocationRoutes <-- EventBus: on_revocation_entry_event() +group Endorse transaction process +RevocationRoutes --> IndyRevocation: send_entry() +IndyRevocation --> Ledger: send_entry() +RevocationRoutes --> TransactionManager: create_record() +RevocationRoutes --> TransactionManager: create_request() +RevocationRoutes --> OutboundHandler: send_outbound_msg() +OutboundHandler --> OtherAgent: send_msg() +OtherAgent --> OtherAgent: endorse_msg() +EndorsedTxnHandler <-- OtherAgent: send_msg() +TransactionManager <-- EndorsedTxnHandler: receive_endorse_response() +TransactionManager <-- EndorsedTxnHandler: complete_transaction() +Ledger <-- TransactionManager: txn_submit() +TransactionManager --> TransactionManager: endorsed_txn_post_processing() + +' Notify that the revocation entry is completed (no one listens to this notification yet) +TransactionManager --> EventBus: notify_revocation_entry_endorsed_event() +end + +@enduml diff --git a/docs/assets/endorse-public-did.png b/docs/assets/endorse-public-did.png new file mode 100644 index 0000000000..275b4ab6de Binary files /dev/null and b/docs/assets/endorse-public-did.png differ diff --git a/docs/assets/endorse-public-did.puml b/docs/assets/endorse-public-did.puml new file mode 100644 index 0000000000..63de78bb50 --- /dev/null +++ b/docs/assets/endorse-public-did.puml @@ -0,0 +1,53 @@ +@startuml +' List of actors for our use case +actor Admin +participant WalletRoutes +participant IndyWallet +participant LedgerRoutes +participant Ledger +participant TransactionManager +participant EventBus +participant OutboundHandler +participant EndorsedTxnHandler +boundary OtherAgent + +' Sequence for writing a new DID on the ledger (assumes the author already has a DID) +Admin --> WalletRoutes: POST /wallet/did/create +Admin --> LedgerRoutes: POST /ledger/register-nym +group Endorse transaction process +LedgerRoutes --> Ledger: register_nym() +LedgerRoutes --> TransactionManager: create_record() +LedgerRoutes --> TransactionManager: create_request() +LedgerRoutes --> OutboundHandler: send_outbound_msg() +OutboundHandler --> OtherAgent: send_msg() +OtherAgent --> OtherAgent: endorse_msg() +EndorsedTxnHandler <-- OtherAgent: send_msg() +TransactionManager <-- EndorsedTxnHandler: receive_endorse_response() +TransactionManager <-- EndorsedTxnHandler: complete_transaction() +Ledger <-- TransactionManager: txn_submit() +TransactionManager --> TransactionManager: endorsed_txn_post_processing() +TransactionManager --> EventBus: notify_endorse_did_event() +end + +WalletRoutes <-- EventBus: on_register_nym_event() +WalletRoutes --> WalletRoutes:promote_wallet_public_did() +WalletRoutes --> IndyWallet:set_public_did() +group Endorse transaction process +WalletRoutes --> IndyWallet:set_did_endpoint() +IndyWallet --> Ledger:update_endpoint_for_did() +WalletRoutes --> TransactionManager: create_record() +WalletRoutes --> TransactionManager: create_request() +WalletRoutes --> OutboundHandler: send_outbound_msg() +OutboundHandler --> OtherAgent: send_msg() +OtherAgent --> OtherAgent: endorse_msg() +EndorsedTxnHandler <-- OtherAgent: send_msg() +TransactionManager <-- EndorsedTxnHandler: receive_endorse_response() +TransactionManager <-- EndorsedTxnHandler: complete_transaction() +Ledger <-- TransactionManager: txn_submit() +TransactionManager --> TransactionManager: endorsed_txn_post_processing() + +' notification that no one is listening to yet +TransactionManager --> EventBus: notify_endorse_did_attrib_event() +end + +@enduml diff --git a/docs/assets/endorser-design.png b/docs/assets/endorser-design.png new file mode 100644 index 0000000000..1c4b9fc555 Binary files /dev/null and b/docs/assets/endorser-design.png differ diff --git a/docs/assets/endorser-design.puml b/docs/assets/endorser-design.puml new file mode 100644 index 0000000000..39883ea66b --- /dev/null +++ b/docs/assets/endorser-design.puml @@ -0,0 +1,31 @@ +@startuml +interface AdminUser + +interface OtherAgent + +object TransactionRoutes + +object TransactionHandlers + +AdminUser --> TransactionRoutes: invoke_endpoint() + +OtherAgent --> TransactionHandlers: send_message() + +object TransactionManager + +object Wallet + +TransactionManager --> Wallet: manage_records() + +TransactionRoutes --> TransactionManager: invoke_api() +TransactionHandlers --> TransactionManager: handle_msg() + +object EventBus + +TransactionManager --> EventBus: notify() + +interface OtherProtocolRoutes + +OtherProtocolRoutes --> EventBus: subscribe() +EventBus --> OtherProtocolRoutes: notify() +@enduml