Merge branch 'main' into feature/keylist-webhooks

CHANGELOG.md

@@ -1,10 +1,10 @@
-# 0.7.4-RC1
+# 0.7.4-RC2
 
-This release consists largely of internal fixes with a few minor enhancements.
-There have been a lot of groups exercising ACA-Py and the updates made in this
-release are a reflection of those efforts. We have PRs that have been
-contributed by 17 different people, which is likely a record for a single ACA-Py
-release.
+The 0.7.4 release consists largely of internal fixes with a few minor
+enhancements. There have been a lot of groups exercising ACA-Py and the updates
+made in this release are a reflection of those efforts. We have PRs that have
+been contributed by 17 different people, which is likely a record for a single
+ACA-Py release.
 
 The largest enhancement is in the area of the Hyperledger Indy endorser,
 enabling an instance of ACA-Py to act as an Endorser for Indy authors needed
@@ -27,6 +27,16 @@ the team at LISSI for creating the
 to make load testing so easy! And of course to the core ACA-Py team for
 addressing the findings.
 
+The team has worked a lot on evolving the persistent queue (PQ) approach
+available in ACA-Py. We have landed on a design whereby the ability to use
+queues for inbound and outbound messages is within ACA-Py, with a default
+in-memory implementation, and the implementations of external persistent queues
+solutions is handled by referencing a plugin from a separate repository. There
+will shortly be two concrete, out-of-the-box solutions available, one for Kafka
+and one for Redis, and anyone else can implement their own PQ plugin as long as
+it uses the same ACA-Py queuing interface. Look for the new PQ repos shortly
+within Hyperledger Aries.
+
 Several new ways to control ACA-Py configurations were added, including new
 startup parameters, Admin API parameters to control instances of protocols, and
 additional web hook notifications.
@@ -60,6 +70,9 @@ stuff needed for a growing codebase.
   - Feature/undelivered events [\#1694](https://github.com/hyperledger/aries-cloudagent-python/pull/1694) ([mepeltier](https://github.com/mepeltier))
   - Allow use of SEED when creating local wallet DID Issue-1682 Issue-1682 [\#1705](https://github.com/hyperledger/aries-cloudagent-python/pull/1705) ([DaevMithran](https://github.com/DaevMithran))
 
+- Persistent Queues
+  - Redis PQ Cleanup in preparation for enabling the uses of plugin PQ implementations \[Issue\#1659\] [\#1659](https://github.com/hyperledger/aries-cloudagent-python/pull/1690) ([shaangill025](https://github.com/shaangill025))
+
 - Issue Credential, Revocation, Present Proof updates/fixes
   - Fix: DIF proof proposal when creating bound presentation request \[Issue\#1687\] [\#1690](https://github.com/hyperledger/aries-cloudagent-python/pull/1690) ([shaangill025](https://github.com/shaangill025))
   - Fix DIF PresExch and OOB request\_attach delete unused connection [\#1676](https://github.com/hyperledger/aries-cloudagent-python/pull/1676) ([shaangill025](https://github.com/shaangill025))
@@ -68,6 +81,7 @@ stuff needed for a growing codebase.
   - Fixes for credential details in issue-credential webhook responses [\#1668](https://github.com/hyperledger/aries-cloudagent-python/pull/1668) ([andrewwhitehead](https://github.com/andrewwhitehead))
   - Fix: present-proof v2 send-proposal [issue\#1474](https://github.com/hyperledger/aries-cloudagent-python/issues/1474) [\#1667](https://github.com/hyperledger/aries-cloudagent-python/pull/1667) ([shaangill025](https://github.com/shaangill025))
     - Fixes Issue 3b from [\#1597](https://github.com/hyperledger/aries-cloudagent-python/issues/1597): V2 Credential exchange ignores the auto-respond-credential-request
+  - fix: Resolve Revocation Notification environment variable name collision [\#1751](https://github.com/hyperledger/aries-cloudagent-python/pull/1751) ([frostyfrog](https://github.com/frostyfrog))
   - fix: always notify if revocation notification record exists [\#1665](https://github.com/hyperledger/aries-cloudagent-python/pull/1665) ([TimoGlastra](https://github.com/TimoGlastra))
   - Revert change to send\_credential\_ack return value [\#1660](https://github.com/hyperledger/aries-cloudagent-python/pull/1660) ([andrewwhitehead](https://github.com/andrewwhitehead))
   - Fix usage of send\_credential\_ack [\#1653](https://github.com/hyperledger/aries-cloudagent-python/pull/1653) ([andrewwhitehead](https://github.com/andrewwhitehead))
@@ -80,6 +94,7 @@ stuff needed for a growing codebase.
 
 - Mediator updates and fixes
   - feat: allow querying default mediator from base wallet [\#1729](https://github.com/hyperledger/aries-cloudagent-python/pull/1729) ([dbluhm](https://github.com/dbluhm))
+  - Added async with for mediator record delete [\#1749](https://github.com/hyperledger/aries-cloudagent-python/pull/1749) ([dejsenlitro](https://github.com/dejsenlitro))
 
 - Multitenacy updates and fixes
   - feat: create new JWT tokens and invalidate older for multitenancy [\#1725](https://github.com/hyperledger/aries-cloudagent-python/pull/1725) ([TimoGlastra](https://github.com/TimoGlastra))
@@ -96,6 +111,7 @@ stuff needed for a growing codebase.
   - Add an integration test for mixed proof with a revocable cred and a n… [\#1672](https://github.com/hyperledger/aries-cloudagent-python/pull/1672) ([ianco](https://github.com/ianco))
 
 - Documentation and Demo Updates
+  - Fetch from --genesis-url likely to fail in composed container [\#1746](https://github.com/hyperledger/aries-cloudagent-python/pull/1739) ([tdiesler](https://github.com/tdiesler))
   - Fixes logic for web hook formatter in Faber demo [\#1739](https://github.com/hyperledger/aries-cloudagent-python/pull/1739) ([amanji](https://github.com/amanji))
   - Multitenancy Docs Update [\#1706](https://github.com/hyperledger/aries-cloudagent-python/pull/1706) ([MonolithicMonk](https://github.com/MonolithicMonk))
   - [\#1674](https://github.com/hyperledger/aries-cloudagent-python/issue/1674) Add basic DOCKER\_ENV logging for run\_demo [\#1675](https://github.com/hyperledger/aries-cloudagent-python/pull/1675) ([tdiesler](https://github.com/tdiesler))

Multiledger.md

@@ -14,6 +14,7 @@ More background information including problem statement, design (algorithm) and
   - [Read Requests](#read-requests)
     - [For checking ledger in parallel](#for-checking-ledger-in-parallel)
   - [Write Requests](#write-requests)
+- [A Special Warning for TAA Acceptance](#a-special-warning-for-taa-acceptance)
 - [Impact on other ACA-Py function](#impact-on-other-aca-py-function)
 
 ## Usage
@@ -104,6 +105,25 @@ If multiple ledgers are configured then `IndyLedgerRequestsExecutor` service ext
 
 On startup, the first configured applicable ledger is assigned as the `write_ledger` [`BaseLedger`], the selection is dependant on the order (top-down) and whether it is `production` or `non_production`. For instance, considering this [example configuration](#example-config-file), ledger `bcorvinTest` will be set as `write_ledger` as it is the topmost `production` ledger. If no `production` ledgers are included in configuration then the topmost `non_production` ledger is selected.
 
+## A Special Warning for TAA Acceptance
+
+When you run in multi-ledger mode, ACA-Py will use the `pool-name` (or `id`) specified in the ledger configuration file for each ledger.
+
+(When running in single-ledger mode, ACA-Py uses `default` as the ledger name.)
+
+If you are running against a ledger in `write` mode, and the ledger requires you to accept a Transaction Author Agreement (TAA), ACA-Py stores the TAA acceptance
+status in the wallet in a non-secrets record, using the ledger's `pool_name` as a key.
+
+This means that if you are upgrading from single-ledger to multi-ledger mode, you will need to *either*:
+
+- set the `id` for your writable ledger to `default` (in your `ledgers.yaml` file)
+
+*or*:
+
+- re-accept the TAA once you restart your ACA-Py in multi-ledger mode
+
+Once you re-start ACA-Py, you can check the `GET /ledger/taa` endpoint to verify your TAA acceptance status.
+
 ## Impact on other ACA-Py function
 
 There should be no impact/change in functionality to any ACA-Py protocols.

aries_cloudagent/askar/profile.py

@@ -213,6 +213,8 @@ async def _teardown(self, commit: bool = None):
                 await self._handle.commit()
             except AskarError as err:
                 raise ProfileError("Error committing transaction") from err
+        if self._handle:
+            await self._handle.close()
         self._handle = None
         self._check_duration()
 

aries_cloudagent/config/argparse.py

@@ -534,6 +534,20 @@ def add_arguments(self, parser: ArgumentParser):
             ),
         )
 
+        parser.add_argument(
+            "--block-plugin",
+            dest="blocked_plugins",
+            type=str,
+            action="append",
+            required=False,
+            metavar="<module>",
+            env_var="ACAPY_BLOCKED_PLUGIN",
+            help=(
+                "Block <module> plugin module from loading. Multiple "
+                "instances of this parameter can be specified."
+            ),
+        )
+
         parser.add_argument(
             "--plugin-config",
             dest="plugin_config",
@@ -611,6 +625,9 @@ def get_settings(self, args: Namespace) -> dict:
         if args.external_plugins:
             settings["external_plugins"] = args.external_plugins
 
+        if args.blocked_plugins:
+            settings["blocked_plugins"] = args.blocked_plugins
+
         if args.plugin_config:
             with open(args.plugin_config, "r") as stream:
                 settings["plugin_config"] = yaml.safe_load(stream)

aries_cloudagent/config/default_context.py

@@ -111,7 +111,9 @@ async def bind_providers(self, context: InjectionContext):
     async def load_plugins(self, context: InjectionContext):
         """Set up plugin registry and load plugins."""
 
-        plugin_registry = PluginRegistry()
+        plugin_registry = PluginRegistry(
+            blocklist=self.settings.get("blocked_plugins", [])
+        )
         context.injector.bind_instance(PluginRegistry, plugin_registry)
 
         # Register standard protocol plugins

aries_cloudagent/core/plugin_registry.py

@@ -3,7 +3,7 @@
 import logging
 from collections import OrderedDict
 from types import ModuleType
-from typing import Sequence
+from typing import Sequence, Iterable
 
 from ..config.injection_context import InjectionContext
 from ..core.event_bus import EventBus
@@ -19,9 +19,10 @@
 class PluginRegistry:
     """Plugin registry for indexing application plugins."""
 
-    def __init__(self):
+    def __init__(self, blocklist: Iterable[str] = []):
         """Initialize a `PluginRegistry` instance."""
         self._plugins = OrderedDict()
+        self._blocklist = set(blocklist)
 
     @property
     def plugin_names(self) -> Sequence[str]:
@@ -119,6 +120,9 @@ def register_plugin(self, module_name: str) -> ModuleType:
         """Register a plugin module."""
         if module_name in self._plugins:
             mod = self._plugins[module_name]
+        elif module_name in self._blocklist:
+            LOGGER.debug(f"Blocked {module_name} from loading due to blocklist")
+            return None
         else:
             try:
                 mod = ClassLoader.load_module(module_name)

aries_cloudagent/core/tests/test_plugin_registry.py

@@ -15,7 +15,8 @@
 
 class TestPluginRegistry(AsyncTestCase):
     def setUp(self):
-        self.registry = PluginRegistry()
+        self.blocked_module = "blocked_module"
+        self.registry = PluginRegistry(blocklist=[self.blocked_module])
 
         self.context = InjectionContext(enforce_typing=False)
         self.proto_registry = async_mock.MagicMock(
@@ -478,6 +479,23 @@ class MODULE:
             ]
             assert self.registry.register_plugin("dummy") == obj
 
+    async def test_unregister_plugin_has_setup(self):
+        class MODULE:
+            setup = "present"
+
+        obj = MODULE()
+        with async_mock.patch.object(
+            ClassLoader, "load_module", async_mock.MagicMock()
+        ) as load_module:
+            load_module.side_effect = [
+                obj,  # module
+                None,  # routes
+                None,  # message types
+                None,  # definition without versions attr
+            ]
+            assert self.registry.register_plugin(self.blocked_module) == None
+            assert self.blocked_module not in self.registry._plugins.keys()
+
     async def test_register_definitions_malformed(self):
         class MODULE:
             no_setup = "no setup attr"

aries_cloudagent/ledger/indy.py

@@ -5,6 +5,7 @@
 import logging
 import tempfile
 from datetime import date, datetime
+from io import StringIO
 from os import path
 from time import time
 from typing import Sequence, Tuple, Optional
@@ -42,6 +43,17 @@
 GENESIS_TRANSACTION_FILE = "indy_genesis_transactions.txt"
 
 
+def _normalize_txns(txns: str) -> str:
+    """Normalize a set of genesis transactions."""
+    lines = StringIO()
+    for line in txns.splitlines():
+        line = line.strip()
+        if line:
+            lines.write(line)
+            lines.write("\n")
+    return lines.getvalue()
+
+
 class IndySdkLedgerPoolProvider(BaseProvider):
     """Indy ledger pool provider which keys off the selected pool name."""
 
@@ -107,12 +119,28 @@ def __init__(
         self.cache = cache
         self.cache_duration = cache_duration
         self.genesis_transactions = genesis_transactions
+        self.genesis_txns_cache = genesis_transactions
         self.handle = None
         self.name = name
         self.taa_cache = None
         self.read_only = read_only
         self.socks_proxy = socks_proxy
 
+    @property
+    def genesis_txns(self) -> str:
+        """Get the configured genesis transactions."""
+        if not self.genesis_txns_cache:
+            try:
+                txn_path = path.join(
+                    tempfile.gettempdir(), f"{self.name}_{GENESIS_TRANSACTION_FILE}"
+                )
+                self.genesis_txns_cache = _normalize_txns(open(txn_path).read())
+            except FileNotFoundError:
+                raise LedgerConfigError(
+                    "Pool config '%s' not found", self.name
+                ) from None
+        return self.genesis_txns_cache
+
     async def create_pool_config(
         self, genesis_transactions: str, recreate: bool = False
     ):

aries_cloudagent/protocols/revocation_notification/definition.py

@@ -6,5 +6,11 @@
         "minimum_minor_version": 0,
         "current_minor_version": 0,
         "path": "v1_0",
-    }
+    },
+    {
+        "major_version": 2,
+        "minimum_minor_version": 0,
+        "current_minor_version": 0,
+        "path": "v2_0",
+    },
 ]

aries_cloudagent/protocols/revocation_notification/v1_0/models/rev_notification_record.py

@@ -27,6 +27,7 @@ class Meta:
         "rev_reg_id",
         "cred_rev_id",
         "connection_id",
+        "version",
     }
 
     def __init__(
@@ -38,6 +39,7 @@ def __init__(
         connection_id: str = None,
         thread_id: str = None,
         comment: str = None,
+        version: str = None,
         **kwargs,
     ):
         """Construct record."""
@@ -47,6 +49,7 @@ def __init__(
         self.connection_id = connection_id
         self.thread_id = thread_id
         self.comment = comment
+        self.version = version
 
     @property
     def revocation_notification_id(self) -> Optional[str]:
@@ -73,6 +76,7 @@ async def query_by_ids(
             rev_reg_id: the rev reg id by which to filter
         """
         tag_filter = {
+            **{"version": "v1_0"},
             **{"cred_rev_id": cred_rev_id for _ in [""] if cred_rev_id},
             **{"rev_reg_id": rev_reg_id for _ in [""] if rev_reg_id},
         }
@@ -101,6 +105,7 @@ async def query_by_rev_reg_id(
             rev_reg_id: the rev reg id by which to filter
         """
         tag_filter = {
+            **{"version": "v1_0"},
             **{"rev_reg_id": rev_reg_id for _ in [""] if rev_reg_id},
         }
 
@@ -157,3 +162,7 @@ class Meta:
         description="Optional comment to include in revocation notification",
         required=False,
     )
+    version = fields.Str(
+        description="Version of Revocation Notification to send out",
+        required=False,
+    )

aries_cloudagent/protocols/revocation_notification/v1_0/models/tests/test_rev_notification_record.py

@@ -21,6 +21,7 @@ def rec():
         connection_id="mock_connection_id",
         thread_id="mock_thread_id",
         comment="mock_comment",
+        version="v1_0",
     )
 
 
@@ -50,6 +51,7 @@ async def test_storage(profile, rec):
             another = RevNotificationRecord(
                 rev_reg_id="mock_rev_reg_id",
                 cred_rev_id="mock_cred_rev_id",
+                version="v1_0",
             )
             await another.save(session)
             await RevNotificationRecord.query_by_ids(

aries_cloudagent/protocols/revocation_notification/v2_0/handlers/revoke_handler.py

@@ -0,0 +1,44 @@
+"""Handler for revoke message."""
+
+from .....messaging.base_handler import BaseHandler
+from .....messaging.request_context import RequestContext
+from .....messaging.responder import BaseResponder
+
+from ..messages.revoke import Revoke
+
+
+class RevokeHandler(BaseHandler):
+    """Handler for revoke message."""
+
+    RECIEVED_TOPIC = "acapy::revocation-notification-v2::received"
+    WEBHOOK_TOPIC = "acapy::webhook::revocation-notification-v2"
+
+    async def handle(self, context: RequestContext, responder: BaseResponder):
+        """Handle revoke message."""
+        assert isinstance(context.message, Revoke)
+        self._logger.debug(
+            "Received notification of revocation for %s cred %s with comment: %s",
+            context.message.revocation_format,
+            context.message.credential_id,
+            context.message.comment,
+        )
+        # Emit a webhook
+        if context.settings.get("revocation.monitor_notification"):
+            await context.profile.notify(
+                self.WEBHOOK_TOPIC,
+                {
+                    "revocation_format": context.message.revocation_format,
+                    "credential_id": context.message.credential_id,
+                    "comment": context.message.comment,
+                },
+            )
+
+        # Emit an event
+        await context.profile.notify(
+            self.RECIEVED_TOPIC,
+            {
+                "revocation_format": context.message.revocation_format,
+                "credential_id": context.message.credential_id,
+                "comment": context.message.comment,
+            },
+        )