Merge branch 'main' into upd/pyjwt

DevReadMe.md

@@ -18,6 +18,7 @@ See the [README](README.md) for details about this repository and information ab
 - [Developing](#developing)
   - [Prerequisites](#prerequisites)
   - [Running Locally](#running-locally)
+  - [Logging](#logging)
   - [Running Tests](#running-tests)
   - [Running Aries Agent Test Harness Tests](#running-aries-agent-test-harness-tests)
 - [Development Workflow](#development-workflow)

README.md

@@ -78,6 +78,19 @@ An ACA-Py instance puts together an OpenAPI-documented REST interface based on t
 
 Technical note: the administrative API exposed by the agent for the controller to use must be protected with an API key (using the --admin-api-key command line arg) or deliberately left unsecured using the --admin-insecure-mode command line arg. The latter should not be used other than in development if the API is not otherwise secured.
 
+## Troubleshooting
+
+There are a number of resources for getting help with ACA-Py and troubleshooting
+any problems you might run into. The [Troubleshooting](Troubleshooting.md)
+document contains some guidance about issues that have been experienced in the
+past. Feel free to submit PRs to supplement the troubleshooting document!
+Searching the [ACA-Py GitHub
+issues](https://github.com/hyperledger/aries-cloudagent-python/issues) will
+often uncover challenges that others have experienced, often with answers to
+solving those challenges. As well, there is the "aries-cloudagent-python"
+channel on the Hyperledger Discord chat server ([invitation
+here](https://discord.gg/hyperledger)).
+
 ## Credit
 
 The initial implementation of ACA-Py was developed by the Government of British Columbia’s Digital Trust Team in Canada. To learn more about what’s happening with decentralized identity and digital trust in British Columbia, a new website will be launching and the link will be made available here.

Troubleshooting.md

@@ -0,0 +1,109 @@
+# Troubleshooting Aries Cloud Agent Python <!-- omit in toc -->
+
+This document contains some troubleshooting information that contributors to the
+community think may be helpful. Most of the content here assumes the reader has
+gotten started with ACA-Py and has arrived here because of an issue that came up
+in their use of ACA-Py.
+
+Contributions (via pull request) to this document are welcome. Topics added here
+will mostly come from reported issues that contributors think would be helpful
+to the larger community.
+
+## Table of Contents <!-- omit in toc -->
+
+- [Unable to Connect to Ledger](#unable-to-connect-to-ledger)
+  - [Local ledger running?](#local-ledger-running)
+  - [Any Firewalls](#any-firewalls)
+- [Damaged, Unpublishable Revocation Registry](#damaged-unpublishable-revocation-registry)
+
+## Unable to Connect to Ledger
+
+The most common issue hit by first time users is getting an error on startup "unable to connect to ledger". Here are a list of things to check when you see that error.
+
+### Local ledger running?
+
+Unless you specify via startup parameters or environment variables that you are using a public Hyperledger Indy ledger, ACA-Py assumes that you are running a local ledger -- an instance of [von-network](https://github.com/bcgov/von-network).
+If that is the cause -- have you started your local ledger, and did it startup properly.  Things to check:
+
+- Any errors in the startup of von-network?
+- Is the von-network webserver (usually at `https:/localhost:9000`) accessible? If so, can you click on and see the Genesis File?
+- Do you even need a local ledger? If not, you can use a public sandbox ledger,
+  such as the [Dev Greenlight ledger](), likely by just prefacing your ACA-Py
+  command with `LEDGER_URL=http://dev.greenlight.bcovrin.vonx.io`. For example,
+  when running the Alice-Faber demo in the [demo](demo) folder, you can run (for
+  example), the Faber agent using the command:
+  `LEDGER_URL=http://dev.greenlight.bcovrin.vonx.io ./run_demo faber`
+
+### Any Firewalls
+
+Do you have any firewalls in play that might be blocking the ports that are used by the ledger, notably 9701-9708? To access a ledger
+the ACA-Py instance must be able to get to those ports of the ledger, regardless if the ledger is local or remote.
+
+## Damaged, Unpublishable Revocation Registry
+
+We have discovered that in the ACA-Py AnonCreds implementation, it is possible
+to get into a state where the publishing of updates to a Revocation Registry
+(RevReg) is impossible. This can happen where ACA-Py starts to publish an update
+to the RevReg, but the write transaction to the Hyperledger Indy ledger fails
+for some reason. When a credential revocation is published, aca-py (via indy-sdk
+or askar/credx) updates the revocation state in the wallet as well as on the
+ledger.  The revocation state is dependant on whatever the previous revocation
+state is/was, so if the ledger and wallet are mis-matched the publish will fail.
+(Andrew/s PR # 1804 (merged) should mitigate but probably won't completely
+eliminate this from happening).
+
+For example, in case we've seen, the write RevRegEntry transaction failed at the
+ledger because there was a problem with accepting the TAA (Transaction Author
+Agreement). Once the error occurred, the RevReg state held by the ACA-Py agent,
+and the RevReg state on the ledger were different. Even after the ability to
+write to the ledger was restored, the RevReg could still not be published
+because of the differences in the RevReg state. Such a situation can now be
+corrected, as follows:
+
+To address this issue, some new endpoints were added to ACA-Py in Release 0.7.4,
+as follows:
+
+- GET `/revocation/registry/<id>/issued` - counts of the number of issued/revoked
+  within a registry
+- GET `/revocation/registry/<id>/issued/details` - details of all credentials
+  issued/revoked within a registry
+- GET `/revocation/registry/<id>/issued/indy_recs` - calculated rev_reg_delta from
+  the ledger
+  - This is used to compare ledger revoked vs wallet revoked credentials, which
+    is essentially the state of the RevReg on the ledger and in ACA-Py. Where
+    there is a difference, we have an error.
+- PUT `/revocation/registry/<id>/fix-revocation-entry-state` - publish an update
+  to the RevReg state on the ledger to bring it into alignment with what is in
+  the ACA-Py instance.
+  - There is a boolean parameter (`apply_ledger_update`) to control whether the
+    ledger entry actually gets published so, if you are so inclined, you can
+    call the endpoint to see what the transaction would be, before you actually
+    try to do a ledger update.  This will return:
+    - `rev_reg_delta` - same as the ".../indy_recs" endpoint
+    - `accum_calculated` - transaction to write to ledger
+    - `accum_fixed` - If `apply_ledger_update`, the transaction actually written
+      to the ledger
+
+Note that there is (currently) a backlog item to prevent the wallet and ledger
+from getting out of sync (e.g. don't update the ACA-Py RevReg state if the
+ledger write fails), but even after that change is made, having this ability
+will be retained for use if needed.
+
+We originally ran into this due to the TAA acceptance getting lost when
+switching to multi-ledger (as described
+[here](https://github.com/hyperledger/aries-cloudagent-python/blob/main/Multiledger.md#a-special-warning-for-taa-acceptance).
+Note that this is one reason how this "out of sync" scenario can occur, but
+there may be others.
+
+We add an integration test that demonstrates/tests this issue [here](https://github.com/hyperledger/aries-cloudagent-python/blob/main/demo/features/taa-txn-author-acceptance.feature#L67).
+
+To run the scenario either manually or using the integration tests, you can do the following:
+
+- Start von-network in TAA mode:
+  - `./manage start --taa-sample --logs`
+- Start the tails server as usual:
+  - `./manage start --logs`
+- To run the scenario manually, start faber and let the agent know it needs to TAA-accept before doing any ledger writes:
+  - `./run_demo faber --revocation --taa-accept`, and then you can run through all the transactions using the Swagger page.
+- To run the scenario via an integration test, run:
+  - `./run_bdd -t @taa_required`

aries_cloudagent/protocols/present_proof/v1_0/handlers/presentation_request_handler.py

@@ -80,6 +80,10 @@ async def handle(self, context: RequestContext, responder: BaseResponder):
                         "connection_id": connection_id,
                     },
                 )  # holder initiated via proposal
+                presentation_exchange_record.presentation_request = indy_proof_request
+                presentation_exchange_record.presentation_request_dict = (
+                    context.message.serialize()
+                )
         except StorageNotFoundError:  # verifier sent this request free of any proposal
             presentation_exchange_record = V10PresentationExchange(
                 connection_id=connection_id,
@@ -94,7 +98,6 @@ async def handle(self, context: RequestContext, responder: BaseResponder):
                 trace=(context.message._trace is not None),
             )
 
-        presentation_exchange_record.presentation_request = indy_proof_request
         presentation_exchange_record = await presentation_manager.receive_request(
             presentation_exchange_record
         )  # mgr only saves record: on exception, saving state null is hopeless

aries_cloudagent/protocols/present_proof/v1_0/manager.py

@@ -472,7 +472,9 @@ async def send_presentation_ack(
                 return
 
         if responder:
-            presentation_ack_message = PresentationAck()
+            presentation_ack_message = PresentationAck(
+                verification_result=presentation_exchange_record.verified
+            )
             presentation_ack_message._thread = {
                 "thid": presentation_exchange_record.thread_id
             }
@@ -515,7 +517,7 @@ async def receive_presentation_ack(
                     "role": V10PresentationExchange.ROLE_PROVER,
                 },
             )
-
+            presentation_exchange_record.verified = message._verification_result
             presentation_exchange_record.state = (
                 V10PresentationExchange.STATE_PRESENTATION_ACKED
             )

aries_cloudagent/protocols/present_proof/v1_0/messages/presentation_ack.py

@@ -1,6 +1,6 @@
 """Represents an explicit RFC 15 ack message, adopted into present-proof protocol."""
 
-from marshmallow import EXCLUDE
+from marshmallow import EXCLUDE, fields, validate
 
 from ....notification.v1_0.messages.ack import V10Ack, V10AckSchema
 
@@ -21,7 +21,7 @@ class Meta:
         message_type = PRESENTATION_ACK
         schema_class = "PresentationAckSchema"
 
-    def __init__(self, status: str = None, **kwargs):
+    def __init__(self, status: str = None, verification_result: str = None, **kwargs):
         """
         Initialize an explicit ack message instance.
 
@@ -30,6 +30,7 @@ def __init__(self, status: str = None, **kwargs):
 
         """
         super().__init__(status, **kwargs)
+        self._verification_result = verification_result
 
 
 class PresentationAckSchema(V10AckSchema):
@@ -40,3 +41,10 @@ class Meta:
 
         model_class = PresentationAck
         unknown = EXCLUDE
+
+    verification_result = fields.Str(
+        required=False,
+        description="Whether presentation is verified: true or false",
+        example="true",
+        validate=validate.OneOf(["true", "false"]),
+    )

aries_cloudagent/protocols/present_proof/v1_0/tests/test_manager.py

@@ -1301,7 +1301,7 @@ async def test_send_presentation_ack_no_responder(self):
         self.profile.context.injector.clear_binding(BaseResponder)
         await self.manager.send_presentation_ack(exchange)
 
-    async def test_receive_presentation_ack(self):
+    async def test_receive_presentation_ack_a(self):
         connection_record = async_mock.MagicMock(connection_id=CONN_ID)
 
         exchange_dummy = V10PresentationExchange()
@@ -1322,6 +1322,28 @@ async def test_receive_presentation_ack(self):
                 V10PresentationExchange.STATE_PRESENTATION_ACKED
             )
 
+    async def test_receive_presentation_ack_b(self):
+        connection_record = async_mock.MagicMock(connection_id=CONN_ID)
+
+        exchange_dummy = V10PresentationExchange()
+        message = async_mock.MagicMock(_verification_result="true")
+
+        with async_mock.patch.object(
+            V10PresentationExchange, "save", autospec=True
+        ) as save_ex, async_mock.patch.object(
+            V10PresentationExchange, "retrieve_by_tag_filter", autospec=True
+        ) as retrieve_ex:
+            retrieve_ex.return_value = exchange_dummy
+            exchange_out = await self.manager.receive_presentation_ack(
+                message, connection_record
+            )
+            save_ex.assert_called_once()
+
+            assert exchange_out.state == (
+                V10PresentationExchange.STATE_PRESENTATION_ACKED
+            )
+            assert exchange_out.verified == "true"
+
     async def test_receive_problem_report(self):
         connection_id = "connection-id"
         stored_exchange = V10PresentationExchange(

aries_cloudagent/protocols/present_proof/v2_0/manager.py

@@ -409,7 +409,7 @@ async def send_pres_ack(self, pres_ex_record: V20PresExRecord):
         responder = self._profile.inject_or(BaseResponder)
 
         if responder:
-            pres_ack_message = V20PresAck()
+            pres_ack_message = V20PresAck(verification_result=pres_ex_record.verified)
             pres_ack_message._thread = {"thid": pres_ex_record.thread_id}
             pres_ack_message.assign_trace_decorator(
                 self._profile.settings, pres_ex_record.trace
@@ -445,7 +445,7 @@ async def receive_pres_ack(self, message: V20PresAck, conn_record: ConnRecord):
                     "role": V20PresExRecord.ROLE_PROVER,
                 },
             )
-
+            pres_ex_record.verified = message._verification_result
             pres_ex_record.state = V20PresExRecord.STATE_DONE
 
             await pres_ex_record.save(session, reason="receive v2.0 presentation ack")

aries_cloudagent/protocols/present_proof/v2_0/messages/pres_ack.py

@@ -1,6 +1,6 @@
 """Represents an explicit RFC 15 ack message, adopted into present-proof protocol."""
 
-from marshmallow import EXCLUDE
+from marshmallow import EXCLUDE, fields, validate
 
 from ....notification.v1_0.messages.ack import V10Ack, V10AckSchema
 
@@ -19,7 +19,7 @@ class Meta:
         message_type = PRES_20_ACK
         schema_class = "V20PresAckSchema"
 
-    def __init__(self, status: str = None, **kwargs):
+    def __init__(self, status: str = None, verification_result: str = None, **kwargs):
         """
         Initialize an explicit ack message instance.
 
@@ -28,6 +28,7 @@ def __init__(self, status: str = None, **kwargs):
 
         """
         super().__init__(status, **kwargs)
+        self._verification_result = verification_result
 
 
 class V20PresAckSchema(V10AckSchema):
@@ -38,3 +39,10 @@ class Meta:
 
         model_class = V20PresAck
         unknown = EXCLUDE
+
+    verification_result = fields.Str(
+        required=False,
+        description="Whether presentation is verified: true or false",
+        example="true",
+        validate=validate.OneOf(["true", "false"]),
+    )

aries_cloudagent/protocols/present_proof/v2_0/tests/test_manager.py

@@ -2076,13 +2076,31 @@ async def test_send_pres_ack(self):
         messages = responder.messages
         assert len(messages) == 1
 
+        px_rec = V20PresExRecord(verified="true")
+
+        responder = MockResponder()
+        self.profile.context.injector.bind_instance(BaseResponder, responder)
+
+        await self.manager.send_pres_ack(px_rec)
+        messages = responder.messages
+        assert len(messages) == 1
+
+        px_rec = V20PresExRecord(verified="false")
+
+        responder = MockResponder()
+        self.profile.context.injector.bind_instance(BaseResponder, responder)
+
+        await self.manager.send_pres_ack(px_rec)
+        messages = responder.messages
+        assert len(messages) == 1
+
     async def test_send_pres_ack_no_responder(self):
         px_rec = V20PresExRecord()
 
         self.profile.context.injector.clear_binding(BaseResponder)
         await self.manager.send_pres_ack(px_rec)
 
-    async def test_receive_pres_ack(self):
+    async def test_receive_pres_ack_a(self):
         conn_record = async_mock.MagicMock(connection_id=CONN_ID)
 
         px_rec_dummy = V20PresExRecord()
@@ -2099,6 +2117,24 @@ async def test_receive_pres_ack(self):
 
             assert px_rec_out.state == V20PresExRecord.STATE_DONE
 
+    async def test_receive_pres_ack_b(self):
+        conn_record = async_mock.MagicMock(connection_id=CONN_ID)
+
+        px_rec_dummy = V20PresExRecord()
+        message = async_mock.MagicMock(_verification_result="true")
+
+        with async_mock.patch.object(
+            V20PresExRecord, "save", autospec=True
+        ) as save_ex, async_mock.patch.object(
+            V20PresExRecord, "retrieve_by_tag_filter", autospec=True
+        ) as retrieve_ex:
+            retrieve_ex.return_value = px_rec_dummy
+            px_rec_out = await self.manager.receive_pres_ack(message, conn_record)
+            save_ex.assert_called_once()
+
+            assert px_rec_out.state == V20PresExRecord.STATE_DONE
+            assert px_rec_out.verified == "true"
+
     async def test_receive_problem_report(self):
         connection_id = "connection-id"
         stored_exchange = V20PresExRecord(