Merge pull request openwallet-foundation#72 from hyperledger/main

update main

.circleci/config.yml

@@ -2,7 +2,7 @@ version: 2
 jobs:
   agent-build:
     docker:
-      - image: bcgovimages/von-image:py36-1.15-0
+      - image: bcgovimages/von-image:py36-1.15-1
     steps:
       - checkout
       - restore_cache:

.github/workflows/integrationtests.yml

@@ -0,0 +1,27 @@
+name: acapy-integration-tests
+on:
+  workflow_dispatch:
+  pull_request:
+    branches:
+      - main
+defaults:
+  run:
+    shell: bash
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    if: (github.event_name == 'pull_request' && github.repository == 'hyperledger/aries-cloudagent-python') || (github.event_name != 'pull_request')
+    steps:
+      - name: checkout-acapy
+        uses: actions/checkout@v2
+        with:
+          path: acapy
+      - name: run-von-network
+        uses: ./acapy/actions/run-von-network
+      - name: run-indy-tails-server
+        uses: ./acapy/actions/run-indy-tails-server
+      - name: run-integration-tests
+        uses: ./acapy/actions/run-integration-tests
+        # to run with a specific set of tests include the following parameter:
+        # with:
+        #   TEST_SCOPE: "-t @T001-RFC0037"

CHANGELOG.md

@@ -1,6 +1,6 @@
 # 0.6.0
 
-## February 5, 2021
+## February 25, 2021
 
 This is a significant release of ACA-Py with several new features, as well as changes to the internal architecture in order to set the groundwork for using the new shared component libraries: [indy-vdr](https://github.com/hyperledger/indy-vdr), [indy-credx](https://github.com/hyperledger/indy-shared-rs), and [aries-askar](https://github.com/hyperledger/aries-askar).
 

CONTRIBUTING.md

@@ -6,8 +6,9 @@ For significant changes, please open an issue first to discuss the proposed chan
 
 (If you are new to GitHub, you might start with a [basic tutorial](https://help.github.com/articles/set-up-git) and check out a more detailed guide to [pull requests](https://help.github.com/articles/using-pull-requests/).)
 
-Pull requests will be evaluated by the repository guardians on a schedule and if deemed beneficial will be committed to the `main` branch. Pull requests should have a descriptive name and include an summary of all changes made in the pull request description.
+Pull requests will be evaluated by the repository guardians on a schedule and if deemed beneficial will be committed to the `main` branch. Pull requests should have a descriptive name, include an summary of all changes made in the pull request description, and include unit tests that provide good coverage of the feature or fix. A Continuous Integration (CI) pipeline is executed on all PRs before review and contributors are expected to address all CI issues identified. Where appropriate, PRs that impact the
+end-user and developer demos in the repo should include updates or extensions to those demos to cover the new capabilities.
 
 If you would like to propose a significant change, please open an issue first to discuss the work with the community.
 
-All contributors retain the original copyright to their stuff, but by contributing to this project, you grant a world-wide, royalty-free, perpetual, irrevocable, non-exclusive, transferable license to all users **under the terms of the license under which this project is distributed.**
+All contributors retain the original copyright to their stuff, but by contributing to this project, you grant a world-wide, royalty-free, perpetual, irrevocable, non-exclusive, transferable license to all users **under the terms of the [license](./LICENSE) under which this project is distributed.**

DevReadMe.md

@@ -27,11 +27,11 @@ See the [README](README.md) for details about this repository and information ab
 
 Aries Cloud Agent Python (ACA-Py) is a configurable, extensible, non-mobile Aries agent that implements an easy way for developers to build decentralized identity services that use verifiable credentials.
 
-The information on this page assumes you are developer with a background in decentralized identity, Indy, Aries and verifiable credentials. If you aren't familiar with those concepts and projects, please use our [Getting Started Guide](docs/GettingStartedAriesDev/README.md) to learn more.
+The information on this page assumes you are developer with a background in decentralized identity, Indy, Aries and verifiable credentials. If you aren't familiar with those concepts and projects, please use our [Getting Started Guide](/docs/GettingStartedAriesDev/README.md) to learn more.
 
 ## Developer Demos
 
-To put ACA-Py through its paces at the command line, checkout our [demos](docs/GettingStartedAriesDev/AriesDeveloperDemos.md) page.
+To put ACA-Py through its paces at the command line, checkout our [demos](/docs/GettingStartedAriesDev/AriesDeveloperDemos.md) page.
 
 ## Running
 
@@ -205,7 +205,7 @@ Please also refer to the [contributing guidelines](/CONTRIBUTING.md) and [code o
 
 ## Publishing Releases
 
-The [publishing](https://github.com/hyperledger/aries-cloudagent-python/blob/master/PUBLISHING.md) document provides information on tagging a release and publishing the release artifacts to PyPi.
+The [publishing](https://github.com/hyperledger/aries-cloudagent-python/blob/main/PUBLISHING.md) document provides information on tagging a release and publishing the release artifacts to PyPi.
 
 ## Dynamic Injection of Services
 

Mediation.md

@@ -67,7 +67,7 @@ See [Aries RFC 0211: Coordinate Mediation Protocol](https://github.com/hyperledg
 
 ## Mediator Message Flow Overview
 
-![Mediator Message Flow](./docs/assets/mediation-message-flow.png)
+![Mediator Message Flow](/docs/assets/mediation-message-flow.png)
 
 ## Using a Mediator
 

Multitenancy.md

@@ -37,7 +37,7 @@ The wallets used by the different tenants are called **sub wallets**. A sub wall
 
 The **base wallet** however, takes on a different role and has limited functionality. Its main function is to manage the sub wallets, which can be done using the [Multi-tenant Admin API](#multi-tenant-admin-api). It stores all settings and information about the different sub wallets and will route incoming messages to the corresponding sub wallets. See [Message Routing](#message-routing) for more details. All other features are disabled for the base wallet. This means it cannot issue credentials, present proof, or do any of the other actions sub wallets can do. This is to keep a clear hierarchical difference between base and sub wallets
 
-![Multi-tenancy Architecture](./docs/assets/multitenancyDiagram.png)
+![Multi-tenancy Architecture](/docs/assets/multitenancyDiagram.png)
 
 ### Usage
 
@@ -223,4 +223,4 @@ For deterministic JWT creation and verification between restarts and multiple in
 
 When using the SwaggerUI you can click the :lock: icon next to each of the endpoints or the `Authorize` button at the top to set the correct authentication headers. Make sure to also include the `Bearer ` part in the input field. This won't be automatically added.
 
-![](./docs/assets/adminApiAuthentication.png)
+![](/docs/assets/adminApiAuthentication.png)

OutboundQueue.md

@@ -0,0 +1,53 @@
+# Outbound Queues in ACA-py
+
+## Background
+
+By default, messages often stay in ACA-py memory for long periods of time without being delivered. As a result, when the ACA-py Python process is terminated unexpectedly, messages are lost.
+
+But with recent changes, outbound messages can now be sent to a message queue of your choice instead of being delivered by ACA-py. This queue is external to the ACA-py process, and can be configured for the durability requirements you want. This new concept of an "outbound queue" is intended to be an optional replacement to the current ACA-py outbound transport (i.e. option `-ot`, `--outbound-transport`).
+
+If you run an outbound queue, you will also need to run a new service, a delivery agent, to actually deliver the message. See more details below.
+
+## Usage Details
+
+A new set of commandline options have been added to provide a way for users to "opt in" to use of the outbound queue. These new options are as follows:
+
+- `-oq`, `--outbound-queue`: specifies the queue connection details.
+- `-oqp`, `--outbound-queue-prefix`: defines a prefix to use when generating the topic key.
+- `-oqc`, `--outbound-queue-class`: specify the location of a custom queue class.
+
+Only the first, `--outbound-queue`, is required if you would like to opt into the outbound queue to replace `--outbound-transport`. The input for this option takes the form `[protocol]://[host]:[port]`. So for example, if the queue I want to use is Redis, on host `myredis.mydomain.com` using the default port for Redis, the string would be as follows: `redis://myredis.mydomain.com:6379`
+
+The second option, `--outbound-queue-prefix`, specifies the queue topic prefix. The queue topic is generated in the following form: `{prefix}.outbound_transport`. The default value for this commandline option is the value `acapy`, so a queue key of `acapy.outbound_transport` is generated in the case of the default settings. ACA-py will send messages to the queue using this generated key as the topic.
+
+The third option, `--outbound-queue-class`, specifies the queue backend. By default, this is `aries_cloudagent.transport.outbound.queue.redis:RedisOutboundQueue`, which specifies ACA-py's builtin Redis `LIST` backend. Users can define their own class, inheriting from `BaseOutboundQueue`, to implement a queue backend of their choice. This commandline option is the official entrypoint of ACA-py's pluggable queue interface. Developers must specify a Python dotpath to a module importable in the current `PYTHONPATH`, followed by a colon, followed by the name of their custom class.
+
+## Delivery Agent
+
+When using `--outbound-queue` instead of `--outbound-transport`, ACA-py no longer delivers the messages to destinations. Instead, a delivery service ([a prototype can be found here](https://github.com/andrewwhitehead/aca-deliver)) would need to be run. This service should pick up a message from the queue and then deliver that message. 
+
+When running `--outbound-queue`, ACA-py serializes messages to be sent to the queue by using MessagePack. MessagePack is a protocol to serialize content into a compact binary format. ACA-py generates keys in MessagePack as follows:
+- `endpoint` - specifies the endpoint for the message.
+- `headers` - specifies a set of key-value pairs representing message headers.
+- `payload` - the raw binary content of the message.
+
+The delivery service will need to deserialize the binary content on the consuming end. The result will then be a key-value data structure (for example, a `dict` in Python). So the deseralized message, deserialized into a Python `dict` for example, would be in the following form:
+```
+{
+    "headers": {"Content-Type": "..."},
+    "endpoint": "...",
+    "payload": "..."
+}
+```
+The delivery agent should process this message and deliver it to the recipient as appropriate.
+
+## Backend-Specific Notes
+
+### Redis
+
+Value for `--outbound-queue-class` to use this backend:
+- `aries_cloudagent.transport.outbound.queue.redis:RedisOutboundQueue`
+
+This is a queue backend, using the `LIST` data type in Redis. When using Redis, the delivery service consuming this queue in order to send outbound messages over transport will need to pop from the left side of the queue (i.e. the Redis `LPOP` command) to get messages in the order they were sent.
+
+Users will need to configure [Redis persistence](https://redis.io/topics/persistence) to gain message durability benefits in their Redis deployment. Redis by default runs entirely in-memory, so it is subject to the same data loss characteristics as ACA-py unless you also configure it to run in persistence mode.