Multiple issues in the 0.7.3 release

[etschelp]

I'm using this ticket to collect issues that I found in the 0.7.3 release.

1. Tails file upload fails when a credDef is created and multi ledger support is enabled.

See discussion on the chat: https://chat.hyperledger.org/channel/aries-cloudagent-python?msg=XodRcp7dFxmncAWma I still do not get this to work.

2. Revocation notification has a wrong revocation id for V1 exchanges (V2 works as expected)

For example the credential acked event has the following link to the revocation registry:

{
        "rev_reg_id": "7D2jFxSPJP4zTAP9WAcJWo:4:7D2jFxSPJP4zTAP9WAcJWo:3:CL:571:CustomerBankRevoc:CL_ACCUM:01f6b385-28bc-420a-bcc7-00ac282b1d3c",
        "cred_rev_id": "5"
}

But in the revocation event the value of the revocation id becomes 6 instead of 5.

{
    "thread_id": "indy::7D2jFxSPJP4zTAP9WAcJWo:4:7D2jFxSPJP4zTAP9WAcJWo:3:CL:571:CustomerBankRevoc:CL_ACCUM:01f6b385-28bc-420a-bcc7-00ac282b1d3c::6",
    "comment": null
}

3a. V2 Credential exchange ignores the auto-respond-credential-request: false flag

If aca-py receives a V2 credential request it immediately responds even if the auto flag is set to false. Works as expected for V1.

3b. V2 Presentation exchange ignores the auto-respond-presentation-proposal: true flag

If aca-py receives a V2 presentation proposal and the auto respond flag is set to true aca-py does not automatically respond. Works a expected for V1.

4. Wallet type askar has issues when receiving V1 credentials.

Probably same as or related to #1588 Any other time answering a credential offer with a request fails with:

2022-01-13 11:40:34,184 aries_cloudagent.admin.server ERROR Handler error with exception: Record not found.
2022-01-13 11:40:34,184 aries_askar.native.aries_askar.ffi.store INFO 	src/ffi/store.rs:1007 | Closed session SessionHandle(980)
2022-01-13 11:40:34,185 aries_askar.native.aries_askar.backend.db_utils INFO 	src/backend/db_utils.rs:168 | Dropped pool connection
Traceback (most recent call last):
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/protocols/issue_credential/v1_0/routes.py", line 997, in credential_exchange_send_request
    cred_ex_record, connection_record.my_did
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/protocols/issue_credential/v1_0/manager.py", line 471, in create_request
    await cred_ex_record.save(session, reason="create credential request")
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/messaging/models/base_record.py", line 354, in save
    await storage.update_record(record, record.value, record.tags)
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/storage/askar.py", line 121, in update_record
    raise StorageNotFoundError("Record not found") from None
aries_cloudagent.storage.error.StorageNotFoundError: Record not found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/admin/server.py", line 163, in ready_middleware
    return await handler(request)
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/admin/server.py", line 200, in debug_middleware
    return await handler(request)
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aiohttp_apispec/middlewares.py", line 45, in validation_middleware
    return await handler(request)
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/admin/server.py", line 301, in check_token
    return await handler(request)
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/admin/server.py", line 391, in setup_context
    return await task
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/protocols/issue_credential/v1_0/routes.py", line 1017, in credential_exchange_send_request
    outbound_handler,
  File "/home/indy/.pyenv/versions/3.6.13/lib/python3.6/site-packages/aries_cloudagent/protocols/issue_credential/v1_0/__init__.py", line 63, in report_problem
    raise http_error_class(reason=err.roll_up) from err
aiohttp.web_exceptions.HTTPBadRequest: Record not found.

[swcurran]

@shaangill025 and @andrewwhitehead --- can the two of you please look at these? Shaanjot likely the 1, 2 and 3 as they are in the same area, Andrew the fourth because of the Askar connection.

Thanks

[andrewwhitehead]

For #4, it appears that the process is failing when trying to update the credential exchange record because it doesn't exist. This could be because auto-respond was enabled and the exchange has already completed and been removed. I can't see why this behaviour would be any different with the Askar backend.

[etschelp]

For #4, it appears that the process is failing when trying to update the credential exchange record because it doesn't exist. This could be because auto-respond was enabled and the exchange has already completed and been removed. I can't see why this behaviour would be any different with the Askar backend.

It's indeed different. I have two 0.7.3 agents, one running in indy the other in aksar, both have all auto-respond-credential-xxx flags set to false. The indy one works a s expected. The askar one receives the credential offer, but is not able to handle the credential request trigger because the record is missing. I double checked to see if maybe the exchange runs through automatically and the controller reacts on a wrong state, but thats not the case.

[solidnerd]

@etschelp Which Storage Backend are you running in the Acapy. Do you use Postgres or SQLite ?

[etschelp]

@etschelp Which Storage Backend are you running in the Acapy. Do you use Postgres or SQLite ?

Always Postgres

[solidnerd]

@etschelp We saw also errors on the Postgres Backend regarding encoding problems with utf-8 and askar. Did you tried it with sqlite ?

[etschelp]

What are your encoding problems? So far I did not experience any. But your question made me curious, by looking at the init methods in the indy and askar sdk's I do not see that there is any encoding set. Which means for postgres it comes back to the defaults. If you are using the docker image its derived from LANG=en_US.utf8 if not it is ascii, or what was set during initdb, or during createdb. So the first thing that I would check is if the encoding of the wallet db that aca-py is using is correct.

[andrewwhitehead]

Askar (via sqlx) always sets the client encoding to utf8 when creating a connection. What kind of encoding problems are you seeing?

[andrewwhitehead]

On my Mac I can run the performance demo with a test of 300 credential exchanges using the following (with Postgres.app installed): POSTGRES=1 LEDGER_URL=http://test.bcovrin.vonx.io python3 -m runners.performance --wallet-type askar. Note this uses the issue-credential-2.0/send endpoint to automate the send.

Startup duration: 5.05s
Faber      | Connected
Connect duration: 0.17s
Faber      | Publishing test schema
Alice      | Connected
Faber      | Schema ID: VYRv5TfbgWBuuzru4ayZkb:2:degree schema:42.9.95
Faber      | Publishing test credential definition
Faber      | Credential Definition ID: VYRv5TfbgWBuuzru4ayZkb:3:CL:131901:default
Publish duration: 5.73s
Faber      | Started 100 credential exchanges in 2.10s
Faber      | Started 100 credential exchanges in 3.75s
Faber      | Done starting 300 credential exchanges in 9.38s
Issuing credentials   100.0% [===================================================>] 300/300 eta [00:00] 
Receiving credentials 100.0% [===================================================>] 300/300 eta [00:00] 
Faber      | Completed 300 credential exchanges in 10.68s
Faber      | Average time per credential: 0.04s (28.09/s)
Alice      | alice602305 DB                 |             items |        items_tags
Alice      | ==========================================================================================
Alice      | 300 creds                      |      908   6568kB |     3914    848kB
Alice      | 
Faber      | faber446483 DB                 |             items |        items_tags
Faber      | ==========================================================================================
Faber      | 300 creds                      |       19     72kB |       45     72kB

[solidnerd]

@andrewwhitehead @etschelp We fixed by setting the LANG environment variable. I'll open a separate issue there we can discuss it not to derive from the topic. I think it's missing in the Docker Image.

[swcurran]

Can we get an update on this one at the ACA-Pug meeting today/tomorrow (2022.02.08)? We have a WIP PR and some back and forth that may address some of the issue. Ideally, I'd like to get the issues remaining separated into separate issues, because who doesn't like having more issues? 😂 It is easier to track one issue per issue.