fix!: current CType meta schema ignores additional properties in claim contents

[rflechtner]

fixes KILTProtocol/ticket#2378

Introduces a new meta schema to fix the issue that additional properties were not validated.
Newly created CTypes would use this meta schema by default; legacy CTypes are still validated against the old meta schema.

• Makes sure that no additional properties are present on an IClaim
• Allows for selective disclosure (does not require all allowable properties to be present)
• Meta schema id uses content integrity protected URI, ideally uses decentralised solution
• Comes with a strategy for expanding claims to URIs and for checking the same claims on the credentialSubject of a Verifiable Credential
  • Unfortunately it seems that the only viable strategy so far is deriving a schema from the CType.
  • Better compatibility could be achieved by defining an optional @id property on all CTypes (to avoid failing credentialSubjecct verification on VCs, where this would be present after expanding), but this comes at a cost; any IClaim would pass verification even with an extra @id property.
• Upgrade path / interoperability strategy for credentials issued for the old CType version
  • Because the currently used json schema version does not allow defining a CType which derives the full set of properties from another CType (you can only add references for each property individually) the recommended upgrade path would be to create a new CType with the same properties and publish proof that it is equivalent to the old e.g. by publishing code that reproduces the derivation. Creating a CType where each property is derived from the old CType is another option, but checking that no properties were omitted and that the property names are identical is still a manual process. Compound/nested CTypes are also a bit more complicated to handle, so this seems like the less preferable option.
  • If all properties on the new CType are identical, verifiers would only need to add the new CType to the set of accepted CTypes. The credential could then be processed as before, no matter which version of the CType was used.

How to test:

Updated tests to cover both the old and new meta schema and CTypes created based on them.

Checklist:

• I have verified that the code works
• I have verified that the code is easy to understand
  • If not, I have left a well-balanced amount of inline comments
• I have left the code in a better state
• I have documented the changes (where applicable)

[rflechtner]

we should update the hosted version if merged

[ntn-x2]

Looks very good! I would like to understand a bit more how it all works, if you answer my questions. Then I would take a second look after that :)

[ntn-x2]

Question, probably out of scope for this PR: how is a verifier supposed to know what are the nested CTypes? Are they assumed to have this knowledge? Or might they only know the top-level CType they want to verify a credential against?

[rflechtner]

I was thinking the same thing but yes, I think it's out of scope. Maybe we need a CType registry in the sdk? Would also let us validate a CType only once (when adding to the registry) and not every time it is used.

[ntn-x2]

That would make our SDK less stateless, which is something we wanted to strive for as much as possible. it might be a good idea to at least reference this in a comment or something.

[rflechtner]

Guess that's true. The way I see it choosing a CType is a very conscious decision for a verifier, so they'd be familiar with it and know whether or not it is nested. At least the need to known what sort of data they will be working with in their application, so they'd look into that for sure.

[rflechtner]

As far as I can see all the upgrade paths described above but the one where people simply create a new CType based on the old would break quite a few things. So I'd suggest sticking to that. Besides this, I see only two open issue that we should clarify before putting the new CType out there:

1. Do we consider it V1 or is it simply draft-02? (maybe a tech board question?)
2. Do we allow nested properties (objects and arrays as values) even though selective disclosure is only possible for top-level properties? I.e. you can't omit an item from an array or a property from an object.

[ntn-x2]

Do we consider it V1 or is it simply draft-02? (maybe a tech board question?)

I think this version should be quite stable, right? At least as far as our credentials currently allow. If we want to support nested selected disclosure, we would have a v2 out there. So i would be in favour of having this a v1.

Do we allow nested properties (objects and arrays as values) even though selective disclosure is only possible for top-level properties? I.e. you can't omit an item from an array or a property from an object.

I think, for the sake of completeness, we could still allow any property, making it clear (either in the JSDocs or in the documentation itself), that only the top-level property can be revealed or hidden as a whole.

[rflechtner]

Do we allow nested properties (objects and arrays as values) even though selective disclosure is only possible for top-level properties? I.e. you can't omit an item from an array or a property from an object.

I think, for the sake of completeness, we could still allow any property, making it clear (either in the JSDocs or in the documentation itself), that only the top-level property can be revealed or hidden as a whole.

As it turns out, there are quite a few details to figure out if we were to allow this, so I think it should be its own ticket with design phase and all. It would also be something that does not affect existing CTypes at all, so it could be quite simply the next meta schema version. See https://github.com/KILTprotocol/ticket/issues/2456.

[ntn-x2]

Just a question, otherwise LGTM! Very very very very nice job! 👍