Generate test cases for PSA key generation

[gilles-peskine-arm]

tests/scripts/generate_psa_tests.py generates test cases in a systematic way for combinations of key types, algorithms, operations, etc. The following families of test cases are already covered in the development branch or in work in progress:

• Attempting to import or generate a key when support for this key type is not enabled (NotSupported).
• Attempting to perform an operation with a key when the combination of key type, algorithm and operation is either impossible or not enabled (OpFail in PSA storage format: exercise keys (2.26) #4444).
• Importing a key of a supported type (covered via import-save-compare test cases in current storage format tests).

The goal of this task is to add another family of test cases: attempting to generate or derive a key. This includes both positive and negative test cases. This should have its own Python class and its own .data file, and probably its own .function file. Requirements for this task:

• The negative test cases are: key type not supported in the configuration; generation is impossible (public key types, independent of the configuration).
• For the positive test cases, check that the key attributes of the created key are correct.
• This task includes moving the existing support for key generation from NotSupported (added in Fix psa_generate_key(): return PSA_ERROR_INVALID_ARGUMENT for public key #5037) out to the new class. This may require refactoring some code from the NotSupported class so that it's available to the new class. (I think this will make the generation script easier to maintain; if you think it's better to keep all not-supported tests together, please let me know why and let's discuss it.)

Likely follow-ups, out of scope here:

• We'll need similar tests for derivation. They're more complicated because we don't support derivation for all key types and because derivation is itself more complicated.
• Perform some operations on the new key (i.e. call exercise_key). This needs extra machinery to know what algorithms a key type supports, which I'm working on in PSA storage format: exercise keys (2.26) #4444.
• Test writing the generated key to storage and reading it back.
• Remove manually written test cases from test_suite_psa_crypto that are redundant with the new automatic test cases. There's not much to do until we exercise the generated keys here.

[mprse]

@gilles-peskine-arm I would like to clarify and confirm if my understanding is correct.

We want to have test class to verify all cases (positive and negative) of key generation. I need to add test that takes all possible key attributes, tries to create a key based on given attributes and verifies the result (additionally verifies attributes of the created key on success) .

I can see the following key attributes that can be specified:

• life time
• usage flags
• algorithm
• bits
• type

The function may look like this:

/* BEGIN_CASE */
void generate_key( int key_life_time, int key_usage_flags, int key_algorithm, int key_type, int key_bits, int result)
{
    psa_key_attributes_t attributes = PSA_KEY_ATTRIBUTES_INIT;
    mbedtls_svc_key_id_t key_id = INVALID_KEY_ID;

    psa_key_lifetime_t _key_life_time = (psa_key_lifetime_t) key_life_time;
    psa_key_usage_t _key_usage_flags = (psa_key_usage_t) key_usage_flags;
    psa_algorithm_t _key_algorithm = (psa_algorithm_t) key_algorithm;
    psa_key_type_t _key_type = (psa_key_type_t) key_type;
    size_t _key_bits = (size_t) key_bits;
    psa_status_t _result = (psa_status_t) result;

    PSA_ASSERT( psa_crypto_init( ) );
    psa_set_key_life_time( &attributes, _key_life_time );
    psa_set_key_usage_flags( &attributes, _key_usage_flags );
    psa_set_key_algorithm( &attributes, _key_algorithm );
    psa_set_key_type( &attributes, _key_type );
    psa_set_key_bits( &attributes, _key_bits );
    TEST_EQUAL( psa_generate_key( &attributes, &key_id ),
                _result );
    
    // Verify attributs of the created key on success

exit:
    psa_destroy_key( key_id );
    PSA_DONE( );
}
/* END_CASE */

Now for this function I need to create python class to generate test cases data (e.g. class psa_generate_key_tests).

When test function, script and test data is ready I need to refactor class NotSupported and remove key generate tests as they will be already handled by class psa_generate_key_tests.

Key derivation is out of scope for this task? (The goal of this task is to add another family of test cases: attempting to generate or derive a key - but looking at the task name and fallow-up derivation is out of scope. Anyway I will start with key generation.)

Is there a documentation where is specified which key can be created and when we expect error?

[gilles-peskine-arm]

In this task, the only attributes that matter are the key type and size (bits). The goal is to enumerate all key types, and one or more size for each key type (it's neither possible nor useful to enumerate all possible sizes). There's code to enumerate key types in StorageFormat.all_keys_for_types and its auxiliary method keys_for_type:

key_types = sorted(self.constructors.key_types)
for key_type in self.constructors.generate_expressions(key_types):
    kt = crypto_knowledge.KeyType(key_type)
    for bits in kt.sizes_to_test():
        # generate test case(s)

lifetime can be 0 (a volatile key, not using a secure element) for all these tests, and we don't care about the policy (usage flags, algorithms) yet, they'll come with the “Perform some operations on the new key” follow-up.

I need to refactor class NotSupported and remove key generate tests as they will be already handled by class psa_generate_key_tests.

I see two possible ways to structure the script: either group all the generate test cases (both supported and not-supported), or group all the not-supported test cases (both import and generate). I'm not sure what the simplest way is. In the task description I wrote “moving the existing support for key generation from NotSupported (…) out to the new class. (…) I think this will make the generation script easier to maintain” — but now that I start spelling it out I see that this means that “generate test cases(s)” above will have to enumerate all the not-supported cases. (For example, if the key type is PSA_KEY_TYPE_ECC_KEY_PAIR(PSA_CURVE_FAMILY_SECP_R1) and the key size is 384, the generation operation is only supported if both PSA_WANT_KEY_TYPE_ECC_KEY_PAIR and PSA_WANT_ECC_SECP_R1_521 are enabled, meaning that there needs to be two negative test cases, one where the first symbol is defined but the second isn't and one where the first is not defined but the second is.) So maybe it's better after all to keep the not-supported test cases for generate in the not-supported class, and only move the invalid-argument ones to the new class, which would then only generate test cases expecting SUCCESS or INVALID_ARGUMENT.

Key derivation is out of scope for this task?

Right. It should be added later but I think it would add too much to the scope to do it together with generate.

Is there a documentation where is specified which key can be created and when we expect error?

The documentation of psa_generate_key, and of individual key types. The python code already has building blocks like info.constructors, automatic_dependencies and the crypto_knowledge module, so the only thing that's missing for this task is the knowledge of whether it's valid to request the generation of a key. All key types can be generated except for public keys.

[mprse]

@gilles-peskine-arm Thank you for clarification. Now everything is clear to me.