Use simplified encodings for Kyber and Dilithium keys

[randombit]

Currently we follow (just approximately) an internet draft for the X.509 encoding, but using private allocation OIDs.

This is actually the worst possible situation because

• If the draft is finalized in a way that doesn't match exactly what we're doing, then we'll have to support the final RFC as well as our current format. I know we are incompatible with the current draft, and the current draft anyway seems very half-baked with numerous underspecified parts, so I don't think it's even possible to be compatible with the draft without guessing.
• If the draft is abandoned, then we'll be using this one off format that is incompatible with OQS and similar code

To minimize long term pain I think the best course of action is to remove the existing ASN.1 based key encodings for Kyber and Dilithium, and have our private allocation OIDs simply export the "Raw" encoding as used by the reference implementation as an OCTET STRING. That's a key format we'll have to continue to support anyway since it looks like the ecosystem will end up splitting into "raw octets as represented by the reference code" vs "destructuring the public key into ASN.1 for no particular reason". But at least this way we only end up having to support at most one of the latter.

@reneme @securitykernel thoughts on this?

[securitykernel]

That's probably fine. Let's wait what @reneme thinks. He's on a PQC meetup today, so maybe he could also discuss with others.

[reneme]

I tend to agree, too. As long as these algorithms are not-yet-standardized they'll likely be used for proof of concept and experimental work anyway. Compatibility with the reference implementations and OQS should therefore be the first priority. We implemented those extra encodings mainly because "we could" without an actual use case.

That said, I'm wondering about the overall compatibility guarantees we want to give for those algorithms in the near future. After all they might still change during the standardization process. Maybe we should reconsider the experimental modules for Kyber and Dilithium: In that case I wouldn't see much harm for leaving the extra encodings in.

[reneme]

I tend to agree, too. As long as these algorithms are not-yet-standardized they'll likely be used for proof of concept and experimental work anyway. Compatibility with the reference implementations and OQS should therefore be the first priority. We implemented those extra encodings mainly because "we could" without an actual use case.

That said, I'm wondering about the overall compatibility guarantees we want to give for those algorithms in the near future. After all they might still change during the standardization process. Maybe we should reconsider the experimental modules for Kyber and Dilithium: In that case I wouldn't see much harm for leaving the extra encodings in.

[reneme]

Done with the PRs referenced above.