No QSC TLS signature support

[cryptojane94]

Describe the bug
I generated the keys and ceritifcates following the procedure described, but when running the server the connection couldn't be established because I got the error:

error setting certificate
80DBC4E8877F0000:error:0A0000F7:SSL routines:ssl_set_cert_unknown certificate type:ssl/ssl_rsa.c:257:

I have installed Ubuntu 22.04.3 LTS and I am using OpenSSL 3.2.0-dev

[baentsch]

This is a typical error (message) when oqsprovider didn't get properly activated. Please confirm that it is shown active when running openssl list -providers as per USAGE.md

[Krylol94]

Hello,

I encountered the same issue. If openssl list -providers is run with the -provider-path option I can see the OpenSSL OQS provider as active.

[baentsch]

Hello,

I encountered the same issue. If openssl list -providers is run with the -provider-path option I can see the OpenSSL OQS provider as active.

Ahh -- it works when -provider-path is set, but not when unset?! Then you (all) either need to set OPENSSL_MODULES suitably (to the directory where oqsprovider is stored) or ensure "modules" is suitably set in "openssl.cnf" (and you load the proper "openssl.cnf"): https://github.com/open-quantum-safe/oqs-provider/blob/main/USAGE.md#activation

[Krylol94]

Thanks for your reply.

I included the lines:

oqsprovider = oqsprovider_sect
[oqsprovider_sect]
activate = 1

But it's not clear to me if the -provider-path line has to be included after activate = 1:

oqsprovider = oqsprovider_sect
[oqsprovider_sect]
module  = /to/oqs/folder/oqs.so
activate = 1

But it is still not working.

In my previous tests the provider has been always activated with -provider-path . I guess that the issue may be caused my something else.

[baentsch]

But it's not clear to me if the -provider-path line has to be included after activate =

No; "module=" as you did it.

I guess that the issue may be caused my something else.

Are you 100% certain that this "openssl.cnf" is loaded? Set OPENSSL_CONF env var explicitly to be sure. Also check that "ls -l /to/oqs/folder/oqs.so" shows the proper binary.

[Krylol94]

I am sure that openssl.cnf has been loaded correctly. I am not sure how to check the if the binary is the proper one, but up to now I was able to generate the certificates, while the attempt to open a connection fails.

[baentsch]

up to now I was able to generate the certificates, while the attempt to open a connection fails

Well, if you can generate proper QSC certs then the provider is surely loaded. What happens if you (try to) display the cert (via the openssl x509 command)? Also, did you build the provider from source code or are you using our docker image "openquantumsafe/oqs-ossl3" (to eliminate any build errors)?

[Krylol94]

I opened the certificates with openssl x509 -text -in dilithium3_srv.crt -noout and it shows the error 'Unable to load Public Key'.

I build it with the cmake command shown in the README.md

[baentsch]

'Unable to load Public Key'

with 99% certainty means that oqsprovider is not active. What is the output of running openssl list -providers? What are the key environment variables set to? LD_LIBRARY_PATH, OPENSSL_MODULES, OPENSSL_CONF? Please try the docker image.

[DaSoty]

Hi, although @baentsch know infinitely more about this than I do, maybe I can help you with the steps I followed to install it and it worked correctly:

git clone https://github.com/open-quantum-safe/oqs-provider.git
cd oqs-provider
apt remove openssl
./scripts/fullbuild.sh -F -OPENSSL_INSTALL
cmake --install _build
cp .local/bin/* /usr/bin/
nano /root/oqs-provider/.local/ssl/openssl.cnf
./scripts/runtests.sh

If you already have OpenSSL installed and it is in another path such as /usr/local/ssl, I had to modify the fullbuild.sh file as you can see at this discussion. Remember that in this case you will need to modify the file /usr/local/ssl/openssl.cnf.

If I have said anything wrong, @baentsch please correct me.

Thank you so much!

[baentsch]

If you already have OpenSSL installed and it is in another path such as /usr/local/ssl

In such case, please set the environment variable OPENSSL_INSTALL when calling fullbuild.sh e.g., as such: OPENSSL_INSTALL=/usr/local/ssl ./scripts/fullbuild.sh. Then it should not be necessary to change that script.

[Krylol94]

'Unable to load Public Key'

with 99% certainty means that oqsprovider is not active. What is the output of running openssl list -providers? What are the key environment variables set to? LD_LIBRARY_PATH, OPENSSL_MODULES, OPENSSL_CONF? Please try the docker image.

Consider that I have two version of openssl installed:

• OpenSSL 3.0.2, which is the system one, with this one I loaded the provider but again I couldn't start a server connection (the same way as explained before)
• OpenSSL 3.2.0 which has been installed in a separate folder and is loaded with the command showed in the following

In the terminal I type:
LD_LIBRARY_PATH=/to/path/openssl/install/lib64/:$LD_LIBRARY_PATH /to/path/openssl/install/bin/openssl list -providers

The output I obtain is:
Providers:
default
name: OpenSSL Default Provider
version: 3.2.0
status: active
oqsprovider
name: OpenSSL OQS Provider
version: 0.5.2-dev
status: active

I am sure that the 3.2.0-dev version of OpenSSL is run and the provider is loaded, as shown here.

[Krylol94]

Hi, although @baentsch know infinitely more about this than I do, maybe I can help you with the steps I followed to install it and it worked correctly:

git clone https://github.com/open-quantum-safe/oqs-provider.git
cd oqs-provider
apt remove openssl
./scripts/fullbuild.sh -F -OPENSSL_INSTALL
cmake --install _build
cp .local/bin/* /usr/bin/
nano /root/oqs-provider/.local/ssl/openssl.cnf
./scripts/runtests.sh

If you already have OpenSSL installed and it is in another path such as /usr/local/ssl, I had to modify the fullbuild.sh file as you can see at this discussion. Remember that in this case you will need to modify the file /usr/local/ssl/openssl.cnf.

If I have said anything wrong, @baentsch please correct me.

Thank you so much!

I think that this is my case.

[baentsch]

Consider that I have two version of openssl installed:

OpenSSL 3.0.2, which is the system one, with this one I loaded the provider but again I couldn't start a server connection (the same way as explained before)
OpenSSL 3.2.0 which has been installed in a separate folder and is loaded with the command showed in the following

Thanks for the additional check/explanation. So do I understand correctly that you cannot successfully decode an X509 cert and/or start a server when running OpenSSL3.2.0? If so, please show the complete command line and output of trying so.

Or is the statement "with this one I loaded the provider but again I couldn't start a server connection (the same way as explained before)" indeed only applying to OpenSSL3.0.2? In other words, does OpenSSL3.2.0 work as expected in all regards and 3.0.2 does not (as is documented)?

[Krylol94]

Thanks for the additional check/explanation. So do I understand correctly that you cannot successfully decode an X509 cert and/or start a server when running OpenSSL3.2.0? If so, please show the complete command line and output of trying so.

I build again the oqs-provider with:

OPENSSL_INSTALL=/to/path/openssl/install/ssl ./scripts/fullbuild.sh -F

and run again the commands in USAGE.md > Creating keys and certificates and Setting up a (quantum-safe) test server

The output of the commands, considering that OpenSSL3.2.0-dev is used is:

• Decode with LD_LIBRARY_PATH=/to/path/openssl/install/lib64/:$LD_LIBRARY_PATH /to/path/openssl/install/bin/openssl x509 -text -in dilithium3_srv.crt -noout
  with output

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            2b:81:48:ca:c0:21:3c:01:cd:ec:e9:4b:c7:33:cb:2d:ed:f9:77:39
        Signature Algorithm: dilithium3
        Issuer: CN=test CA
        Validity
            Not Before: Aug 16 07:29:37 2023 GMT
            Not After : Aug 15 07:29:37 2024 GMT
        Subject: CN=test server
        Subject Public Key Info:
            Public Key Algorithm: dilithium3
                dilithium3 public key:
                PQ key material:
...

X509v3 extensions:
            X509v3 Subject Key Identifier: 
                1E:7F:9B:8D:1A:21:D5:57:66:3F:1C:68:10:C5:08:A5:60:14:71:96
            X509v3 Authority Key Identifier: 
                6D:3F:CE:65:37:17:50:D2:CA:DB:8A:DB:87:6D:D5:A8:CD:42:E2:97
    Signature Algorithm: dilithium3
    Signature Value:
...

This time is correct.

• Server Connection
  LD_LIBRARY_PATH=/to/path/openssl/install/lib64/:$LD_LIBRARY_PATH /to/path/openssl/install/bin/openssl s_server -cert dilithium3_srv.crt -key dilithium3_srv.key -www -tls1_3 -groups kyber768:frodo640shake

with output:

Using default temp DH parameters
error setting certificate
80FB761AA37F0000:error:0A0000F7:SSL routines:ssl_set_cert:unknown certificate type:ssl/ssl_rsa.c:257:

Or is the statement "with this one I loaded the provider but again I couldn't start a server connection (the same way as explained before)" indeed only applying to OpenSSL3.0.2? In other words, does OpenSSL3.2.0 work as expected in all regards and 3.0.2 does not (as is documented)?

The problem is that I have the same issue with both OpenSSL3.0.2 and OpenSSL3.2.0 when a server connection is started.

Thanks a lot.

[baentsch]

This time is correct.

Good to hear. What changed compared to the first try? Can this be applied to the server start too?

The problem is that I have the same issue with both OpenSSL3.0.2 and OpenSSL3.2.0 when a server connection is started.

Again, this code path can only be taken if the provider is not active -- or the certificate chain is bad(ly built). Again, please use our ready-built dockerfile to eliminate the latter as an error cause (and to validate your scripts). Remote debugging with too many "moving parts" is just not really doable.

[Krylol94]

This time is correct.

Good to hear. What changed compared to the first try? Can this be applied to the server start too?

The problem is that I have the same issue with both OpenSSL3.0.2 and OpenSSL3.2.0 when a server connection is started.

I used the wrong OpenSSL version, unfortunately the server connection is still not working.

Again, this code path can only be taken if the provider is not active -- or the certificate chain is bad(ly built). Again, please use our ready-built dockerfile to eliminate the latter as an error cause (and to validate your scripts). Remote debugging with too many "moving parts" is just not really doable.

How should I use it?

Thanks for your answers.

[baentsch]

I used the wrong OpenSSL version, unfortunately the server connection is still not working.

So are you now using the correct version (3.2.0-dev)? And are you certain to have generated all certificates with the correct OpenSSL version? Verify the chain before use.

How should I use it?

Usage is documented (https://hub.docker.com/repository/docker/openquantumsafe/oqs-ossl3/general)

[Krylol94]

So are you now using the correct version (3.2.0-dev)? And are you certain to have generated all certificates with the correct OpenSSL version? Verify the chain before use.

Now I am sure about it, the certificates has been generated with the correct OpenSSL version and again, the OQS provider is shown as actived.

How should I use it?

Usage is documented (https://hub.docker.com/repository/docker/openquantumsafe/oqs-ossl3/general)

In docker I tried to replicate the example is USAGE.md with certificate generation and the attempt to establish a server connection:

• The certificates looks like the one I generated before, I suppose that this step has been done correctly:
• The server connection shows else,

when openssl x509 -req -in dilithium3_srv.csr -out dilithium3_srv.crt -CA dilithium3_CA.crt -CAkey dilithium3_CA.key -CAcreateserial -days 365 is issued in the docker container I obtain:

Using default temp DH parameters
488BB3F8027F0000:error:80000062:system library:BIO_bind:Address in use:crypto/bio/bio_sock2.c:240:calling bind()
488BB3F8027F0000:error:10000075:BIO routines:BIO_bind:unable to bind socket:crypto/bio/bio_sock2.c:242:
   0 items in the session cache
   0 client connects (SSL_connect())
   0 client renegotiates (SSL_connect())
   0 client connects that finished
   0 server accepts (SSL_accept())
   0 server renegotiates (SSL_accept())
   0 server accepts that finished
   0 session cache hits
   0 session cache misses
   0 session cache timeouts
   0 callback cache hits
   0 cache full overflows (128 allowed)

I am not sure if I am not using properly docker.

[baentsch]

The error message above cannot be returned when issuing an openssl x509 command, but only when setting up a server (openssl s_server). In in that case it is indicative of a server already using the same port -- which is the case if you started the docker without parameter: That image auto-starts a QSC-enabled TLS server. You can avoid that by passing the parameter sh to the docker invocation (docker run -it openquantumsafe/oqs-ossl3 sh). Please read up a bit on docker usage: It's really worthwhile knowing the basics of that -- if only to gain platform independence for your work.

[cryptobsv]

I see the same error as described in the original post. I'm on oqsprovider commit f463621 and openssl commit 79cdbe893da0c613db97356d05c0b088e885707f. I have built and installed openssl locally (in addition to my system installation). I have modified openssl.cnf, so oqsprovider is found and used. For the following, I'm in the installation directory of this local openssl version.

$ LD_LIBRARY_PATH=$PWD/lib64 bin/openssl list -providers
Providers:
  default
    name: OpenSSL Default Provider
    version: 3.2.0
    status: active
  oqsprovider
    name: OpenSSL OQS Provider
    version: 0.5.2-dev
    status: active

$ LD_LIBRARY_PATH=$PWD/lib64 bin/openssl req -x509 -new -newkey dilithium2 -keyout key.pem -out cert.pem -nodes -subj "/CN=oqstest" -days 365                                               
-----

$ LD_LIBRARY_PATH=$PWD/lib64 bin/openssl verify cert.pem
cert.pem: OK

$ LD_LIBRARY_PATH=$PWD/lib64 bin/openssl s_server -cert cert.pem -key key.pem -curves kyber512 -www -tls1_3 -accept 127.0.0.1:8080
Using default temp DH parameters
error setting certificate
800B0AFC147F0000:error:0A0000F7:SSL routines:ssl_set_cert:unknown certificate type:ssl/ssl_rsa.c:257:

I have used strace on the last command (i.e. openssl s_server) and verified that the local versions of libcrypto and libssl are loaded, that oqsprovider is found and that the files with certificate and key are found. I have also tried to replace the cert and key with the pre-generated ones from your docker container with oqsprovider, but got the same error.

I can see that openssl s_serveris running in your docker container. Which commit ids of openssl and oqsprovider did you use to build this container?

Is there anything I can do to diagnose this more precisely? Thanks.

[baentsch]

Which commit ids of openssl and oqsprovider did you use to build this container?

CI built the container: The log of the build is this -- but I fail to spot the commit id :-( On the binary, I'm afraid I'm not aware of a mechanism to determine the commit id in an openssl build. oqsprovider is built from commit f463621 (available via openssl list -verbose -providers).

Is there anything I can do to diagnose this more precisely? Thanks.

Not really -- this is pretty much all I'd have recommended. Just one question for me to try to reproduce: Which environment/platform are you using? OS, compiler, etc.

[cryptobsv]

$ uname -a                                          
Linux kali 6.3.0-kali1-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.3.7-1kali1 (2023-06-29) x86_64 GNU/Linux

$ gcc --version                                                                              
gcc (Debian 13.1.0-6) 13.1.0

Additional info: I built openssl with debug info and stopped in ssl_cert_lookup_by_pkey(). I found that ctx->sigalg_list_len is zero. From there onwards, it is clear the error occurs. However, I don't understand why this variable is zero.

$ LD_LIBRARY_PATH=$PWD/lib64 bin/openssl list -signature-algorithms

lists, among many others:

  dilithium2 @ oqsprovider

Do you know where ctx->sigalg_list_len is initialized? Should I have a look there?

[baentsch]

Do you know where ctx->sigalg_list_len is initialized?

I should as I added that code to OpenSSL in openssl/openssl#19312: https://github.com/openssl/openssl/blob/6404d064b8012a2c353603a3b3effa6289313d61/ssl/t1_lib.c#L662

Should I have a look there?

Indeed that'd be the place to understand why the OQS sigalgs don't get "accepted" in your setup.

[cryptobsv]

It seems, the code referenced in the previous comment never runs in my setup. To investigate, I set a breakpoint on ssl_load_sigalgs(). Through a chain of functions this calls oqs_provider_get_capabilities() in oqsprovider. There, it seems, OSSL_CAPABILITY_TLS_SIGALG_NAME is undefined. So, when asked for the capability "TLS-SIGALG", the function returns 0.
Hence, I suspect something is wrong with my build configuration. Shouldn't OSSL_CAPABILITY_TLS_SIGALG_NAME be defined?

[baentsch]

Very good analysis, Thanks @cryptobsv !

Shouldn't OSSL_CAPABILITY_TLS_SIGALG_NAME be defined?

Absolutely. Presence of that (OpenSSL) define is prerequisite for proper pluggable signature support: https://www.openssl.org/docs/manmaster/man7/provider-base.html. It would be a serious bummer for that define to not be available (in "master"/3.2.0-dev); unavailability of it in OpenSSL 3.1 or 3.0 in turn would be expected --- so my question is whether you have compiled oqs-provider using OpenSSL3.0 or 3.1 header files? Then indeed things would fail in just the way described: QSC certificates worked since OpenSSL3.0, but QSC-TLS-session setup only since/in/with 3.2(-dev).

Before you state it, I'll say it: Building oqsprovider with such old openssl code should at least issue a clear warning at build time about features not supported due to the given openssl (build) version. Will add that.

[cryptobsv]

Yes, this is it. I built oqsprovider with scripts/fullbuild.sh. I had OPENSSL_INSTALL point to my local openssl 3.2.0-dev. I had a look into _build/CMakeCache.txt. The cmake variables OPEN_SSL_CRYPTO_LIBRARY, OPENSSL_ROOT_DIR and OPENSSL_SSL_LIBRARY were all pointing to the right place, i.e. my local openssl 3.2.0-dev installation. However, OPENSSL_INCLUDE_DIR pointed to /usr/include. Hence, the headers of my openssl system installation were used. I changed this manually to point to the include directory of my local openssl installation, rebuilt oqsprovider, with cmake --build . and now everything works.
Perhaps OPENSSL_INCLUDE_DIR should be set in scripts/fullbuild.sh or in some CMakeLists.txt", depending on OPENSSL_INSTALL`?
Anyway, thanks for the quick responses.

[baentsch]

Thanks for the thorough analysis/"attribution research", @cryptobsv ! I now wonder what caused OPENSSL_INCLUDE_DIR to be set to the wrong place. Is that an env var (pre)set in your environment? Are you aware of what software set that (value)? In my understanding, this variable should be set by the cmake command "find_package" -- which in turn is driven by the variable OPENSSL_ROOT_DIR -- and that is set by fullbuild.sh.

Final question (also to @cryptojane94 ): Have you run ./scripts/runtests.sh -V (verbose output)? Did that output inform you about an OpenSSL version mismatch -- most notably in the TLS run test?

[cryptobsv]

env|grep OPENSSL is empty.
I agree, according to the cmake documentation, find_package() should set the paths correctly, but for some reason cmake seems to mix the path to the libraries of my user installation of openssl with the path to include of my system installation. When I add

message(STATUS "OPENSSL_CRYPTO_LIBRARY=${OPENSSL_CRYPTO_LIBRARY}")
message(STATUS "OPENSSL_INCLUDE_DIR=${OPENSSL_INCLUDE_DIR}")

right after find_package(OpenSSL 3.0 REQUIRED) in CMakeLists.txt, I see

-- OPENSSL_CRYPTO_LIBRARY=/home/and/so/on/openssl-local/lib64/libcrypto.so
-- OPENSSL_INCLUDE_DIR=/usr/include

in the output of scripts/fullbuild.sh. I'm using cmake version 3.26.4.
Yes, scripts/runtests.sh -V reports TLS-SIG handshake test not enabled. Update OpenSSL to more current version.
During my initial investigation I also ran runtests.sh, but without -V. Perhaps runtests.sh could report the number of skipped tests in the bottom line. I just saw 100% passed and stopped looking further in that direction. A hint like "n tests skipped. Run with -V for details" or similar might help.
I'm happy to run further diagnostics, if it helps.

[baentsch]

cmake seems to mix the path to the libraries of my user installation of openssl with the path to include of my system installation.

Indeed, it looks like a cmake bug. When I add the same output in my setup (Linux Mint 20.1, cmake 3.16.3), both show the same (correct OPENSSL_ROOT_DIR-based) path.

A hint like "n tests skipped. Run with -V for details" or similar might help.

Yup - Wanted to do that when adding the warning about (having to) skip that test, but ctest simply "swallows" all output if not run with "-V". If you'd have a suggestion how else to make this visible, I'm all ears. A rather crude approach would be to set -V as default in "runtests.sh", of course.

[baentsch]

FWIW and for the fun of it, I started a docker image of your Linux ("Kali") distro, installed cmake git ninja-build and tried to build oqs-provider. This already fails in liboqs:

cmake ..
-- The C compiler identification is GNU 13.1.0
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test CC_SUPPORTS_WA_NOEXECSTACK
-- Performing Test CC_SUPPORTS_WA_NOEXECSTACK - Success
-- Performing Test LD_SUPPORTS_WL_Z_NOEXECSTACK
-- Performing Test LD_SUPPORTS_WL_Z_NOEXECSTACK - Success
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Alg enablement unchanged
CMake Error at /usr/share/cmake-3.26/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the
  system variable OPENSSL_ROOT_DIR (missing: OPENSSL_CRYPTO_LIBRARY
  OPENSSL_INCLUDE_DIR) (Required is at least version "1.1.1")
Call Stack (most recent call first):
  /usr/share/cmake-3.26/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.26/Modules/FindOpenSSL.cmake:670 (find_package_handle_standard_args)
  CMakeLists.txt:133 (find_package)


-- Configuring incomplete, errors occurred!

openssl version
OpenSSL 3.0.10 1 Aug 2023 (Library: OpenSSL 3.0.10 1 Aug 2023)

--> This distro has OpenSSL installed and cmake (find_packages) fails to locate it.

I begin to wonder how you even got as far as you did(?). The underlying issue seems to be none of OQS, though.

@cryptojane94 : As you use a different distro, can you confirm your problem is indeed of the same nature (compiling oqs-provider against an old openssl?) Which cmake version are you using?

[Krylol94]

Thanks for your replies @baentsch and @cryptobsv .

Final question (also to @cryptojane94 ): Have you run ./scripts/runtests.sh -V (verbose output)? Did that output inform you about an OpenSSL version mismatch -- most notably in the TLS run test?

The output I got is:
Start 4: oqs_tlssig

4: Test command: /pwd/oqs-provider/_build/test/oqs_test_tlssig "oqsprovider" "/home/kristjane/oqs-provider/test/openssl-ca.cnf" "/pwd/oqs-provider/_build/test/tmp"
4: Environment variables:
4: OPENSSL_MODULES=/pwd/oqs-provider/_build/lib
4: Test timeout computed to be: 10000000
4: TLS-SIG handshake test not enabled. Update OpenSSL to more current version.
4: Test passed
4/5 Test #4: oqs_tlssig ....................... Passed 0.01 sec

I guess this is suspicious.

[cryptobsv]

I begin to wonder how you even got as far as you did(?).

Now I can't reproduce your issue. :-) Never mind.

The underlying issue seems to be none of OQS, though.

I agree. It's not worth following it up any further.

Just one small update: Building with PKG_CONFIG_PATH=/path/to/openssl/user/installation/lib64/pkgconfig scripts/fullbuild.sh works for me and is more elegant than my previous quick-and-dirty approach of editing CMakeCache.txt. Note that I am not setting OPENSSL_INSTALL or similar at all here.

[baentsch]

I guess this is suspicious.

More than suspicious :-) Rather tell-tale: It's clearly saying that the OpenSSL version you built against is not supporting provider-based TLS signatures (i.e., is not 3.2.0-dev). You may want to try out the approach explained by @cryptobsv above (and/or set OPENSSL_INSTALL if your cmake is working OK). With the additions in #235 I'll consider this issue resolved then. Thanks for your heads-up(s) and diligence finding out the root cause here!

[baentsch]

Closing the issue due to inactivity (and after adding #235 for a telltale warning). Please re-open with new findings if necessary.