tls grade: cryptcheck.fr or ssllabs ?

[dalf]

Being the one first criteria to sort the instance list, the tls grade measure has to be reliable, otherwise the order will be "blinking". Unfortunately, this is not the case right now using cryptcheck:

• no result will be returned if a server has an ipv6 miss-configuration.
• some other cases, I can't determine.

It is possible to run cryptcheck locally using docker and / or send pull request to the git repository.
But, it seems there is a consensus around ssllab, at least in the current instance list.

Here some information ssllab:

• ssllab API: https://github.com/ssllabs/ssllabs-scan/blob/master/ssllabs-api-docs-v3.md
• ssllab Terms and Conditions: https://www.ssllabs.com/downloads/Qualys_SSL_Labs_Terms_of_Use.pdf

Extract :

...
You are not allowed, without our express permission, to:

• use the API for commercial purposes;
• use the API on a public web site;
• publish any information received from us via the APIs without the owner’s express permission;
• distribute, proxy, or otherwise make the API available for access or use by any person or entity other than your authorized employees, including but not limited to acting as a service bureau or developing a competing product or service offering.

...

[unixfox]

I don't think there is a consensus around Qualys SSL labs, it's probably due to the fact that ssllabs is more popular than cryptcheck.

SSL Labs really lacks of SSL/TLS good practices, it's just in 2020 that they started giving a lower score for using TLS 1.0 & TLS 1.1: https://blog.qualys.com/ssllabs/2018/11/19/grade-change-for-tls-1-0-and-tls-1-1-protocols
Whereas the PCI asked in 2018 everyone to stop using TLS 1.0: https://blog.pcisecuritystandards.org/are-you-ready-for-30-june-2018-sayin-goodbye-to-ssl-early-tls

We shouldn't trust SSL Labs for measuring the quality of the encryption of a Searx instance. aeris himself agree that nobody should use Qualys SSL Labs: https://mastodon.social/@fschaap/5000393

As for the arguments against cryptcheck:

no result will be returned if a server has an ipv6 miss-configuration.

I find this completely normal, the owner of the Searx instance has to correctly configure his website. We shouldn't follow the fallback to IPv4 mechanism.

Try to SSH to a domain that have a miss-configuration for its IPv6 record but the IPv4 record is valid, it will just timeout without falling back to IPv4.
And what about the people that are behind a NAT64&DNS64 network? They will feel the same thing, the browser will timeout due to the IPv6 miss-configuration because their computer doesn't have any valid IPv4.

some other cases, I can't determine.

I only encountered one, a case where a scan wouldn't finish. If that's becoming more and more frequent, it's better to open an issue marked as a bug.
Apart from that, if you could give more details maybe we could ask aeris to fix the potential bugs.

EDIT: I saw a bunch of question mark on searx.space and I think that's what you related to "other cases".
I don't really know why your program can't fetch the API because for every website with a "?" I do have correct results from the API. For example "searx.foo.li" has a question mark but the API is able to return the results of the scan: https://cryptcheck.fr/https/searx.foo.li.json

[dalf]

EDIT: I saw a bunch of question mark on searx.space and I think that's what you related to "other cases".
I don't really know why your program can't fetch the API because for every website with a "?" I do have correct results from the API. For example "searx.foo.li" has a question mark but the API is able to return the results of the scan: https://cryptcheck.fr/https/searx.foo.li.json

For some hosts, I think it is a timeout:

• refresh
• timeout after 1 minute --> '?'
• the result arrives later --> the link works.

For some other hosts, it is the IPv6 configuration problem.

Anyway, it seems that if the process stops for some reason, it is not possible to restart even using the .../refresh URL. So there are some hosts which will never get a result.

Example with search.mdosch.de:

• https://cryptcheck.fr/https/search.mdosch.de the check has started 3 days ago.
• docker run --network host --rm aeris22/cryptcheck https search.mdosch.de -qj works as expected and quickly (BTW --no-ipv6 is ignored)

So I end up thinking it is better to run aeris22/cryptcheck docker image directly. I wish to run searx-stats2 without docker access, so a small wrapper using docker-py needs to be written.

[unixfox]

Maybe you could use podman instead? This way you don't have to run a docker daemon all the time and you can launch the container with a non-root account: https://github.com/containers/python-podman

I was also thinking of running our own cryptcheck on Heroku for example? Maybe the official cryptcheck just have too much load to handle?

EDIT: Temporary solution to have full non-root in podman python: containers/python-podman#16 (comment)

[dalf]

I've tried podman:

• thank you about the comment to run podman without root access.
• about ipv6 : podman and IPv6 containers/podman#3245
• otherwise I'm not sure it is possible to run podman container using the host network: https://github.com/containers/python-podman/blob/b28a35c49651be328086532d444fce3f30bbc6fd/podman/libs/images.py#L39 (using python, using command line works as expected).
• may be it requires a deeper investigation, but I haven't found a way to get the stdout for the container ( https://github.com/containers/python-podman/blob/master/examples/eg_attach.py doesn't help)

A quick and dirty using docker & flask:

#!/usr/bin/env python3

import docker
import docker.errors
import json
import logging
from flask import Flask

logger = logging.getLogger('cryptcheck')

PORT = 50000
IMAGE = 'aeris22/cryptcheck:latest'
COMMAND = 'https $HOSTNAME --quiet --json'

app = Flask(__name__)

@app.route('/<hostname>')
def index(hostname):
    logging.debug('cryptcheck %s', hostname)
    try:
        client = docker.from_env()
        c = client.containers.run(image=IMAGE,
                                  command=COMMAND.replace('$HOSTNAME', hostname),
                                  auto_remove=True,
                                  network_mode='host')
        return c.decode('utf-8')
    except docker.errors.ContainerError as e:
        return str(e), 500
    except Exception as e:
        return str(e), 503


def run():
    logger.debug('starting webserver on port %s', PORT)
    app.run(
        port=PORT,
        host='127.0.0.1',
        threaded=True
    )


if __name__ == "__main__":
    run()

Note:

• nothing new: the aeris22/cryptcheck image ignores --no-ipv6 command line parameter: so if there is a ipv6 miss-configuration, there is no result even for the ipv4 addresses (with / without --no-ipv6 ).
• most probably it is not possible to implement https://github.com/dalf/searx-stats2/issues/1 with podman / docker. But the TLS grade doesn't require to be fetched from every location.

[dalf]

WIP: https://github.com/dalf/cryptcheck-backend (perhaps it is clunky: I've never written Ruby code).

http://localhost:8888/https/<host> returns the cryptcheck result (no cache, no database).
The response time may be very long (perhaps 2 or 3 minutes in some edge cases).

Using this docker image, it is possible to call cryptcheck without the root permission.

[dalf]

cryptcheck-backend is reliabled since it has been used, so I've merged the PR #15
I think this issue can be close.
Note: the project now depends on cryptcheck-backend which requires docker (or padman or anything compatible with docker).