Point extension #517

gstarovo · 2024-04-12T15:03:10Z

fixes #103

This change is

tomato42

we should have test coverage to verify that

it is actually negotiated (compressed representation is used)
that if one side doesn't advertise support for compressed, that uncompressed is used

Reviewed 5 of 5 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: all files reviewed, 22 unresolved discussions (waiting on @gstarovo)

.github/workflows/ci.yml line 8 at r1 (raw file):

    - master
    - tlslite-ng-0.7
    - format_extension

we don't need this here, please squash it with the second commit

tlslite/handshakesettings.py line 411 at r1 (raw file):

        self.keyExchangeNames = list(KEY_EXCHANGE_NAMES)
        self.cipherImplementations = list(CIPHER_IMPLEMENTATIONS)
        self.ecPointFormats = [ECPointFormat.uncompressed,

nit: trailing whitespace

also, all new class fields should use snake_case not camelCase

tlslite/handshakesettings.py line 413 at r2 (raw file):

        self.ecPointFormats = [ECPointFormat.uncompressed, 
                               ECPointFormat.ansiX962_compressed_char2,
                               ECPointFormat.ansiX962_compressed_prime]

the values should be copied and validated that they are recognised, check how other settings are handled

tlslite/keyexchange.py line 707 at r1 (raw file):

        kex = ECDHKeyExchange(self.group_id, self.serverHello.server_version)
        self.ecdhXs = kex.get_random_private_key()
        ext_negotiated = 0

this shouldn't be a 0, it should be ECPointFormat.uncompressed to document that this is the fallback value

tlslite/keyexchange.py line 714 at r1 (raw file):

            for ext in ext_c.formats:
                if ext in ext_s.formats and ext_negotiated is None:
                    ext_negotiated = ext

we don't need to set ext_negotiated to None
we're looking for first match, so after we find a value that is in both, we should break from the loop

tlslite/keyexchange.py line 734 at r1 (raw file):

        kex = ECDHKeyExchange(self.group_id, self.serverHello.server_version)
        ext_supported = [0]

we should be explicit that this is ECPointFormat.uncompressed

tlslite/keyexchange.py line 761 at r1 (raw file):

        ecdhXc = kex.get_random_private_key()
        ext_negotiated = 0
        ext_supported = [0]

same here, please use ECPointFormat.uncompressed

tlslite/keyexchange.py line 767 at r1 (raw file):

            ext_supported = []
            for ext in ext_c.formats:
                ext_negotiated = None

shouldn't this be outside the loop?

tlslite/keyexchange.py line 957 at r1 (raw file):

        return bytesToNumber(getRandomBytes(needed_bytes))

    def calc_public_value(self, private, frm_negotiated=None):

the doc string should explain that this is added here for API compat and has no effect on FFDH

tlslite/keyexchange.py line 981 at r1 (raw file):

        return bytesToNumber(peer_share)

    def calc_shared_key(self, private, peer_share, frm_supported_=None):

doc string should explain that this is added for API compatibility, not because it's needed for FFDH

tlslite/keyexchange.py line 1040 at r1 (raw file):

    @staticmethod
    def _get_ext_name(ext):
        """Get extension name from the numeric value."""

it's not "extension name", it's "point format"?

tlslite/keyexchange.py line 1041 at r1 (raw file):

    def _get_ext_name(ext):
        """Get extension name from the numeric value."""
        transform = {0: 'uncompressed', 1: 'compressed', 2: 'compressed'}

here too it would be better to use defines from ECPointFormat instead of magic values

tlslite/keyexchange.py line 1046 at r1 (raw file):

    def calc_public_value(self, private, frm_negotiated=0):
        """Calculate public value for given private key."""
        extension = self._get_ext_name(frm_negotiated)

I think point_fmt would work better here as the variable name

tlslite/keyexchange.py line 1057 at r1 (raw file):

            return bytearray(point.to_bytes(encoding=extension))

    def calc_shared_key(self, private, peer_share, frm_supported_=set([0])):

why _ at the end of frm_supported_?

also, the parameters should be documented in the doc string

tlslite/tlsconnection.py line 748 at r1 (raw file):

                group_id = getattr(GroupName, group_name)
                key_share = self._genKeyShareEntry(group_id, (3, 4),
                                                   settings.ecPointFormats[0])

why this is needed?

tlslite/tlsconnection.py line 766 at r1 (raw file):

            groups.extend(self._curveNamesToList(settings))
            extensions.append(ECPointFormatsExtension().\
                              create(settings.ecPointFormats))

if the list is empty, we shouldn't send an extensions

tlslite/tlsconnection.py line 987 at r1 (raw file):

                            ext_negotiated = ext
                key_share = self._genKeyShareEntry(group_id, (3, 4),
                                                   ext_negotiated)

key shares are TLS 1.2 specific, they don't use point format negotiation

see section 4.2.8.2 of RFC 8446:

   Note: Versions of TLS prior to 1.3 permitted point format
   negotiation; TLS 1.3 removes this feature in favor of a single point
   format for each curve.

tlslite/tlsconnection.py line 1187 at r1 (raw file):

    @classmethod
    def _genKeyShareEntry(cls, group, version, ext_negotiated):

same here, not needed

tlslite/tlsconnection.py line 1224 at r1 (raw file):

                                                   "advertised group.")
            kex = self._getKEX(sr_kex.group, self.version)
            ext_supported = set([0])

please us a define

tlslite/tlsconnection.py line 2731 at r1 (raw file):

            ext_negotiated = 0
            ext_supported = set([0])

please use defines

tlslite/tlsconnection.py line 2745 at r1 (raw file):

                        "No negotiated point extension")

            key_share = self._genKeyShareEntry(selected_group,

again, key shares are tls 1.3 specific

unit_tests/test_tlslite_keyexchange.py line 2532 at r1 (raw file):

        with self.assertRaises(NotImplementedError):
            kex.calc_shared_key(None, None, None)

those changes shouldn't be needed, the code should work the same as it did before introduction of the options

tomato42

Reviewed 9 of 9 files at r3, all commit messages.
Reviewable status: all files reviewed, 17 unresolved discussions (waiting on @gstarovo)

scripts/tls.py line 420 at r3 (raw file):

                                for item in cipher.split(',')]
    # CHANGED
    settings.ec_point_formats = []

this seems like debugging code we shouldn't be including...

scripts/tls.py line 575 at r3 (raw file):

    # CHANGED

    settings.ec_point_formats = [2, 0]

same here, looks like debugging code to me...

tests/tlstest.py line 312 at r3 (raw file):

    connection.handshakeClientCert(settings=settings)
    testConnClient(connection)
    assert connection.session.ec_point_format == ECPointFormat.ansiX962_compressed_char2

This is actually incorrect: secp256r1, secp384r1 and secp521r1 can use either uncompressed or ansiX962_compressed_prime, they can't use ansiX962_compressed_char2

the char2 encoding is for curves like the sect163r2or the sect233r1, but don't bother searching for them: we don't have support for "characteristic-2" curves, only for "prime field" curves

tlslite/handshakesettings.py line 361 at r3 (raw file):

    :vartype ec_point_formats: list
    :ivat ec_point_formats: Enabeled point format extension for

nit: Enabled
nit: :ivar

tlslite/handshakesettings.py line 609 at r3 (raw file):

                not 64 <= other.record_size_limit <= 2**14 + 1:
            raise ValueError("record_size_limit cannot exceed 2**14+1 bytes")

nit: whitespace

tlslite/handshakesettings.py line 613 at r3 (raw file):

                      i not in EC_POINT_FORMATS]
        if bad_ec_ext:
            raise ValueError("Unknown ec point format extension: "

not a fan of phrasing it like this... maybe "Unknown EC point formats provided: {0}" ?

tlslite/session.py line 78 at r3 (raw file):

    :vartype ec_point_format: int
    :ivar ec_point_format: used ec point extension format; 
        created for testing

don't think we need the last part; but I think I'd rephrase it to something like: "Used EC point format for the ECDH key exchange"

tlslite/tlsconnection.py line 663 at r3 (raw file):

        if ext_c and ext_s:
            ext_ec_point = [i for i in ext_c.formats \
                        if i in ext_s.formats][0]

If we go for iterators, then probably we should go for a solution that uses them fully, something like:

ext_ec_point = next((i for i in ext_c.formats if i in ext_s.formats)

and speaking of corner cases: we probably should handle the situation when there is no overlap: that's a protocol violation, but we need to detect it and handle correctly

tlslite/tlsconnection.py line 776 at r3 (raw file):

            else:
                extensions.append(ECPointFormatsExtension().\
                                create(list([ECPointFormat.uncompressed])))

this will still send the extension... why?

tlslite/tlsconnection.py line 2287 at r3 (raw file):

            else:
                extensions.append(ECPointFormatsExtension().\
                                create(list([ECPointFormat.uncompressed])))

didn't we agree that if the list is empty, the extension should be omitted?

tlslite/tlsconnection.py line 2432 at r3 (raw file):

        ext_ec_point = ECPointFormat.uncompressed
        if ext_c and ext_s:
            ext_ec_point = [i for i in ext_c.formats if i in ext_s.formats][0]

same here, next() will be more pythonic

unit_tests/test_tlslite_keyexchange.py line 23 at r3 (raw file):

from tlslite.constants import CipherSuite, CertificateType, AlertDescription, \
        HashAlgorithm, SignatureAlgorithm, GroupName, ECCurveType, \
        SignatureScheme, ECPointFormat

unused?

unit_tests/test_tlslite_keyexchange.py line 37 at r3 (raw file):

from tlslite import VerifierDB
from tlslite.extensions import SupportedGroupsExtension, SNIExtension, \
        ECPointFormatsExtension

unused?

tomato42

Reviewed 6 of 6 files at r4, all commit messages.
Reviewable status: all files reviewed, 9 unresolved discussions (waiting on @gstarovo)

tests/tlstest.py line 310 at r4 (raw file):

    settings.maxVersion = (3, 3)
    settings.eccCurves = ["secp256r1", "secp384r1", "secp521r1", "x25519", "x448"]
    settings.ec_point_formats = [ECPointFormat.ansiX962_compressed_prime, ECPointFormat.uncompressed]

why we need to set it? shouldn't this be the default?

tests/tlstest.py line 2217 at r4 (raw file):

    settings.maxVersion = (3, 3)
    settings.eccCurves = ["secp256r1", "secp384r1", "secp521r1", "x25519", "x448"]
    settings.ec_point_formats = [ECPointFormat.ansiX962_compressed_prime, ECPointFormat.uncompressed]

same here, this should be the default...

tests/tlstest.py line 2221 at r4 (raw file):

                            privateKey=x509ecdsaKey, settings=settings)
    testConnServer(connection)
    print(connection.session.ec_point_format)

leftover debug ?

tlslite/tlsconnection.py line 664 at r4 (raw file):

            ext_ec_point = next((i for i in ext_c.formats \
                                 if i in ext_s.formats), \
                                    ECPointFormat.uncompressed)

no, this will just use uncompressed when there's no overlap, we can't do it like this, when there's no overlap we need to send an illegal_parameter alert

tlslite/tlsconnection.py line 2429 at r4 (raw file):

            ext_ec_point = next((i for i in ext_c.formats \
                                 if i in ext_s.formats),\
                                      ECPointFormat.uncompressed)

same here, we need to abort in case of no overlap

tomato42 · 2024-05-10T11:48:07Z

failure in test (py3.6 with tackpy, ubuntu-20.04, 3.6, tackpy) seems to be relevant

gstarovo · 2024-05-10T12:57:13Z

failure in test (py3.6 with tackpy, ubuntu-20.04, 3.6, tackpy) seems to be relevant

Yes, yes, I'm trying to understand why it is only with Python 3.6, or maybe the problem is somewhere else.

tomato42 · 2024-05-10T13:23:43Z

I don't think it's py3.6 related, I think it's tackpy related

tomato42 · 2024-05-14T11:43:10Z

line too long errors from codeclimate are valid

tomato42

Reviewed 5 of 7 files at r7, 1 of 1 files at r8, 1 of 1 files at r9, 1 of 1 files at r10, all commit messages.
Reviewable status: all files reviewed, 5 unresolved discussions (waiting on @gstarovo)

tlslite/keyexchange.py line 1040 at r1 (raw file):

Previously, tomato42 (Hubert Kario) wrote…

it's not "extension name", it's "point format"?

this doesn't look to be fixed...

tlslite/keyexchange.py line 1114 at r10 (raw file):

            ecdh.load_received_public_key_bytes(peer_share,
                                                valid_encodings= \
                                                    valid_encodings)

nit: as codeclimate indicates: over indented (also, the \ is not necessary inside ()

tlslite/tlsconnection.py line 667 at r10 (raw file):

                    ext_ec_point = next((i for i in ext_c.formats \
                                        if i in ext_s.formats))

nit: leftover whitespace

tlslite/tlsconnection.py line 2440 at r10 (raw file):

                    ext_ec_point = next((i for i in ext_c.formats \
                                        if i in ext_s.formats))

nit: leftover whitespace

tomato42

the code itself is fine, but as it is now, it breaks ECDH in tlsfuzzer (as we've discussed in person), so technically it's incomplete

Reviewed 2 of 2 files at r11, all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @gstarovo)

tomato42

Reviewed 2 of 2 files at r12, all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @gstarovo)

tomato42 · 2024-06-12T14:06:29Z

@gstarovo that fixes the tlsfuzzer issue? should I try it and if it works, merge this PR, or do you want to to some additional tweaks?

tomato42 · 2024-08-26T14:16:24Z

line too long issues from codeclimate are valid

tomato42

Reviewed 3 of 4 files at r13, 1 of 1 files at r14, all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @gstarovo)

tomato42

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @gstarovo)

tlslite/keyexchange.py line 1065 at r14 (raw file):

    def calc_public_value(self,
                          private,
                          frm_negotiated=ECPointFormat.uncompressed):

sorry for rejecting it this late, but I've been working on point format handling for TLS 1.3 post-quantum hybrid key shares, and I've noticed that for test coverage, we need to be able to both send the shares in all the different formats (including the ones that aren't specified in TLS, like "raw" and "hybrid"), and similarly reject formats we don't want (and default to accepting only uncompressed)

so I think doing it like this: a9af7a3 will be cleaner

tlslite/keyexchange.py line 1079 at r14 (raw file):

    def calc_shared_key(self, private, peer_share,
                        frm_supported=set([ECPointFormat.uncompressed])):

as in the other comment: a list/set of strings is better here, not TLS identifiers

tomato42

Reviewed 7 of 7 files at r15, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @gstarovo)

tlslite/keyexchange.py line 1136 at r15 (raw file):

            raise TLSIllegalParameterException("Invalid ECC point")
        except DecodeError:
            raise TLSDecodeError("Unexpected error")

If the doc above is an indication, this should be something like "Empty point format extension"?

tomato42

Reviewed 1 of 1 files at r16, 1 of 1 files at r17, all commit messages.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @gstarovo)

tlslite/keyexchange.py line 717 at r16 (raw file):

        if ext_c:
            if ext_c.formats == []:
                raise TLSDecodeError("The compression list is empty.")

I think something like "Point formats extension is empty." would be more descriptive

tlslite/tlsconnection.py line 4414 at r17 (raw file):

        except DecodeError as alert:
            for result in self._sendError(
                    AlertDescription.illegal_parameter,

either this should be decode_error or the DecodeError exception should have been caught earlier and turned into TLSIllegalParameterException...

what's the situation that leads to this code?

tlslite/tlsconnection.py line 4420 at r17 (raw file):

            alert = Alert().create(AlertDescription.illegal_parameter,
                                           AlertLevel.fatal)
            for result in self._sendMsg(alert):

why it's using _sendMsg not _sendError?

tlslite/tlsconnection.py line 4426 at r17 (raw file):

            alert = Alert().create(AlertDescription.decode_error,
                                           AlertLevel.fatal)
            for result in self._sendMsg(alert):

same here, wh not _sendError?

tomato42

Reviewed 2 of 2 files at r19, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @gstarovo)

tlslite/keyexchange.py line 717 at r16 (raw file):

Previously, tomato42 (Alicja Kario) wrote…

I think something like "Point formats extension is empty." would be more descriptive

sorry, looks like wasn't clear: sending decode_error alert in such case is correct, the issue was only with the message associated with it

tlslite/keyexchange.py line 1138 at r19 (raw file):

            raise TLSIllegalParameterException("Invalid ECC point")
        except DecodeError:
            raise TLSIllegalParameterException("Empty point format extension")

same here, I think that TLSDecodeError was correct before

https://www.rfc-editor.org/rfc/rfc8422.html#section-5.1.2 specifies it as:

           enum {
               uncompressed (0),
               deprecated (1..2),
               reserved (248..255)
           } ECPointFormat;
           struct {
               ECPointFormat ec_point_format_list<1..2^8-1>
           } ECPointFormatList;

so an empty one should result in a decode_error not illegal_parameter

tomato42

Reviewed 2 of 2 files at r20, all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @gstarovo)

check if the client adverties the uncompressed point format extension error when uncompressed is not supported fix: changes in accepting the format(form ECPointFormat to string) of ec format. fix: tests, keyShares added to tests for checking ecc point extension

…ormat is not found; alert decode_error is added when the list of ecc extenison is empty fix: message alert description

… to check of client hello by server; that way the server will abort the connection after bad client hello, not after server key exchange

tomato42

side note: I have no problem with fixup commits, but the CI will verify that the test coverage passes after each and every one of them, so it may be a good idea to rebase and apply those fixup patches to the base commit that they're changing

Reviewed 3 of 5 files at r21, 2 of 2 files at r25, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @gstarovo)

tlslite/tlsconnection.py line 3441 at r25 (raw file):

        ecExt = clientHello.getExtension(ExtensionType.ec_point_formats)
        if ecExt:
            if ecExt.formats == []:

emptiness of a collection in python is generally tested by using if not ecExt.formats:

tomato42 · 2024-11-28T13:13:29Z

...and the CI failures seem to be related to your changes

tomato42 requested changes Apr 12, 2024

View reviewed changes

gstarovo force-pushed the point_extension branch from dd118da to d683eb5 Compare April 17, 2024 11:57

gstarovo force-pushed the point_extension branch 2 times, most recently from aca7813 to 076956c Compare April 29, 2024 13:04

tomato42 requested changes May 3, 2024

View reviewed changes

tomato42 requested changes May 9, 2024

View reviewed changes

gstarovo force-pushed the point_extension branch from 5937e12 to a505851 Compare May 10, 2024 12:43

gstarovo force-pushed the point_extension branch from ff7afb5 to 4a2d3df Compare May 14, 2024 09:16

gstarovo force-pushed the point_extension branch from a724ea3 to e380425 Compare May 30, 2024 09:51

tomato42 requested changes May 30, 2024

View reviewed changes

tomato42 reviewed May 31, 2024

View reviewed changes

tomato42 previously approved these changes Jun 12, 2024

View reviewed changes

gstarovo force-pushed the point_extension branch from dd3ba8a to 82bfabc Compare June 27, 2024 11:06

gstarovo dismissed tomato42’s stale review via f40e1eb August 21, 2024 07:38

gstarovo force-pushed the point_extension branch 5 times, most recently from 7bffdb9 to 2659e77 Compare August 26, 2024 13:50

gstarovo force-pushed the point_extension branch from 2659e77 to 818a4de Compare August 26, 2024 14:28

tomato42 approved these changes Aug 29, 2024

View reviewed changes

tomato42 requested changes Oct 15, 2024

View reviewed changes

gstarovo force-pushed the point_extension branch 2 times, most recently from b1de13b to cc82cfe Compare October 22, 2024 13:50

gstarovo force-pushed the point_extension branch 2 times, most recently from 1a2a156 to aeb79a3 Compare October 23, 2024 09:39

tomato42 requested changes Oct 23, 2024

View reviewed changes

tomato42 requested changes Nov 1, 2024

View reviewed changes

gstarovo force-pushed the point_extension branch from 88e049e to 7814a60 Compare November 5, 2024 10:51

tomato42 requested changes Nov 6, 2024

View reviewed changes

tomato42 previously approved these changes Nov 12, 2024

View reviewed changes

gstarovo dismissed tomato42’s stale review via b814eca November 27, 2024 15:55

gstarovo added 5 commits November 27, 2024 16:58

fix: decode error wording

d590dc6

fix: alert illegal_extension is added due to rfc, when uncompressed f…

2a21d8a

…ormat is not found; alert decode_error is added when the list of ecc extenison is empty fix: message alert description

feat: TLSDecodeError instead of TLSIllegalParameter

08ee649

fix: moved control of uncompressed point ext from server key exchange…

f59374b

… to check of client hello by server; that way the server will abort the connection after bad client hello, not after server key exchange

gstarovo force-pushed the point_extension branch from b814eca to f59374b Compare November 27, 2024 15:59

tomato42 requested changes Nov 28, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Point extension #517

Point extension #517

gstarovo commented Apr 12, 2024 •

edited by tomato42

Loading

tomato42 left a comment

tomato42 left a comment

tomato42 left a comment

tomato42 commented May 10, 2024

gstarovo commented May 10, 2024

tomato42 commented May 10, 2024

tomato42 commented May 14, 2024

tomato42 left a comment

tomato42 left a comment

tomato42 left a comment

tomato42 commented Jun 12, 2024

tomato42 commented Aug 26, 2024

tomato42 left a comment

tomato42 left a comment

tomato42 left a comment

tomato42 left a comment

tomato42 left a comment

tomato42 left a comment

tomato42 left a comment

tomato42 commented Nov 28, 2024

Point extension #517

Are you sure you want to change the base?

Point extension #517

Conversation

gstarovo commented Apr 12, 2024 • edited by tomato42 Loading

tomato42 left a comment

Choose a reason for hiding this comment

tomato42 left a comment

Choose a reason for hiding this comment

tomato42 left a comment

Choose a reason for hiding this comment

tomato42 commented May 10, 2024

gstarovo commented May 10, 2024

tomato42 commented May 10, 2024

tomato42 commented May 14, 2024

tomato42 left a comment

Choose a reason for hiding this comment

tomato42 left a comment

Choose a reason for hiding this comment

tomato42 left a comment

Choose a reason for hiding this comment

tomato42 commented Jun 12, 2024

tomato42 commented Aug 26, 2024

tomato42 left a comment

Choose a reason for hiding this comment

tomato42 left a comment

Choose a reason for hiding this comment

tomato42 left a comment

Choose a reason for hiding this comment

tomato42 left a comment

Choose a reason for hiding this comment

tomato42 left a comment

Choose a reason for hiding this comment

tomato42 left a comment

Choose a reason for hiding this comment

tomato42 left a comment

Choose a reason for hiding this comment

tomato42 commented Nov 28, 2024

gstarovo commented Apr 12, 2024 •

edited by tomato42

Loading