Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

packages.matrix.org signing key will expire on 2023-04-15 #10389

Closed
richvdh opened this issue Jul 14, 2021 · 31 comments
Closed

packages.matrix.org signing key will expire on 2023-04-15 #10389

richvdh opened this issue Jul 14, 2021 · 31 comments
Assignees
Labels
A-Packaging Our Debian packages, docker images; or issues relevant to downstream packagers T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. Z-Future-Maintenance Things that can't yet be done, but will need cleaning up in a couple of months/releases

Comments

@richvdh
Copy link
Member

richvdh commented Jul 14, 2021

The GPG key (F473DD4473365DE1) used to sign the Debian repository at https://packages.matrix.org has an expiry date of 2023-04-15. We'll need to do something about this well in advance.

Related: matrix-org/matrix.org#973

@richvdh
Copy link
Member Author

richvdh commented Jul 14, 2021

Ideas are:

  • remove the expiry date, and tell everyone to download the public key again
  • Create a keyring package which matrix-synapse-py3 depends on or recommends, and which contains the latest copy of the key.

The advantage of the second option is that it won't require any action by users. It's a fairly commonly-used pattern; see for example:

  • Brave, who have installation instructions very much like ours, but brave-browser depends on brave-keyring which includes an up-to-date copy of /usr/share/keyrings/brave-browser-archive-keyring.gpg.

@richvdh richvdh added the A-Packaging Our Debian packages, docker images; or issues relevant to downstream packagers label Jul 14, 2021
@richvdh
Copy link
Member Author

richvdh commented Jul 14, 2021

a keyring package would also be a good place to ensure that the revoked key (C35EB17E1EAE708E6603A9B3AD0592FE47F0DF61) is not trusted.

@erikjohnston erikjohnston added the T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. label Jul 26, 2021
@callahad callahad added the Z-Future-Maintenance Things that can't yet be done, but will need cleaning up in a couple of months/releases label Jul 29, 2021
@callahad callahad added this to the Revisit: Yearly milestone Sep 15, 2021
@richvdh
Copy link
Member Author

richvdh commented Nov 17, 2022

Just a reminder that this will need sorting in Q1 next year.

@reivilibre
Copy link
Contributor

Seems like our solution has caused an error in some cases: element-hq/element-web#15176

@DMRobertson
Copy link
Contributor

What's the status---do we still need to generate/renew keys?

@richvdh
Copy link
Member Author

richvdh commented Mar 16, 2023

we still need to republish the existing keys with an updated expiry date. This is on me.

@richvdh richvdh self-assigned this Mar 16, 2023
@t3chguy
Copy link
Member

t3chguy commented Mar 16, 2023

@richvdh for the uninitiated could you log the steps for me to blindly follow for packages.element.io please :)

@DMRobertson
Copy link
Contributor

Seems like our solution has caused an error in some cases: element-hq/element-web#15176

FTR this was fixed by a change to the way we build the keyring package.

@richvdh
Copy link
Member Author

richvdh commented Mar 16, 2023

The steps are as follows, given a gpg data directory at /path/to/gnupg-directory which has the master keys:

  1. Take a backup:

    cp -r /path/to/gnupg-directory.2023-03-16
  2. Tell GPG to work with this directory, not ~/.gnupg:

    export GNUPGHOME=/path/to/gnupg-directory
  3. List current keys. Obviously not critical, but helpful in what follows:

    gpg -k --with-subkey-fingerprints  --list-options show-unusable-subkeys=yes

    This will show something like:

    /path/to/gnupg-directory
    ------------------------------------------------
    pub   rsa4096 2019-04-15 [SC] [expires: 2024-04-13]
          AAF9AE843A7584B5A3E4CD2BCF45A512DE2DA058
    uid           [ultimate] matrix.org packages <[email protected]>
    sub   rsa3072 2019-04-15 [S] [expires: 2023-04-15]
          5586CCC0CBBBEFC7A25811ADF473DD4473365DE1
    
    pub   rsa4096 2019-04-15 [SC] [expires: 2024-04-13]
          12D4CD600C2240A9F4A82071D7B0B66941D01538
    uid           [ultimate] riot.im packages <[email protected]>
    sub   rsa3072 2019-04-15 [S] [expires: 2023-04-15]
          75741890063E5E9A46135D01C2850B265AC085BD
    

    This shows two primary keys (one for matrix.org with fingerprint ...A058, one for riot.im with fingerprint ...1538), each of which has a single subkey. The primary keys exist only to sign the subkeys: their private parts live under lock and key and only come out every two years to update the expiry dates. The subkeys are what we actually sign the repositories with.

    In this case we can see that the primary keys will expire in a year too, so while I have the key out of the safe I will update the expiry date on that as well.

  4. Start editing the keys:

    gpg (GnuPG) 2.2.27; Copyright (C) 2021 Free Software Foundation, Inc.
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    
    Secret key is available.
    
    sec  rsa4096/CF45A512DE2DA058
         created: 2019-04-15  expires: 2024-04-13  usage: SC  
         trust: ultimate      validity: ultimate
    ssb  rsa3072/F473DD4473365DE1
         created: 2019-04-15  expires: 2023-04-15  usage: S   
    [ultimate] (1). matrix.org packages <[email protected]>
    
    gpg> 
    
  5. Update the expiry date on the primary key. As above, not normally required. (This will prompt for a passphrase which you should have obtained elsewhere.)

    gpg> expire
    Changing expiration time for the primary key.
    Please specify how long the key should be valid.
          0 = key does not expire
       <n>  = key expires in n days
       <n>w = key expires in n weeks
       <n>m = key expires in n months
       <n>y = key expires in n years
    Key is valid for? (0) 10y
    Key expires at Sun 13 Mar 2033 18:10:32 GMT
    Is this correct? (y/N) y
    
    sec  rsa4096/CF45A512DE2DA058
         created: 2019-04-15  expires: 2033-03-13  usage: SC  
         trust: ultimate      validity: ultimate
    ssb  rsa3072/F473DD4473365DE1
         created: 2019-04-15  expires: 2023-04-15  usage: S   
    [ultimate] (1). matrix.org packages <[email protected]>
    
  6. Repeat for sub key:

    gpg> key 1
    
    sec  rsa4096/CF45A512DE2DA058
        created: 2019-04-15  expires: 2033-03-13  usage: SC  
        trust: ultimate      validity: ultimate
    ssb* rsa3072/F473DD4473365DE1
        created: 2019-04-15  expires: 2023-04-15  usage: S   
    [ultimate] (1). matrix.org packages <[email protected]>
    
    gpg> expire
    Changing expiration time for a subkey.
    Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
    Key is valid for? (0) 2y
    Key expires at Sat 15 Mar 2025 18:10:59 GMT
    Is this correct? (y/N) y
    
    sec  rsa4096/CF45A512DE2DA058
        created: 2019-04-15  expires: 2033-03-13  usage: SC  
        trust: ultimate      validity: ultimate
    ssb* rsa3072/F473DD4473365DE1
        created: 2019-04-15  expires: 2025-03-15  usage: S   
    [ultimate] (1). matrix.org packages <[email protected]>
    
  7. Save changes:

    gpg> save
    
  8. We can now export new copies of the public keys, and publish them to the key servers:

    gpg --export [email protected] > matrix-org-archive-keyring.gpg
    gpg --export [email protected] | curl -T - https://keys.openpgp.org

    The .gpg file is published to packages.matrix.org (or packages.element.io). Publishing to https://keys.openpgp.org isn't strictly required, but is generally helpful for people looking for the key. (Note that the curl returns a validation link that you have to click).

@richvdh
Copy link
Member Author

richvdh commented Mar 16, 2023

I now realise this is also all written down at https://gitlab.matrix.org/new-vector/internal/-/wikis/packages#renewing-the-debian-package-signing-keys (for element employees)

@richvdh
Copy link
Member Author

richvdh commented Mar 16, 2023

The public key at https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg is now updated. @matrix-org/synapse-core: please could you update the keyring package?

@richvdh richvdh reopened this Mar 16, 2023
@erikjohnston
Copy link
Member

I've updated the keyring package, so this should be done

@richvdh
Copy link
Member Author

richvdh commented Apr 17, 2023

If anybody is seeing errors along the lines of the following:

W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://packages.matrix.org/debian bullseye InRelease: The following signatures were invalid: EXPKEYSIG F473DD4473365DE1 matrix.org packages <[email protected]>

... then that suggests you have an outdated copy of the matrix-org-archive-keyring package.

To fix it:

  1. make sure that /etc/apt/sources.list.d/matrix-org.list contains a a line matching:

    deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ <release> main
    

    If there are lines with a different signed-by section, update them.

  2. Run the following:

    sudo wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
    sudo apt update
    sudo apt install matrix-synapse-py3 matrix-org-archive-keyring
    

@DMRobertson
Copy link
Contributor

The expiry date has now passed.

Do we need to say anything about this in the release/upgrade notes, or should this be transparent to users of our deb packages?

@t3chguy
Copy link
Member

t3chguy commented Apr 17, 2023

It'll be transparent to users who kept up to date, a keyring package was added as a recommended dependency which will keep the gpg keyring up to date. For users upgrading from old versions or with custom gpg setups will need to manually update the keyring.

@gigawhitlocks
Copy link

It was not transparent for me, on an Ubuntu LTS using managed releases.

@hex-m
Copy link

hex-m commented Apr 18, 2023

Probably related? I get the this error from the Element repository on my (Debian 11) Desktop.

W: GPG error: https://packages.element.io/debian default Release: The following signatures were invalid: EXPKEYSIG C2850B265AC085BD riot.im packages <[email protected]>
E: The repository 'https://packages.riot.im/debian default Release' is no longer signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

Downloading the latest version of the keyring according to the documentation did not solve the problem.

@t3chguy
Copy link
Member

t3chguy commented Apr 18, 2023

@hex-m that's element-hq/element-desktop#807

@richvdh
Copy link
Member Author

richvdh commented Apr 18, 2023

It was not transparent for me, on an Ubuntu LTS using managed releases.

I'm not sure what "managed releases" mean, but it's possible you either haven't upgraded synapse recently, or you upgraded without installing recommended packages.

@helium75
Copy link

I updated the keyring but still have the same issue.

@tleydxdy
Copy link

seems like for me the source.list need to be updated to include signed-by like so
deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/

@richvdh
Copy link
Member Author

richvdh commented Apr 20, 2023

seems like for me the source.list need to be updated to include signed-by like so:

deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/

Yes, that is what it says at https://matrix-org.github.io/synapse/latest/setup/installation.html#matrixorg-packages. If your sources.list doesn't match the documentation, for sure you will have problems.

@tleydxdy
Copy link

yeah I think mine is left over from years ago

@richvdh
Copy link
Member Author

richvdh commented Apr 24, 2023

yeah I think mine is left over from years ago

suggest you fix it, otherwise you will have the same problem in two years' time

@dannyp777
Copy link

Hi, I upgraded to Ubuntu Lunar last Friday and noticed apt-get complaining about the F473DD4473365DE1 key for the matrix repository "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.asc] https://packages.matrix.org/debian lunar main" .
I downloaded matrix-org-archive-keyring.asc manually from the webpage but it didn't work.
However the gpg version of it has worked after changing my "signed-by" to use the gpg file instead of the asc file.
So it looks to me like the asc file is not in-sync with the gpg file.

@richvdh
Copy link
Member Author

richvdh commented Apr 26, 2023

@dannyp777 your sources.list is incorrect. You'll need to make it use matrix-org-archive-keyring.gpg, per the documentation.

I've removed the outdated .asc file from packages.matrix.org - thanks for letting us know.

@ghost
Copy link

ghost commented Jun 4, 2023

wget -qO - https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg | sudo apt-key add -
is done for me

@richvdh
Copy link
Member Author

richvdh commented Jun 5, 2023

wget -qO - https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg | sudo apt-key add - is done for me

Please don't do this. It means that the matrix.org keyring becomes trusted to replace any package on your system.

@ghost
Copy link

ghost commented Jun 5, 2023

wget -qO - https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg | sudo apt-key add - is done for me

Please don't do this. It means that the matrix.org keyring becomes trusted to replace any package on your system.

XD Sorry for my poor Linux level

@Ademan
Copy link

Ademan commented Jun 9, 2023

I missed the window between when the new keyring package became available, and when the repo key was changed.

I understand how I can update to use the new key (already done) but it kind of sucks that my chain of trust is now broken. Would you consider providing a human readable message signed by the old key which indicates the new key is trustworthy? (going forward)

Obviously missing the window is entirely on me, but it would be nice to be able to recover the chain of trust in that event.

@richvdh
Copy link
Member Author

richvdh commented Jun 12, 2023

The key has not been changed. Just the expiry date.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A-Packaging Our Debian packages, docker images; or issues relevant to downstream packagers T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. Z-Future-Maintenance Things that can't yet be done, but will need cleaning up in a couple of months/releases
Projects
None yet
Development

No branches or pull requests