Release 2024-07-08 - (expected chart version 5.4.0)

[zebot]

[2024-07-08] (Chart Release 5.4.0)

Release notes

• Phone registration and login is not supported anymore. All API endpoints dealing with phone numbers and phone activation codes now fail with a 400 error. Brig options related to phone number support have now been deleted, namely:
  • setTwilio
  • setNexmo
  • setAllowlistPhonePrefixes. ([WPB-9065] Stop supporting phone numbers in supported client API versions #4045)

API changes

• Internal API endpoints related to phone numbers have been removed.

  In brig:

  • iGetPhonePrefix
  • iDeletePhonePrefix
  • iPostPhonePrefix.

  In stern:

  • get-users-by-phone
  • put-phone. ([WPB-9065] Stop supporting phone numbers in supported client API versions #4045)

Features

• charts/coturn: support putting coturn into 'drain' mode when terminating pods, denying new incoming client connections. This speeds up graceful coturn restarts significantly. (WPB 2690 #4098)

• Set SFT usernames's shared field according to team settings (WPB-6954 Set SFT username's shared field according to team setting #4117)

• Updated the mlsE2EId feature config with two additional fields crlProxy and useProxyOnMobile (WPB-8824 MLS E2EID CRL proxy flag #4051)

• reject MLS messages for future epochs (reject MLS messages for future epochs #4110)

• Introduce more configuration options to the coturn helm chart (coturn: Add IP configuration options #4083)

• Update email templates to v1.0.121. (Update email templates to v1.0.121. #4064)

• Support connecting to RabbitMQ over TLS. See "Configure RabbitMQ" section in the documentation for details. (RabbitMQ TLS #4094)

• Support connecting to Redis over TLS

  It can be enabled by setting these options on the wire-server helm chart:

  gundeck:
    config:
      redis:
        enableTls: true
  
        # When custom CAs are required, one of these must be set:
        tlsCa: <PEM encoded CA certificates>
        tlsCaSecretRef:
          name: <Name of the secret>
          key: <Key in the secret containing pem encoded CA Cert>
  
        # When TLS needs to be used without verification:
        insecureSkipVerifyTls: true

  ([WPB-6718] Implement TLS support in Wire's Redis interface #4016)

Bug fixes and other updates

• fixed stern endpoint /i/users/meta-info (WPB-9677 fix stern endpoint /i/users/meta-info #4101)

• Log password reset errors instead of propagating them (WPB-5491 Log password reset errors instead of propagating them #4114)

• Log request ids in brig. (Move password reset code to AuthenticationSubsystem #4086)

• Do not set update origin "scim" in public brig api. (Do not set update origin "scim" in public brig api. #4072)

• Disabling legalhold before user's approval doesn't result in an error ([WPB-9685] don't react with "legalhold already disabled" when on pending state #4104)

• Make scim-delete-user idempotent. Hide information about existing users (make delete idempotent) (Fix scim logic: delete dangling external_ids if they happen. #4120)

• Expose /providers/assets via nginz (WPB-9062 Provider API asset upload #4082)

• federator: Expect a client certificate to be the certificate chain

  Without this openssl doesn't forward to whole chain causing mTLS to not succeed. (federator: Expect a client certificate to be the certificate chain #4089)

• Only resend proposals once after external commit (Only resend proposals once after external commit #4103)

• gundeck: Better tolerance for redis-cluster restarts (WPB-9102 gundeck: Better tolerance for redis-cluster restarts #4084)

• GHC does not support repeated --with-rtsopts options, and it simply applies the last one. This means many of the baked-in options were actually not being passed, including -N for some of the services and -T for cannon. (Fix repeated with-rtsopts option #4118)

• Ensure that a Request ID is logged whenever unexpected errors are caught in any service ([WPB-7161] Fix Request ID logging in all services #4059)

• charts/coturn: use allowed dir to write PID file (WPB 2690 #4098)

• Make pending LH requests (with no LH devices listening yet) not throw LH policy errors. This helps eg. in cases where a LH request is issued to the wrong user by accident, and the user can clear up the mistake. (Treat pending legalhold devices as not having a legalhold device #4056)

Documentation

• Adjust documentation for migrated helm charts (Update docs: Migrated Helm charts (#3931) #4058)

Internal changes

• Adapt EJPD data to current requirements. (Add conversations to full ejpd info. #3945)

• Port team feature tests to the integration package (WPB-6442 migrate all team feature tests to integration package part 1 #4063)

• Ported flaky legalhold test to the new integration test suite (WPB-8757 Port flaky legalhold test from galley to integration #4057)

• Added profile update operations to the user subsystem. (User subsystem: add profile update operations #4046)

• Introduce authentication subsystem with password reset. (Move password reset code to AuthenticationSubsystem #4086)

• update nixpkgs and hence GHC version as well as some other tooling. ([WPB-8943] ghc 9.4 -> 9.6, nixpkgs bump #4071)

• nginz: Added allowlisted_fqdn_origins to nginx_conf value (WPB-9495: nginz: configure extra origins #4087)

• Add weeder for dead code elimination. (Add weeder (dead code elimination tool) to dev environment #4088)

• Introduce email subsystem (Introduce EmailSmsSubsystem, use it for a few emails and password reset SMS #4111)

• replace cabal.project.local template and update cabal.project ([chore] replace cabal.project.local template and update cabal.project #4119)

• Add HTTP proxy in the local setup for elasticsearch in federation-v0. This makes it possible to use a single elasticsearch instance for both the main backends and federation-v0. (dockerephemeral environment: give ES an http interface (needed for old brig) #4062)

• federator: Add metrics for garbage collections and unexpected errors that were caught (Define metrics using Prometheus.unsafeRegister instead of having the metrics-core wrapper #4085)

• federator: Simplify polysemy setup to make it similar to other services so the
  interpreter is only used for hoisting the servant application and not explicitly
  inside handler of an endpoint ([WPB-7161] Fix Request ID logging in all services #4059)

• Added prometheus enable and datacenter size variables for k8ssandra-test-cluster helm chart. (Update k8ssandra test cluster chart #4011)

• Make Handle type abstract to guarantee it always contains valid Handles. (Make Handle newtype abstract #4076)

• metrics-core: Delete Data.Metrics in favour of defining metrics closer to where they are being emitted (Define metrics using Prometheus.unsafeRegister instead of having the metrics-core wrapper #4085)

• add more metadata into the meta attribute of all nix derivations produced locally ([feat] add more metadata in nix to own code #4069)

• Do not log anything when warp kills a worker thread. ( catchErrors middleware: Don't create responses for ThreadKilled errors  #4112)

• Introduce VerificationCodSubsystem (Introduce VerificationCodeSubsystem #4121)

• add tests for bots that use self-signed certs and add documentation on why we cannot test the bots to work with PKI (Add more fingerprint verification tests for bots #4027)

[lwille]

⚠️ setting gundeck.config.redis.tlsCa leads to error:

 Error: template: wire-server/charts/gundeck/templates/redis-ca-secret.yaml:14:20: executing "wire-server/charts/gundeck/templates/redis-ca-secret.yaml" at <.Values.redis.tlsCa>: nil pointer evaluating interface {}.tlsCa

this is because charts/gundeck/templates/redis-ca-secret.yaml references .Values.redis.tlsCa instead of .Values.config.redis.tlsCa. Fix is underway.