Merge branch 'master' into develop

.github/workflows/release-artifacts.yml

@@ -5,7 +5,7 @@ name: Build release artifacts
 on:
   # we build on PRs and develop to (hopefully) get early warning
   # of things breaking (but only build one set of debs). PRs skip
-  # building wheels on macOS & ARM.
+  # building wheels on ARM.
   pull_request:
   push:
     branches: ["develop", "release-*"]
@@ -111,20 +111,14 @@ jobs:
     runs-on: ${{ matrix.os }}
     strategy:
       matrix:
-        os: [ubuntu-22.04, macos-13]
+        os: [ubuntu-22.04]
         arch: [x86_64, aarch64]
         # is_pr is a flag used to exclude certain jobs from the matrix on PRs.
         # It is not read by the rest of the workflow.
         is_pr:
           - ${{ startsWith(github.ref, 'refs/pull/') }}
 
         exclude:
-          # Don't build macos wheels on PR CI.
-          - is_pr: true
-            os: "macos-13"
-          # Don't build aarch64 wheels on mac.
-          - os: "macos-13"
-            arch: aarch64
           # Don't build aarch64 wheels on PR CI.
           - is_pr: true
             arch: aarch64

CHANGES.md

@@ -1,3 +1,53 @@
+# Synapse 1.120.2 (2024-12-03)
+
+This version has building of wheels for macOS disabled.
+It is functionally identical to 1.120.1, which contains multiple security fixes.
+If you are already using 1.120.1, there is no need to upgrade to this version.
+
+
+
+# Synapse 1.120.1 (2024-12-03)
+
+This patch release fixes multiple security vulnerabilities, some affecting all prior versions of Synapse. Server administrators are encouraged to update Synapse as soon as possible. We are not aware of these vulnerabilities being exploited in the wild.
+
+Administrators who are unable to update Synapse may use the workarounds described in the linked GitHub Security Advisory below.
+
+### Security advisory
+
+The following issues are fixed in 1.120.1.
+
+- [GHSA-rfq8-j7rh-8hf2](https://github.com/element-hq/synapse/security/advisories/GHSA-rfq8-j7rh-8hf2) / [CVE-2024-52805](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-52805): **Unsupported content types can lead to memory exhaustion**
+
+  Synapse instances which have a high `max_upload_size` and which don't have a reverse proxy in front of them that would otherwise limit upload size are affected.
+
+  Fixed by [4b7154c58501b4bf5e1c2d6c11ebef96529f2fdf](https://github.com/element-hq/synapse/commit/4b7154c58501b4bf5e1c2d6c11ebef96529f2fdf).
+
+- [GHSA-f3r3-h2mq-hx2h](https://github.com/element-hq/synapse/security/advisories/GHSA-f3r3-h2mq-hx2h) / [CVE-2024-52815](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-52815): **Malicious invites via federation can break a user's sync**
+
+  Fixed by [d82e1ed357b7ee21dff83d06cba7a67840cfd464](https://github.com/element-hq/synapse/commit/d82e1ed357b7ee21dff83d06cba7a67840cfd464).
+
+- [GHSA-vp6v-whfm-rv3g](https://github.com/element-hq/synapse/security/advisories/GHSA-vp6v-whfm-rv3g) / [CVE-2024-53863](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-53863): **Synapse can be forced to thumbnail unexpected file formats, invoking potentially untrustworthy decoders**
+
+  Synapse instances can disable dynamic thumbnailing by setting `dynamic_thumbnails` to `false` in the configuration file.
+
+  Fixed by [b64a4e5fbbbf119b6c65aedf0d999b4237d55503](https://github.com/element-hq/synapse/commit/b64a4e5fbbbf119b6c65aedf0d999b4237d55503).
+
+- [GHSA-56w4-5538-8v8h](https://github.com/element-hq/synapse/security/advisories/GHSA-56w4-5538-8v8h) / [CVE-2024-53867](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-53867): **The Sliding Sync feature on Synapse versions between 1.113.0rc1 and 1.120.0 can leak partial room state changes to users no longer in a room**
+
+  Non-state events, like messages, are unaffected.
+
+  Synapse instances can disable the Sliding Sync feature by setting `experimental_features.msc3575_enabled` to `false` in the configuration file.
+
+  Fixed by [4daa533e82f345ce87b9495d31781af570ba3ead](https://github.com/element-hq/synapse/commit/4daa533e82f345ce87b9495d31781af570ba3ead).
+
+See the advisories for more details. If you have any questions, email [security at element.io](mailto:[email protected]).
+
+### Bugfixes
+
+- Fix release process to not create duplicate releases. ([\#17970](https://github.com/element-hq/synapse/issues/17970))
+
+
+
 # Synapse 1.120.0 (2024-11-26)
 
 ### Bugfixes

debian/changelog

@@ -1,3 +1,15 @@
+matrix-synapse-py3 (1.120.2) stable; urgency=medium
+
+  * New synapse release 1.120.2.
+
+ -- Synapse Packaging team <[email protected]>  Tue, 03 Dec 2024 15:43:37 +0000
+
+matrix-synapse-py3 (1.120.1) stable; urgency=medium
+
+  * New synapse release 1.120.1.
+
+ -- Synapse Packaging team <[email protected]>  Tue, 03 Dec 2024 09:07:57 +0000
+
 matrix-synapse-py3 (1.120.0) stable; urgency=medium
 
   * New synapse release 1.120.0.

pyproject.toml

@@ -97,7 +97,7 @@ module-name = "synapse.synapse_rust"
 
 [tool.poetry]
 name = "matrix-synapse"
-version = "1.120.0"
+version = "1.120.2"
 description = "Homeserver for the Matrix decentralised comms protocol"
 authors = ["Matrix.org Team and Contributors <[email protected]>"]
 license = "AGPL-3.0-or-later"

synapse/federation/transport/server/federation.py

@@ -509,6 +509,9 @@ async def on_PUT(
         event = content["event"]
         invite_room_state = content.get("invite_room_state", [])
 
+        if not isinstance(invite_room_state, list):
+            invite_room_state = []
+
         # Synapse expects invite_room_state to be in unsigned, as it is in v1
         # API
 

synapse/handlers/federation.py

@@ -880,6 +880,9 @@ async def do_knock(
         if stripped_room_state is None:
             raise KeyError("Missing 'knock_room_state' field in send_knock response")
 
+        if not isinstance(stripped_room_state, list):
+            raise TypeError("'knock_room_state' has wrong type")
+
         event.unsigned["knock_room_state"] = stripped_room_state
 
         context = EventContext.for_outlier(self._storage_controllers)

synapse/handlers/sliding_sync/__init__.py

@@ -39,6 +39,7 @@
     trace,
 )
 from synapse.storage.databases.main.roommember import extract_heroes_from_room_summary
+from synapse.storage.databases.main.state_deltas import StateDelta
 from synapse.storage.databases.main.stream import PaginateFunction
 from synapse.storage.roommember import (
     MemberSummary,
@@ -48,6 +49,7 @@
     MutableStateMap,
     PersistedEventPosition,
     Requester,
+    RoomStreamToken,
     SlidingSyncStreamToken,
     StateMap,
     StrCollection,
@@ -470,6 +472,64 @@ async def get_current_state_at(
 
         return state_map
 
+    @trace
+    async def get_current_state_deltas_for_room(
+        self,
+        room_id: str,
+        room_membership_for_user_at_to_token: RoomsForUserType,
+        from_token: RoomStreamToken,
+        to_token: RoomStreamToken,
+    ) -> List[StateDelta]:
+        """
+        Get the state deltas between two tokens taking into account the user's
+        membership. If the user is LEAVE/BAN, we will only get the state deltas up to
+        their LEAVE/BAN event (inclusive).
+
+        (> `from_token` and <= `to_token`)
+        """
+        membership = room_membership_for_user_at_to_token.membership
+        # We don't know how to handle `membership` values other than these. The
+        # code below would need to be updated.
+        assert membership in (
+            Membership.JOIN,
+            Membership.INVITE,
+            Membership.KNOCK,
+            Membership.LEAVE,
+            Membership.BAN,
+        )
+
+        # People shouldn't see past their leave/ban event
+        if membership in (
+            Membership.LEAVE,
+            Membership.BAN,
+        ):
+            to_bound = (
+                room_membership_for_user_at_to_token.event_pos.to_room_stream_token()
+            )
+        # If we are participating in the room, we can get the latest current state in
+        # the room
+        elif membership == Membership.JOIN:
+            to_bound = to_token
+        # We can only rely on the stripped state included in the invite/knock event
+        # itself so there will never be any state deltas to send down.
+        elif membership in (Membership.INVITE, Membership.KNOCK):
+            return []
+        else:
+            # We don't know how to handle this type of membership yet
+            #
+            # FIXME: We should use `assert_never` here but for some reason
+            # the exhaustive matching doesn't recognize the `Never` here.
+            # assert_never(membership)
+            raise AssertionError(
+                f"Unexpected membership {membership} that we don't know how to handle yet"
+            )
+
+        return await self.store.get_current_state_deltas_for_room(
+            room_id=room_id,
+            from_token=from_token,
+            to_token=to_bound,
+        )
+
     @trace
     async def get_room_sync_data(
         self,
@@ -755,13 +815,19 @@ async def get_room_sync_data(
 
             stripped_state = []
             if invite_or_knock_event.membership == Membership.INVITE:
-                stripped_state.extend(
-                    invite_or_knock_event.unsigned.get("invite_room_state", [])
+                invite_state = invite_or_knock_event.unsigned.get(
+                    "invite_room_state", []
                 )
+                if not isinstance(invite_state, list):
+                    invite_state = []
+
+                stripped_state.extend(invite_state)
             elif invite_or_knock_event.membership == Membership.KNOCK:
-                stripped_state.extend(
-                    invite_or_knock_event.unsigned.get("knock_room_state", [])
-                )
+                knock_state = invite_or_knock_event.unsigned.get("knock_room_state", [])
+                if not isinstance(knock_state, list):
+                    knock_state = []
+
+                stripped_state.extend(knock_state)
 
             stripped_state.append(strip_event(invite_or_knock_event))
 
@@ -790,8 +856,9 @@ async def get_room_sync_data(
             # TODO: Limit the number of state events we're about to send down
             # the room, if its too many we should change this to an
             # `initial=True`?
-            deltas = await self.store.get_current_state_deltas_for_room(
+            deltas = await self.get_current_state_deltas_for_room(
                 room_id=room_id,
+                room_membership_for_user_at_to_token=room_membership_for_user_at_to_token,
                 from_token=from_bound,
                 to_token=to_token.room_key,
             )

synapse/http/site.py

@@ -21,6 +21,7 @@
 import contextlib
 import logging
 import time
+from http import HTTPStatus
 from typing import TYPE_CHECKING, Any, Generator, Optional, Tuple, Union
 
 import attr
@@ -139,6 +140,41 @@ def __repr__(self) -> str:
             self.synapse_site.site_tag,
         )
 
+    # Twisted machinery: this method is called by the Channel once the full request has
+    # been received, to dispatch the request to a resource.
+    #
+    # We're patching Twisted to bail/abort early when we see someone trying to upload
+    # `multipart/form-data` so we can avoid Twisted parsing the entire request body into
+    # in-memory (specific problem of this specific `Content-Type`). This protects us
+    # from an attacker uploading something bigger than the available RAM and crashing
+    # the server with a `MemoryError`, or carefully block just enough resources to cause
+    # all other requests to fail.
+    #
+    # FIXME: This can be removed once we Twisted releases a fix and we update to a
+    # version that is patched
+    def requestReceived(self, command: bytes, path: bytes, version: bytes) -> None:
+        if command == b"POST":
+            ctype = self.requestHeaders.getRawHeaders(b"content-type")
+            if ctype and b"multipart/form-data" in ctype[0]:
+                self.method, self.uri = command, path
+                self.clientproto = version
+                self.code = HTTPStatus.UNSUPPORTED_MEDIA_TYPE.value
+                self.code_message = bytes(
+                    HTTPStatus.UNSUPPORTED_MEDIA_TYPE.phrase, "ascii"
+                )
+                self.responseHeaders.setRawHeaders(b"content-length", [b"0"])
+
+                logger.warning(
+                    "Aborting connection from %s because `content-type: multipart/form-data` is unsupported: %s %s",
+                    self.client,
+                    command,
+                    path,
+                )
+                self.write(b"")
+                self.loseConnection()
+                return
+        return super().requestReceived(command, path, version)
+
     def handleContentChunk(self, data: bytes) -> None:
         # we should have a `content` by now.
         assert self.content, "handleContentChunk() called before gotLength()"

synapse/media/thumbnailer.py

@@ -67,6 +67,11 @@ class ThumbnailError(Exception):
 class Thumbnailer:
     FORMATS = {"image/jpeg": "JPEG", "image/png": "PNG"}
 
+    # Which image formats we allow Pillow to open.
+    # This should intentionally be kept restrictive, because the decoder of any
+    # format in this list becomes part of our trusted computing base.
+    PILLOW_FORMATS = ("jpeg", "png", "webp", "gif")
+
     @staticmethod
     def set_limits(max_image_pixels: int) -> None:
         Image.MAX_IMAGE_PIXELS = max_image_pixels
@@ -76,7 +81,7 @@ def __init__(self, input_path: str):
         self._closed = False
 
         try:
-            self.image = Image.open(input_path)
+            self.image = Image.open(input_path, formats=self.PILLOW_FORMATS)
         except OSError as e:
             # If an error occurs opening the image, a thumbnail won't be able to
             # be generated.

synapse/push/push_tools.py

@@ -74,9 +74,13 @@ async def get_context_for_event(
 
         room_state = []
         if ev.content.get("membership") == Membership.INVITE:
-            room_state = ev.unsigned.get("invite_room_state", [])
+            invite_room_state = ev.unsigned.get("invite_room_state", [])
+            if isinstance(invite_room_state, list):
+                room_state = invite_room_state
         elif ev.content.get("membership") == Membership.KNOCK:
-            room_state = ev.unsigned.get("knock_room_state", [])
+            knock_room_state = ev.unsigned.get("knock_room_state", [])
+            if isinstance(knock_room_state, list):
+                room_state = knock_room_state
 
         # Ideally we'd reuse the logic in `calculate_room_name`, but that gets
         # complicated to handle partial events vs pulling events from the DB.

synapse/rest/client/sync.py

@@ -436,7 +436,12 @@ async def encode_invited(
             )
             unsigned = dict(invite.get("unsigned", {}))
             invite["unsigned"] = unsigned
-            invited_state = list(unsigned.pop("invite_room_state", []))
+
+            invited_state = unsigned.pop("invite_room_state", [])
+            if not isinstance(invited_state, list):
+                invited_state = []
+
+            invited_state = list(invited_state)
             invited_state.append(invite)
             invited[room.room_id] = {"invite_state": {"events": invited_state}}
 
@@ -476,7 +481,10 @@ async def encode_knocked(
             # Extract the stripped room state from the unsigned dict
             # This is for clients to get a little bit of information about
             # the room they've knocked on, without revealing any sensitive information
-            knocked_state = list(unsigned.pop("knock_room_state", []))
+            knocked_state = unsigned.pop("knock_room_state", [])
+            if not isinstance(knocked_state, list):
+                knocked_state = []
+            knocked_state = list(knocked_state)
 
             # Append the actual knock membership event itself as well. This provides
             # the client with:

synapse/storage/databases/main/state_deltas.py

@@ -243,6 +243,13 @@ async def get_current_state_deltas_for_room(
 
         (> `from_token` and <= `to_token`)
         """
+        # We can bail early if the `from_token` is after the `to_token`
+        if (
+            to_token is not None
+            and from_token is not None
+            and to_token.is_before_or_eq(from_token)
+        ):
+            return []
 
         if (
             from_token is not None