Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Add type annotations to synapse.metrics #10847

Merged
merged 19 commits into from
Nov 17, 2021
Merged

Conversation

squahtx
Copy link
Contributor

@squahtx squahtx commented Sep 17, 2021

No description provided.

@squahtx squahtx force-pushed the squah/synapse_metrics_typings branch from 3254187 to 5447ea9 Compare September 20, 2021 12:06
@squahtx squahtx requested a review from a team September 20, 2021 12:08
@squahtx squahtx force-pushed the squah/synapse_metrics_typings branch from 5447ea9 to 94ef546 Compare September 20, 2021 12:42
@squahtx squahtx force-pushed the squah/synapse_metrics_typings branch from 94ef546 to f058e4f Compare September 20, 2021 13:10
synapse/groups/attestations.py Outdated Show resolved Hide resolved
@@ -53,7 +66,7 @@

class RegistryProxy:
@staticmethod
def collect():
def collect() -> Iterable[Metric]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would think this needs to be a Generator, but maybe not?

This is true for a variety of spots we use yield.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Or maybe I can significantly simplify type hints in other spots!)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opted for using the most basic type that callers want, to minimize assumptions, rather than documenting the concrete return type.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All Generators are Iterators (and all Iterators are Iterable), and in this case this seems cleaner than saying a Generator[Metric, None, None].

@@ -179,7 +179,7 @@ def __getattr__(self, name):


# The type of entry which goes on our after_callbacks and exception_callbacks lists.
_CallbackListEntry = Tuple[Callable[..., None], Iterable[Any], Dict[str, Any]]
_CallbackListEntry = Tuple[Callable[..., object], Iterable[Any], Dict[str, Any]]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious why this is object and not Any?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's to make mypy complain if we ever try to use the return values of the callbacks.

Everything is assignable to object, but not the other way around, whereas Any is assignable to everything without any checks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

object is the supertype of everything is what I think you're saying, so we'd get shouted at if we tried to do x: int = (something returning object) whereas Any is entirely untyped & unchecked.

I think this makes sense.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean we should generally prefer object over Any?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't a clue. I imagine it's on a case by case basis. Any is certainly a last resort for me, since it's equivalent to not annotating types at all.

"""Decorator that wraps a function that gets called as a background
process.

Equivalent of calling the function with `run_as_background_process`
"""

def wrap_as_background_process_inner(func):
# NB: Return type is incorrect and should be F with a Deferred[Optional[R]] return
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should work, but maybe not? This makes me a bit nervous that we're not mentioning this returns a Deferred.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what to do about this. I updated the docstring above to call the change of return type out.

We can either annotate the return type correctly and lose the types of the arguments for the decorated function,
or annotate the return type incorrectly and preserve the types of the arguments.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel kind of bad about lying but appreciate the dilemma. :/

I'm not really sure what's best, though. Is lying going to cause type errors for users of the function?

I would probably make the docstring louder about the return type being a lie, though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to improve the docstring further

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine as - is.

question though, what happens if you bound F to return an Awaitable?

F = TypeVar("F", bound=Callable[..., Awaitable])

(or something similar?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine as - is.

to be clear: I'm saying I'm happy with the PR as it stands.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question though, what happens if you bound F to return an Awaitable?
F = TypeVar("F", bound=Callable[..., Awaitable])

Ooh. mypy throws up 3 errors which all look trivial to fix:

synapse/util/caches/expiringcache.py:86: error: Argument 2 to "run_as_background_process" has incompatible type "Callable[[], None]"; expected "Callable[..., Awaitable[Any]]"  [arg-type]
synapse/app/_base.py:404: error: Value of type variable "F" of function cannot be "Callable[[VarArg(Any), KwArg(Any)], None]"  [type-var]
synapse/handlers/typing.py:92: error: Value of type variable "F" of function cannot be "Callable[[FollowerTypingHandler], None]"  [type-var]

So it sounds like the cleanest thing to do is to restrict run_as_background_process to only accept Awaitable-returning functions and drop the whole maybe_awaitable thing. I'll make the change when I find some time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've gotten rid of the maybe_awaitable.

synapse/metrics/background_process_metrics.py Outdated Show resolved Hide resolved


@overload
def run_as_background_process( # type: ignore[misc]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's being ignored here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's to suppress error: Overloaded function signatures 1 and 2 overlap with incompatible return types [misc]. I shall add a comment

I initially tried a Union but ran into a bug/deficiency in mypy where it couldn't figure out what R was at every call site.
#10847 (comment)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you have a tracking link to e.g. a mypy issue or something of the sort, could be good. (and maybe add a remark that you tried Union[Awaitable[R], R] or whatever, to save someone for thinking 'ha! I can see how to do that one!' and falling into the same pit)

I'm a bit sad Union[Awaitable[R], R] wouldn't have done the trick though, not your fault though

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to improve the comments further

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if updating mypy in #11006 will have changed this situation at all?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good shout! I gave it another go and mypy is still unhappy without the overloads (version 0.910?)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shame. Worth a try!

@clokep clokep requested a review from a team September 20, 2021 15:30
Copy link
Contributor

@reivilibre reivilibre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all seems to make sense. I'm a little bit concerned about the lie and would probably want to ask someone else on the team about that ...

mypy.ini Outdated Show resolved Hide resolved
@@ -53,7 +66,7 @@

class RegistryProxy:
@staticmethod
def collect():
def collect() -> Iterable[Metric]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All Generators are Iterators (and all Iterators are Iterable), and in this case this seems cleaner than saying a Generator[Metric, None, None].

Comment on lines 116 to 117
# Placeholder for `InFlightGauge._metrics_class`, which is created dynamically
_MetricsEntry = Any
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this common practice in our code?

Is it possible/useful to create a Protocol for it (if we have no other choice)?

Right now, Any doesn't tell mypy (or me, or my IDE) a whole lot, though I do appreciate that you've taken the time to common it out into a common label :).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah I see the fields are created dynamically. Makes sense, even if a bit sad.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm taking a look at adding more typing to synapse.utils and got stuck on annotating metrics here. I think it should be one of these dynamically-created types but can't annotate it meaningfully.

So +1 to any other smart approaches!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Python typings are really frustrating. The equivalent in typescript would be something like: this

Perhaps we can add a type parameter to InFlightGauge and make the calling code provide an appropriate Protocol?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. InFlightGauge now takes a type parameter, so now those callbacks can have proper type annotations.
mypy forced me to annotate in_flight in synapse/util/metrics.py (sorry!).

Two caveats:

  • mypy won't allow the type parameter to be constrained to a Protocol
  • there's no check that the names passed to the InFlightGauge constructor match the attributes in the protocol

F = TypeVar("F", bound=Callable[..., Any])


def runUntilCurrentTimer(reactor: ReactorBase, func: F) -> F:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@@ -41,7 +43,7 @@
MINUS_INF = float("-inf")


def floatToGoString(d):
def floatToGoString(d: Any) -> str:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we come up with something a bit stronger than Any here?
Union[float, str] (if that's what is needed)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's typeof(Sample.value) and I had a really hard time figuring out what that was, since prometheus_client doesn't have typings.

Though it seems that prometheus_client pinky promises that it's an int or float:
https://github.com/prometheus/client_python/blob/master/prometheus_client/samples.py#L35-L39
I shall update the function!



@overload
def run_as_background_process( # type: ignore[misc]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you have a tracking link to e.g. a mypy issue or something of the sort, could be good. (and maybe add a remark that you tried Union[Awaitable[R], R] or whatever, to save someone for thinking 'ha! I can see how to do that one!' and falling into the same pit)

I'm a bit sad Union[Awaitable[R], R] wouldn't have done the trick though, not your fault though

"""Decorator that wraps a function that gets called as a background
process.

Equivalent of calling the function with `run_as_background_process`
"""

def wrap_as_background_process_inner(func):
# NB: Return type is incorrect and should be F with a Deferred[Optional[R]] return
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel kind of bad about lying but appreciate the dilemma. :/

I'm not really sure what's best, though. Is lying going to cause type errors for users of the function?

I would probably make the docstring louder about the return type being a lie, though.

@@ -179,7 +179,7 @@ def __getattr__(self, name):


# The type of entry which goes on our after_callbacks and exception_callbacks lists.
_CallbackListEntry = Tuple[Callable[..., None], Iterable[Any], Dict[str, Any]]
_CallbackListEntry = Tuple[Callable[..., object], Iterable[Any], Dict[str, Any]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

object is the supertype of everything is what I think you're saying, so we'd get shouted at if we tried to do x: int = (something returning object) whereas Any is entirely untyped & unchecked.

I think this makes sense.

@reivilibre reivilibre requested a review from a team September 22, 2021 12:03
DMRobertson pushed a commit that referenced this pull request Sep 22, 2021
Didn't get this to pass `no-untyped-def`, think I'll need to watch #10847
Copy link
Contributor

@reivilibre reivilibre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

going to collect some thoughts about whether we feel that it's OK to lie; otherwise LGTM

synapse/metrics/_exposition.py Show resolved Hide resolved
synapse/metrics/__init__.py Show resolved Hide resolved
DMRobertson pushed a commit that referenced this pull request Oct 6, 2021
The following modules now pass `disallow_untyped_defs`:

* synapse.util.caches.cached_call 
* synapse.util.caches.lrucache
* synapse.util.caches.response_cache 
* synapse.util.caches.stream_change_cache
* synapse.util.caches.ttlcache pass
* synapse.util.daemonize
* synapse.util.patch_inline_callbacks pass `no-untyped-defs`
* synapse.util.versionstring

Additional typing in synapse.util.metrics. Didn't get this to pass `no-untyped-defs`, think I'll need to watch #10847
synapse/metrics/__init__.py Show resolved Hide resolved


@overload
def run_as_background_process( # type: ignore[misc]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if updating mypy in #11006 will have changed this situation at all?

@erikjohnston erikjohnston added the X-Awaiting-Changes A contributed PR which needs changes and re-review before it can be merged label Nov 1, 2021
@DMRobertson
Copy link
Contributor

DMRobertson commented Nov 12, 2021

What was the status of this change? I think we were keen to see it done and wouldn't want it to fall through the cracks!

Edit: I guess Olivier wanted to think things over in #10847 (review)

@richvdh
Copy link
Member

richvdh commented Nov 15, 2021

Edit: I guess Olivier wanted to think things over in #10847 (review)

NB that edits don't seen by those of us that read our GH notifications via email.

I think this is somewhere between @squahtx and @reivilibre to sort out currently.

@reivilibre
Copy link
Contributor

NB that edits don't seen by those of us that read our GH notifications via email.

I think this is somewhere between @squahtx and @reivilibre to sort out currently.

Thanks for pointing this out to me! I didn't realise I was holding it up.

I think the only thing on my mind was #10847 (comment) really — if other people are happy with an innocent lie, then I'm happy to roll it through (and it sounds like, bear in mind that we can one day use ParamSpec?)

@squahtx
Copy link
Contributor Author

squahtx commented Nov 15, 2021

Awaiting changes from myself, to restrict run_as_background_process to Awaitable-returning functions only.

@squahtx squahtx removed the X-Awaiting-Changes A contributed PR which needs changes and re-review before it can be merged label Nov 15, 2021
Copy link
Contributor

@reivilibre reivilibre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Callable[..., Awaitable[R]],
Callable[..., R],
],
func: Callable[..., Awaitable[Optional[R]]],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No problem at all, but curious if this needed to be Optional[R] — I naïvely expect it would be fine to let it be R since R is variable.

It won't matter anyway since the return position of a Callable is contravariant (fn(...) -> R is-a fn(...) -> Optional[R]), and this function always returns Optional[R] regardless because of the error handling.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, it ought to be Awaitable[R]

mypy gets unhappy elsewhere if I change it:

synapse/metrics/background_process_metrics.py:273: error: Argument 2 to "run_as_background_process" has incompatible type "F"; expected "Callable[..., Awaitable[R]]"  [arg-type]

@squahtx squahtx merged commit 84fac0f into develop Nov 17, 2021
@squahtx squahtx deleted the squah/synapse_metrics_typings branch November 17, 2021 19:07
babolivier added a commit to matrix-org/synapse-dinsic that referenced this pull request Dec 6, 2021
Synapse 1.48.0 (2021-11-30)
===========================

This release removes support for the long-deprecated `trust_identity_server_for_password_resets` configuration flag.

This release also fixes some performance issues with some background database updates introduced in Synapse 1.47.0.

No significant changes since 1.48.0rc1.

Synapse 1.48.0rc1 (2021-11-25)
==============================

Features
--------

- Experimental support for the thread relation defined in [MSC3440](matrix-org/matrix-spec-proposals#3440). ([\#11161](matrix-org/synapse#11161))
- Support filtering by relation senders & types per [MSC3440](matrix-org/matrix-spec-proposals#3440). ([\#11236](matrix-org/synapse#11236))
- Add support for the `/_matrix/client/v3` and `/_matrix/media/v3` APIs from Matrix v1.1. ([\#11318](matrix-org/synapse#11318), [\#11371](matrix-org/synapse#11371))
- Support the stable version of [MSC2778](matrix-org/matrix-spec-proposals#2778): the `m.login.application_service` login type. Contributed by @tulir. ([\#11335](matrix-org/synapse#11335))
- Add a new version of delete room admin API `DELETE /_synapse/admin/v2/rooms/<room_id>` to run it in the background. Contributed by @dklimpel. ([\#11223](matrix-org/synapse#11223))
- Allow the admin [Delete Room API](https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#delete-room-api) to block a room without the need to join it. ([\#11228](matrix-org/synapse#11228))
- Add an admin API to un-shadow-ban a user. ([\#11347](matrix-org/synapse#11347))
- Add an admin API to run background database schema updates. ([\#11352](matrix-org/synapse#11352))
- Add an admin API for blocking a room. ([\#11324](matrix-org/synapse#11324))
- Update the JWT login type to support custom a `sub` claim. ([\#11361](matrix-org/synapse#11361))
- Store and allow querying of arbitrary event relations. ([\#11391](matrix-org/synapse#11391))

Bugfixes
--------

- Fix a long-standing bug wherein display names or avatar URLs containing null bytes cause an internal server error when stored in the DB. ([\#11230](matrix-org/synapse#11230))
- Prevent [MSC2716](matrix-org/matrix-spec-proposals#2716) historical state events from being pushed to an application service via `/transactions`. ([\#11265](matrix-org/synapse#11265))
- Fix a long-standing bug where uploading extremely thin images (e.g. 1000x1) would fail. Contributed by @Neeeflix. ([\#11288](matrix-org/synapse#11288))
- Fix a bug, introduced in Synapse 1.46.0, which caused the `check_3pid_auth` and `on_logged_out` callbacks in legacy password authentication provider modules to not be registered. Modules using the generic module interface were not affected. ([\#11340](matrix-org/synapse#11340))
- Fix a bug introduced in 1.41.0 where space hierarchy responses would be incorrectly reused if multiple users were to make the same request at the same time. ([\#11355](matrix-org/synapse#11355))
- Fix a bug introduced in 1.45.0 where the `read_templates` method of the module API would error. ([\#11377](matrix-org/synapse#11377))
- Fix an issue introduced in 1.47.0 which prevented servers re-joining rooms they had previously left, if their signing keys were replaced. ([\#11379](matrix-org/synapse#11379))
- Fix a bug introduced in 1.13.0 where creating and publishing a room could cause errors if `room_list_publication_rules` is configured. ([\#11392](matrix-org/synapse#11392))
- Improve performance of various background database updates. ([\#11421](matrix-org/synapse#11421), [\#11422](matrix-org/synapse#11422))

Improved Documentation
----------------------

- Suggest users of the Debian packages add configuration to `/etc/matrix-synapse/conf.d/` to prevent, upon upgrade, being asked to choose between their configuration and the maintainer's. ([\#11281](matrix-org/synapse#11281))
- Fix typos in the documentation for the `username_available` admin API. Contributed by Stanislav Motylkov. ([\#11286](matrix-org/synapse#11286))
- Add Single Sign-On, SAML and CAS pages to the documentation. ([\#11298](matrix-org/synapse#11298))
- Change the word 'Home server' as one word 'homeserver' in documentation. ([\#11320](matrix-org/synapse#11320))
- Fix missing quotes for wildcard domains in `federation_certificate_verification_whitelist`. ([\#11381](matrix-org/synapse#11381))

Deprecations and Removals
-------------------------

- Remove deprecated `trust_identity_server_for_password_resets` configuration flag. ([\#11333](matrix-org/synapse#11333), [\#11395](matrix-org/synapse#11395))

Internal Changes
----------------

- Add type annotations to `synapse.metrics`. ([\#10847](matrix-org/synapse#10847))
- Split out federated PDU retrieval function into a non-cached version. ([\#11242](matrix-org/synapse#11242))
- Clean up code relating to to-device messages and sending ephemeral events to application services. ([\#11247](matrix-org/synapse#11247))
- Fix a small typo in the error response when a relation type other than 'm.annotation' is passed to `GET /rooms/{room_id}/aggregations/{event_id}`. ([\#11278](matrix-org/synapse#11278))
- Drop unused database tables `room_stats_historical` and `user_stats_historical`. ([\#11280](matrix-org/synapse#11280))
- Require all files in synapse/ and tests/ to pass mypy unless specifically excluded. ([\#11282](matrix-org/synapse#11282), [\#11285](matrix-org/synapse#11285), [\#11359](matrix-org/synapse#11359))
- Add missing type hints to `synapse.app`. ([\#11287](matrix-org/synapse#11287))
- Remove unused parameters on `FederationEventHandler._check_event_auth`. ([\#11292](matrix-org/synapse#11292))
- Add type hints to `synapse._scripts`. ([\#11297](matrix-org/synapse#11297))
- Fix an issue which prevented the `remove_deleted_devices_from_device_inbox` background database schema update from running when updating from a recent Synapse version. ([\#11303](matrix-org/synapse#11303))
- Add type hints to storage classes. ([\#11307](matrix-org/synapse#11307), [\#11310](matrix-org/synapse#11310), [\#11311](matrix-org/synapse#11311), [\#11312](matrix-org/synapse#11312), [\#11313](matrix-org/synapse#11313), [\#11314](matrix-org/synapse#11314), [\#11316](matrix-org/synapse#11316), [\#11322](matrix-org/synapse#11322), [\#11332](matrix-org/synapse#11332), [\#11339](matrix-org/synapse#11339), [\#11342](matrix-org/synapse#11342))
- Add type hints to `synapse.util`. ([\#11321](matrix-org/synapse#11321), [\#11328](matrix-org/synapse#11328))
- Improve type annotations in Synapse's test suite. ([\#11323](matrix-org/synapse#11323), [\#11330](matrix-org/synapse#11330))
- Test that room alias deletion works as intended. ([\#11327](matrix-org/synapse#11327))
- Add type annotations for some methods and properties in the module API. ([\#11341](matrix-org/synapse#11341))
- Fix running `scripts-dev/complement.sh`, which was broken in v1.47.0rc1. ([\#11368](matrix-org/synapse#11368))
- Rename internal functions for token generation to better reflect what they do. ([\#11369](matrix-org/synapse#11369), [\#11370](matrix-org/synapse#11370))
- Add type hints to configuration classes. ([\#11377](matrix-org/synapse#11377))
- Publish a `develop` image to Docker Hub. ([\#11380](matrix-org/synapse#11380))
- Keep fallback key marked as used if it's re-uploaded. ([\#11382](matrix-org/synapse#11382))
- Use `auto_attribs` on the `attrs` class `RefreshTokenLookupResult`. ([\#11386](matrix-org/synapse#11386))
- Rename unstable `access_token_lifetime` configuration option to `refreshable_access_token_lifetime` to make it clear it only concerns refreshable access tokens. ([\#11388](matrix-org/synapse#11388))
- Do not run the broken MSC2716 tests when running `scripts-dev/complement.sh`. ([\#11389](matrix-org/synapse#11389))
- Remove dead code from supporting ACME. ([\#11393](matrix-org/synapse#11393))
- Refactor including the bundled relations when serializing an event. ([\#11408](matrix-org/synapse#11408))
squahtx pushed a commit that referenced this pull request Dec 7, 2021
After #10847, `looping_background_call` would print an error in the logs
every time a non-async function was called. Since the error would be
caught and ignored immediately, there were no other side effects.
squahtx added a commit that referenced this pull request Dec 7, 2021
After #10847, `looping_background_call` would print an error in the logs
every time a non-async function was called. Since the error would be
caught and ignored immediately, there were no other side effects.
Fizzadar added a commit to Fizzadar/synapse that referenced this pull request Jan 5, 2022
Synapse 1.48.0 (2021-11-30)
===========================

This release removes support for the long-deprecated `trust_identity_server_for_password_resets` configuration flag.

This release also fixes some performance issues with some background database updates introduced in Synapse 1.47.0.

No significant changes since 1.48.0rc1.

Synapse 1.48.0rc1 (2021-11-25)
==============================

Features
--------

- Experimental support for the thread relation defined in [MSC3440](matrix-org/matrix-spec-proposals#3440). ([\matrix-org#11161](matrix-org#11161))
- Support filtering by relation senders & types per [MSC3440](matrix-org/matrix-spec-proposals#3440). ([\matrix-org#11236](matrix-org#11236))
- Add support for the `/_matrix/client/v3` and `/_matrix/media/v3` APIs from Matrix v1.1. ([\matrix-org#11318](matrix-org#11318), [\matrix-org#11371](matrix-org#11371))
- Support the stable version of [MSC2778](matrix-org/matrix-spec-proposals#2778): the `m.login.application_service` login type. Contributed by @tulir. ([\matrix-org#11335](matrix-org#11335))
- Add a new version of delete room admin API `DELETE /_synapse/admin/v2/rooms/<room_id>` to run it in the background. Contributed by @dklimpel. ([\matrix-org#11223](matrix-org#11223))
- Allow the admin [Delete Room API](https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#delete-room-api) to block a room without the need to join it. ([\matrix-org#11228](matrix-org#11228))
- Add an admin API to un-shadow-ban a user. ([\matrix-org#11347](matrix-org#11347))
- Add an admin API to run background database schema updates. ([\matrix-org#11352](matrix-org#11352))
- Add an admin API for blocking a room. ([\matrix-org#11324](matrix-org#11324))
- Update the JWT login type to support custom a `sub` claim. ([\matrix-org#11361](matrix-org#11361))
- Store and allow querying of arbitrary event relations. ([\matrix-org#11391](matrix-org#11391))

Bugfixes
--------

- Fix a long-standing bug wherein display names or avatar URLs containing null bytes cause an internal server error when stored in the DB. ([\matrix-org#11230](matrix-org#11230))
- Prevent [MSC2716](matrix-org/matrix-spec-proposals#2716) historical state events from being pushed to an application service via `/transactions`. ([\matrix-org#11265](matrix-org#11265))
- Fix a long-standing bug where uploading extremely thin images (e.g. 1000x1) would fail. Contributed by @Neeeflix. ([\matrix-org#11288](matrix-org#11288))
- Fix a bug, introduced in Synapse 1.46.0, which caused the `check_3pid_auth` and `on_logged_out` callbacks in legacy password authentication provider modules to not be registered. Modules using the generic module interface were not affected. ([\matrix-org#11340](matrix-org#11340))
- Fix a bug introduced in 1.41.0 where space hierarchy responses would be incorrectly reused if multiple users were to make the same request at the same time. ([\matrix-org#11355](matrix-org#11355))
- Fix a bug introduced in 1.45.0 where the `read_templates` method of the module API would error. ([\matrix-org#11377](matrix-org#11377))
- Fix an issue introduced in 1.47.0 which prevented servers re-joining rooms they had previously left, if their signing keys were replaced. ([\matrix-org#11379](matrix-org#11379))
- Fix a bug introduced in 1.13.0 where creating and publishing a room could cause errors if `room_list_publication_rules` is configured. ([\matrix-org#11392](matrix-org#11392))
- Improve performance of various background database updates. ([\matrix-org#11421](matrix-org#11421), [\matrix-org#11422](matrix-org#11422))

Improved Documentation
----------------------

- Suggest users of the Debian packages add configuration to `/etc/matrix-synapse/conf.d/` to prevent, upon upgrade, being asked to choose between their configuration and the maintainer's. ([\matrix-org#11281](matrix-org#11281))
- Fix typos in the documentation for the `username_available` admin API. Contributed by Stanislav Motylkov. ([\matrix-org#11286](matrix-org#11286))
- Add Single Sign-On, SAML and CAS pages to the documentation. ([\matrix-org#11298](matrix-org#11298))
- Change the word 'Home server' as one word 'homeserver' in documentation. ([\matrix-org#11320](matrix-org#11320))
- Fix missing quotes for wildcard domains in `federation_certificate_verification_whitelist`. ([\matrix-org#11381](matrix-org#11381))

Deprecations and Removals
-------------------------

- Remove deprecated `trust_identity_server_for_password_resets` configuration flag. ([\matrix-org#11333](matrix-org#11333), [\matrix-org#11395](matrix-org#11395))

Internal Changes
----------------

- Add type annotations to `synapse.metrics`. ([\matrix-org#10847](matrix-org#10847))
- Split out federated PDU retrieval function into a non-cached version. ([\matrix-org#11242](matrix-org#11242))
- Clean up code relating to to-device messages and sending ephemeral events to application services. ([\matrix-org#11247](matrix-org#11247))
- Fix a small typo in the error response when a relation type other than 'm.annotation' is passed to `GET /rooms/{room_id}/aggregations/{event_id}`. ([\matrix-org#11278](matrix-org#11278))
- Drop unused database tables `room_stats_historical` and `user_stats_historical`. ([\matrix-org#11280](matrix-org#11280))
- Require all files in synapse/ and tests/ to pass mypy unless specifically excluded. ([\matrix-org#11282](matrix-org#11282), [\matrix-org#11285](matrix-org#11285), [\matrix-org#11359](matrix-org#11359))
- Add missing type hints to `synapse.app`. ([\matrix-org#11287](matrix-org#11287))
- Remove unused parameters on `FederationEventHandler._check_event_auth`. ([\matrix-org#11292](matrix-org#11292))
- Add type hints to `synapse._scripts`. ([\matrix-org#11297](matrix-org#11297))
- Fix an issue which prevented the `remove_deleted_devices_from_device_inbox` background database schema update from running when updating from a recent Synapse version. ([\matrix-org#11303](matrix-org#11303))
- Add type hints to storage classes. ([\matrix-org#11307](matrix-org#11307), [\matrix-org#11310](matrix-org#11310), [\matrix-org#11311](matrix-org#11311), [\matrix-org#11312](matrix-org#11312), [\matrix-org#11313](matrix-org#11313), [\matrix-org#11314](matrix-org#11314), [\matrix-org#11316](matrix-org#11316), [\matrix-org#11322](matrix-org#11322), [\matrix-org#11332](matrix-org#11332), [\matrix-org#11339](matrix-org#11339), [\matrix-org#11342](matrix-org#11342))
- Add type hints to `synapse.util`. ([\matrix-org#11321](matrix-org#11321), [\matrix-org#11328](matrix-org#11328))
- Improve type annotations in Synapse's test suite. ([\matrix-org#11323](matrix-org#11323), [\matrix-org#11330](matrix-org#11330))
- Test that room alias deletion works as intended. ([\matrix-org#11327](matrix-org#11327))
- Add type annotations for some methods and properties in the module API. ([\matrix-org#11341](matrix-org#11341))
- Fix running `scripts-dev/complement.sh`, which was broken in v1.47.0rc1. ([\matrix-org#11368](matrix-org#11368))
- Rename internal functions for token generation to better reflect what they do. ([\matrix-org#11369](matrix-org#11369), [\matrix-org#11370](matrix-org#11370))
- Add type hints to configuration classes. ([\matrix-org#11377](matrix-org#11377))
- Publish a `develop` image to Docker Hub. ([\matrix-org#11380](matrix-org#11380))
- Keep fallback key marked as used if it's re-uploaded. ([\matrix-org#11382](matrix-org#11382))
- Use `auto_attribs` on the `attrs` class `RefreshTokenLookupResult`. ([\matrix-org#11386](matrix-org#11386))
- Rename unstable `access_token_lifetime` configuration option to `refreshable_access_token_lifetime` to make it clear it only concerns refreshable access tokens. ([\matrix-org#11388](matrix-org#11388))
- Do not run the broken MSC2716 tests when running `scripts-dev/complement.sh`. ([\matrix-org#11389](matrix-org#11389))
- Remove dead code from supporting ACME. ([\matrix-org#11393](matrix-org#11393))
- Refactor including the bundled relations when serializing an event. ([\matrix-org#11408](matrix-org#11408))
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants