Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Soft-fail spammy events received over federation #10263

Merged
merged 2 commits into from
Jun 29, 2021
Merged

Conversation

richvdh
Copy link
Member

@richvdh richvdh commented Jun 28, 2021

Soft-failing spam which arrives over federation achieves two things:

  • the content will not be served to the clients of local users
  • spam events will not be used as prev-events for future events we generate.

The second point is useful in the following situation. Suppose we have three servers in a room: spammy.com, speedy.com and slow.com. spammy creates thousands of events and sends them out to the other servers. speedy accepts them all (possibly redacting them or similar measures) and starts generating its own events (possibly including things like server ACLs). slow.com can't keep up with the traffic from spammy.com so when it sees events from speedy.com it sees unknown events which it needs to backfill, so traffic from speedy.com to slow.com is severely impacted too.

On the other hand, if speedy.com soft-fails the spam, slow.com will be able to better handle later events from speedy.com.

@richvdh richvdh requested a review from a team June 28, 2021 15:59
Copy link
Contributor

@babolivier babolivier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm otherwise

synapse/federation/federation_base.py Outdated Show resolved Hide resolved
@richvdh richvdh merged commit a0ed0f3 into develop Jun 29, 2021
@richvdh richvdh deleted the rav/soft_fail_spam branch June 29, 2021 10:08
babolivier added a commit that referenced this pull request Jul 7, 2021
Synapse 1.38.0rc1 (2021-07-06)
==============================

This release includes a database schema update which could result in elevated disk usage. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#upgrading-to-v1380) for more information.

Features
--------

- Implement refresh tokens as specified by [MSC2918](matrix-org/matrix-spec-proposals#2918). ([\#9450](#9450))
- Add support for evicting cache entries based on last access time. ([\#10205](#10205))
- Omit empty fields from the `/sync` response. Contributed by @deepbluev7. ([\#10214](#10214))
- Improve validation on federation `send_{join,leave,knock}` endpoints. ([\#10225](#10225), [\#10243](#10243))
- Add SSO `external_ids` to the Query User Account admin API. ([\#10261](#10261))
- Mark events received over federation which fail a spam check as "soft-failed". ([\#10263](#10263))
- Add metrics for new inbound federation staging area. ([\#10284](#10284))
- Add script to print information about recently registered users. ([\#10290](#10290))

Bugfixes
--------

- Fix a long-standing bug which meant that invite rejections and knocks were not sent out over federation in a timely manner. ([\#10223](#10223))
- Fix a bug introduced in v1.26.0 where only users who have set profile information could be deactivated with erasure enabled. ([\#10252](#10252))
- Fix a long-standing bug where Synapse would return errors after 2<sup>31</sup> events were handled by the server. ([\#10264](#10264), [\#10267](#10267), [\#10282](#10282), [\#10286](#10286), [\#10291](#10291), [\#10314](#10314), [\#10326](#10326))
- Fix the prometheus `synapse_federation_server_pdu_process_time` metric. Broke in v1.37.1. ([\#10279](#10279))
- Ensure that inbound events from federation that were being processed when Synapse was restarted get promptly processed on start up. ([\#10303](#10303))

Improved Documentation
----------------------

- Move the upgrade notes to [docs/upgrade.md](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md) and convert them to markdown. ([\#10166](#10166))
- Choose Welcome & Overview as the default page for synapse documentation website. ([\#10242](#10242))
- Adjust the URL in the README.rst file to point to irc.libera.chat. ([\#10258](#10258))
- Fix homeserver config option name in presence router documentation. ([\#10288](#10288))
- Fix link pointing at the wrong section in the modules documentation page. ([\#10302](#10302))

Internal Changes
----------------

- Drop `Origin` and `Accept` from the value of the `Access-Control-Allow-Headers` response header. ([\#10114](#10114))
- Add type hints to the federation servlets. ([\#10213](#10213))
- Improve the reliability of auto-joining remote rooms. ([\#10237](#10237))
- Update the release script to use the semver terminology and determine the release branch based on the next version. ([\#10239](#10239))
- Fix type hints for computing auth events. ([\#10253](#10253))
- Improve the performance of the spaces summary endpoint by only recursing into spaces (and not rooms in general). ([\#10256](#10256))
- Move event authentication methods from `Auth` to `EventAuthHandler`. ([\#10268](#10268))
- Re-enable a SyTest after it has been fixed. ([\#10292](#10292))
aaronraimist added a commit to aaronraimist/synapse that referenced this pull request Jul 13, 2021
Synapse 1.38.0 (2021-07-13)
===========================

This release includes a database schema update which could result in elevated disk usage. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#upgrading-to-v1380) for more information.

No significant changes since 1.38.0rc3.

Synapse 1.38.0rc3 (2021-07-13)
==============================

Internal Changes
----------------

- Build the Debian packages in CI. ([\matrix-org#10247](matrix-org#10247), [\matrix-org#10379](matrix-org#10379))

Synapse 1.38.0rc2 (2021-07-09)
==============================

Bugfixes
--------

- Fix bug where inbound federation in a room could be delayed due to not correctly dropping a lock. Introduced in v1.37.1. ([\matrix-org#10336](matrix-org#10336))

Improved Documentation
----------------------

- Update links to documentation in the sample config. Contributed by @dklimpel. ([\matrix-org#10287](matrix-org#10287))
- Fix broken links in [INSTALL.md](INSTALL.md). Contributed by @dklimpel. ([\matrix-org#10331](matrix-org#10331))

Synapse 1.38.0rc1 (2021-07-06)
==============================

Features
--------

- Implement refresh tokens as specified by [MSC2918](matrix-org/matrix-spec-proposals#2918). ([\matrix-org#9450](matrix-org#9450))
- Add support for evicting cache entries based on last access time. ([\matrix-org#10205](matrix-org#10205))
- Omit empty fields from the `/sync` response. Contributed by @deepbluev7. ([\matrix-org#10214](matrix-org#10214))
- Improve validation on federation `send_{join,leave,knock}` endpoints. ([\matrix-org#10225](matrix-org#10225), [\matrix-org#10243](matrix-org#10243))
- Add SSO `external_ids` to the Query User Account admin API. ([\matrix-org#10261](matrix-org#10261))
- Mark events received over federation which fail a spam check as "soft-failed". ([\matrix-org#10263](matrix-org#10263))
- Add metrics for new inbound federation staging area. ([\matrix-org#10284](matrix-org#10284))
- Add script to print information about recently registered users. ([\matrix-org#10290](matrix-org#10290))

Bugfixes
--------

- Fix a long-standing bug which meant that invite rejections and knocks were not sent out over federation in a timely manner. ([\matrix-org#10223](matrix-org#10223))
- Fix a bug introduced in v1.26.0 where only users who have set profile information could be deactivated with erasure enabled. ([\matrix-org#10252](matrix-org#10252))
- Fix a long-standing bug where Synapse would return errors after 2<sup>31</sup> events were handled by the server. ([\matrix-org#10264](matrix-org#10264), [\matrix-org#10267](matrix-org#10267), [\matrix-org#10282](matrix-org#10282), [\matrix-org#10286](matrix-org#10286), [\matrix-org#10291](matrix-org#10291), [\matrix-org#10314](matrix-org#10314), [\matrix-org#10326](matrix-org#10326))
- Fix the prometheus `synapse_federation_server_pdu_process_time` metric. Broke in v1.37.1. ([\matrix-org#10279](matrix-org#10279))
- Ensure that inbound events from federation that were being processed when Synapse was restarted get promptly processed on start up. ([\matrix-org#10303](matrix-org#10303))

Improved Documentation
----------------------

- Move the upgrade notes to [docs/upgrade.md](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md) and convert them to markdown. ([\matrix-org#10166](matrix-org#10166))
- Choose Welcome & Overview as the default page for synapse documentation website. ([\matrix-org#10242](matrix-org#10242))
- Adjust the URL in the README.rst file to point to irc.libera.chat. ([\matrix-org#10258](matrix-org#10258))
- Fix homeserver config option name in presence router documentation. ([\matrix-org#10288](matrix-org#10288))
- Fix link pointing at the wrong section in the modules documentation page. ([\matrix-org#10302](matrix-org#10302))

Internal Changes
----------------

- Drop `Origin` and `Accept` from the value of the `Access-Control-Allow-Headers` response header. ([\matrix-org#10114](matrix-org#10114))
- Add type hints to the federation servlets. ([\matrix-org#10213](matrix-org#10213))
- Improve the reliability of auto-joining remote rooms. ([\matrix-org#10237](matrix-org#10237))
- Update the release script to use the semver terminology and determine the release branch based on the next version. ([\matrix-org#10239](matrix-org#10239))
- Fix type hints for computing auth events. ([\matrix-org#10253](matrix-org#10253))
- Improve the performance of the spaces summary endpoint by only recursing into spaces (and not rooms in general). ([\matrix-org#10256](matrix-org#10256))
- Move event authentication methods from `Auth` to `EventAuthHandler`. ([\matrix-org#10268](matrix-org#10268))
- Re-enable a SyTest after it has been fixed. ([\matrix-org#10292](matrix-org#10292))
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Jul 17, 2021
Synapse 1.38.0 (2021-07-13)
===========================

This release includes a database schema update which could result in elevated disk usage. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#upgrading-to-v1380) for more information.

No significant changes since 1.38.0rc3.


Synapse 1.38.0rc3 (2021-07-13)
==============================

Internal Changes
----------------

- Build the Debian packages in CI. ([\#10247](matrix-org/synapse#10247), [\#10379](matrix-org/synapse#10379))


Synapse 1.38.0rc2 (2021-07-09)
==============================

Bugfixes
--------

- Fix bug where inbound federation in a room could be delayed due to not correctly dropping a lock. Introduced in v1.37.1. ([\#10336](matrix-org/synapse#10336))


Improved Documentation
----------------------

- Update links to documentation in the sample config. Contributed by @dklimpel. ([\#10287](matrix-org/synapse#10287))
- Fix broken links in [INSTALL.md](INSTALL.md). Contributed by @dklimpel. ([\#10331](matrix-org/synapse#10331))


Synapse 1.38.0rc1 (2021-07-06)
==============================

Features
--------

- Implement refresh tokens as specified by [MSC2918](matrix-org/matrix-spec-proposals#2918). ([\#9450](matrix-org/synapse#9450))
- Add support for evicting cache entries based on last access time. ([\#10205](matrix-org/synapse#10205))
- Omit empty fields from the `/sync` response. Contributed by @deepbluev7. ([\#10214](matrix-org/synapse#10214))
- Improve validation on federation `send_{join,leave,knock}` endpoints. ([\#10225](matrix-org/synapse#10225), [\#10243](matrix-org/synapse#10243))
- Add SSO `external_ids` to the Query User Account admin API. ([\#10261](matrix-org/synapse#10261))
- Mark events received over federation which fail a spam check as "soft-failed". ([\#10263](matrix-org/synapse#10263))
- Add metrics for new inbound federation staging area. ([\#10284](matrix-org/synapse#10284))
- Add script to print information about recently registered users. ([\#10290](matrix-org/synapse#10290))


Bugfixes
--------

- Fix a long-standing bug which meant that invite rejections and knocks were not sent out over federation in a timely manner. ([\#10223](matrix-org/synapse#10223))
- Fix a bug introduced in v1.26.0 where only users who have set profile information could be deactivated with erasure enabled. ([\#10252](matrix-org/synapse#10252))
- Fix a long-standing bug where Synapse would return errors after 2<sup>31</sup> events were handled by the server. ([\#10264](matrix-org/synapse#10264), [\#10267](matrix-org/synapse#10267), [\#10282](matrix-org/synapse#10282), [\#10286](matrix-org/synapse#10286), [\#10291](matrix-org/synapse#10291), [\#10314](matrix-org/synapse#10314), [\#10326](matrix-org/synapse#10326))
- Fix the prometheus `synapse_federation_server_pdu_process_time` metric. Broke in v1.37.1. ([\#10279](matrix-org/synapse#10279))
- Ensure that inbound events from federation that were being processed when Synapse was restarted get promptly processed on start up. ([\#10303](matrix-org/synapse#10303))


Improved Documentation
----------------------

- Move the upgrade notes to [docs/upgrade.md](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md) and convert them to markdown. ([\#10166](matrix-org/synapse#10166))
- Choose Welcome & Overview as the default page for synapse documentation website. ([\#10242](matrix-org/synapse#10242))
- Adjust the URL in the README.rst file to point to irc.libera.chat. ([\#10258](matrix-org/synapse#10258))
- Fix homeserver config option name in presence router documentation. ([\#10288](matrix-org/synapse#10288))
- Fix link pointing at the wrong section in the modules documentation page. ([\#10302](matrix-org/synapse#10302))


Internal Changes
----------------

- Drop `Origin` and `Accept` from the value of the `Access-Control-Allow-Headers` response header. ([\#10114](matrix-org/synapse#10114))
- Add type hints to the federation servlets. ([\#10213](matrix-org/synapse#10213))
- Improve the reliability of auto-joining remote rooms. ([\#10237](matrix-org/synapse#10237))
- Update the release script to use the semver terminology and determine the release branch based on the next version. ([\#10239](matrix-org/synapse#10239))
- Fix type hints for computing auth events. ([\#10253](matrix-org/synapse#10253))
- Improve the performance of the spaces summary endpoint by only recursing into spaces (and not rooms in general). ([\#10256](matrix-org/synapse#10256))
- Move event authentication methods from `Auth` to `EventAuthHandler`. ([\#10268](matrix-org/synapse#10268))
- Re-enable a SyTest after it has been fixed. ([\#10292](matrix-org/synapse#10292))
babolivier added a commit to matrix-org/synapse-dinsic that referenced this pull request Sep 1, 2021
Synapse 1.38.0 (2021-07-13)
===========================

This release includes a database schema update which could result in elevated disk usage. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#upgrading-to-v1380) for more information.

No significant changes since 1.38.0rc3.

Synapse 1.38.0rc3 (2021-07-13)
==============================

Internal Changes
----------------

- Build the Debian packages in CI. ([\#10247](matrix-org/synapse#10247), [\#10379](matrix-org/synapse#10379))

Synapse 1.38.0rc2 (2021-07-09)
==============================

Bugfixes
--------

- Fix bug where inbound federation in a room could be delayed due to not correctly dropping a lock. Introduced in v1.37.1. ([\#10336](matrix-org/synapse#10336))

Improved Documentation
----------------------

- Update links to documentation in the sample config. Contributed by @dklimpel. ([\#10287](matrix-org/synapse#10287))
- Fix broken links in [INSTALL.md](INSTALL.md). Contributed by @dklimpel. ([\#10331](matrix-org/synapse#10331))

Synapse 1.38.0rc1 (2021-07-06)
==============================

Features
--------

- Implement refresh tokens as specified by [MSC2918](matrix-org/matrix-spec-proposals#2918). ([\#9450](matrix-org/synapse#9450))
- Add support for evicting cache entries based on last access time. ([\#10205](matrix-org/synapse#10205))
- Omit empty fields from the `/sync` response. Contributed by @deepbluev7. ([\#10214](matrix-org/synapse#10214))
- Improve validation on federation `send_{join,leave,knock}` endpoints. ([\#10225](matrix-org/synapse#10225), [\#10243](matrix-org/synapse#10243))
- Add SSO `external_ids` to the Query User Account admin API. ([\#10261](matrix-org/synapse#10261))
- Mark events received over federation which fail a spam check as "soft-failed". ([\#10263](matrix-org/synapse#10263))
- Add metrics for new inbound federation staging area. ([\#10284](matrix-org/synapse#10284))
- Add script to print information about recently registered users. ([\#10290](matrix-org/synapse#10290))

Bugfixes
--------

- Fix a long-standing bug which meant that invite rejections and knocks were not sent out over federation in a timely manner. ([\#10223](matrix-org/synapse#10223))
- Fix a bug introduced in v1.26.0 where only users who have set profile information could be deactivated with erasure enabled. ([\#10252](matrix-org/synapse#10252))
- Fix a long-standing bug where Synapse would return errors after 2<sup>31</sup> events were handled by the server. ([\#10264](matrix-org/synapse#10264), [\#10267](matrix-org/synapse#10267), [\#10282](matrix-org/synapse#10282), [\#10286](matrix-org/synapse#10286), [\#10291](matrix-org/synapse#10291), [\#10314](matrix-org/synapse#10314), [\#10326](matrix-org/synapse#10326))
- Fix the prometheus `synapse_federation_server_pdu_process_time` metric. Broke in v1.37.1. ([\#10279](matrix-org/synapse#10279))
- Ensure that inbound events from federation that were being processed when Synapse was restarted get promptly processed on start up. ([\#10303](matrix-org/synapse#10303))

Improved Documentation
----------------------

- Move the upgrade notes to [docs/upgrade.md](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md) and convert them to markdown. ([\#10166](matrix-org/synapse#10166))
- Choose Welcome & Overview as the default page for synapse documentation website. ([\#10242](matrix-org/synapse#10242))
- Adjust the URL in the README.rst file to point to irc.libera.chat. ([\#10258](matrix-org/synapse#10258))
- Fix homeserver config option name in presence router documentation. ([\#10288](matrix-org/synapse#10288))
- Fix link pointing at the wrong section in the modules documentation page. ([\#10302](matrix-org/synapse#10302))

Internal Changes
----------------

- Drop `Origin` and `Accept` from the value of the `Access-Control-Allow-Headers` response header. ([\#10114](matrix-org/synapse#10114))
- Add type hints to the federation servlets. ([\#10213](matrix-org/synapse#10213))
- Improve the reliability of auto-joining remote rooms. ([\#10237](matrix-org/synapse#10237))
- Update the release script to use the semver terminology and determine the release branch based on the next version. ([\#10239](matrix-org/synapse#10239))
- Fix type hints for computing auth events. ([\#10253](matrix-org/synapse#10253))
- Improve the performance of the spaces summary endpoint by only recursing into spaces (and not rooms in general). ([\#10256](matrix-org/synapse#10256))
- Move event authentication methods from `Auth` to `EventAuthHandler`. ([\#10268](matrix-org/synapse#10268))
- Re-enable a SyTest after it has been fixed. ([\#10292](matrix-org/synapse#10292))
Fizzadar pushed a commit to Fizzadar/synapse that referenced this pull request Oct 26, 2021
Synapse 1.38.0 (2021-07-13)
===========================

This release includes a database schema update which could result in elevated disk usage. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#upgrading-to-v1380) for more information.

No significant changes since 1.38.0rc3.

Synapse 1.38.0rc3 (2021-07-13)
==============================

Internal Changes
----------------

- Build the Debian packages in CI. ([\matrix-org#10247](matrix-org#10247), [\matrix-org#10379](matrix-org#10379))

Synapse 1.38.0rc2 (2021-07-09)
==============================

Bugfixes
--------

- Fix bug where inbound federation in a room could be delayed due to not correctly dropping a lock. Introduced in v1.37.1. ([\matrix-org#10336](matrix-org#10336))

Improved Documentation
----------------------

- Update links to documentation in the sample config. Contributed by @dklimpel. ([\matrix-org#10287](matrix-org#10287))
- Fix broken links in [INSTALL.md](INSTALL.md). Contributed by @dklimpel. ([\matrix-org#10331](matrix-org#10331))

Synapse 1.38.0rc1 (2021-07-06)
==============================

Features
--------

- Implement refresh tokens as specified by [MSC2918](matrix-org/matrix-spec-proposals#2918). ([\matrix-org#9450](matrix-org#9450))
- Add support for evicting cache entries based on last access time. ([\matrix-org#10205](matrix-org#10205))
- Omit empty fields from the `/sync` response. Contributed by @deepbluev7. ([\matrix-org#10214](matrix-org#10214))
- Improve validation on federation `send_{join,leave,knock}` endpoints. ([\matrix-org#10225](matrix-org#10225), [\matrix-org#10243](matrix-org#10243))
- Add SSO `external_ids` to the Query User Account admin API. ([\matrix-org#10261](matrix-org#10261))
- Mark events received over federation which fail a spam check as "soft-failed". ([\matrix-org#10263](matrix-org#10263))
- Add metrics for new inbound federation staging area. ([\matrix-org#10284](matrix-org#10284))
- Add script to print information about recently registered users. ([\matrix-org#10290](matrix-org#10290))

Bugfixes
--------

- Fix a long-standing bug which meant that invite rejections and knocks were not sent out over federation in a timely manner. ([\matrix-org#10223](matrix-org#10223))
- Fix a bug introduced in v1.26.0 where only users who have set profile information could be deactivated with erasure enabled. ([\matrix-org#10252](matrix-org#10252))
- Fix a long-standing bug where Synapse would return errors after 2<sup>31</sup> events were handled by the server. ([\matrix-org#10264](matrix-org#10264), [\matrix-org#10267](matrix-org#10267), [\matrix-org#10282](matrix-org#10282), [\matrix-org#10286](matrix-org#10286), [\matrix-org#10291](matrix-org#10291), [\matrix-org#10314](matrix-org#10314), [\matrix-org#10326](matrix-org#10326))
- Fix the prometheus `synapse_federation_server_pdu_process_time` metric. Broke in v1.37.1. ([\matrix-org#10279](matrix-org#10279))
- Ensure that inbound events from federation that were being processed when Synapse was restarted get promptly processed on start up. ([\matrix-org#10303](matrix-org#10303))

Improved Documentation
----------------------

- Move the upgrade notes to [docs/upgrade.md](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md) and convert them to markdown. ([\matrix-org#10166](matrix-org#10166))
- Choose Welcome & Overview as the default page for synapse documentation website. ([\matrix-org#10242](matrix-org#10242))
- Adjust the URL in the README.rst file to point to irc.libera.chat. ([\matrix-org#10258](matrix-org#10258))
- Fix homeserver config option name in presence router documentation. ([\matrix-org#10288](matrix-org#10288))
- Fix link pointing at the wrong section in the modules documentation page. ([\matrix-org#10302](matrix-org#10302))

Internal Changes
----------------

- Drop `Origin` and `Accept` from the value of the `Access-Control-Allow-Headers` response header. ([\matrix-org#10114](matrix-org#10114))
- Add type hints to the federation servlets. ([\matrix-org#10213](matrix-org#10213))
- Improve the reliability of auto-joining remote rooms. ([\matrix-org#10237](matrix-org#10237))
- Update the release script to use the semver terminology and determine the release branch based on the next version. ([\matrix-org#10239](matrix-org#10239))
- Fix type hints for computing auth events. ([\matrix-org#10253](matrix-org#10253))
- Improve the performance of the spaces summary endpoint by only recursing into spaces (and not rooms in general). ([\matrix-org#10256](matrix-org#10256))
- Move event authentication methods from `Auth` to `EventAuthHandler`. ([\matrix-org#10268](matrix-org#10268))
- Re-enable a SyTest after it has been fixed. ([\matrix-org#10292](matrix-org#10292))
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants