mirror of
https://github.com/element-hq/synapse
synced 2024-06-30 10:13:29 +00:00
Compare commits
45 commits
662ac539d6
...
b1e98eec0e
Author | SHA1 | Date | |
---|---|---|---|
|
b1e98eec0e | ||
|
7c23b77f3e | ||
|
77e1414b08 | ||
|
222f5b120b | ||
|
27756c9fdf | ||
|
cc5e5893fe | ||
|
7c169f4d2c | ||
|
f75da9cc53 | ||
|
75c19bf57a | ||
|
b924a8e1a9 | ||
|
a8dcd686fb | ||
|
315b8d2032 | ||
|
9f47513458 | ||
|
ef7fbdfebd | ||
|
9cf0ef9c70 | ||
|
a023538822 | ||
|
f79dbd0f61 | ||
|
c89fea3fd1 | ||
|
554a92601a | ||
|
a98cb87bee | ||
|
6e8af83193 | ||
|
805e6c9a8f | ||
|
3c61ddbbc9 | ||
|
ae4c236a6d | ||
|
930a64b6c1 | ||
|
7a11c0ac4f | ||
|
cf711ac03c | ||
|
700d2cc4a0 | ||
|
1e74b50dc6 | ||
|
7c2d8f1f01 | ||
|
118b734081 | ||
|
7a6186b888 | ||
|
452a59f887 | ||
|
adeedb7b7c | ||
|
7c5fb13f7b | ||
|
f8d57ce656 | ||
|
13ed84c573 | ||
|
4243c1f074 | ||
|
3239b7459c | ||
|
c99203d98c | ||
|
9104a9f0d0 | ||
|
a412a5829d | ||
|
7ef89b985d | ||
|
bdf82efea5 | ||
|
afaf2d9388 |
2
.github/ISSUE_TEMPLATE.md
vendored
2
.github/ISSUE_TEMPLATE.md
vendored
|
@ -2,4 +2,4 @@
|
|||
(using a matrix.org account if necessary). We do not use GitHub issues for
|
||||
support.
|
||||
|
||||
**If you want to report a security issue** please see https://matrix.org/security-disclosure-policy/
|
||||
**If you want to report a security issue** please see https://element.io/security/security-disclosure-policy
|
||||
|
|
2
.github/ISSUE_TEMPLATE/BUG_REPORT.yml
vendored
2
.github/ISSUE_TEMPLATE/BUG_REPORT.yml
vendored
|
@ -7,7 +7,7 @@ body:
|
|||
**THIS IS NOT A SUPPORT CHANNEL!**
|
||||
**IF YOU HAVE SUPPORT QUESTIONS ABOUT RUNNING OR CONFIGURING YOUR OWN HOME SERVER**, please ask in **[#synapse:matrix.org](https://matrix.to/#/#synapse:matrix.org)** (using a matrix.org account if necessary).
|
||||
|
||||
If you want to report a security issue, please see https://matrix.org/security-disclosure-policy/
|
||||
If you want to report a security issue, please see https://element.io/security/security-disclosure-policy
|
||||
|
||||
This is a bug report form. By following the instructions below and completing the sections with your information, you will help the us to get all the necessary data to fix your issue.
|
||||
|
||||
|
|
20
.github/workflows/tests.yml
vendored
20
.github/workflows/tests.yml
vendored
|
@ -21,6 +21,7 @@ jobs:
|
|||
trial: ${{ !startsWith(github.ref, 'refs/pull/') || steps.filter.outputs.trial }}
|
||||
integration: ${{ !startsWith(github.ref, 'refs/pull/') || steps.filter.outputs.integration }}
|
||||
linting: ${{ !startsWith(github.ref, 'refs/pull/') || steps.filter.outputs.linting }}
|
||||
linting_readme: ${{ !startsWith(github.ref, 'refs/pull/') || steps.filter.outputs.linting_readme }}
|
||||
steps:
|
||||
- uses: dorny/paths-filter@v3
|
||||
id: filter
|
||||
|
@ -72,6 +73,9 @@ jobs:
|
|||
- 'pyproject.toml'
|
||||
- 'poetry.lock'
|
||||
- '.github/workflows/tests.yml'
|
||||
|
||||
linting_readme:
|
||||
- 'README.rst'
|
||||
|
||||
check-sampleconfig:
|
||||
runs-on: ubuntu-latest
|
||||
|
@ -269,6 +273,20 @@ jobs:
|
|||
|
||||
- run: cargo fmt --check
|
||||
|
||||
# This is to detect issues with the rst file, which can otherwise cause issues
|
||||
# when uploading packages to PyPi.
|
||||
lint-readme:
|
||||
runs-on: ubuntu-latest
|
||||
needs: changes
|
||||
if: ${{ needs.changes.outputs.linting_readme == 'true' }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.x"
|
||||
- run: "pip install rstcheck"
|
||||
- run: "rstcheck --report-level=WARNING README.rst"
|
||||
|
||||
# Dummy step to gate other tests on without repeating the whole list
|
||||
linting-done:
|
||||
if: ${{ !cancelled() }} # Run this even if prior jobs were skipped
|
||||
|
@ -284,6 +302,7 @@ jobs:
|
|||
- lint-clippy
|
||||
- lint-clippy-nightly
|
||||
- lint-rustfmt
|
||||
- lint-readme
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: matrix-org/done-action@v2
|
||||
|
@ -301,6 +320,7 @@ jobs:
|
|||
lint-clippy
|
||||
lint-clippy-nightly
|
||||
lint-rustfmt
|
||||
lint-readme
|
||||
|
||||
|
||||
calculate-test-jobs:
|
||||
|
|
91
CHANGES.md
91
CHANGES.md
|
@ -1,3 +1,94 @@
|
|||
# Synapse 1.110.0rc2 (2024-06-26)
|
||||
|
||||
### Internal Changes
|
||||
|
||||
- Fix uploading packages to PyPi. ([\#17363](https://github.com/element-hq/synapse/issues/17363))
|
||||
|
||||
|
||||
|
||||
|
||||
# Synapse 1.110.0rc1 (2024-06-26)
|
||||
|
||||
### Features
|
||||
|
||||
- Add initial implementation of an experimental [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) Sliding Sync `/sync` endpoint. ([\#17187](https://github.com/element-hq/synapse/issues/17187))
|
||||
- Add experimental support for [MSC3823](https://github.com/matrix-org/matrix-spec-proposals/pull/3823) - Account suspension. ([\#17255](https://github.com/element-hq/synapse/issues/17255))
|
||||
- Improve ratelimiting in Synapse. ([\#17256](https://github.com/element-hq/synapse/issues/17256))
|
||||
- Add support for the unstable [MSC4151](https://github.com/matrix-org/matrix-spec-proposals/pull/4151) report room API. ([\#17270](https://github.com/element-hq/synapse/issues/17270), [\#17296](https://github.com/element-hq/synapse/issues/17296))
|
||||
- Filter for public and empty rooms added to Admin-API [List Room API](https://element-hq.github.io/synapse/latest/admin_api/rooms.html#list-room-api). ([\#17276](https://github.com/element-hq/synapse/issues/17276))
|
||||
- Add `is_dm` filtering to experimental [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) Sliding Sync `/sync` endpoint. ([\#17277](https://github.com/element-hq/synapse/issues/17277))
|
||||
- Add `is_encrypted` filtering to experimental [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) Sliding Sync `/sync` endpoint. ([\#17281](https://github.com/element-hq/synapse/issues/17281))
|
||||
- Include user membership in events served to clients, per [MSC4115](https://github.com/matrix-org/matrix-spec-proposals/pull/4115). ([\#17282](https://github.com/element-hq/synapse/issues/17282))
|
||||
- Do not require user-interactive authentication for uploading cross-signing keys for the first time, per [MSC3967](https://github.com/matrix-org/matrix-spec-proposals/pull/3967). ([\#17284](https://github.com/element-hq/synapse/issues/17284))
|
||||
- Add `stream_ordering` sort to experimental [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) Sliding Sync `/sync` endpoint. ([\#17293](https://github.com/element-hq/synapse/issues/17293))
|
||||
- `register_new_matrix_user` now supports a --password-file flag, which
|
||||
is useful for scripting. ([\#17294](https://github.com/element-hq/synapse/issues/17294))
|
||||
- `register_new_matrix_user` now supports a --exists-ok flag to allow registration of users that already exist in the database.
|
||||
This is useful for scripts that bootstrap user accounts with initial passwords. ([\#17304](https://github.com/element-hq/synapse/issues/17304))
|
||||
- Add support for via query parameter from [MSC4156](https://github.com/matrix-org/matrix-spec-proposals/pull/4156). ([\#17322](https://github.com/element-hq/synapse/issues/17322))
|
||||
- Add `is_invite` filtering to experimental [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) Sliding Sync `/sync` endpoint. ([\#17335](https://github.com/element-hq/synapse/issues/17335))
|
||||
- Support [MSC3916](https://github.com/matrix-org/matrix-spec-proposals/blob/rav/authentication-for-media/proposals/3916-authentication-for-media.md) by adding a federation /download endpoint. ([\#17350](https://github.com/element-hq/synapse/issues/17350))
|
||||
|
||||
### Bugfixes
|
||||
|
||||
- Fix searching for users with their exact localpart whose ID includes a hyphen. ([\#17254](https://github.com/element-hq/synapse/issues/17254))
|
||||
- Fix wrong retention policy being used when filtering events. ([\#17272](https://github.com/element-hq/synapse/issues/17272))
|
||||
- Fix bug where OTKs were not always included in `/sync` response when using workers. ([\#17275](https://github.com/element-hq/synapse/issues/17275))
|
||||
- Fix a long-standing bug where an invalid 'from' parameter to [`/notifications`](https://spec.matrix.org/v1.10/client-server-api/#get_matrixclientv3notifications) would result in an Internal Server Error. ([\#17283](https://github.com/element-hq/synapse/issues/17283))
|
||||
- Fix edge case in `/sync` returning the wrong the state when using sharded event persisters. ([\#17295](https://github.com/element-hq/synapse/issues/17295))
|
||||
- Add initial implementation of an experimental [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) Sliding Sync `/sync` endpoint. ([\#17301](https://github.com/element-hq/synapse/issues/17301))
|
||||
- Fix email notification subject when invited to a space. ([\#17336](https://github.com/element-hq/synapse/issues/17336))
|
||||
|
||||
### Improved Documentation
|
||||
|
||||
- Add missing quotes for example for `exclude_rooms_from_sync`. ([\#17308](https://github.com/element-hq/synapse/issues/17308))
|
||||
- Update header in the README to visually fix the the auto-generated table of contents. ([\#17329](https://github.com/element-hq/synapse/issues/17329))
|
||||
- Fix stale references to the Foundation's Security Disclosure Policy. ([\#17341](https://github.com/element-hq/synapse/issues/17341))
|
||||
- Add default values for `rc_invites.per_issuer` to docs. ([\#17347](https://github.com/element-hq/synapse/issues/17347))
|
||||
- Fix an error in the docs for `search_all_users` parameter under `user_directory`. ([\#17348](https://github.com/element-hq/synapse/issues/17348))
|
||||
|
||||
### Internal Changes
|
||||
|
||||
- Remove unused `expire_access_token` option in the Synapse Docker config file. Contributed by @AaronDewes. ([\#17198](https://github.com/element-hq/synapse/issues/17198))
|
||||
- Use fully-qualified `PersistedEventPosition` when returning `RoomsForUser` to facilitate proper comparisons and `RoomStreamToken` generation. ([\#17265](https://github.com/element-hq/synapse/issues/17265))
|
||||
- Add debug logging for when room keys are uploaded, including whether they are replacing other room keys. ([\#17266](https://github.com/element-hq/synapse/issues/17266))
|
||||
- Handle OTK uploads off master. ([\#17271](https://github.com/element-hq/synapse/issues/17271))
|
||||
- Don't try and resync devices for remote users whose servers are marked as down. ([\#17273](https://github.com/element-hq/synapse/issues/17273))
|
||||
- Re-organize Pydantic models and types used in handlers. ([\#17279](https://github.com/element-hq/synapse/issues/17279))
|
||||
- Expose the worker instance that persisted the event on `event.internal_metadata.instance_name`. ([\#17300](https://github.com/element-hq/synapse/issues/17300))
|
||||
- Update the README with Element branding, improve headers and fix the #synapse:matrix.org support room link rendering. ([\#17324](https://github.com/element-hq/synapse/issues/17324))
|
||||
- Change path of the experimental [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) Sliding Sync implementation to `/org.matrix.simplified_msc3575/sync` since our simplified API is slightly incompatible with what's in the current MSC. ([\#17331](https://github.com/element-hq/synapse/issues/17331))
|
||||
- Handle device lists notifications for large accounts more efficiently in worker mode. ([\#17333](https://github.com/element-hq/synapse/issues/17333), [\#17358](https://github.com/element-hq/synapse/issues/17358))
|
||||
- Do not block event sending/receiving while calculating large event auth chains. ([\#17338](https://github.com/element-hq/synapse/issues/17338))
|
||||
- Tidy up `parse_integer` docs and call sites to reflect the fact that they require non-negative integers by default, and bring `parse_integer_from_args` default in alignment. Contributed by Denis Kasak (@dkasak). ([\#17339](https://github.com/element-hq/synapse/issues/17339))
|
||||
|
||||
|
||||
|
||||
### Updates to locked dependencies
|
||||
|
||||
* Bump authlib from 1.3.0 to 1.3.1. ([\#17343](https://github.com/element-hq/synapse/issues/17343))
|
||||
* Bump dawidd6/action-download-artifact from 3.1.4 to 5. ([\#17289](https://github.com/element-hq/synapse/issues/17289))
|
||||
* Bump dawidd6/action-download-artifact from 5 to 6. ([\#17313](https://github.com/element-hq/synapse/issues/17313))
|
||||
* Bump docker/build-push-action from 5 to 6. ([\#17312](https://github.com/element-hq/synapse/issues/17312))
|
||||
* Bump jinja2 from 3.1.3 to 3.1.4. ([\#17287](https://github.com/element-hq/synapse/issues/17287))
|
||||
* Bump lazy_static from 1.4.0 to 1.5.0. ([\#17355](https://github.com/element-hq/synapse/issues/17355))
|
||||
* Bump msgpack from 1.0.7 to 1.0.8. ([\#17317](https://github.com/element-hq/synapse/issues/17317))
|
||||
* Bump netaddr from 1.2.1 to 1.3.0. ([\#17353](https://github.com/element-hq/synapse/issues/17353))
|
||||
* Bump packaging from 24.0 to 24.1. ([\#17352](https://github.com/element-hq/synapse/issues/17352))
|
||||
* Bump phonenumbers from 8.13.37 to 8.13.39. ([\#17315](https://github.com/element-hq/synapse/issues/17315))
|
||||
* Bump regex from 1.10.4 to 1.10.5. ([\#17290](https://github.com/element-hq/synapse/issues/17290))
|
||||
* Bump requests from 2.31.0 to 2.32.2. ([\#17345](https://github.com/element-hq/synapse/issues/17345))
|
||||
* Bump sentry-sdk from 2.1.1 to 2.3.1. ([\#17263](https://github.com/element-hq/synapse/issues/17263))
|
||||
* Bump sentry-sdk from 2.3.1 to 2.6.0. ([\#17351](https://github.com/element-hq/synapse/issues/17351))
|
||||
* Bump tornado from 6.4 to 6.4.1. ([\#17344](https://github.com/element-hq/synapse/issues/17344))
|
||||
* Bump mypy from 1.8.0 to 1.9.0. ([\#17297](https://github.com/element-hq/synapse/issues/17297))
|
||||
* Bump types-jsonschema from 4.21.0.20240311 to 4.22.0.20240610. ([\#17288](https://github.com/element-hq/synapse/issues/17288))
|
||||
* Bump types-netaddr from 1.2.0.20240219 to 1.3.0.20240530. ([\#17314](https://github.com/element-hq/synapse/issues/17314))
|
||||
* Bump types-pillow from 10.2.0.20240423 to 10.2.0.20240520. ([\#17285](https://github.com/element-hq/synapse/issues/17285))
|
||||
* Bump types-pyyaml from 6.0.12.12 to 6.0.12.20240311. ([\#17316](https://github.com/element-hq/synapse/issues/17316))
|
||||
* Bump typing-extensions from 4.11.0 to 4.12.2. ([\#17354](https://github.com/element-hq/synapse/issues/17354))
|
||||
* Bump urllib3 from 2.0.7 to 2.2.2. ([\#17346](https://github.com/element-hq/synapse/issues/17346))
|
||||
|
||||
# Synapse 1.109.0 (2024-06-18)
|
||||
|
||||
### Internal Changes
|
||||
|
|
4
Cargo.lock
generated
4
Cargo.lock
generated
|
@ -212,9 +212,9 @@ dependencies = [
|
|||
|
||||
[[package]]
|
||||
name = "lazy_static"
|
||||
version = "1.4.0"
|
||||
version = "1.5.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e2abad23fbc42b3700f2f279844dc832adb2b2eb069b2df918f455c4e18cc646"
|
||||
checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe"
|
||||
|
||||
[[package]]
|
||||
name = "libc"
|
||||
|
|
71
README.rst
71
README.rst
|
@ -1,21 +1,34 @@
|
|||
=========================================================================
|
||||
Synapse |support| |development| |documentation| |license| |pypi| |python|
|
||||
=========================================================================
|
||||
.. image:: https://github.com/element-hq/product/assets/87339233/7abf477a-5277-47f3-be44-ea44917d8ed7
|
||||
:height: 60px
|
||||
|
||||
Synapse is an open-source `Matrix <https://matrix.org/>`_ homeserver written and
|
||||
maintained by the Matrix.org Foundation. We began rapid development in 2014,
|
||||
reaching v1.0.0 in 2019. Development on Synapse and the Matrix protocol itself continues
|
||||
in earnest today.
|
||||
**Element Synapse - Matrix homeserver implementation**
|
||||
|
||||
Briefly, Matrix is an open standard for communications on the internet, supporting
|
||||
federation, encryption and VoIP. Matrix.org has more to say about the `goals of the
|
||||
Matrix project <https://matrix.org/docs/guides/introduction>`_, and the `formal specification
|
||||
<https://spec.matrix.org/>`_ describes the technical details.
|
||||
|support| |development| |documentation| |license| |pypi| |python|
|
||||
|
||||
Synapse is an open source `Matrix <https://matrix.org>`__ homeserver
|
||||
implementation, written and maintained by `Element <https://element.io>`_.
|
||||
`Matrix <https://github.com/matrix-org>`__ is the open standard for
|
||||
secure and interoperable real time communications. You can directly run
|
||||
and manage the source code in this repository, available under an AGPL
|
||||
license. There is no support provided from Element unless you have a
|
||||
subscription.
|
||||
|
||||
Subscription alternative
|
||||
========================
|
||||
|
||||
Alternatively, for those that need an enterprise-ready solution, Element
|
||||
Server Suite (ESS) is `available as a subscription <https://element.io/pricing>`_.
|
||||
ESS builds on Synapse to offer a complete Matrix-based backend including the full
|
||||
`Admin Console product <https://element.io/enterprise-functionality/admin-console>`_,
|
||||
giving admins the power to easily manage an organization-wide
|
||||
deployment. It includes advanced identity management, auditing,
|
||||
moderation and data retention options as well as Long Term Support and
|
||||
SLAs. ESS can be used to support any Matrix-based frontend client.
|
||||
|
||||
.. contents::
|
||||
|
||||
Installing and configuration
|
||||
============================
|
||||
🛠️ Installing and configuration
|
||||
===============================
|
||||
|
||||
The Synapse documentation describes `how to install Synapse <https://element-hq.github.io/synapse/latest/setup/installation.html>`_. We recommend using
|
||||
`Docker images <https://element-hq.github.io/synapse/latest/setup/installation.html#docker-images-and-ansible-playbooks>`_ or `Debian packages from Matrix.org
|
||||
|
@ -105,8 +118,8 @@ Following this advice ensures that even if an XSS is found in Synapse, the
|
|||
impact to other applications will be minimal.
|
||||
|
||||
|
||||
Testing a new installation
|
||||
==========================
|
||||
🧪 Testing a new installation
|
||||
=============================
|
||||
|
||||
The easiest way to try out your new Synapse installation is by connecting to it
|
||||
from a web client.
|
||||
|
@ -159,8 +172,20 @@ the form of::
|
|||
As when logging in, you will need to specify a "Custom server". Specify your
|
||||
desired ``localpart`` in the 'User name' box.
|
||||
|
||||
Troubleshooting and support
|
||||
===========================
|
||||
🎯 Troubleshooting and support
|
||||
==============================
|
||||
|
||||
🚀 Professional support
|
||||
-----------------------
|
||||
|
||||
Enterprise quality support for Synapse including SLAs is available as part of an
|
||||
`Element Server Suite (ESS) <https://element.io/pricing>` subscription.
|
||||
|
||||
If you are an existing ESS subscriber then you can raise a `support request <https://ems.element.io/support>`
|
||||
and access the `knowledge base <https://ems-docs.element.io>`.
|
||||
|
||||
🤝 Community support
|
||||
--------------------
|
||||
|
||||
The `Admin FAQ <https://element-hq.github.io/synapse/latest/usage/administration/admin_faq.html>`_
|
||||
includes tips on dealing with some common problems. For more details, see
|
||||
|
@ -176,8 +201,8 @@ issues for support requests, only for bug reports and feature requests.
|
|||
.. |docs| replace:: ``docs``
|
||||
.. _docs: docs
|
||||
|
||||
Identity Servers
|
||||
================
|
||||
🪪 Identity Servers
|
||||
===================
|
||||
|
||||
Identity servers have the job of mapping email addresses and other 3rd Party
|
||||
IDs (3PIDs) to Matrix user IDs, as well as verifying the ownership of 3PIDs
|
||||
|
@ -206,8 +231,8 @@ an email address with your account, or send an invite to another user via their
|
|||
email address.
|
||||
|
||||
|
||||
Development
|
||||
===========
|
||||
🛠️ Development
|
||||
==============
|
||||
|
||||
We welcome contributions to Synapse from the community!
|
||||
The best place to get started is our
|
||||
|
@ -225,8 +250,8 @@ Alongside all that, join our developer community on Matrix:
|
|||
`#synapse-dev:matrix.org <https://matrix.to/#/#synapse-dev:matrix.org>`_, featuring real humans!
|
||||
|
||||
|
||||
.. |support| image:: https://img.shields.io/matrix/synapse:matrix.org?label=support&logo=matrix
|
||||
:alt: (get support on #synapse:matrix.org)
|
||||
.. |support| image:: https://img.shields.io/badge/matrix-community%20support-success
|
||||
:alt: (get community support in #synapse:matrix.org)
|
||||
:target: https://matrix.to/#/#synapse:matrix.org
|
||||
|
||||
.. |development| image:: https://img.shields.io/matrix/synapse-dev:matrix.org?label=development&logo=matrix
|
||||
|
|
|
@ -1 +0,0 @@
|
|||
Add initial implementation of an experimental [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) Sliding Sync `/sync` endpoint.
|
|
@ -1 +0,0 @@
|
|||
Fix searching for users with their exact localpart whose ID includes a hyphen.
|
|
@ -1 +0,0 @@
|
|||
Improve ratelimiting in Synapse (#17256).
|
|
@ -1 +0,0 @@
|
|||
Use fully-qualified `PersistedEventPosition` when returning `RoomsForUser` to facilitate proper comparisons and `RoomStreamToken` generation.
|
|
@ -1 +0,0 @@
|
|||
Add debug logging for when room keys are uploaded, including whether they are replacing other room keys.
|
|
@ -1 +0,0 @@
|
|||
Add support for the unstable [MSC4151](https://github.com/matrix-org/matrix-spec-proposals/pull/4151) report room API.
|
|
@ -1 +0,0 @@
|
|||
Handle OTK uploads off master.
|
|
@ -1 +0,0 @@
|
|||
Fix wrong retention policy being used when filtering events.
|
|
@ -1 +0,0 @@
|
|||
Don't try and resync devices for remote users whose servers are marked as down.
|
|
@ -1 +0,0 @@
|
|||
Fix bug where OTKs were not always included in `/sync` response when using workers.
|
|
@ -1 +0,0 @@
|
|||
Add `is_dm` filtering to experimental [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) Sliding Sync `/sync` endpoint.
|
|
@ -1 +0,0 @@
|
|||
Re-organize Pydantic models and types used in handlers.
|
|
@ -1 +0,0 @@
|
|||
Add `is_encrypted` filtering to experimental [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) Sliding Sync `/sync` endpoint.
|
|
@ -1 +0,0 @@
|
|||
Include user membership in events served to clients, per MSC4115.
|
|
@ -1 +0,0 @@
|
|||
Do not require user-interactive authentication for uploading cross-signing keys for the first time, per MSC3967.
|
|
@ -1 +0,0 @@
|
|||
Add `stream_ordering` sort to experimental [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) Sliding Sync `/sync` endpoint.
|
|
@ -1,2 +0,0 @@
|
|||
`register_new_matrix_user` now supports a --password-file flag, which
|
||||
is useful for scripting.
|
|
@ -1 +0,0 @@
|
|||
Fix edge case in `/sync` returning the wrong the state when using sharded event persisters.
|
|
@ -1 +0,0 @@
|
|||
Add support for the unstable [MSC4151](https://github.com/matrix-org/matrix-spec-proposals/pull/4151) report room API.
|
|
@ -1 +0,0 @@
|
|||
Bump `mypy` from 1.8.0 to 1.9.0.
|
|
@ -1 +0,0 @@
|
|||
Expose the worker instance that persisted the event on `event.internal_metadata.instance_name`.
|
|
@ -1 +0,0 @@
|
|||
Add initial implementation of an experimental [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) Sliding Sync `/sync` endpoint.
|
|
@ -1 +0,0 @@
|
|||
Add missing quotes for example for `exclude_rooms_from_sync`.
|
|
@ -1 +0,0 @@
|
|||
Add support for via query parameter from MSC415.
|
|
@ -1 +0,0 @@
|
|||
This is a changelog so tests will run.
|
2
changelog.d/17326.feature
Normal file
2
changelog.d/17326.feature
Normal file
|
@ -0,0 +1,2 @@
|
|||
Add initial implementation of delayed events (Futures) as proposed by [MSC4140](https://github.com/matrix-org/matrix-spec-proposals/pull/4140).
|
||||
|
1
changelog.d/17362.bugfix
Normal file
1
changelog.d/17362.bugfix
Normal file
|
@ -0,0 +1 @@
|
|||
Fix rare race which causes no new to-device messages to be received from remote server.
|
1
changelog.d/17363.misc
Normal file
1
changelog.d/17363.misc
Normal file
|
@ -0,0 +1 @@
|
|||
Fix uploading packages to PyPi.
|
1
changelog.d/17367.misc
Normal file
1
changelog.d/17367.misc
Normal file
|
@ -0,0 +1 @@
|
|||
Add CI check for the README.
|
13
debian/changelog
vendored
13
debian/changelog
vendored
|
@ -1,8 +1,15 @@
|
|||
matrix-synapse-py3 (1.109.0+nmu1) UNRELEASED; urgency=medium
|
||||
matrix-synapse-py3 (1.110.0~rc2) stable; urgency=medium
|
||||
|
||||
* `register_new_matrix_user` now supports a --password-file flag.
|
||||
* New Synapse release 1.110.0rc2.
|
||||
|
||||
-- Synapse Packaging team <packages@matrix.org> Tue, 18 Jun 2024 13:29:36 +0100
|
||||
-- Synapse Packaging team <packages@matrix.org> Wed, 26 Jun 2024 18:14:48 +0200
|
||||
|
||||
matrix-synapse-py3 (1.110.0~rc1) stable; urgency=medium
|
||||
|
||||
* `register_new_matrix_user` now supports a --password-file and a --exists-ok flag.
|
||||
* New Synapse release 1.110.0rc1.
|
||||
|
||||
-- Synapse Packaging team <packages@matrix.org> Wed, 26 Jun 2024 14:07:56 +0200
|
||||
|
||||
matrix-synapse-py3 (1.109.0) stable; urgency=medium
|
||||
|
||||
|
|
3
debian/register_new_matrix_user.ronn
vendored
3
debian/register_new_matrix_user.ronn
vendored
|
@ -48,6 +48,9 @@ A sample YAML file accepted by `register_new_matrix_user` is described below:
|
|||
Shared secret as defined in server config file. This is an optional
|
||||
parameter as it can be also supplied via the YAML file.
|
||||
|
||||
* `--exists-ok`:
|
||||
Do not fail if the user already exists. The user account will be not updated in this case.
|
||||
|
||||
* `server_url`:
|
||||
URL of the home server. Defaults to 'https://localhost:8448'.
|
||||
|
||||
|
|
|
@ -176,7 +176,6 @@ app_service_config_files:
|
|||
{% endif %}
|
||||
|
||||
macaroon_secret_key: "{{ SYNAPSE_MACAROON_SECRET_KEY }}"
|
||||
expire_access_token: False
|
||||
|
||||
## Signing Keys ##
|
||||
|
||||
|
|
|
@ -36,6 +36,10 @@ The following query parameters are available:
|
|||
- the room's name,
|
||||
- the local part of the room's canonical alias, or
|
||||
- the complete (local and server part) room's id (case sensitive).
|
||||
* `public_rooms` - Optional flag to filter public rooms. If `true`, only public rooms are queried. If `false`, public rooms are excluded from
|
||||
the query. When the flag is absent (the default), **both** public and non-public rooms are included in the search results.
|
||||
* `empty_rooms` - Optional flag to filter empty rooms. A room is empty if joined_members is zero. If `true`, only empty rooms are queried. If `false`, empty rooms are excluded from
|
||||
the query. When the flag is absent (the default), **both** empty and non-empty rooms are included in the search results.
|
||||
|
||||
Defaults to no filtering.
|
||||
|
||||
|
|
|
@ -1662,6 +1662,19 @@ rc_registration_token_validity:
|
|||
burst_count: 6
|
||||
```
|
||||
---
|
||||
### `rc_future_token_validity`
|
||||
|
||||
This option checks the validity of future tokens that ratelimits requests based on
|
||||
the client's IP address.
|
||||
Defaults to `per_second: 0.1`, `burst_count: 5`.
|
||||
|
||||
Example configuration:
|
||||
```yaml
|
||||
rc_future_token_validity:
|
||||
per_second: 0.3
|
||||
burst_count: 6
|
||||
```
|
||||
---
|
||||
### `rc_login`
|
||||
|
||||
This option specifies several limits for login:
|
||||
|
@ -1759,8 +1772,9 @@ rc_3pid_validation:
|
|||
### `rc_invites`
|
||||
|
||||
This option sets ratelimiting how often invites can be sent in a room or to a
|
||||
specific user. `per_room` defaults to `per_second: 0.3`, `burst_count: 10` and
|
||||
`per_user` defaults to `per_second: 0.003`, `burst_count: 5`.
|
||||
specific user. `per_room` defaults to `per_second: 0.3`, `burst_count: 10`,
|
||||
`per_user` defaults to `per_second: 0.003`, `burst_count: 5`, and `per_issuer`
|
||||
defaults to `per_second: 0.3`, `burst_count: 10`.
|
||||
|
||||
Client requests that invite user(s) when [creating a
|
||||
room](https://spec.matrix.org/v1.2/client-server-api/#post_matrixclientv3createroom)
|
||||
|
@ -2718,7 +2732,7 @@ Example configuration:
|
|||
session_lifetime: 24h
|
||||
```
|
||||
---
|
||||
### `refresh_access_token_lifetime`
|
||||
### `refreshable_access_token_lifetime`
|
||||
|
||||
Time that an access token remains valid for, if the session is using refresh tokens.
|
||||
|
||||
|
@ -3806,7 +3820,8 @@ This setting defines options related to the user directory.
|
|||
This option has the following sub-options:
|
||||
* `enabled`: Defines whether users can search the user directory. If false then
|
||||
empty responses are returned to all queries. Defaults to true.
|
||||
* `search_all_users`: Defines whether to search all users visible to your HS at the time the search is performed. If set to true, will return all users who share a room with the user from the homeserver.
|
||||
* `search_all_users`: Defines whether to search all users visible to your homeserver at the time the search is performed.
|
||||
If set to true, will return all users known to the homeserver matching the search query.
|
||||
If false, search results will only contain users
|
||||
visible in public rooms and users sharing a room with the requester.
|
||||
Defaults to false.
|
||||
|
|
|
@ -62,6 +62,6 @@ following documentation:
|
|||
|
||||
## Reporting a security vulnerability
|
||||
|
||||
If you've found a security issue in Synapse or any other Matrix.org Foundation
|
||||
project, please report it to us in accordance with our [Security Disclosure
|
||||
Policy](https://www.matrix.org/security-disclosure-policy/). Thank you!
|
||||
If you've found a security issue in Synapse or any other Element project,
|
||||
please report it to us in accordance with our [Security Disclosure
|
||||
Policy](https://element.io/security/security-disclosure-policy). Thank you!
|
||||
|
|
76
poetry.lock
generated
76
poetry.lock
generated
|
@ -35,13 +35,13 @@ tests-no-zope = ["attrs[tests-mypy]", "cloudpickle", "hypothesis", "pympler", "p
|
|||
|
||||
[[package]]
|
||||
name = "authlib"
|
||||
version = "1.3.0"
|
||||
version = "1.3.1"
|
||||
description = "The ultimate Python library in building OAuth and OpenID Connect servers and clients."
|
||||
optional = true
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "Authlib-1.3.0-py2.py3-none-any.whl", hash = "sha256:9637e4de1fb498310a56900b3e2043a206b03cb11c05422014b0302cbc814be3"},
|
||||
{file = "Authlib-1.3.0.tar.gz", hash = "sha256:959ea62a5b7b5123c5059758296122b57cd2585ae2ed1c0622c21b371ffdae06"},
|
||||
{file = "Authlib-1.3.1-py2.py3-none-any.whl", hash = "sha256:d35800b973099bbadc49b42b256ecb80041ad56b7fe1216a362c7943c088f377"},
|
||||
{file = "authlib-1.3.1.tar.gz", hash = "sha256:7ae843f03c06c5c0debd63c9db91f9fda64fa62a42a77419fa15fbb7e7a58917"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
|
@ -1461,13 +1461,13 @@ test = ["lxml", "pytest (>=4.6)", "pytest-cov"]
|
|||
|
||||
[[package]]
|
||||
name = "netaddr"
|
||||
version = "1.2.1"
|
||||
version = "1.3.0"
|
||||
description = "A network address manipulation library for Python"
|
||||
optional = false
|
||||
python-versions = ">=3.7"
|
||||
files = [
|
||||
{file = "netaddr-1.2.1-py3-none-any.whl", hash = "sha256:bd9e9534b0d46af328cf64f0e5a23a5a43fca292df221c85580b27394793496e"},
|
||||
{file = "netaddr-1.2.1.tar.gz", hash = "sha256:6eb8fedf0412c6d294d06885c110de945cf4d22d2b510d0404f4e06950857987"},
|
||||
{file = "netaddr-1.3.0-py3-none-any.whl", hash = "sha256:c2c6a8ebe5554ce33b7d5b3a306b71bbb373e000bbbf2350dd5213cc56e3dbbe"},
|
||||
{file = "netaddr-1.3.0.tar.gz", hash = "sha256:5c3c3d9895b551b763779ba7db7a03487dc1f8e3b385af819af341ae9ef6e48a"},
|
||||
]
|
||||
|
||||
[package.extras]
|
||||
|
@ -1488,13 +1488,13 @@ tests = ["Sphinx", "doubles", "flake8", "flake8-quotes", "gevent", "mock", "pyte
|
|||
|
||||
[[package]]
|
||||
name = "packaging"
|
||||
version = "24.0"
|
||||
version = "24.1"
|
||||
description = "Core utilities for Python packages"
|
||||
optional = false
|
||||
python-versions = ">=3.7"
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "packaging-24.0-py3-none-any.whl", hash = "sha256:2ddfb553fdf02fb784c234c7ba6ccc288296ceabec964ad2eae3777778130bc5"},
|
||||
{file = "packaging-24.0.tar.gz", hash = "sha256:eb82c5e3e56209074766e6885bb04b8c38a0c015d0a30036ebe7ece34c9989e9"},
|
||||
{file = "packaging-24.1-py3-none-any.whl", hash = "sha256:5b8f2217dbdbd2f7f384c41c628544e6d52f2d0f53c6d0c3ea61aa5d1d7ff124"},
|
||||
{file = "packaging-24.1.tar.gz", hash = "sha256:026ed72c8ed3fcce5bf8950572258698927fd1dbda10a5e981cdf0ac37f4f002"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
@ -2157,13 +2157,13 @@ rpds-py = ">=0.7.0"
|
|||
|
||||
[[package]]
|
||||
name = "requests"
|
||||
version = "2.31.0"
|
||||
version = "2.32.2"
|
||||
description = "Python HTTP for Humans."
|
||||
optional = false
|
||||
python-versions = ">=3.7"
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "requests-2.31.0-py3-none-any.whl", hash = "sha256:58cd2187c01e70e6e26505bca751777aa9f2ee0b7f4300988b709f44e013003f"},
|
||||
{file = "requests-2.31.0.tar.gz", hash = "sha256:942c5a758f98d790eaed1a29cb6eefc7ffb0d1cf7af05c3d2791656dbd6ad1e1"},
|
||||
{file = "requests-2.32.2-py3-none-any.whl", hash = "sha256:fc06670dd0ed212426dfeb94fc1b983d917c4f9847c863f313c9dfaaffb7c23c"},
|
||||
{file = "requests-2.32.2.tar.gz", hash = "sha256:dd951ff5ecf3e3b3aa26b40703ba77495dab41da839ae72ef3c8e5d8e2433289"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
|
@ -2387,13 +2387,13 @@ doc = ["Sphinx", "sphinx-rtd-theme"]
|
|||
|
||||
[[package]]
|
||||
name = "sentry-sdk"
|
||||
version = "2.3.1"
|
||||
version = "2.6.0"
|
||||
description = "Python client for Sentry (https://sentry.io)"
|
||||
optional = true
|
||||
python-versions = ">=3.6"
|
||||
files = [
|
||||
{file = "sentry_sdk-2.3.1-py2.py3-none-any.whl", hash = "sha256:c5aeb095ba226391d337dd42a6f9470d86c9fc236ecc71cfc7cd1942b45010c6"},
|
||||
{file = "sentry_sdk-2.3.1.tar.gz", hash = "sha256:139a71a19f5e9eb5d3623942491ce03cf8ebc14ea2e39ba3e6fe79560d8a5b1f"},
|
||||
{file = "sentry_sdk-2.6.0-py2.py3-none-any.whl", hash = "sha256:422b91cb49378b97e7e8d0e8d5a1069df23689d45262b86f54988a7db264e874"},
|
||||
{file = "sentry_sdk-2.6.0.tar.gz", hash = "sha256:65cc07e9c6995c5e316109f138570b32da3bd7ff8d0d0ee4aaf2628c3dd8127d"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
|
@ -2598,22 +2598,22 @@ files = [
|
|||
|
||||
[[package]]
|
||||
name = "tornado"
|
||||
version = "6.4"
|
||||
version = "6.4.1"
|
||||
description = "Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed."
|
||||
optional = true
|
||||
python-versions = ">= 3.8"
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "tornado-6.4-cp38-abi3-macosx_10_9_universal2.whl", hash = "sha256:02ccefc7d8211e5a7f9e8bc3f9e5b0ad6262ba2fbb683a6443ecc804e5224ce0"},
|
||||
{file = "tornado-6.4-cp38-abi3-macosx_10_9_x86_64.whl", hash = "sha256:27787de946a9cffd63ce5814c33f734c627a87072ec7eed71f7fc4417bb16263"},
|
||||
{file = "tornado-6.4-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f7894c581ecdcf91666a0912f18ce5e757213999e183ebfc2c3fdbf4d5bd764e"},
|
||||
{file = "tornado-6.4-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e43bc2e5370a6a8e413e1e1cd0c91bedc5bd62a74a532371042a18ef19e10579"},
|
||||
{file = "tornado-6.4-cp38-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f0251554cdd50b4b44362f73ad5ba7126fc5b2c2895cc62b14a1c2d7ea32f212"},
|
||||
{file = "tornado-6.4-cp38-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:fd03192e287fbd0899dd8f81c6fb9cbbc69194d2074b38f384cb6fa72b80e9c2"},
|
||||
{file = "tornado-6.4-cp38-abi3-musllinux_1_1_i686.whl", hash = "sha256:88b84956273fbd73420e6d4b8d5ccbe913c65d31351b4c004ae362eba06e1f78"},
|
||||
{file = "tornado-6.4-cp38-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:71ddfc23a0e03ef2df1c1397d859868d158c8276a0603b96cf86892bff58149f"},
|
||||
{file = "tornado-6.4-cp38-abi3-win32.whl", hash = "sha256:6f8a6c77900f5ae93d8b4ae1196472d0ccc2775cc1dfdc9e7727889145c45052"},
|
||||
{file = "tornado-6.4-cp38-abi3-win_amd64.whl", hash = "sha256:10aeaa8006333433da48dec9fe417877f8bcc21f48dda8d661ae79da357b2a63"},
|
||||
{file = "tornado-6.4.tar.gz", hash = "sha256:72291fa6e6bc84e626589f1c29d90a5a6d593ef5ae68052ee2ef000dfd273dee"},
|
||||
{file = "tornado-6.4.1-cp38-abi3-macosx_10_9_universal2.whl", hash = "sha256:163b0aafc8e23d8cdc3c9dfb24c5368af84a81e3364745ccb4427669bf84aec8"},
|
||||
{file = "tornado-6.4.1-cp38-abi3-macosx_10_9_x86_64.whl", hash = "sha256:6d5ce3437e18a2b66fbadb183c1d3364fb03f2be71299e7d10dbeeb69f4b2a14"},
|
||||
{file = "tornado-6.4.1-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e2e20b9113cd7293f164dc46fffb13535266e713cdb87bd2d15ddb336e96cfc4"},
|
||||
{file = "tornado-6.4.1-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:8ae50a504a740365267b2a8d1a90c9fbc86b780a39170feca9bcc1787ff80842"},
|
||||
{file = "tornado-6.4.1-cp38-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:613bf4ddf5c7a95509218b149b555621497a6cc0d46ac341b30bd9ec19eac7f3"},
|
||||
{file = "tornado-6.4.1-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:25486eb223babe3eed4b8aecbac33b37e3dd6d776bc730ca14e1bf93888b979f"},
|
||||
{file = "tornado-6.4.1-cp38-abi3-musllinux_1_2_i686.whl", hash = "sha256:454db8a7ecfcf2ff6042dde58404164d969b6f5d58b926da15e6b23817950fc4"},
|
||||
{file = "tornado-6.4.1-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:a02a08cc7a9314b006f653ce40483b9b3c12cda222d6a46d4ac63bb6c9057698"},
|
||||
{file = "tornado-6.4.1-cp38-abi3-win32.whl", hash = "sha256:d9a566c40b89757c9aa8e6f032bcdb8ca8795d7c1a9762910c722b1635c9de4d"},
|
||||
{file = "tornado-6.4.1-cp38-abi3-win_amd64.whl", hash = "sha256:b24b8982ed444378d7f21d563f4180a2de31ced9d8d84443907a0a64da2072e7"},
|
||||
{file = "tornado-6.4.1.tar.gz", hash = "sha256:92d3ab53183d8c50f8204a51e6f91d18a15d5ef261e84d452800d4ff6fc504e9"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
@ -2917,13 +2917,13 @@ files = [
|
|||
|
||||
[[package]]
|
||||
name = "typing-extensions"
|
||||
version = "4.11.0"
|
||||
version = "4.12.2"
|
||||
description = "Backported and Experimental Type Hints for Python 3.8+"
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "typing_extensions-4.11.0-py3-none-any.whl", hash = "sha256:c1f94d72897edaf4ce775bb7558d5b79d8126906a14ea5ed1635921406c0387a"},
|
||||
{file = "typing_extensions-4.11.0.tar.gz", hash = "sha256:83f085bd5ca59c80295fc2a82ab5dac679cbe02b9f33f7d83af68e241bea51b0"},
|
||||
{file = "typing_extensions-4.12.2-py3-none-any.whl", hash = "sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d"},
|
||||
{file = "typing_extensions-4.12.2.tar.gz", hash = "sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
@ -2939,18 +2939,18 @@ files = [
|
|||
|
||||
[[package]]
|
||||
name = "urllib3"
|
||||
version = "2.0.7"
|
||||
version = "2.2.2"
|
||||
description = "HTTP library with thread-safe connection pooling, file post, and more."
|
||||
optional = false
|
||||
python-versions = ">=3.7"
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "urllib3-2.0.7-py3-none-any.whl", hash = "sha256:fdb6d215c776278489906c2f8916e6e7d4f5a9b602ccbcfdf7f016fc8da0596e"},
|
||||
{file = "urllib3-2.0.7.tar.gz", hash = "sha256:c97dfde1f7bd43a71c8d2a58e369e9b2bf692d1334ea9f9cae55add7d0dd0f84"},
|
||||
{file = "urllib3-2.2.2-py3-none-any.whl", hash = "sha256:a448b2f64d686155468037e1ace9f2d2199776e17f0a46610480d311f73e3472"},
|
||||
{file = "urllib3-2.2.2.tar.gz", hash = "sha256:dd505485549a7a552833da5e6063639d0d177c04f23bc3864e41e5dc5f612168"},
|
||||
]
|
||||
|
||||
[package.extras]
|
||||
brotli = ["brotli (>=1.0.9)", "brotlicffi (>=0.8.0)"]
|
||||
secure = ["certifi", "cryptography (>=1.9)", "idna (>=2.0.0)", "pyopenssl (>=17.1.0)", "urllib3-secure-extra"]
|
||||
h2 = ["h2 (>=4,<5)"]
|
||||
socks = ["pysocks (>=1.5.6,!=1.5.7,<2.0)"]
|
||||
zstd = ["zstandard (>=0.18.0)"]
|
||||
|
||||
|
|
|
@ -96,7 +96,7 @@ module-name = "synapse.synapse_rust"
|
|||
|
||||
[tool.poetry]
|
||||
name = "matrix-synapse"
|
||||
version = "1.109.0"
|
||||
version = "1.110.0rc2"
|
||||
description = "Homeserver for the Matrix decentralised comms protocol"
|
||||
authors = ["Matrix.org Team and Contributors <packages@matrix.org>"]
|
||||
license = "AGPL-3.0-or-later"
|
||||
|
|
|
@ -52,6 +52,7 @@ def request_registration(
|
|||
user_type: Optional[str] = None,
|
||||
_print: Callable[[str], None] = print,
|
||||
exit: Callable[[int], None] = sys.exit,
|
||||
exists_ok: bool = False,
|
||||
) -> None:
|
||||
url = "%s/_synapse/admin/v1/register" % (server_location.rstrip("/"),)
|
||||
|
||||
|
@ -97,6 +98,10 @@ def request_registration(
|
|||
r = requests.post(url, json=data)
|
||||
|
||||
if r.status_code != 200:
|
||||
response = r.json()
|
||||
if exists_ok and response["errcode"] == "M_USER_IN_USE":
|
||||
_print("User already exists. Skipping.")
|
||||
return
|
||||
_print("ERROR! Received %d %s" % (r.status_code, r.reason))
|
||||
if 400 <= r.status_code < 500:
|
||||
try:
|
||||
|
@ -115,6 +120,7 @@ def register_new_user(
|
|||
shared_secret: str,
|
||||
admin: Optional[bool],
|
||||
user_type: Optional[str],
|
||||
exists_ok: bool = False,
|
||||
) -> None:
|
||||
if not user:
|
||||
try:
|
||||
|
@ -154,7 +160,13 @@ def register_new_user(
|
|||
admin = False
|
||||
|
||||
request_registration(
|
||||
user, password, server_location, shared_secret, bool(admin), user_type
|
||||
user,
|
||||
password,
|
||||
server_location,
|
||||
shared_secret,
|
||||
bool(admin),
|
||||
user_type,
|
||||
exists_ok=exists_ok,
|
||||
)
|
||||
|
||||
|
||||
|
@ -173,6 +185,11 @@ def main() -> None:
|
|||
default=None,
|
||||
help="Local part of the new user. Will prompt if omitted.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--exists-ok",
|
||||
action="store_true",
|
||||
help="Do not fail if user already exists.",
|
||||
)
|
||||
password_group = parser.add_mutually_exclusive_group()
|
||||
password_group.add_argument(
|
||||
"-p",
|
||||
|
@ -192,6 +209,7 @@ def main() -> None:
|
|||
default=None,
|
||||
help="User type as specified in synapse.api.constants.UserTypes",
|
||||
)
|
||||
|
||||
admin_group = parser.add_mutually_exclusive_group()
|
||||
admin_group.add_argument(
|
||||
"-a",
|
||||
|
@ -281,7 +299,15 @@ def main() -> None:
|
|||
if args.admin or args.no_admin:
|
||||
admin = args.admin
|
||||
|
||||
register_new_user(args.user, password, server_url, secret, admin, args.user_type)
|
||||
register_new_user(
|
||||
args.user,
|
||||
password,
|
||||
server_url,
|
||||
secret,
|
||||
admin,
|
||||
args.user_type,
|
||||
exists_ok=args.exists_ok,
|
||||
)
|
||||
|
||||
|
||||
def _read_file(file_path: Any, config_path: str) -> str:
|
||||
|
|
|
@ -433,10 +433,32 @@ class ExperimentalConfig(Config):
|
|||
("experimental", "msc4108_delegation_endpoint"),
|
||||
)
|
||||
|
||||
self.msc3823_account_suspension = experimental.get(
|
||||
"msc3823_account_suspension", False
|
||||
)
|
||||
|
||||
self.msc3916_authenticated_media_enabled = experimental.get(
|
||||
"msc3916_authenticated_media_enabled", False
|
||||
)
|
||||
|
||||
# MSC4140: Delayed events (Futures)
|
||||
# The maximum allowed delay for timeout futures.
|
||||
try:
|
||||
self.msc4140_max_future_timeout_duration = int(
|
||||
experimental["msc4140_max_future_timeout_duration"]
|
||||
)
|
||||
if self.msc4140_max_future_timeout_duration < 0:
|
||||
raise ValueError
|
||||
except ValueError:
|
||||
raise ConfigError(
|
||||
"Timeout duration must be a positive integer",
|
||||
("experimental", "msc4140_max_future_timeout_duration"),
|
||||
)
|
||||
except KeyError:
|
||||
self.msc4140_max_future_timeout_duration = (
|
||||
10 * 365 * 24 * 60 * 60 * 1000
|
||||
) # 10 years
|
||||
|
||||
# MSC4151: Report room API (Client-Server API)
|
||||
self.msc4151_enabled: bool = experimental.get("msc4151_enabled", False)
|
||||
|
||||
|
|
|
@ -124,6 +124,13 @@ class RatelimitConfig(Config):
|
|||
defaults={"per_second": 0.1, "burst_count": 5},
|
||||
)
|
||||
|
||||
# Ratelimit requests to send/cancel/refresh futures (MSC4140):
|
||||
self.rc_future_token_validity = RatelimitSettings.parse(
|
||||
config,
|
||||
"rc_future_token_validity",
|
||||
defaults={"per_second": 0.1, "burst_count": 5},
|
||||
)
|
||||
|
||||
# It is reasonable to login with a bunch of devices at once (i.e. when
|
||||
# setting up an account), but it is *not* valid to continually be
|
||||
# logging into new devices.
|
||||
|
|
|
@ -33,6 +33,7 @@ from synapse.federation.transport.server.federation import (
|
|||
FEDERATION_SERVLET_CLASSES,
|
||||
FederationAccountStatusServlet,
|
||||
FederationUnstableClientKeysClaimServlet,
|
||||
FederationUnstableMediaDownloadServlet,
|
||||
)
|
||||
from synapse.http.server import HttpServer, JsonResource
|
||||
from synapse.http.servlet import (
|
||||
|
@ -315,6 +316,13 @@ def register_servlets(
|
|||
):
|
||||
continue
|
||||
|
||||
if servletclass == FederationUnstableMediaDownloadServlet:
|
||||
if (
|
||||
not hs.config.server.enable_media_repo
|
||||
or not hs.config.experimental.msc3916_authenticated_media_enabled
|
||||
):
|
||||
continue
|
||||
|
||||
servletclass(
|
||||
hs=hs,
|
||||
authenticator=authenticator,
|
||||
|
|
|
@ -360,13 +360,29 @@ class BaseFederationServlet:
|
|||
"request"
|
||||
)
|
||||
return None
|
||||
if (
|
||||
func.__self__.__class__.__name__ # type: ignore
|
||||
== "FederationUnstableMediaDownloadServlet"
|
||||
):
|
||||
response = await func(
|
||||
origin, content, request, *args, **kwargs
|
||||
)
|
||||
else:
|
||||
response = await func(
|
||||
origin, content, request.args, *args, **kwargs
|
||||
)
|
||||
else:
|
||||
if (
|
||||
func.__self__.__class__.__name__ # type: ignore
|
||||
== "FederationUnstableMediaDownloadServlet"
|
||||
):
|
||||
response = await func(
|
||||
origin, content, request, *args, **kwargs
|
||||
)
|
||||
else:
|
||||
response = await func(
|
||||
origin, content, request.args, *args, **kwargs
|
||||
)
|
||||
else:
|
||||
response = await func(
|
||||
origin, content, request.args, *args, **kwargs
|
||||
)
|
||||
finally:
|
||||
# if we used the origin's context as the parent, add a new span using
|
||||
# the servlet span as a parent, so that we have a link
|
||||
|
|
|
@ -44,10 +44,13 @@ from synapse.federation.transport.server._base import (
|
|||
)
|
||||
from synapse.http.servlet import (
|
||||
parse_boolean_from_args,
|
||||
parse_integer,
|
||||
parse_integer_from_args,
|
||||
parse_string_from_args,
|
||||
parse_strings_from_args,
|
||||
)
|
||||
from synapse.http.site import SynapseRequest
|
||||
from synapse.media._base import DEFAULT_MAX_TIMEOUT_MS, MAXIMUM_ALLOWED_MAX_TIMEOUT_MS
|
||||
from synapse.types import JsonDict
|
||||
from synapse.util import SYNAPSE_VERSION
|
||||
from synapse.util.ratelimitutils import FederationRateLimiter
|
||||
|
@ -787,6 +790,43 @@ class FederationAccountStatusServlet(BaseFederationServerServlet):
|
|||
return 200, {"account_statuses": statuses, "failures": failures}
|
||||
|
||||
|
||||
class FederationUnstableMediaDownloadServlet(BaseFederationServerServlet):
|
||||
"""
|
||||
Implementation of new federation media `/download` endpoint outlined in MSC3916. Returns
|
||||
a multipart/mixed response consisting of a JSON object and the requested media
|
||||
item. This endpoint only returns local media.
|
||||
"""
|
||||
|
||||
PATH = "/media/download/(?P<media_id>[^/]*)"
|
||||
PREFIX = FEDERATION_UNSTABLE_PREFIX + "/org.matrix.msc3916"
|
||||
RATELIMIT = True
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
hs: "HomeServer",
|
||||
ratelimiter: FederationRateLimiter,
|
||||
authenticator: Authenticator,
|
||||
server_name: str,
|
||||
):
|
||||
super().__init__(hs, authenticator, ratelimiter, server_name)
|
||||
self.media_repo = self.hs.get_media_repository()
|
||||
|
||||
async def on_GET(
|
||||
self,
|
||||
origin: Optional[str],
|
||||
content: Literal[None],
|
||||
request: SynapseRequest,
|
||||
media_id: str,
|
||||
) -> None:
|
||||
max_timeout_ms = parse_integer(
|
||||
request, "timeout_ms", default=DEFAULT_MAX_TIMEOUT_MS
|
||||
)
|
||||
max_timeout_ms = min(max_timeout_ms, MAXIMUM_ALLOWED_MAX_TIMEOUT_MS)
|
||||
await self.media_repo.get_local_media(
|
||||
request, media_id, None, max_timeout_ms, federation=True
|
||||
)
|
||||
|
||||
|
||||
FEDERATION_SERVLET_CLASSES: Tuple[Type[BaseFederationServlet], ...] = (
|
||||
FederationSendServlet,
|
||||
FederationEventServlet,
|
||||
|
@ -818,4 +858,5 @@ FEDERATION_SERVLET_CLASSES: Tuple[Type[BaseFederationServlet], ...] = (
|
|||
FederationV1SendKnockServlet,
|
||||
FederationMakeKnockServlet,
|
||||
FederationAccountStatusServlet,
|
||||
FederationUnstableMediaDownloadServlet,
|
||||
)
|
||||
|
|
329
synapse/handlers/futures.py
Normal file
329
synapse/handlers/futures.py
Normal file
|
@ -0,0 +1,329 @@
|
|||
#
|
||||
# This file is licensed under the Affero General Public License (AGPL) version 3.
|
||||
#
|
||||
# Copyright (C) 2024 New Vector, Ltd
|
||||
#
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Affero General Public License as
|
||||
# published by the Free Software Foundation, either version 3 of the
|
||||
# License, or (at your option) any later version.
|
||||
#
|
||||
# See the GNU Affero General Public License for more details:
|
||||
# <https://www.gnu.org/licenses/agpl-3.0.html>.
|
||||
#
|
||||
# Originally licensed under the Apache License, Version 2.0:
|
||||
# <http://www.apache.org/licenses/LICENSE-2.0>.
|
||||
#
|
||||
#
|
||||
|
||||
import logging
|
||||
from http import HTTPStatus
|
||||
from typing import TYPE_CHECKING, Dict, List, Optional
|
||||
|
||||
from twisted.internet.interfaces import IDelayedCall
|
||||
|
||||
from synapse.api.constants import EventTypes
|
||||
from synapse.api.errors import Codes, NotFoundError, ShadowBanError, SynapseError
|
||||
from synapse.logging.opentracing import set_tag
|
||||
from synapse.metrics.background_process_metrics import run_as_background_process
|
||||
from synapse.storage.databases.main.futures import (
|
||||
EventType,
|
||||
FutureID,
|
||||
FutureTokenType,
|
||||
StateKey,
|
||||
Timeout,
|
||||
Timestamp,
|
||||
)
|
||||
from synapse.types import JsonDict, Requester, RoomID, UserID, create_requester
|
||||
from synapse.util.stringutils import random_string
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from synapse.server import HomeServer
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class FuturesHandler:
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.store = hs.get_datastores().main
|
||||
self.config = hs.config
|
||||
self.clock = hs.get_clock()
|
||||
self.request_ratelimiter = hs.get_request_ratelimiter()
|
||||
self.event_creation_handler = hs.get_event_creation_handler()
|
||||
self.room_member_handler = hs.get_room_member_handler()
|
||||
|
||||
self._hostname = hs.hostname
|
||||
|
||||
self._futures: Dict[FutureID, IDelayedCall] = {}
|
||||
|
||||
async def _schedule_db_futures() -> None:
|
||||
all_future_timestamps = await self.store.get_all_future_timestamps()
|
||||
|
||||
# Get the time after awaiting to increase accuracy.
|
||||
# For even more accuracy, could get the time on each loop iteration.
|
||||
current_ts = self._get_current_ts()
|
||||
|
||||
for future_id, ts in all_future_timestamps:
|
||||
timeout = _get_timeout_between(current_ts, ts)
|
||||
if timeout > 0:
|
||||
self._schedule_future(future_id, timeout)
|
||||
else:
|
||||
logger.info("Scheduling timeout for future_id %s now", future_id)
|
||||
run_as_background_process(
|
||||
"_send_future",
|
||||
self._send_future,
|
||||
future_id,
|
||||
)
|
||||
|
||||
self._initialized_from_db = run_as_background_process(
|
||||
"_schedule_db_futures", _schedule_db_futures
|
||||
)
|
||||
|
||||
async def add_future(
|
||||
self,
|
||||
requester: Requester,
|
||||
*,
|
||||
room_id: str,
|
||||
event_type: str,
|
||||
state_key: Optional[str],
|
||||
origin_server_ts: Optional[int],
|
||||
content: JsonDict,
|
||||
timeout: Optional[int],
|
||||
group_id: Optional[str],
|
||||
is_custom_endpoint: bool = False, # TODO: Remove this eventually
|
||||
) -> JsonDict:
|
||||
"""Creates a new future, and if it is a timeout future, schedules it to be sent.
|
||||
|
||||
Params:
|
||||
requester: The initial requester of the future.
|
||||
room_id: The room where the future event should be sent.
|
||||
event_type: The event type of the future event.
|
||||
state_key: The state key of the future event, or None if it is not a state event.
|
||||
origin_server_ts: The custom timestamp to send the future event with.
|
||||
If None, the timestamp will be the actual time when the future is sent.
|
||||
content: The content of the future event.
|
||||
timeout: How long (in milliseconds) to wait before automatically sending the future.
|
||||
If None, the future will never be automatically sent.
|
||||
group_id: The future group this future belongs to.
|
||||
If None, the future will be added to a new group with an auto-generated ID.
|
||||
|
||||
Returns:
|
||||
A mapping of tokens that can be used in requests to send, cancel, or refresh the future.
|
||||
"""
|
||||
if timeout is not None:
|
||||
max_timeout = self.config.experimental.msc4140_max_future_timeout_duration
|
||||
if timeout > max_timeout:
|
||||
raise SynapseError(
|
||||
HTTPStatus.BAD_REQUEST,
|
||||
f"'{'future_timeout' if not is_custom_endpoint else 'timeout'}' is too large",
|
||||
Codes.INVALID_PARAM,
|
||||
{
|
||||
"max_timeout_duration": max_timeout,
|
||||
},
|
||||
)
|
||||
else:
|
||||
if group_id is None:
|
||||
raise SynapseError(
|
||||
HTTPStatus.BAD_REQUEST,
|
||||
"At least one of 'future_timeout' and 'future_group_id' is required",
|
||||
Codes.MISSING_PARAM,
|
||||
)
|
||||
|
||||
await self.request_ratelimiter.ratelimit(requester)
|
||||
await self._initialized_from_db
|
||||
|
||||
if (
|
||||
timeout is None
|
||||
and group_id is not None
|
||||
and not await self.store.has_group_id(requester.user, group_id)
|
||||
):
|
||||
raise SynapseError(
|
||||
HTTPStatus.CONFLICT,
|
||||
# TODO: Think of a better wording for this
|
||||
"The given 'future_group_id' does not exist and cannot be created with an action future",
|
||||
Codes.INVALID_PARAM,
|
||||
)
|
||||
|
||||
(
|
||||
future_id,
|
||||
group_id,
|
||||
send_token,
|
||||
cancel_token,
|
||||
refresh_token,
|
||||
) = await self.store.add_future(
|
||||
user_id=requester.user,
|
||||
group_id=group_id,
|
||||
room_id=RoomID.from_string(room_id),
|
||||
event_type=event_type,
|
||||
state_key=state_key,
|
||||
origin_server_ts=origin_server_ts,
|
||||
content=content,
|
||||
timeout=timeout,
|
||||
timestamp=timeout + self.clock.time_msec() if timeout is not None else None,
|
||||
)
|
||||
|
||||
if timeout is not None:
|
||||
self._schedule_future(future_id, Timeout(timeout))
|
||||
|
||||
ret = {
|
||||
"future_group_id": group_id,
|
||||
"send_token": send_token,
|
||||
"cancel_token": cancel_token,
|
||||
}
|
||||
# TODO: type hint for non-None refresh_token return value when timeout argument is non-None
|
||||
if refresh_token is not None:
|
||||
ret["refresh_token"] = refresh_token
|
||||
return ret
|
||||
|
||||
async def use_future_token(self, token: str) -> None:
|
||||
"""Executes the appropriate action for the given token.
|
||||
|
||||
Params:
|
||||
token: The token of the future to act on.
|
||||
"""
|
||||
await self._initialized_from_db
|
||||
|
||||
future_id, future_token_type = await self.store.get_future_by_token(token)
|
||||
if future_token_type == FutureTokenType.SEND:
|
||||
await self._send_future(future_id)
|
||||
elif future_token_type == FutureTokenType.CANCEL:
|
||||
await self._cancel_future(future_id)
|
||||
elif future_token_type == FutureTokenType.REFRESH:
|
||||
await self._refresh_future(future_id)
|
||||
|
||||
async def _send_future(self, future_id: FutureID) -> None:
|
||||
await self._initialized_from_db
|
||||
|
||||
args = await self.store.pop_future_event(future_id)
|
||||
|
||||
self._unschedule_future(future_id)
|
||||
await self._send_event(*args)
|
||||
|
||||
async def _send_future_on_timeout(self, future_id: FutureID) -> None:
|
||||
await self._initialized_from_db
|
||||
|
||||
try:
|
||||
args = await self.store.pop_future_event(future_id)
|
||||
except NotFoundError:
|
||||
logger.warning(
|
||||
"future_id %s missing from DB on timeout, so it must have been cancelled/sent",
|
||||
future_id,
|
||||
)
|
||||
else:
|
||||
await self._send_event(*args)
|
||||
|
||||
async def _cancel_future(self, future_id: FutureID) -> None:
|
||||
await self._initialized_from_db
|
||||
|
||||
await self.store.remove_future(future_id)
|
||||
|
||||
self._unschedule_future(future_id)
|
||||
|
||||
async def _refresh_future(self, future_id: FutureID) -> None:
|
||||
await self._initialized_from_db
|
||||
|
||||
timeout = await self.store.update_future_timestamp(
|
||||
future_id,
|
||||
self._get_current_ts(),
|
||||
)
|
||||
|
||||
self._unschedule_future(future_id)
|
||||
self._schedule_future(future_id, timeout)
|
||||
|
||||
def _schedule_future(
|
||||
self,
|
||||
future_id: FutureID,
|
||||
timeout: Timeout,
|
||||
) -> None:
|
||||
"""NOTE: Should not be called with a future_id that isn't in the DB, or with a negative timeout."""
|
||||
delay_sec = timeout / 1000
|
||||
|
||||
logger.info(
|
||||
"Scheduling timeout for future_id %d in %.3fs", future_id, delay_sec
|
||||
)
|
||||
|
||||
self._futures[future_id] = self.clock.call_later(
|
||||
delay_sec,
|
||||
run_as_background_process,
|
||||
"_send_future_on_timeout",
|
||||
self._send_future_on_timeout,
|
||||
future_id,
|
||||
)
|
||||
|
||||
def _unschedule_future(self, future_id: FutureID) -> None:
|
||||
try:
|
||||
delayed_call = self._futures.pop(future_id)
|
||||
self.clock.cancel_call_later(delayed_call)
|
||||
except KeyError:
|
||||
logger.debug("future_id %s was not mapped to a delayed call", future_id)
|
||||
|
||||
async def get_all_futures_for_user(self, requester: Requester) -> List[JsonDict]:
|
||||
"""Return all pending futures requested by the given user."""
|
||||
await self.request_ratelimiter.ratelimit(requester)
|
||||
await self._initialized_from_db
|
||||
return await self.store.get_all_futures_for_user(requester.user)
|
||||
|
||||
# TODO: Consider getting a list of all timeout futures that were in this one's group, and cancel their delayed calls
|
||||
async def _send_event(
|
||||
self,
|
||||
user_localpart: str,
|
||||
room_id: RoomID,
|
||||
event_type: EventType,
|
||||
state_key: Optional[StateKey],
|
||||
origin_server_ts: Optional[Timestamp],
|
||||
content: JsonDict,
|
||||
# TODO: support guests
|
||||
# is_guest: bool,
|
||||
txn_id: Optional[str] = None,
|
||||
) -> None:
|
||||
user_id = UserID(user_localpart, self._hostname)
|
||||
requester = create_requester(
|
||||
user_id,
|
||||
# is_guest=is_guest,
|
||||
)
|
||||
|
||||
try:
|
||||
if state_key is not None and event_type == EventTypes.Member:
|
||||
membership = content.get("membership", None)
|
||||
event_id, _ = await self.room_member_handler.update_membership(
|
||||
requester,
|
||||
target=UserID.from_string(state_key),
|
||||
room_id=str(room_id),
|
||||
action=membership,
|
||||
content=content,
|
||||
origin_server_ts=origin_server_ts,
|
||||
)
|
||||
else:
|
||||
event_dict: JsonDict = {
|
||||
"type": event_type,
|
||||
"content": content,
|
||||
"room_id": room_id.to_string(),
|
||||
"sender": user_id.to_string(),
|
||||
}
|
||||
|
||||
if state_key is not None:
|
||||
event_dict["state_key"] = state_key
|
||||
|
||||
if origin_server_ts is not None:
|
||||
event_dict["origin_server_ts"] = origin_server_ts
|
||||
|
||||
(
|
||||
event,
|
||||
_,
|
||||
) = await self.event_creation_handler.create_and_send_nonmember_event(
|
||||
requester,
|
||||
event_dict,
|
||||
txn_id=txn_id,
|
||||
)
|
||||
event_id = event.event_id
|
||||
except ShadowBanError:
|
||||
event_id = "$" + random_string(43)
|
||||
|
||||
set_tag("event_id", event_id)
|
||||
|
||||
def _get_current_ts(self) -> Timestamp:
|
||||
return Timestamp(self.clock.time_msec())
|
||||
|
||||
|
||||
def _get_timeout_between(from_ts: Timestamp, to_ts: Timestamp) -> Timeout:
|
||||
return Timeout(to_ts - from_ts)
|
|
@ -642,6 +642,17 @@ class EventCreationHandler:
|
|||
"""
|
||||
await self.auth_blocking.check_auth_blocking(requester=requester)
|
||||
|
||||
if event_dict["type"] == EventTypes.Message:
|
||||
requester_suspended = await self.store.get_user_suspended_status(
|
||||
requester.user.to_string()
|
||||
)
|
||||
if requester_suspended:
|
||||
raise SynapseError(
|
||||
403,
|
||||
"Sending messages while account is suspended is not allowed.",
|
||||
Codes.USER_ACCOUNT_SUSPENDED,
|
||||
)
|
||||
|
||||
if event_dict["type"] == EventTypes.Create and event_dict["state_key"] == "":
|
||||
room_version_id = event_dict["content"]["room_version"]
|
||||
maybe_room_version_obj = KNOWN_ROOM_VERSIONS.get(room_version_id)
|
||||
|
|
|
@ -554,7 +554,7 @@ class SlidingSyncHandler:
|
|||
|
||||
# Flatten out the map
|
||||
dm_room_id_set = set()
|
||||
if dm_map:
|
||||
if isinstance(dm_map, dict):
|
||||
for room_ids in dm_map.values():
|
||||
# Account data should be a list of room IDs. Ignore anything else
|
||||
if isinstance(room_ids, list):
|
||||
|
@ -593,8 +593,21 @@ class SlidingSyncHandler:
|
|||
):
|
||||
filtered_room_id_set.remove(room_id)
|
||||
|
||||
if filters.is_invite:
|
||||
raise NotImplementedError()
|
||||
# Filter for rooms that the user has been invited to
|
||||
if filters.is_invite is not None:
|
||||
# Make a copy so we don't run into an error: `Set changed size during
|
||||
# iteration`, when we filter out and remove items
|
||||
for room_id in list(filtered_room_id_set):
|
||||
room_for_user = sync_room_map[room_id]
|
||||
# If we're looking for invite rooms, filter out rooms that the user is
|
||||
# not invited to and vice versa
|
||||
if (
|
||||
filters.is_invite and room_for_user.membership != Membership.INVITE
|
||||
) or (
|
||||
not filters.is_invite
|
||||
and room_for_user.membership == Membership.INVITE
|
||||
):
|
||||
filtered_room_id_set.remove(room_id)
|
||||
|
||||
if filters.room_types:
|
||||
raise NotImplementedError()
|
||||
|
|
|
@ -119,14 +119,15 @@ def parse_integer(
|
|||
default: value to use if the parameter is absent, defaults to None.
|
||||
required: whether to raise a 400 SynapseError if the parameter is absent,
|
||||
defaults to False.
|
||||
negative: whether to allow negative integers, defaults to True.
|
||||
negative: whether to allow negative integers, defaults to False (disallowing
|
||||
negatives).
|
||||
Returns:
|
||||
An int value or the default.
|
||||
|
||||
Raises:
|
||||
SynapseError: if the parameter is absent and required, if the
|
||||
parameter is present and not an integer, or if the
|
||||
parameter is illegitimate negative.
|
||||
parameter is illegitimately negative.
|
||||
"""
|
||||
args: Mapping[bytes, Sequence[bytes]] = request.args # type: ignore
|
||||
return parse_integer_from_args(args, name, default, required, negative)
|
||||
|
@ -164,7 +165,7 @@ def parse_integer_from_args(
|
|||
name: str,
|
||||
default: Optional[int] = None,
|
||||
required: bool = False,
|
||||
negative: bool = True,
|
||||
negative: bool = False,
|
||||
) -> Optional[int]:
|
||||
"""Parse an integer parameter from the request string
|
||||
|
||||
|
@ -174,7 +175,8 @@ def parse_integer_from_args(
|
|||
default: value to use if the parameter is absent, defaults to None.
|
||||
required: whether to raise a 400 SynapseError if the parameter is absent,
|
||||
defaults to False.
|
||||
negative: whether to allow negative integers, defaults to True.
|
||||
negative: whether to allow negative integers, defaults to False (disallowing
|
||||
negatives).
|
||||
|
||||
Returns:
|
||||
An int value or the default.
|
||||
|
@ -182,7 +184,7 @@ def parse_integer_from_args(
|
|||
Raises:
|
||||
SynapseError: if the parameter is absent and required, if the
|
||||
parameter is present and not an integer, or if the
|
||||
parameter is illegitimate negative.
|
||||
parameter is illegitimately negative.
|
||||
"""
|
||||
name_bytes = name.encode("ascii")
|
||||
|
||||
|
|
|
@ -25,7 +25,16 @@ import os
|
|||
import urllib
|
||||
from abc import ABC, abstractmethod
|
||||
from types import TracebackType
|
||||
from typing import Awaitable, Dict, Generator, List, Optional, Tuple, Type
|
||||
from typing import (
|
||||
TYPE_CHECKING,
|
||||
Awaitable,
|
||||
Dict,
|
||||
Generator,
|
||||
List,
|
||||
Optional,
|
||||
Tuple,
|
||||
Type,
|
||||
)
|
||||
|
||||
import attr
|
||||
|
||||
|
@ -37,8 +46,13 @@ from synapse.api.errors import Codes, cs_error
|
|||
from synapse.http.server import finish_request, respond_with_json
|
||||
from synapse.http.site import SynapseRequest
|
||||
from synapse.logging.context import make_deferred_yieldable
|
||||
from synapse.util import Clock
|
||||
from synapse.util.stringutils import is_ascii
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from synapse.storage.databases.main.media_repository import LocalMedia
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# list all text content types that will have the charset default to UTF-8 when
|
||||
|
@ -260,6 +274,68 @@ def _can_encode_filename_as_token(x: str) -> bool:
|
|||
return True
|
||||
|
||||
|
||||
async def respond_with_multipart_responder(
|
||||
clock: Clock,
|
||||
request: SynapseRequest,
|
||||
responder: "Optional[Responder]",
|
||||
media_info: "LocalMedia",
|
||||
) -> None:
|
||||
"""
|
||||
Responds to requests originating from the federation media `/download` endpoint by
|
||||
streaming a multipart/mixed response
|
||||
|
||||
Args:
|
||||
clock:
|
||||
request: the federation request to respond to
|
||||
responder: the responder which will send the response
|
||||
media_info: metadata about the media item
|
||||
"""
|
||||
if not responder:
|
||||
respond_404(request)
|
||||
return
|
||||
|
||||
# If we have a responder we *must* use it as a context manager.
|
||||
with responder:
|
||||
if request._disconnected:
|
||||
logger.warning(
|
||||
"Not sending response to request %s, already disconnected.", request
|
||||
)
|
||||
return
|
||||
|
||||
from synapse.media.media_storage import MultipartFileConsumer
|
||||
|
||||
# note that currently the json_object is just {}, this will change when linked media
|
||||
# is implemented
|
||||
multipart_consumer = MultipartFileConsumer(
|
||||
clock, request, media_info.media_type, {}, media_info.media_length
|
||||
)
|
||||
|
||||
logger.debug("Responding to media request with responder %s", responder)
|
||||
if media_info.media_length is not None:
|
||||
content_length = multipart_consumer.content_length()
|
||||
assert content_length is not None
|
||||
request.setHeader(b"Content-Length", b"%d" % (content_length,))
|
||||
|
||||
request.setHeader(
|
||||
b"Content-Type",
|
||||
b"multipart/mixed; boundary=%s" % multipart_consumer.boundary,
|
||||
)
|
||||
|
||||
try:
|
||||
await responder.write_to_consumer(multipart_consumer)
|
||||
except Exception as e:
|
||||
# The majority of the time this will be due to the client having gone
|
||||
# away. Unfortunately, Twisted simply throws a generic exception at us
|
||||
# in that case.
|
||||
logger.warning("Failed to write to consumer: %s %s", type(e), e)
|
||||
|
||||
# Unregister the producer, if it has one, so Twisted doesn't complain
|
||||
if request.producer:
|
||||
request.unregisterProducer()
|
||||
|
||||
finish_request(request)
|
||||
|
||||
|
||||
async def respond_with_responder(
|
||||
request: SynapseRequest,
|
||||
responder: "Optional[Responder]",
|
||||
|
|
|
@ -54,6 +54,7 @@ from synapse.media._base import (
|
|||
ThumbnailInfo,
|
||||
get_filename_from_headers,
|
||||
respond_404,
|
||||
respond_with_multipart_responder,
|
||||
respond_with_responder,
|
||||
)
|
||||
from synapse.media.filepath import MediaFilePaths
|
||||
|
@ -429,6 +430,7 @@ class MediaRepository:
|
|||
media_id: str,
|
||||
name: Optional[str],
|
||||
max_timeout_ms: int,
|
||||
federation: bool = False,
|
||||
) -> None:
|
||||
"""Responds to requests for local media, if exists, or returns 404.
|
||||
|
||||
|
@ -440,6 +442,7 @@ class MediaRepository:
|
|||
the filename in the Content-Disposition header of the response.
|
||||
max_timeout_ms: the maximum number of milliseconds to wait for the
|
||||
media to be uploaded.
|
||||
federation: whether the local media being fetched is for a federation request
|
||||
|
||||
Returns:
|
||||
Resolves once a response has successfully been written to request
|
||||
|
@ -460,9 +463,14 @@ class MediaRepository:
|
|||
file_info = FileInfo(None, media_id, url_cache=bool(url_cache))
|
||||
|
||||
responder = await self.media_storage.fetch_media(file_info)
|
||||
await respond_with_responder(
|
||||
request, responder, media_type, media_length, upload_name
|
||||
)
|
||||
if federation:
|
||||
await respond_with_multipart_responder(
|
||||
self.clock, request, responder, media_info
|
||||
)
|
||||
else:
|
||||
await respond_with_responder(
|
||||
request, responder, media_type, media_length, upload_name
|
||||
)
|
||||
|
||||
async def get_remote_media(
|
||||
self,
|
||||
|
|
|
@ -19,9 +19,12 @@
|
|||
#
|
||||
#
|
||||
import contextlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
from contextlib import closing
|
||||
from io import BytesIO
|
||||
from types import TracebackType
|
||||
from typing import (
|
||||
IO,
|
||||
|
@ -30,24 +33,35 @@ from typing import (
|
|||
AsyncIterator,
|
||||
BinaryIO,
|
||||
Callable,
|
||||
List,
|
||||
Optional,
|
||||
Sequence,
|
||||
Tuple,
|
||||
Type,
|
||||
Union,
|
||||
cast,
|
||||
)
|
||||
from uuid import uuid4
|
||||
|
||||
import attr
|
||||
from zope.interface import implementer
|
||||
|
||||
from twisted.internet import interfaces
|
||||
from twisted.internet.defer import Deferred
|
||||
from twisted.internet.interfaces import IConsumer
|
||||
from twisted.protocols.basic import FileSender
|
||||
|
||||
from synapse.api.errors import NotFoundError
|
||||
from synapse.logging.context import defer_to_thread, make_deferred_yieldable
|
||||
from synapse.logging.context import (
|
||||
defer_to_thread,
|
||||
make_deferred_yieldable,
|
||||
run_in_background,
|
||||
)
|
||||
from synapse.logging.opentracing import start_active_span, trace, trace_with_opname
|
||||
from synapse.util import Clock
|
||||
from synapse.util.file_consumer import BackgroundFileConsumer
|
||||
|
||||
from ..types import JsonDict
|
||||
from ._base import FileInfo, Responder
|
||||
from .filepath import MediaFilePaths
|
||||
|
||||
|
@ -57,6 +71,8 @@ if TYPE_CHECKING:
|
|||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
CRLF = b"\r\n"
|
||||
|
||||
|
||||
class MediaStorage:
|
||||
"""Responsible for storing/fetching files from local sources.
|
||||
|
@ -174,7 +190,7 @@ class MediaStorage:
|
|||
and configured storage providers.
|
||||
|
||||
Args:
|
||||
file_info
|
||||
file_info: Metadata about the media file
|
||||
|
||||
Returns:
|
||||
Returns a Responder if the file was found, otherwise None.
|
||||
|
@ -316,7 +332,7 @@ class FileResponder(Responder):
|
|||
"""Wraps an open file that can be sent to a request.
|
||||
|
||||
Args:
|
||||
open_file: A file like object to be streamed ot the client,
|
||||
open_file: A file like object to be streamed to the client,
|
||||
is closed when finished streaming.
|
||||
"""
|
||||
|
||||
|
@ -370,3 +386,240 @@ class ReadableFileWrapper:
|
|||
|
||||
# We yield to the reactor by sleeping for 0 seconds.
|
||||
await self.clock.sleep(0)
|
||||
|
||||
|
||||
@implementer(interfaces.IConsumer)
|
||||
@implementer(interfaces.IPushProducer)
|
||||
class MultipartFileConsumer:
|
||||
"""Wraps a given consumer so that any data that gets written to it gets
|
||||
converted to a multipart format.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
clock: Clock,
|
||||
wrapped_consumer: interfaces.IConsumer,
|
||||
file_content_type: str,
|
||||
json_object: JsonDict,
|
||||
content_length: Optional[int] = None,
|
||||
) -> None:
|
||||
self.clock = clock
|
||||
self.wrapped_consumer = wrapped_consumer
|
||||
self.json_field = json_object
|
||||
self.json_field_written = False
|
||||
self.content_type_written = False
|
||||
self.file_content_type = file_content_type
|
||||
self.boundary = uuid4().hex.encode("ascii")
|
||||
|
||||
# The producer that registered with us, and if it's a push or pull
|
||||
# producer.
|
||||
self.producer: Optional["interfaces.IProducer"] = None
|
||||
self.streaming: Optional[bool] = None
|
||||
|
||||
# Whether the wrapped consumer has asked us to pause.
|
||||
self.paused = False
|
||||
|
||||
self.length = content_length
|
||||
|
||||
### IConsumer APIs ###
|
||||
|
||||
def registerProducer(
|
||||
self, producer: "interfaces.IProducer", streaming: bool
|
||||
) -> None:
|
||||
"""
|
||||
Register to receive data from a producer.
|
||||
|
||||
This sets self to be a consumer for a producer. When this object runs
|
||||
out of data (as when a send(2) call on a socket succeeds in moving the
|
||||
last data from a userspace buffer into a kernelspace buffer), it will
|
||||
ask the producer to resumeProducing().
|
||||
|
||||
For L{IPullProducer} providers, C{resumeProducing} will be called once
|
||||
each time data is required.
|
||||
|
||||
For L{IPushProducer} providers, C{pauseProducing} will be called
|
||||
whenever the write buffer fills up and C{resumeProducing} will only be
|
||||
called when it empties. The consumer will only call C{resumeProducing}
|
||||
to balance a previous C{pauseProducing} call; the producer is assumed
|
||||
to start in an un-paused state.
|
||||
|
||||
@param streaming: C{True} if C{producer} provides L{IPushProducer},
|
||||
C{False} if C{producer} provides L{IPullProducer}.
|
||||
|
||||
@raise RuntimeError: If a producer is already registered.
|
||||
"""
|
||||
self.producer = producer
|
||||
self.streaming = streaming
|
||||
|
||||
self.wrapped_consumer.registerProducer(self, True)
|
||||
|
||||
# kick off producing if `self.producer` is not a streaming producer
|
||||
if not streaming:
|
||||
self.resumeProducing()
|
||||
|
||||
def unregisterProducer(self) -> None:
|
||||
"""
|
||||
Stop consuming data from a producer, without disconnecting.
|
||||
"""
|
||||
self.wrapped_consumer.write(CRLF + b"--" + self.boundary + b"--" + CRLF)
|
||||
self.wrapped_consumer.unregisterProducer()
|
||||
self.paused = True
|
||||
|
||||
def write(self, data: bytes) -> None:
|
||||
"""
|
||||
The producer will write data by calling this method.
|
||||
|
||||
The implementation must be non-blocking and perform whatever
|
||||
buffering is necessary. If the producer has provided enough data
|
||||
for now and it is a L{IPushProducer}, the consumer may call its
|
||||
C{pauseProducing} method.
|
||||
"""
|
||||
if not self.json_field_written:
|
||||
self.wrapped_consumer.write(CRLF + b"--" + self.boundary + CRLF)
|
||||
|
||||
content_type = Header(b"Content-Type", b"application/json")
|
||||
self.wrapped_consumer.write(bytes(content_type) + CRLF)
|
||||
|
||||
json_field = json.dumps(self.json_field)
|
||||
json_bytes = json_field.encode("utf-8")
|
||||
self.wrapped_consumer.write(CRLF + json_bytes)
|
||||
self.wrapped_consumer.write(CRLF + b"--" + self.boundary + CRLF)
|
||||
|
||||
self.json_field_written = True
|
||||
|
||||
# if we haven't written the content type yet, do so
|
||||
if not self.content_type_written:
|
||||
type = self.file_content_type.encode("utf-8")
|
||||
content_type = Header(b"Content-Type", type)
|
||||
self.wrapped_consumer.write(bytes(content_type) + CRLF + CRLF)
|
||||
self.content_type_written = True
|
||||
|
||||
self.wrapped_consumer.write(data)
|
||||
|
||||
### IPushProducer APIs ###
|
||||
|
||||
def stopProducing(self) -> None:
|
||||
"""
|
||||
Stop producing data.
|
||||
|
||||
This tells a producer that its consumer has died, so it must stop
|
||||
producing data for good.
|
||||
"""
|
||||
assert self.producer is not None
|
||||
|
||||
self.paused = True
|
||||
self.producer.stopProducing()
|
||||
|
||||
def pauseProducing(self) -> None:
|
||||
"""
|
||||
Pause producing data.
|
||||
|
||||
Tells a producer that it has produced too much data to process for
|
||||
the time being, and to stop until C{resumeProducing()} is called.
|
||||
"""
|
||||
assert self.producer is not None
|
||||
|
||||
self.paused = True
|
||||
|
||||
if self.streaming:
|
||||
cast("interfaces.IPushProducer", self.producer).pauseProducing()
|
||||
else:
|
||||
self.paused = True
|
||||
|
||||
def resumeProducing(self) -> None:
|
||||
"""
|
||||
Resume producing data.
|
||||
|
||||
This tells a producer to re-add itself to the main loop and produce
|
||||
more data for its consumer.
|
||||
"""
|
||||
assert self.producer is not None
|
||||
|
||||
if self.streaming:
|
||||
cast("interfaces.IPushProducer", self.producer).resumeProducing()
|
||||
else:
|
||||
# If the producer is not a streaming producer we need to start
|
||||
# repeatedly calling `resumeProducing` in a loop.
|
||||
run_in_background(self._resumeProducingRepeatedly)
|
||||
|
||||
def content_length(self) -> Optional[int]:
|
||||
"""
|
||||
Calculate the content length of the multipart response
|
||||
in bytes.
|
||||
"""
|
||||
if not self.length:
|
||||
return None
|
||||
# calculate length of json field and content-type header
|
||||
json_field = json.dumps(self.json_field)
|
||||
json_bytes = json_field.encode("utf-8")
|
||||
json_length = len(json_bytes)
|
||||
|
||||
type = self.file_content_type.encode("utf-8")
|
||||
content_type = Header(b"Content-Type", type)
|
||||
type_length = len(bytes(content_type))
|
||||
|
||||
# 154 is the length of the elements that aren't variable, ie
|
||||
# CRLFs and boundary strings, etc
|
||||
self.length += json_length + type_length + 154
|
||||
|
||||
return self.length
|
||||
|
||||
### Internal APIs. ###
|
||||
|
||||
async def _resumeProducingRepeatedly(self) -> None:
|
||||
assert self.producer is not None
|
||||
assert not self.streaming
|
||||
|
||||
producer = cast("interfaces.IPullProducer", self.producer)
|
||||
|
||||
self.paused = False
|
||||
while not self.paused:
|
||||
producer.resumeProducing()
|
||||
await self.clock.sleep(0)
|
||||
|
||||
|
||||
class Header:
|
||||
"""
|
||||
`Header` This class is a tiny wrapper that produces
|
||||
request headers. We can't use standard python header
|
||||
class because it encodes unicode fields using =? bla bla ?=
|
||||
encoding, which is correct, but no one in HTTP world expects
|
||||
that, everyone wants utf-8 raw bytes. (stolen from treq.multipart)
|
||||
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
name: bytes,
|
||||
value: Any,
|
||||
params: Optional[List[Tuple[Any, Any]]] = None,
|
||||
):
|
||||
self.name = name
|
||||
self.value = value
|
||||
self.params = params or []
|
||||
|
||||
def add_param(self, name: Any, value: Any) -> None:
|
||||
self.params.append((name, value))
|
||||
|
||||
def __bytes__(self) -> bytes:
|
||||
with closing(BytesIO()) as h:
|
||||
h.write(self.name + b": " + escape(self.value).encode("us-ascii"))
|
||||
if self.params:
|
||||
for name, val in self.params:
|
||||
h.write(b"; ")
|
||||
h.write(escape(name).encode("us-ascii"))
|
||||
h.write(b"=")
|
||||
h.write(b'"' + escape(val).encode("utf-8") + b'"')
|
||||
h.seek(0)
|
||||
return h.read()
|
||||
|
||||
|
||||
def escape(value: Union[str, bytes]) -> str:
|
||||
"""
|
||||
This function prevents header values from corrupting the request,
|
||||
a newline in the file name parameter makes form-data request unreadable
|
||||
for a majority of parsers. (stolen from treq.multipart)
|
||||
"""
|
||||
if isinstance(value, bytes):
|
||||
value = value.decode("utf-8")
|
||||
return value.replace("\r", "").replace("\n", "").replace('"', '\\"')
|
||||
|
|
|
@ -28,7 +28,7 @@ import jinja2
|
|||
from markupsafe import Markup
|
||||
from prometheus_client import Counter
|
||||
|
||||
from synapse.api.constants import EventTypes, Membership, RoomTypes
|
||||
from synapse.api.constants import EventContentFields, EventTypes, Membership, RoomTypes
|
||||
from synapse.api.errors import StoreError
|
||||
from synapse.config.emailconfig import EmailSubjectConfig
|
||||
from synapse.events import EventBase
|
||||
|
@ -716,7 +716,8 @@ class Mailer:
|
|||
)
|
||||
if (
|
||||
create_event
|
||||
and create_event.content.get("room_type") == RoomTypes.SPACE
|
||||
and create_event.content.get(EventContentFields.ROOM_TYPE)
|
||||
== RoomTypes.SPACE
|
||||
):
|
||||
return self.email_subjects.invite_from_person_to_space % {
|
||||
"person": inviter_name,
|
||||
|
|
|
@ -114,13 +114,19 @@ class ReplicationDataHandler:
|
|||
"""
|
||||
all_room_ids: Set[str] = set()
|
||||
if stream_name == DeviceListsStream.NAME:
|
||||
if any(row.entity.startswith("@") and not row.is_signature for row in rows):
|
||||
if any(not row.is_signature and not row.hosts_calculated for row in rows):
|
||||
prev_token = self.store.get_device_stream_token()
|
||||
all_room_ids = await self.store.get_all_device_list_changes(
|
||||
prev_token, token
|
||||
)
|
||||
self.store.device_lists_in_rooms_have_changed(all_room_ids, token)
|
||||
|
||||
# If we're sending federation we need to update the device lists
|
||||
# outbound pokes stream change cache with updated hosts.
|
||||
if self.send_handler and any(row.hosts_calculated for row in rows):
|
||||
hosts = await self.store.get_destinations_for_device(token)
|
||||
self.store.device_lists_outbound_pokes_have_changed(hosts, token)
|
||||
|
||||
self.store.process_replication_rows(stream_name, instance_name, token, rows)
|
||||
# NOTE: this must be called after process_replication_rows to ensure any
|
||||
# cache invalidations are first handled before any stream ID advances.
|
||||
|
@ -433,12 +439,11 @@ class FederationSenderHandler:
|
|||
# The entities are either user IDs (starting with '@') whose devices
|
||||
# have changed, or remote servers that we need to tell about
|
||||
# changes.
|
||||
hosts = {
|
||||
row.entity
|
||||
for row in rows
|
||||
if not row.entity.startswith("@") and not row.is_signature
|
||||
}
|
||||
await self.federation_sender.send_device_messages(hosts, immediate=False)
|
||||
if any(row.hosts_calculated for row in rows):
|
||||
hosts = await self.store.get_destinations_for_device(token)
|
||||
await self.federation_sender.send_device_messages(
|
||||
hosts, immediate=False
|
||||
)
|
||||
|
||||
elif stream_name == ToDeviceStream.NAME:
|
||||
# The to_device stream includes stuff to be pushed to both local
|
||||
|
|
|
@ -549,10 +549,14 @@ class DeviceListsStream(_StreamFromIdGen):
|
|||
|
||||
@attr.s(slots=True, frozen=True, auto_attribs=True)
|
||||
class DeviceListsStreamRow:
|
||||
entity: str
|
||||
user_id: str
|
||||
# Indicates that a user has signed their own device with their user-signing key
|
||||
is_signature: bool
|
||||
|
||||
# Indicates if this is a notification that we've calculated the hosts we
|
||||
# need to send the update to.
|
||||
hosts_calculated: bool
|
||||
|
||||
NAME = "device_lists"
|
||||
ROW_TYPE = DeviceListsStreamRow
|
||||
|
||||
|
@ -594,13 +598,13 @@ class DeviceListsStream(_StreamFromIdGen):
|
|||
upper_limit_token = min(upper_limit_token, signatures_to_token)
|
||||
|
||||
device_updates = [
|
||||
(stream_id, (entity, False))
|
||||
for stream_id, (entity,) in device_updates
|
||||
(stream_id, (entity, False, hosts))
|
||||
for stream_id, (entity, hosts) in device_updates
|
||||
if stream_id <= upper_limit_token
|
||||
]
|
||||
|
||||
signatures_updates = [
|
||||
(stream_id, (entity, True))
|
||||
(stream_id, (entity, True, False))
|
||||
for stream_id, (entity,) in signatures_updates
|
||||
if stream_id <= upper_limit_token
|
||||
]
|
||||
|
|
|
@ -34,6 +34,7 @@ from synapse.rest.client import (
|
|||
directory,
|
||||
events,
|
||||
filter,
|
||||
futures,
|
||||
initial_sync,
|
||||
keys,
|
||||
knock,
|
||||
|
@ -103,6 +104,7 @@ class ClientRestResource(JsonResource):
|
|||
events.register_servlets(hs, client_resource)
|
||||
|
||||
room.register_servlets(hs, client_resource)
|
||||
futures.register_servlets(hs, client_resource)
|
||||
login.register_servlets(hs, client_resource)
|
||||
profile.register_servlets(hs, client_resource)
|
||||
presence.register_servlets(hs, client_resource)
|
||||
|
|
|
@ -101,6 +101,7 @@ from synapse.rest.admin.users import (
|
|||
ResetPasswordRestServlet,
|
||||
SearchUsersRestServlet,
|
||||
ShadowBanRestServlet,
|
||||
SuspendAccountRestServlet,
|
||||
UserAdminServlet,
|
||||
UserByExternalId,
|
||||
UserByThreePid,
|
||||
|
@ -327,6 +328,8 @@ def register_servlets(hs: "HomeServer", http_server: HttpServer) -> None:
|
|||
BackgroundUpdateRestServlet(hs).register(http_server)
|
||||
BackgroundUpdateStartJobRestServlet(hs).register(http_server)
|
||||
ExperimentalFeaturesRestServlet(hs).register(http_server)
|
||||
if hs.config.experimental.msc3823_account_suspension:
|
||||
SuspendAccountRestServlet(hs).register(http_server)
|
||||
|
||||
|
||||
def register_servlets_for_client_rest_resource(
|
||||
|
|
|
@ -61,8 +61,8 @@ class ListDestinationsRestServlet(RestServlet):
|
|||
async def on_GET(self, request: SynapseRequest) -> Tuple[int, JsonDict]:
|
||||
await assert_requester_is_admin(self._auth, request)
|
||||
|
||||
start = parse_integer(request, "from", default=0, negative=False)
|
||||
limit = parse_integer(request, "limit", default=100, negative=False)
|
||||
start = parse_integer(request, "from", default=0)
|
||||
limit = parse_integer(request, "limit", default=100)
|
||||
|
||||
destination = parse_string(request, "destination")
|
||||
|
||||
|
@ -181,8 +181,8 @@ class DestinationMembershipRestServlet(RestServlet):
|
|||
if not await self._store.is_destination_known(destination):
|
||||
raise NotFoundError("Unknown destination")
|
||||
|
||||
start = parse_integer(request, "from", default=0, negative=False)
|
||||
limit = parse_integer(request, "limit", default=100, negative=False)
|
||||
start = parse_integer(request, "from", default=0)
|
||||
limit = parse_integer(request, "limit", default=100)
|
||||
|
||||
direction = parse_enum(request, "dir", Direction, default=Direction.FORWARDS)
|
||||
|
||||
|
|
|
@ -311,8 +311,8 @@ class DeleteMediaByDateSize(RestServlet):
|
|||
) -> Tuple[int, JsonDict]:
|
||||
await assert_requester_is_admin(self.auth, request)
|
||||
|
||||
before_ts = parse_integer(request, "before_ts", required=True, negative=False)
|
||||
size_gt = parse_integer(request, "size_gt", default=0, negative=False)
|
||||
before_ts = parse_integer(request, "before_ts", required=True)
|
||||
size_gt = parse_integer(request, "size_gt", default=0)
|
||||
keep_profiles = parse_boolean(request, "keep_profiles", default=True)
|
||||
|
||||
if before_ts < 30000000000: # Dec 1970 in milliseconds, Aug 2920 in seconds
|
||||
|
@ -377,8 +377,8 @@ class UserMediaRestServlet(RestServlet):
|
|||
if user is None:
|
||||
raise NotFoundError("Unknown user")
|
||||
|
||||
start = parse_integer(request, "from", default=0, negative=False)
|
||||
limit = parse_integer(request, "limit", default=100, negative=False)
|
||||
start = parse_integer(request, "from", default=0)
|
||||
limit = parse_integer(request, "limit", default=100)
|
||||
|
||||
# If neither `order_by` nor `dir` is set, set the default order
|
||||
# to newest media is on top for backward compatibility.
|
||||
|
@ -421,8 +421,8 @@ class UserMediaRestServlet(RestServlet):
|
|||
if user is None:
|
||||
raise NotFoundError("Unknown user")
|
||||
|
||||
start = parse_integer(request, "from", default=0, negative=False)
|
||||
limit = parse_integer(request, "limit", default=100, negative=False)
|
||||
start = parse_integer(request, "from", default=0)
|
||||
limit = parse_integer(request, "limit", default=100)
|
||||
|
||||
# If neither `order_by` nor `dir` is set, set the default order
|
||||
# to newest media is on top for backward compatibility.
|
||||
|
|
|
@ -35,6 +35,7 @@ from synapse.http.servlet import (
|
|||
ResolveRoomIdMixin,
|
||||
RestServlet,
|
||||
assert_params_in_dict,
|
||||
parse_boolean,
|
||||
parse_enum,
|
||||
parse_integer,
|
||||
parse_json,
|
||||
|
@ -242,13 +243,23 @@ class ListRoomRestServlet(RestServlet):
|
|||
errcode=Codes.INVALID_PARAM,
|
||||
)
|
||||
|
||||
public_rooms = parse_boolean(request, "public_rooms")
|
||||
empty_rooms = parse_boolean(request, "empty_rooms")
|
||||
|
||||
direction = parse_enum(request, "dir", Direction, default=Direction.FORWARDS)
|
||||
reverse_order = True if direction == Direction.BACKWARDS else False
|
||||
|
||||
# Return list of rooms according to parameters
|
||||
rooms, total_rooms = await self.store.get_rooms_paginate(
|
||||
start, limit, order_by, reverse_order, search_term
|
||||
start,
|
||||
limit,
|
||||
order_by,
|
||||
reverse_order,
|
||||
search_term,
|
||||
public_rooms,
|
||||
empty_rooms,
|
||||
)
|
||||
|
||||
response = {
|
||||
# next_token should be opaque, so return a value the client can parse
|
||||
"offset": start,
|
||||
|
|
|
@ -63,10 +63,10 @@ class UserMediaStatisticsRestServlet(RestServlet):
|
|||
),
|
||||
)
|
||||
|
||||
start = parse_integer(request, "from", default=0, negative=False)
|
||||
limit = parse_integer(request, "limit", default=100, negative=False)
|
||||
from_ts = parse_integer(request, "from_ts", default=0, negative=False)
|
||||
until_ts = parse_integer(request, "until_ts", negative=False)
|
||||
start = parse_integer(request, "from", default=0)
|
||||
limit = parse_integer(request, "limit", default=100)
|
||||
from_ts = parse_integer(request, "from_ts", default=0)
|
||||
until_ts = parse_integer(request, "until_ts")
|
||||
|
||||
if until_ts is not None:
|
||||
if until_ts <= from_ts:
|
||||
|
|
|
@ -27,11 +27,13 @@ from typing import TYPE_CHECKING, Dict, List, Optional, Tuple, Union
|
|||
|
||||
import attr
|
||||
|
||||
from synapse._pydantic_compat import HAS_PYDANTIC_V2
|
||||
from synapse.api.constants import Direction, UserTypes
|
||||
from synapse.api.errors import Codes, NotFoundError, SynapseError
|
||||
from synapse.http.servlet import (
|
||||
RestServlet,
|
||||
assert_params_in_dict,
|
||||
parse_and_validate_json_object_from_request,
|
||||
parse_boolean,
|
||||
parse_enum,
|
||||
parse_integer,
|
||||
|
@ -49,10 +51,17 @@ from synapse.rest.client._base import client_patterns
|
|||
from synapse.storage.databases.main.registration import ExternalIDReuseException
|
||||
from synapse.storage.databases.main.stats import UserSortOrder
|
||||
from synapse.types import JsonDict, JsonMapping, UserID
|
||||
from synapse.types.rest import RequestBodyModel
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from synapse.server import HomeServer
|
||||
|
||||
if TYPE_CHECKING or HAS_PYDANTIC_V2:
|
||||
from pydantic.v1 import StrictBool
|
||||
else:
|
||||
from pydantic import StrictBool
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
|
@ -90,8 +99,8 @@ class UsersRestServletV2(RestServlet):
|
|||
async def on_GET(self, request: SynapseRequest) -> Tuple[int, JsonDict]:
|
||||
await assert_requester_is_admin(self.auth, request)
|
||||
|
||||
start = parse_integer(request, "from", default=0, negative=False)
|
||||
limit = parse_integer(request, "limit", default=100, negative=False)
|
||||
start = parse_integer(request, "from", default=0)
|
||||
limit = parse_integer(request, "limit", default=100)
|
||||
|
||||
user_id = parse_string(request, "user_id")
|
||||
name = parse_string(request, "name", encoding="utf-8")
|
||||
|
@ -732,6 +741,36 @@ class DeactivateAccountRestServlet(RestServlet):
|
|||
return HTTPStatus.OK, {"id_server_unbind_result": id_server_unbind_result}
|
||||
|
||||
|
||||
class SuspendAccountRestServlet(RestServlet):
|
||||
PATTERNS = admin_patterns("/suspend/(?P<target_user_id>[^/]*)$")
|
||||
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self.auth = hs.get_auth()
|
||||
self.is_mine = hs.is_mine
|
||||
self.store = hs.get_datastores().main
|
||||
|
||||
class PutBody(RequestBodyModel):
|
||||
suspend: StrictBool
|
||||
|
||||
async def on_PUT(
|
||||
self, request: SynapseRequest, target_user_id: str
|
||||
) -> Tuple[int, JsonDict]:
|
||||
requester = await self.auth.get_user_by_req(request)
|
||||
await assert_user_is_admin(self.auth, requester)
|
||||
|
||||
if not self.is_mine(UserID.from_string(target_user_id)):
|
||||
raise SynapseError(HTTPStatus.BAD_REQUEST, "Can only suspend local users")
|
||||
|
||||
if not await self.store.get_user_by_id(target_user_id):
|
||||
raise NotFoundError("User not found")
|
||||
|
||||
body = parse_and_validate_json_object_from_request(request, self.PutBody)
|
||||
suspend = body.suspend
|
||||
await self.store.set_user_suspended_status(target_user_id, suspend)
|
||||
|
||||
return HTTPStatus.OK, {f"user_{target_user_id}_suspended": suspend}
|
||||
|
||||
|
||||
class AccountValidityRenewServlet(RestServlet):
|
||||
PATTERNS = admin_patterns("/account_validity/validity$")
|
||||
|
||||
|
|
90
synapse/rest/client/futures.py
Normal file
90
synapse/rest/client/futures.py
Normal file
|
@ -0,0 +1,90 @@
|
|||
#
|
||||
# This file is licensed under the Affero General Public License (AGPL) version 3.
|
||||
#
|
||||
# Copyright (C) 2024 New Vector, Ltd
|
||||
#
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Affero General Public License as
|
||||
# published by the Free Software Foundation, either version 3 of the
|
||||
# License, or (at your option) any later version.
|
||||
#
|
||||
# See the GNU Affero General Public License for more details:
|
||||
# <https://www.gnu.org/licenses/agpl-3.0.html>.
|
||||
#
|
||||
# Originally licensed under the Apache License, Version 2.0:
|
||||
# <http://www.apache.org/licenses/LICENSE-2.0>.
|
||||
#
|
||||
#
|
||||
|
||||
""" This module contains REST servlets to do with futures: /future/<paths> """
|
||||
import logging
|
||||
from typing import TYPE_CHECKING, List, Tuple
|
||||
|
||||
from synapse.api.errors import NotFoundError
|
||||
from synapse.api.ratelimiting import Ratelimiter
|
||||
from synapse.http.server import HttpServer
|
||||
from synapse.http.servlet import RestServlet
|
||||
from synapse.http.site import SynapseRequest
|
||||
from synapse.rest.client._base import client_patterns
|
||||
from synapse.types import JsonDict
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from synapse.server import HomeServer
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# TODO: Needs unit testing
|
||||
class FuturesTokenServlet(RestServlet):
|
||||
PATTERNS = client_patterns(
|
||||
r"/org\.matrix\.msc4140/future/(?P<token>[^/]*)$", releases=(), v1=False
|
||||
)
|
||||
CATEGORY = "Future management requests"
|
||||
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
super().__init__()
|
||||
self.futures_handler = hs.get_futures_handler()
|
||||
|
||||
self._ratelimiter = Ratelimiter(
|
||||
store=hs.get_datastores().main,
|
||||
clock=hs.get_clock(),
|
||||
cfg=hs.config.ratelimiting.rc_future_token_validity,
|
||||
)
|
||||
|
||||
async def on_POST(
|
||||
self, request: SynapseRequest, token: str
|
||||
) -> Tuple[int, JsonDict]:
|
||||
# Ratelimit by address, since this is an unauthenticated request
|
||||
ratelimit_key = (request.getClientAddress().host,)
|
||||
# Check if we should be ratelimited due to too many previous failed attempts
|
||||
await self._ratelimiter.ratelimit(None, ratelimit_key, update=False)
|
||||
|
||||
try:
|
||||
await self.futures_handler.use_future_token(token)
|
||||
return 200, {}
|
||||
except NotFoundError as e:
|
||||
# Update the ratelimiter to say we failed (`can_do_action` doesn't raise).
|
||||
await self._ratelimiter.can_do_action(None, ratelimit_key)
|
||||
# TODO: Decide if the error code should be left at 404, instead of 410 as per the MSC
|
||||
e.code = 410
|
||||
raise
|
||||
|
||||
|
||||
# TODO: Needs unit testing
|
||||
class FuturesServlet(RestServlet):
|
||||
PATTERNS = client_patterns(r"/org\.matrix\.msc4140/future$", releases=(), v1=False)
|
||||
CATEGORY = "Future management requests"
|
||||
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
super().__init__()
|
||||
self.auth = hs.get_auth()
|
||||
self.futures_handler = hs.get_futures_handler()
|
||||
|
||||
async def on_GET(self, request: SynapseRequest) -> Tuple[int, List[JsonDict]]:
|
||||
requester = await self.auth.get_user_by_req(request)
|
||||
return 200, await self.futures_handler.get_all_futures_for_user(requester)
|
||||
|
||||
|
||||
def register_servlets(hs: "HomeServer", http_server: HttpServer) -> None:
|
||||
FuturesTokenServlet(hs).register(http_server)
|
||||
FuturesServlet(hs).register(http_server)
|
|
@ -32,6 +32,7 @@ from synapse.http.servlet import RestServlet, parse_integer, parse_string
|
|||
from synapse.http.site import SynapseRequest
|
||||
from synapse.types import JsonDict
|
||||
|
||||
from ...api.errors import SynapseError
|
||||
from ._base import client_patterns
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
@ -56,7 +57,22 @@ class NotificationsServlet(RestServlet):
|
|||
requester = await self.auth.get_user_by_req(request)
|
||||
user_id = requester.user.to_string()
|
||||
|
||||
from_token = parse_string(request, "from", required=False)
|
||||
# While this is intended to be "string" to clients, the 'from' token
|
||||
# is actually based on a numeric ID. So it must parse to an int.
|
||||
from_token_str = parse_string(request, "from", required=False)
|
||||
if from_token_str is not None:
|
||||
# Parse to an integer.
|
||||
try:
|
||||
from_token = int(from_token_str)
|
||||
except ValueError:
|
||||
# If it doesn't parse to an integer, then this cannot possibly be a valid
|
||||
# pagination token, as we only hand out integers.
|
||||
raise SynapseError(
|
||||
400, 'Query parameter "from" contains unrecognised token'
|
||||
)
|
||||
else:
|
||||
from_token = None
|
||||
|
||||
limit = parse_integer(request, "limit", default=50)
|
||||
only = parse_string(request, "only", required=False)
|
||||
|
||||
|
|
|
@ -108,6 +108,19 @@ class ProfileDisplaynameRestServlet(RestServlet):
|
|||
|
||||
propagate = _read_propagate(self.hs, request)
|
||||
|
||||
requester_suspended = (
|
||||
await self.hs.get_datastores().main.get_user_suspended_status(
|
||||
requester.user.to_string()
|
||||
)
|
||||
)
|
||||
|
||||
if requester_suspended:
|
||||
raise SynapseError(
|
||||
403,
|
||||
"Updating displayname while account is suspended is not allowed.",
|
||||
Codes.USER_ACCOUNT_SUSPENDED,
|
||||
)
|
||||
|
||||
await self.profile_handler.set_displayname(
|
||||
user, requester, new_name, is_admin, propagate=propagate
|
||||
)
|
||||
|
@ -167,6 +180,19 @@ class ProfileAvatarURLRestServlet(RestServlet):
|
|||
|
||||
propagate = _read_propagate(self.hs, request)
|
||||
|
||||
requester_suspended = (
|
||||
await self.hs.get_datastores().main.get_user_suspended_status(
|
||||
requester.user.to_string()
|
||||
)
|
||||
)
|
||||
|
||||
if requester_suspended:
|
||||
raise SynapseError(
|
||||
403,
|
||||
"Updating avatar URL while account is suspended is not allowed.",
|
||||
Codes.USER_ACCOUNT_SUSPENDED,
|
||||
)
|
||||
|
||||
await self.profile_handler.set_avatar_url(
|
||||
user, requester, new_avatar_url, is_admin, propagate=propagate
|
||||
)
|
||||
|
|
|
@ -2,7 +2,7 @@
|
|||
# This file is licensed under the Affero General Public License (AGPL) version 3.
|
||||
#
|
||||
# Copyright 2014-2016 OpenMarket Ltd
|
||||
# Copyright (C) 2023 New Vector, Ltd
|
||||
# Copyright (C) 2023-2024 New Vector, Ltd
|
||||
#
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Affero General Public License as
|
||||
|
@ -36,6 +36,7 @@ from synapse.api.constants import Direction, EventTypes, Membership
|
|||
from synapse.api.errors import (
|
||||
AuthError,
|
||||
Codes,
|
||||
InvalidAPICallError,
|
||||
InvalidClientCredentialsError,
|
||||
MissingClientTokenError,
|
||||
ShadowBanError,
|
||||
|
@ -64,7 +65,14 @@ from synapse.metrics.background_process_metrics import run_as_background_process
|
|||
from synapse.rest.client._base import client_patterns
|
||||
from synapse.rest.client.transactions import HttpTransactionCache
|
||||
from synapse.streams.config import PaginationConfig
|
||||
from synapse.types import JsonDict, Requester, StreamToken, ThirdPartyInstanceID, UserID
|
||||
from synapse.types import (
|
||||
JsonDict,
|
||||
Requester,
|
||||
StrCollection,
|
||||
StreamToken,
|
||||
ThirdPartyInstanceID,
|
||||
UserID,
|
||||
)
|
||||
from synapse.types.state import StateFilter
|
||||
from synapse.util.cancellation import cancellable
|
||||
from synapse.util.stringutils import parse_and_validate_server_name, random_string
|
||||
|
@ -409,6 +417,261 @@ class RoomSendEventRestServlet(TransactionRestServlet):
|
|||
)
|
||||
|
||||
|
||||
# TODO: Needs unit testing
|
||||
class RoomStateFutureRestServlet(RestServlet):
|
||||
CATEGORY = "Future sending requests"
|
||||
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
super().__init__()
|
||||
self.auth = hs.get_auth()
|
||||
self.futures_handler = hs.get_futures_handler()
|
||||
|
||||
def register(self, http_server: HttpServer) -> None:
|
||||
# /org.matrix.msc4140/rooms/$roomid/state_future/$eventtype
|
||||
no_state_key = r"/org\.matrix\.msc4140/rooms/(?P<room_id>[^/]*)/state_future/(?P<event_type>[^/]*)$"
|
||||
|
||||
# /org.matrix.msc4140/rooms/$roomid/state_future/$eventtype/$statekey
|
||||
state_key = (
|
||||
r"/org\.matrix\.msc4140/rooms/(?P<room_id>[^/]*)/state_future/"
|
||||
"(?P<event_type>[^/]*)/(?P<state_key>[^/]*)"
|
||||
)
|
||||
http_server.register_paths(
|
||||
"PUT",
|
||||
client_patterns(state_key, releases=(), v1=False),
|
||||
self.on_PUT,
|
||||
self.__class__.__name__,
|
||||
)
|
||||
http_server.register_paths(
|
||||
"PUT",
|
||||
client_patterns(no_state_key, releases=(), v1=False),
|
||||
self.on_PUT_no_state_key,
|
||||
self.__class__.__name__,
|
||||
)
|
||||
|
||||
def on_PUT_no_state_key(
|
||||
self, request: SynapseRequest, room_id: str, event_type: str
|
||||
) -> Awaitable[Tuple[int, JsonDict]]:
|
||||
return self.on_PUT(request, room_id, event_type, "")
|
||||
|
||||
async def on_PUT(
|
||||
self,
|
||||
request: SynapseRequest,
|
||||
room_id: str,
|
||||
event_type: str,
|
||||
state_key: str,
|
||||
txn_id: Optional[str] = None,
|
||||
) -> Tuple[int, JsonDict]:
|
||||
requester = await self.auth.get_user_by_req(request, allow_guest=True)
|
||||
|
||||
if txn_id:
|
||||
set_tag("txn_id", txn_id)
|
||||
|
||||
ret = await self.futures_handler.add_future(
|
||||
requester,
|
||||
room_id=room_id,
|
||||
event_type=event_type,
|
||||
state_key=state_key,
|
||||
origin_server_ts=(
|
||||
parse_integer(request, "ts") if requester.app_service else None
|
||||
),
|
||||
content=parse_json_object_from_request(request),
|
||||
timeout=parse_integer(request, "future_timeout"),
|
||||
group_id=parse_string(request, "future_group_id"),
|
||||
)
|
||||
|
||||
for k, v in ret.items():
|
||||
set_tag(k, v)
|
||||
return 200, ret
|
||||
|
||||
|
||||
# TODO: Needs unit testing
|
||||
class RoomSendFutureRestServlet(TransactionRestServlet):
|
||||
CATEGORY = "Future sending requests"
|
||||
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
super().__init__(hs)
|
||||
self.auth = hs.get_auth()
|
||||
self.futures_handler = hs.get_futures_handler()
|
||||
|
||||
def register(self, http_server: HttpServer) -> None:
|
||||
# /org.matrix.msc4140/rooms/$roomid/send_future/$event_type[/$txn_id]
|
||||
PATTERNS = r"/org\.matrix\.msc4140/rooms/(?P<room_id>[^/]*)/send_future/(?P<event_type>[^/]*)"
|
||||
register_txn_path(self, PATTERNS, http_server, releases=(), v1=False)
|
||||
|
||||
async def _do(
|
||||
self,
|
||||
request: SynapseRequest,
|
||||
requester: Requester,
|
||||
room_id: str,
|
||||
event_type: str,
|
||||
) -> Tuple[int, JsonDict]:
|
||||
ret = await self.futures_handler.add_future(
|
||||
requester,
|
||||
room_id=room_id,
|
||||
event_type=event_type,
|
||||
state_key=None,
|
||||
origin_server_ts=(
|
||||
parse_integer(request, "ts") if requester.app_service else None
|
||||
),
|
||||
content=parse_json_object_from_request(request),
|
||||
timeout=parse_integer(request, "future_timeout"),
|
||||
group_id=parse_string(request, "future_group_id"),
|
||||
)
|
||||
|
||||
for k, v in ret.items():
|
||||
set_tag(k, v)
|
||||
return 200, ret
|
||||
|
||||
async def on_POST(
|
||||
self,
|
||||
request: SynapseRequest,
|
||||
room_id: str,
|
||||
event_type: str,
|
||||
) -> Tuple[int, JsonDict]:
|
||||
requester = await self.auth.get_user_by_req(request, allow_guest=True)
|
||||
return await self._do(request, requester, room_id, event_type)
|
||||
|
||||
async def on_PUT(
|
||||
self, request: SynapseRequest, room_id: str, event_type: str, txn_id: str
|
||||
) -> Tuple[int, JsonDict]:
|
||||
requester = await self.auth.get_user_by_req(request, allow_guest=True)
|
||||
set_tag("txn_id", txn_id)
|
||||
|
||||
return await self.txns.fetch_or_execute_request(
|
||||
request,
|
||||
requester,
|
||||
self._do,
|
||||
request,
|
||||
requester,
|
||||
room_id,
|
||||
event_type,
|
||||
)
|
||||
|
||||
|
||||
# TODO: Remove in favour of the Room{Send,State}FutureRestServlets. Otherwise, this needs unit testing
|
||||
class RoomFutureRestServlet(TransactionRestServlet):
|
||||
CATEGORY = "Future sending requests"
|
||||
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
super().__init__(hs)
|
||||
self.auth = hs.get_auth()
|
||||
self.futures_handler = hs.get_futures_handler()
|
||||
|
||||
def register(self, http_server: HttpServer) -> None:
|
||||
# NOTE: Difference from MSC = remove /send part of path, to avoid ambiguity with regular event sending
|
||||
# /org.matrix.msc4140/rooms/$roomid/future[/$txn_id]
|
||||
PATTERNS = r"/org\.matrix\.msc4140/rooms/(?P<room_id>[^/]*)/future"
|
||||
register_txn_path(self, PATTERNS, http_server, releases=(), v1=False)
|
||||
|
||||
async def _do(
|
||||
self,
|
||||
request: SynapseRequest,
|
||||
requester: Requester,
|
||||
room_id: str,
|
||||
) -> Tuple[int, JsonDict]:
|
||||
body = parse_json_object_from_request(request)
|
||||
|
||||
try:
|
||||
timeout = int(body["timeout"])
|
||||
except KeyError:
|
||||
raise SynapseError(400, "'timeout' is missing", Codes.MISSING_PARAM)
|
||||
except Exception:
|
||||
raise SynapseError(400, "'timeout' is not an integer", Codes.INVALID_PARAM)
|
||||
|
||||
try:
|
||||
timeout_item: JsonDict = {
|
||||
str(k): v for k, v in body["send_on_timeout"].items()
|
||||
}
|
||||
except KeyError:
|
||||
raise SynapseError(400, "'send_on_timeout' is missing", Codes.MISSING_PARAM)
|
||||
except Exception:
|
||||
raise InvalidAPICallError("'send_on_timeout' is not valid JSON")
|
||||
|
||||
try:
|
||||
send_on_action: Dict[str, JsonDict] = {
|
||||
str(action_name): {str(k): v for k, v in action_item.items()}
|
||||
for action_name, action_item in body.get("send_on_action", {}).items()
|
||||
}
|
||||
except Exception:
|
||||
raise InvalidAPICallError(
|
||||
"'send_on_action' is not a JSON mapping of action names to event content"
|
||||
)
|
||||
|
||||
# TODO: Should action events be able to have a different ts value than the timeout event?
|
||||
origin_server_ts = (
|
||||
parse_integer(request, "ts") if requester.app_service else None
|
||||
)
|
||||
|
||||
# TODO: send_now. But first validate send_on_timeout & send_on_action
|
||||
|
||||
send_on_timeout_resp = await self.futures_handler.add_future(
|
||||
requester,
|
||||
room_id=room_id,
|
||||
event_type=str(timeout_item["type"]),
|
||||
state_key=(
|
||||
str(timeout_item["state_key"]) if "state_key" in timeout_item else None
|
||||
),
|
||||
origin_server_ts=origin_server_ts,
|
||||
content=timeout_item["content"],
|
||||
timeout=timeout,
|
||||
group_id=None,
|
||||
is_custom_endpoint=True,
|
||||
)
|
||||
group_id = str(send_on_timeout_resp.pop("future_group_id"))
|
||||
ret = {"send_on_timeout": send_on_timeout_resp}
|
||||
|
||||
if send_on_action:
|
||||
send_on_action_resp: Dict[str, Dict[str, JsonDict]] = {}
|
||||
for action_name, action_item in send_on_action.items():
|
||||
action_item_resp = await self.futures_handler.add_future(
|
||||
requester,
|
||||
room_id=room_id,
|
||||
event_type=str(action_item["type"]),
|
||||
state_key=(
|
||||
str(action_item["state_key"])
|
||||
if "state_key" in action_item
|
||||
else None
|
||||
),
|
||||
origin_server_ts=origin_server_ts,
|
||||
content=action_item["content"],
|
||||
timeout=None,
|
||||
group_id=group_id,
|
||||
is_custom_endpoint=True,
|
||||
)
|
||||
try:
|
||||
del action_item_resp["future_group_id"]
|
||||
except KeyError:
|
||||
pass
|
||||
send_on_action_resp[action_name] = action_item_resp
|
||||
|
||||
ret["send_on_action"] = send_on_action_resp
|
||||
|
||||
return 200, ret
|
||||
|
||||
async def on_POST(
|
||||
self,
|
||||
request: SynapseRequest,
|
||||
room_id: str,
|
||||
) -> Tuple[int, JsonDict]:
|
||||
requester = await self.auth.get_user_by_req(request, allow_guest=True)
|
||||
return await self._do(request, requester, room_id)
|
||||
|
||||
async def on_PUT(
|
||||
self, request: SynapseRequest, room_id: str, txn_id: str
|
||||
) -> Tuple[int, JsonDict]:
|
||||
requester = await self.auth.get_user_by_req(request, allow_guest=True)
|
||||
set_tag("txn_id", txn_id)
|
||||
|
||||
return await self.txns.fetch_or_execute_request(
|
||||
request,
|
||||
requester,
|
||||
self._do,
|
||||
request,
|
||||
requester,
|
||||
room_id,
|
||||
)
|
||||
|
||||
|
||||
# TODO: Needs unit testing for room ID + alias joins
|
||||
class JoinRoomAliasServlet(ResolveRoomIdMixin, TransactionRestServlet):
|
||||
CATEGORY = "Event sending requests"
|
||||
|
@ -510,7 +773,7 @@ class PublicRoomListRestServlet(RestServlet):
|
|||
if server:
|
||||
raise e
|
||||
|
||||
limit: Optional[int] = parse_integer(request, "limit", 0, negative=False)
|
||||
limit: Optional[int] = parse_integer(request, "limit", 0)
|
||||
since_token = parse_string(request, "since")
|
||||
|
||||
if limit == 0:
|
||||
|
@ -1120,6 +1383,20 @@ class RoomRedactEventRestServlet(TransactionRestServlet):
|
|||
) -> Tuple[int, JsonDict]:
|
||||
content = parse_json_object_from_request(request)
|
||||
|
||||
requester_suspended = await self._store.get_user_suspended_status(
|
||||
requester.user.to_string()
|
||||
)
|
||||
|
||||
if requester_suspended:
|
||||
event = await self._store.get_event(event_id, allow_none=True)
|
||||
if event:
|
||||
if event.sender != requester.user.to_string():
|
||||
raise SynapseError(
|
||||
403,
|
||||
"You can only redact your own events while account is suspended.",
|
||||
Codes.USER_ACCOUNT_SUSPENDED,
|
||||
)
|
||||
|
||||
# Ensure the redacts property in the content matches the one provided in
|
||||
# the URL.
|
||||
room_version = await self._store.get_room_version(room_id)
|
||||
|
@ -1330,6 +1607,9 @@ def register_txn_path(
|
|||
servlet: RestServlet,
|
||||
regex_string: str,
|
||||
http_server: HttpServer,
|
||||
releases: StrCollection = ("r0", "v3"),
|
||||
unstable: bool = True,
|
||||
v1: bool = True,
|
||||
) -> None:
|
||||
"""Registers a transaction-based path.
|
||||
|
||||
|
@ -1348,13 +1628,13 @@ def register_txn_path(
|
|||
raise RuntimeError("on_POST and on_PUT must exist when using register_txn_path")
|
||||
http_server.register_paths(
|
||||
"POST",
|
||||
client_patterns(regex_string + "$", v1=True),
|
||||
client_patterns(regex_string + "$", releases, unstable, v1),
|
||||
on_POST,
|
||||
servlet.__class__.__name__,
|
||||
)
|
||||
http_server.register_paths(
|
||||
"PUT",
|
||||
client_patterns(regex_string + "/(?P<txn_id>[^/]*)$", v1=True),
|
||||
client_patterns(regex_string + "/(?P<txn_id>[^/]*)$", releases, unstable, v1),
|
||||
on_PUT,
|
||||
servlet.__class__.__name__,
|
||||
)
|
||||
|
@ -1430,16 +1710,7 @@ class RoomHierarchyRestServlet(RestServlet):
|
|||
requester = await self._auth.get_user_by_req(request, allow_guest=True)
|
||||
|
||||
max_depth = parse_integer(request, "max_depth")
|
||||
if max_depth is not None and max_depth < 0:
|
||||
raise SynapseError(
|
||||
400, "'max_depth' must be a non-negative integer", Codes.BAD_JSON
|
||||
)
|
||||
|
||||
limit = parse_integer(request, "limit")
|
||||
if limit is not None and limit <= 0:
|
||||
raise SynapseError(
|
||||
400, "'limit' must be a positive integer", Codes.BAD_JSON
|
||||
)
|
||||
|
||||
return 200, await self._room_summary_handler.get_room_hierarchy(
|
||||
requester,
|
||||
|
@ -1498,12 +1769,14 @@ class RoomSummaryRestServlet(ResolveRoomIdMixin, RestServlet):
|
|||
|
||||
def register_servlets(hs: "HomeServer", http_server: HttpServer) -> None:
|
||||
RoomStateEventRestServlet(hs).register(http_server)
|
||||
RoomStateFutureRestServlet(hs).register(http_server)
|
||||
RoomMemberListRestServlet(hs).register(http_server)
|
||||
JoinedRoomMemberListRestServlet(hs).register(http_server)
|
||||
RoomMessageListRestServlet(hs).register(http_server)
|
||||
JoinRoomAliasServlet(hs).register(http_server)
|
||||
RoomMembershipRestServlet(hs).register(http_server)
|
||||
RoomSendEventRestServlet(hs).register(http_server)
|
||||
RoomSendFutureRestServlet(hs).register(http_server)
|
||||
PublicRoomListRestServlet(hs).register(http_server)
|
||||
RoomStateRestServlet(hs).register(http_server)
|
||||
RoomRedactEventRestServlet(hs).register(http_server)
|
||||
|
@ -1518,6 +1791,7 @@ def register_servlets(hs: "HomeServer", http_server: HttpServer) -> None:
|
|||
SearchRestServlet(hs).register(http_server)
|
||||
RoomCreateRestServlet(hs).register(http_server)
|
||||
TimestampLookupRestServlet(hs).register(http_server)
|
||||
RoomFutureRestServlet(hs).register(http_server)
|
||||
|
||||
# Some servlets only get registered for the main process.
|
||||
if hs.config.worker.worker_app is None:
|
||||
|
|
|
@ -864,7 +864,7 @@ class SlidingSyncRestServlet(RestServlet):
|
|||
"""
|
||||
|
||||
PATTERNS = client_patterns(
|
||||
"/org.matrix.msc3575/sync$", releases=[], v1=False, unstable=True
|
||||
"/org.matrix.simplified_msc3575/sync$", releases=[], v1=False, unstable=True
|
||||
)
|
||||
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
|
|
|
@ -149,6 +149,8 @@ class VersionsRestServlet(RestServlet):
|
|||
is not None
|
||||
)
|
||||
),
|
||||
# MSC4140: Delayed events (Futures)
|
||||
"org.matrix.msc4140": True,
|
||||
# MSC4151: Report room API (Client-Server API)
|
||||
"org.matrix.msc4151": self.config.experimental.msc4151_enabled,
|
||||
},
|
||||
|
|
|
@ -76,6 +76,7 @@ from synapse.handlers.event_auth import EventAuthHandler
|
|||
from synapse.handlers.events import EventHandler, EventStreamHandler
|
||||
from synapse.handlers.federation import FederationHandler
|
||||
from synapse.handlers.federation_event import FederationEventHandler
|
||||
from synapse.handlers.futures import FuturesHandler
|
||||
from synapse.handlers.identity import IdentityHandler
|
||||
from synapse.handlers.initial_sync import InitialSyncHandler
|
||||
from synapse.handlers.message import EventCreationHandler, MessageHandler
|
||||
|
@ -248,6 +249,7 @@ class HomeServer(metaclass=abc.ABCMeta):
|
|||
"account_validity",
|
||||
"auth",
|
||||
"deactivate_account",
|
||||
"futures",
|
||||
"message",
|
||||
"pagination",
|
||||
"profile",
|
||||
|
@ -936,3 +938,7 @@ class HomeServer(metaclass=abc.ABCMeta):
|
|||
@cache_in_self
|
||||
def get_task_scheduler(self) -> TaskScheduler:
|
||||
return TaskScheduler(self)
|
||||
|
||||
@cache_in_self
|
||||
def get_futures_handler(self) -> FuturesHandler:
|
||||
return FuturesHandler(self)
|
||||
|
|
|
@ -617,6 +617,17 @@ class EventsPersistenceStorageController:
|
|||
room_id, chunk
|
||||
)
|
||||
|
||||
with Measure(self._clock, "calculate_chain_cover_index_for_events"):
|
||||
# We now calculate chain ID/sequence numbers for any state events we're
|
||||
# persisting. We ignore out of band memberships as we're not in the room
|
||||
# and won't have their auth chain (we'll fix it up later if we join the
|
||||
# room).
|
||||
#
|
||||
# See: docs/auth_chain_difference_algorithm.md
|
||||
new_event_links = await self.persist_events_store.calculate_chain_cover_index_for_events(
|
||||
room_id, [e for e, _ in chunk]
|
||||
)
|
||||
|
||||
await self.persist_events_store._persist_events_and_state_updates(
|
||||
room_id,
|
||||
chunk,
|
||||
|
@ -624,6 +635,7 @@ class EventsPersistenceStorageController:
|
|||
new_forward_extremities=new_forward_extremities,
|
||||
use_negative_stream_ordering=backfilled,
|
||||
inhibit_local_membership_updates=backfilled,
|
||||
new_event_links=new_event_links,
|
||||
)
|
||||
|
||||
return replaced_events
|
||||
|
|
|
@ -54,6 +54,7 @@ from .events_bg_updates import EventsBackgroundUpdatesStore
|
|||
from .events_forward_extremities import EventForwardExtremitiesStore
|
||||
from .experimental_features import ExperimentalFeaturesStore
|
||||
from .filtering import FilteringWorkerStore
|
||||
from .futures import FuturesStore
|
||||
from .keys import KeyStore
|
||||
from .lock import LockStore
|
||||
from .media_repository import MediaRepositoryStore
|
||||
|
@ -156,6 +157,7 @@ class DataStore(
|
|||
LockStore,
|
||||
SessionStore,
|
||||
TaskSchedulerWorkerStore,
|
||||
FuturesStore,
|
||||
):
|
||||
def __init__(
|
||||
self,
|
||||
|
|
|
@ -825,14 +825,13 @@ class DeviceInboxWorkerStore(SQLBaseStore):
|
|||
# Check if we've already inserted a matching message_id for that
|
||||
# origin. This can happen if the origin doesn't receive our
|
||||
# acknowledgement from the first time we received the message.
|
||||
already_inserted = self.db_pool.simple_select_one_txn(
|
||||
already_inserted = self.db_pool.simple_select_list_txn(
|
||||
txn,
|
||||
table="device_federation_inbox",
|
||||
keyvalues={"origin": origin, "message_id": message_id},
|
||||
retcols=("message_id",),
|
||||
allow_none=True,
|
||||
)
|
||||
if already_inserted is not None:
|
||||
if already_inserted:
|
||||
return
|
||||
|
||||
# Add an entry for this message_id so that we know we've processed
|
||||
|
|
|
@ -164,22 +164,24 @@ class DeviceWorkerStore(RoomMemberWorkerStore, EndToEndKeyWorkerStore):
|
|||
prefilled_cache=user_signature_stream_prefill,
|
||||
)
|
||||
|
||||
(
|
||||
device_list_federation_prefill,
|
||||
device_list_federation_list_id,
|
||||
) = self.db_pool.get_cache_dict(
|
||||
db_conn,
|
||||
"device_lists_outbound_pokes",
|
||||
entity_column="destination",
|
||||
stream_column="stream_id",
|
||||
max_value=device_list_max,
|
||||
limit=10000,
|
||||
)
|
||||
self._device_list_federation_stream_cache = StreamChangeCache(
|
||||
"DeviceListFederationStreamChangeCache",
|
||||
device_list_federation_list_id,
|
||||
prefilled_cache=device_list_federation_prefill,
|
||||
)
|
||||
self._device_list_federation_stream_cache = None
|
||||
if hs.should_send_federation():
|
||||
(
|
||||
device_list_federation_prefill,
|
||||
device_list_federation_list_id,
|
||||
) = self.db_pool.get_cache_dict(
|
||||
db_conn,
|
||||
"device_lists_outbound_pokes",
|
||||
entity_column="destination",
|
||||
stream_column="stream_id",
|
||||
max_value=device_list_max,
|
||||
limit=10000,
|
||||
)
|
||||
self._device_list_federation_stream_cache = StreamChangeCache(
|
||||
"DeviceListFederationStreamChangeCache",
|
||||
device_list_federation_list_id,
|
||||
prefilled_cache=device_list_federation_prefill,
|
||||
)
|
||||
|
||||
if hs.config.worker.run_background_tasks:
|
||||
self._clock.looping_call(
|
||||
|
@ -207,23 +209,30 @@ class DeviceWorkerStore(RoomMemberWorkerStore, EndToEndKeyWorkerStore):
|
|||
) -> None:
|
||||
for row in rows:
|
||||
if row.is_signature:
|
||||
self._user_signature_stream_cache.entity_has_changed(row.entity, token)
|
||||
self._user_signature_stream_cache.entity_has_changed(row.user_id, token)
|
||||
continue
|
||||
|
||||
# The entities are either user IDs (starting with '@') whose devices
|
||||
# have changed, or remote servers that we need to tell about
|
||||
# changes.
|
||||
if row.entity.startswith("@"):
|
||||
self._device_list_stream_cache.entity_has_changed(row.entity, token)
|
||||
self.get_cached_devices_for_user.invalidate((row.entity,))
|
||||
self._get_cached_user_device.invalidate((row.entity,))
|
||||
self.get_device_list_last_stream_id_for_remote.invalidate((row.entity,))
|
||||
|
||||
else:
|
||||
self._device_list_federation_stream_cache.entity_has_changed(
|
||||
row.entity, token
|
||||
if not row.hosts_calculated:
|
||||
self._device_list_stream_cache.entity_has_changed(row.user_id, token)
|
||||
self.get_cached_devices_for_user.invalidate((row.user_id,))
|
||||
self._get_cached_user_device.invalidate((row.user_id,))
|
||||
self.get_device_list_last_stream_id_for_remote.invalidate(
|
||||
(row.user_id,)
|
||||
)
|
||||
|
||||
def device_lists_outbound_pokes_have_changed(
|
||||
self, destinations: StrCollection, token: int
|
||||
) -> None:
|
||||
assert self._device_list_federation_stream_cache is not None
|
||||
|
||||
for destination in destinations:
|
||||
self._device_list_federation_stream_cache.entity_has_changed(
|
||||
destination, token
|
||||
)
|
||||
|
||||
def device_lists_in_rooms_have_changed(
|
||||
self, room_ids: StrCollection, token: int
|
||||
) -> None:
|
||||
|
@ -363,6 +372,11 @@ class DeviceWorkerStore(RoomMemberWorkerStore, EndToEndKeyWorkerStore):
|
|||
EDU contents.
|
||||
"""
|
||||
now_stream_id = self.get_device_stream_token()
|
||||
if from_stream_id == now_stream_id:
|
||||
return now_stream_id, []
|
||||
|
||||
if self._device_list_federation_stream_cache is None:
|
||||
raise Exception("Func can only be used on federation senders")
|
||||
|
||||
has_changed = self._device_list_federation_stream_cache.has_entity_changed(
|
||||
destination, int(from_stream_id)
|
||||
|
@ -1018,10 +1032,10 @@ class DeviceWorkerStore(RoomMemberWorkerStore, EndToEndKeyWorkerStore):
|
|||
# This query Does The Right Thing where it'll correctly apply the
|
||||
# bounds to the inner queries.
|
||||
sql = """
|
||||
SELECT stream_id, entity FROM (
|
||||
SELECT stream_id, user_id AS entity FROM device_lists_stream
|
||||
SELECT stream_id, user_id, hosts FROM (
|
||||
SELECT stream_id, user_id, false AS hosts FROM device_lists_stream
|
||||
UNION ALL
|
||||
SELECT stream_id, destination AS entity FROM device_lists_outbound_pokes
|
||||
SELECT DISTINCT stream_id, user_id, true AS hosts FROM device_lists_outbound_pokes
|
||||
) AS e
|
||||
WHERE ? < stream_id AND stream_id <= ?
|
||||
ORDER BY stream_id ASC
|
||||
|
@ -1577,6 +1591,14 @@ class DeviceWorkerStore(RoomMemberWorkerStore, EndToEndKeyWorkerStore):
|
|||
get_device_list_changes_in_room_txn,
|
||||
)
|
||||
|
||||
async def get_destinations_for_device(self, stream_id: int) -> StrCollection:
|
||||
return await self.db_pool.simple_select_onecol(
|
||||
table="device_lists_outbound_pokes",
|
||||
keyvalues={"stream_id": stream_id},
|
||||
retcol="destination",
|
||||
desc="get_destinations_for_device",
|
||||
)
|
||||
|
||||
|
||||
class DeviceBackgroundUpdateStore(SQLBaseStore):
|
||||
def __init__(
|
||||
|
@ -2109,18 +2131,18 @@ class DeviceStore(DeviceWorkerStore, DeviceBackgroundUpdateStore):
|
|||
user_id: str,
|
||||
device_id: str,
|
||||
hosts: Collection[str],
|
||||
stream_ids: List[int],
|
||||
stream_id: int,
|
||||
context: Optional[Dict[str, str]],
|
||||
) -> None:
|
||||
for host in hosts:
|
||||
txn.call_after(
|
||||
self._device_list_federation_stream_cache.entity_has_changed,
|
||||
host,
|
||||
stream_ids[-1],
|
||||
)
|
||||
if self._device_list_federation_stream_cache:
|
||||
for host in hosts:
|
||||
txn.call_after(
|
||||
self._device_list_federation_stream_cache.entity_has_changed,
|
||||
host,
|
||||
stream_id,
|
||||
)
|
||||
|
||||
now = self._clock.time_msec()
|
||||
stream_id_iterator = iter(stream_ids)
|
||||
|
||||
encoded_context = json_encoder.encode(context)
|
||||
mark_sent = not self.hs.is_mine_id(user_id)
|
||||
|
@ -2129,7 +2151,7 @@ class DeviceStore(DeviceWorkerStore, DeviceBackgroundUpdateStore):
|
|||
(
|
||||
destination,
|
||||
self._instance_name,
|
||||
next(stream_id_iterator),
|
||||
stream_id,
|
||||
user_id,
|
||||
device_id,
|
||||
mark_sent,
|
||||
|
@ -2314,22 +2336,22 @@ class DeviceStore(DeviceWorkerStore, DeviceBackgroundUpdateStore):
|
|||
return
|
||||
|
||||
def add_device_list_outbound_pokes_txn(
|
||||
txn: LoggingTransaction, stream_ids: List[int]
|
||||
txn: LoggingTransaction, stream_id: int
|
||||
) -> None:
|
||||
self._add_device_outbound_poke_to_stream_txn(
|
||||
txn,
|
||||
user_id=user_id,
|
||||
device_id=device_id,
|
||||
hosts=hosts,
|
||||
stream_ids=stream_ids,
|
||||
stream_id=stream_id,
|
||||
context=context,
|
||||
)
|
||||
|
||||
async with self._device_list_id_gen.get_next_mult(len(hosts)) as stream_ids:
|
||||
async with self._device_list_id_gen.get_next() as stream_id:
|
||||
return await self.db_pool.runInteraction(
|
||||
"add_device_list_outbound_pokes",
|
||||
add_device_list_outbound_pokes_txn,
|
||||
stream_ids,
|
||||
stream_id,
|
||||
)
|
||||
|
||||
async def add_remote_device_list_to_pending(
|
||||
|
|
|
@ -123,9 +123,9 @@ class EndToEndKeyWorkerStore(EndToEndKeyBackgroundStore, CacheInvalidationWorker
|
|||
if stream_name == DeviceListsStream.NAME:
|
||||
for row in rows:
|
||||
assert isinstance(row, DeviceListsStream.DeviceListsStreamRow)
|
||||
if row.entity.startswith("@"):
|
||||
if not row.hosts_calculated:
|
||||
self._get_e2e_device_keys_for_federation_query_inner.invalidate(
|
||||
(row.entity,)
|
||||
(row.user_id,)
|
||||
)
|
||||
|
||||
super().process_replication_rows(stream_name, instance_name, token, rows)
|
||||
|
|
|
@ -148,6 +148,10 @@ class EventFederationWorkerStore(SignatureWorkerStore, EventsWorkerStore, SQLBas
|
|||
500000, "_event_auth_cache", size_callback=len
|
||||
)
|
||||
|
||||
# Flag used by unit tests to disable fallback when there is no chain cover
|
||||
# index.
|
||||
self.tests_allow_no_chain_cover_index = True
|
||||
|
||||
self._clock.looping_call(self._get_stats_for_federation_staging, 30 * 1000)
|
||||
|
||||
if isinstance(self.database_engine, PostgresEngine):
|
||||
|
@ -220,8 +224,10 @@ class EventFederationWorkerStore(SignatureWorkerStore, EventsWorkerStore, SQLBas
|
|||
)
|
||||
except _NoChainCoverIndex:
|
||||
# For whatever reason we don't actually have a chain cover index
|
||||
# for the events in question, so we fall back to the old method.
|
||||
pass
|
||||
# for the events in question, so we fall back to the old method
|
||||
# (except in tests)
|
||||
if not self.tests_allow_no_chain_cover_index:
|
||||
raise
|
||||
|
||||
return await self.db_pool.runInteraction(
|
||||
"get_auth_chain_ids",
|
||||
|
@ -271,7 +277,7 @@ class EventFederationWorkerStore(SignatureWorkerStore, EventsWorkerStore, SQLBas
|
|||
if events_missing_chain_info:
|
||||
# This can happen due to e.g. downgrade/upgrade of the server. We
|
||||
# raise an exception and fall back to the previous algorithm.
|
||||
logger.info(
|
||||
logger.error(
|
||||
"Unexpectedly found that events don't have chain IDs in room %s: %s",
|
||||
room_id,
|
||||
events_missing_chain_info,
|
||||
|
@ -482,8 +488,10 @@ class EventFederationWorkerStore(SignatureWorkerStore, EventsWorkerStore, SQLBas
|
|||
)
|
||||
except _NoChainCoverIndex:
|
||||
# For whatever reason we don't actually have a chain cover index
|
||||
# for the events in question, so we fall back to the old method.
|
||||
pass
|
||||
# for the events in question, so we fall back to the old method
|
||||
# (except in tests)
|
||||
if not self.tests_allow_no_chain_cover_index:
|
||||
raise
|
||||
|
||||
return await self.db_pool.runInteraction(
|
||||
"get_auth_chain_difference",
|
||||
|
@ -710,7 +718,7 @@ class EventFederationWorkerStore(SignatureWorkerStore, EventsWorkerStore, SQLBas
|
|||
if events_missing_chain_info - event_to_auth_ids.keys():
|
||||
# Uh oh, we somehow haven't correctly done the chain cover index,
|
||||
# bail and fall back to the old method.
|
||||
logger.info(
|
||||
logger.error(
|
||||
"Unexpectedly found that events don't have chain IDs in room %s: %s",
|
||||
room_id,
|
||||
events_missing_chain_info - event_to_auth_ids.keys(),
|
||||
|
|
|
@ -1829,7 +1829,7 @@ class EventPushActionsWorkerStore(ReceiptsWorkerStore, StreamWorkerStore, SQLBas
|
|||
async def get_push_actions_for_user(
|
||||
self,
|
||||
user_id: str,
|
||||
before: Optional[str] = None,
|
||||
before: Optional[int] = None,
|
||||
limit: int = 50,
|
||||
only_highlight: bool = False,
|
||||
) -> List[UserPushAction]:
|
||||
|
|
|
@ -34,7 +34,6 @@ from typing import (
|
|||
Optional,
|
||||
Set,
|
||||
Tuple,
|
||||
Union,
|
||||
cast,
|
||||
)
|
||||
|
||||
|
@ -100,6 +99,23 @@ class DeltaState:
|
|||
return not self.to_delete and not self.to_insert and not self.no_longer_in_room
|
||||
|
||||
|
||||
@attr.s(slots=True, auto_attribs=True)
|
||||
class NewEventChainLinks:
|
||||
"""Information about new auth chain links that need to be added to the DB.
|
||||
|
||||
Attributes:
|
||||
chain_id, sequence_number: the IDs corresponding to the event being
|
||||
inserted, and the starting point of the links
|
||||
links: Lists the links that need to be added, 2-tuple of the chain
|
||||
ID/sequence number of the end point of the link.
|
||||
"""
|
||||
|
||||
chain_id: int
|
||||
sequence_number: int
|
||||
|
||||
links: List[Tuple[int, int]] = attr.Factory(list)
|
||||
|
||||
|
||||
class PersistEventsStore:
|
||||
"""Contains all the functions for writing events to the database.
|
||||
|
||||
|
@ -148,6 +164,7 @@ class PersistEventsStore:
|
|||
*,
|
||||
state_delta_for_room: Optional[DeltaState],
|
||||
new_forward_extremities: Optional[Set[str]],
|
||||
new_event_links: Dict[str, NewEventChainLinks],
|
||||
use_negative_stream_ordering: bool = False,
|
||||
inhibit_local_membership_updates: bool = False,
|
||||
) -> None:
|
||||
|
@ -217,6 +234,7 @@ class PersistEventsStore:
|
|||
inhibit_local_membership_updates=inhibit_local_membership_updates,
|
||||
state_delta_for_room=state_delta_for_room,
|
||||
new_forward_extremities=new_forward_extremities,
|
||||
new_event_links=new_event_links,
|
||||
)
|
||||
persist_event_counter.inc(len(events_and_contexts))
|
||||
|
||||
|
@ -243,6 +261,87 @@ class PersistEventsStore:
|
|||
(room_id,), frozenset(new_forward_extremities)
|
||||
)
|
||||
|
||||
async def calculate_chain_cover_index_for_events(
|
||||
self, room_id: str, events: Collection[EventBase]
|
||||
) -> Dict[str, NewEventChainLinks]:
|
||||
# Filter to state events, and ensure there are no duplicates.
|
||||
state_events = []
|
||||
seen_events = set()
|
||||
for event in events:
|
||||
if not event.is_state() or event.event_id in seen_events:
|
||||
continue
|
||||
|
||||
state_events.append(event)
|
||||
seen_events.add(event.event_id)
|
||||
|
||||
if not state_events:
|
||||
return {}
|
||||
|
||||
return await self.db_pool.runInteraction(
|
||||
"_calculate_chain_cover_index_for_events",
|
||||
self.calculate_chain_cover_index_for_events_txn,
|
||||
room_id,
|
||||
state_events,
|
||||
)
|
||||
|
||||
def calculate_chain_cover_index_for_events_txn(
|
||||
self, txn: LoggingTransaction, room_id: str, state_events: Collection[EventBase]
|
||||
) -> Dict[str, NewEventChainLinks]:
|
||||
# We now calculate chain ID/sequence numbers for any state events we're
|
||||
# persisting. We ignore out of band memberships as we're not in the room
|
||||
# and won't have their auth chain (we'll fix it up later if we join the
|
||||
# room).
|
||||
#
|
||||
# See: docs/auth_chain_difference_algorithm.md
|
||||
|
||||
# We ignore legacy rooms that we aren't filling the chain cover index
|
||||
# for.
|
||||
row = self.db_pool.simple_select_one_txn(
|
||||
txn,
|
||||
table="rooms",
|
||||
keyvalues={"room_id": room_id},
|
||||
retcols=("room_id", "has_auth_chain_index"),
|
||||
allow_none=True,
|
||||
)
|
||||
if row is None or row[1] is False:
|
||||
return {}
|
||||
|
||||
# Filter out events that we've already calculated.
|
||||
rows = self.db_pool.simple_select_many_txn(
|
||||
txn,
|
||||
table="event_auth_chains",
|
||||
column="event_id",
|
||||
iterable=[e.event_id for e in state_events],
|
||||
keyvalues={},
|
||||
retcols=("event_id",),
|
||||
)
|
||||
already_persisted_events = {event_id for event_id, in rows}
|
||||
state_events = [
|
||||
event
|
||||
for event in state_events
|
||||
if event.event_id not in already_persisted_events
|
||||
]
|
||||
|
||||
if not state_events:
|
||||
return {}
|
||||
|
||||
# We need to know the type/state_key and auth events of the events we're
|
||||
# calculating chain IDs for. We don't rely on having the full Event
|
||||
# instances as we'll potentially be pulling more events from the DB and
|
||||
# we don't need the overhead of fetching/parsing the full event JSON.
|
||||
event_to_types = {e.event_id: (e.type, e.state_key) for e in state_events}
|
||||
event_to_auth_chain = {e.event_id: e.auth_event_ids() for e in state_events}
|
||||
event_to_room_id = {e.event_id: e.room_id for e in state_events}
|
||||
|
||||
return self._calculate_chain_cover_index(
|
||||
txn,
|
||||
self.db_pool,
|
||||
self.store.event_chain_id_gen,
|
||||
event_to_room_id,
|
||||
event_to_types,
|
||||
event_to_auth_chain,
|
||||
)
|
||||
|
||||
async def _get_events_which_are_prevs(self, event_ids: Iterable[str]) -> List[str]:
|
||||
"""Filter the supplied list of event_ids to get those which are prev_events of
|
||||
existing (non-outlier/rejected) events.
|
||||
|
@ -358,6 +457,7 @@ class PersistEventsStore:
|
|||
inhibit_local_membership_updates: bool,
|
||||
state_delta_for_room: Optional[DeltaState],
|
||||
new_forward_extremities: Optional[Set[str]],
|
||||
new_event_links: Dict[str, NewEventChainLinks],
|
||||
) -> None:
|
||||
"""Insert some number of room events into the necessary database tables.
|
||||
|
||||
|
@ -466,7 +566,9 @@ class PersistEventsStore:
|
|||
# Insert into event_to_state_groups.
|
||||
self._store_event_state_mappings_txn(txn, events_and_contexts)
|
||||
|
||||
self._persist_event_auth_chain_txn(txn, [e for e, _ in events_and_contexts])
|
||||
self._persist_event_auth_chain_txn(
|
||||
txn, [e for e, _ in events_and_contexts], new_event_links
|
||||
)
|
||||
|
||||
# _store_rejected_events_txn filters out any events which were
|
||||
# rejected, and returns the filtered list.
|
||||
|
@ -496,7 +598,11 @@ class PersistEventsStore:
|
|||
self,
|
||||
txn: LoggingTransaction,
|
||||
events: List[EventBase],
|
||||
new_event_links: Dict[str, NewEventChainLinks],
|
||||
) -> None:
|
||||
if new_event_links:
|
||||
self._persist_chain_cover_index(txn, self.db_pool, new_event_links)
|
||||
|
||||
# We only care about state events, so this if there are no state events.
|
||||
if not any(e.is_state() for e in events):
|
||||
return
|
||||
|
@ -519,60 +625,6 @@ class PersistEventsStore:
|
|||
],
|
||||
)
|
||||
|
||||
# We now calculate chain ID/sequence numbers for any state events we're
|
||||
# persisting. We ignore out of band memberships as we're not in the room
|
||||
# and won't have their auth chain (we'll fix it up later if we join the
|
||||
# room).
|
||||
#
|
||||
# See: docs/auth_chain_difference_algorithm.md
|
||||
|
||||
# We ignore legacy rooms that we aren't filling the chain cover index
|
||||
# for.
|
||||
rows = cast(
|
||||
List[Tuple[str, Optional[Union[int, bool]]]],
|
||||
self.db_pool.simple_select_many_txn(
|
||||
txn,
|
||||
table="rooms",
|
||||
column="room_id",
|
||||
iterable={event.room_id for event in events if event.is_state()},
|
||||
keyvalues={},
|
||||
retcols=("room_id", "has_auth_chain_index"),
|
||||
),
|
||||
)
|
||||
rooms_using_chain_index = {
|
||||
room_id for room_id, has_auth_chain_index in rows if has_auth_chain_index
|
||||
}
|
||||
|
||||
state_events = {
|
||||
event.event_id: event
|
||||
for event in events
|
||||
if event.is_state() and event.room_id in rooms_using_chain_index
|
||||
}
|
||||
|
||||
if not state_events:
|
||||
return
|
||||
|
||||
# We need to know the type/state_key and auth events of the events we're
|
||||
# calculating chain IDs for. We don't rely on having the full Event
|
||||
# instances as we'll potentially be pulling more events from the DB and
|
||||
# we don't need the overhead of fetching/parsing the full event JSON.
|
||||
event_to_types = {
|
||||
e.event_id: (e.type, e.state_key) for e in state_events.values()
|
||||
}
|
||||
event_to_auth_chain = {
|
||||
e.event_id: e.auth_event_ids() for e in state_events.values()
|
||||
}
|
||||
event_to_room_id = {e.event_id: e.room_id for e in state_events.values()}
|
||||
|
||||
self._add_chain_cover_index(
|
||||
txn,
|
||||
self.db_pool,
|
||||
self.store.event_chain_id_gen,
|
||||
event_to_room_id,
|
||||
event_to_types,
|
||||
event_to_auth_chain,
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def _add_chain_cover_index(
|
||||
cls,
|
||||
|
@ -583,6 +635,35 @@ class PersistEventsStore:
|
|||
event_to_types: Dict[str, Tuple[str, str]],
|
||||
event_to_auth_chain: Dict[str, StrCollection],
|
||||
) -> None:
|
||||
"""Calculate and persist the chain cover index for the given events.
|
||||
|
||||
Args:
|
||||
event_to_room_id: Event ID to the room ID of the event
|
||||
event_to_types: Event ID to type and state_key of the event
|
||||
event_to_auth_chain: Event ID to list of auth event IDs of the
|
||||
event (events with no auth events can be excluded).
|
||||
"""
|
||||
|
||||
new_event_links = cls._calculate_chain_cover_index(
|
||||
txn,
|
||||
db_pool,
|
||||
event_chain_id_gen,
|
||||
event_to_room_id,
|
||||
event_to_types,
|
||||
event_to_auth_chain,
|
||||
)
|
||||
cls._persist_chain_cover_index(txn, db_pool, new_event_links)
|
||||
|
||||
@classmethod
|
||||
def _calculate_chain_cover_index(
|
||||
cls,
|
||||
txn: LoggingTransaction,
|
||||
db_pool: DatabasePool,
|
||||
event_chain_id_gen: SequenceGenerator,
|
||||
event_to_room_id: Dict[str, str],
|
||||
event_to_types: Dict[str, Tuple[str, str]],
|
||||
event_to_auth_chain: Dict[str, StrCollection],
|
||||
) -> Dict[str, NewEventChainLinks]:
|
||||
"""Calculate the chain cover index for the given events.
|
||||
|
||||
Args:
|
||||
|
@ -590,6 +671,10 @@ class PersistEventsStore:
|
|||
event_to_types: Event ID to type and state_key of the event
|
||||
event_to_auth_chain: Event ID to list of auth event IDs of the
|
||||
event (events with no auth events can be excluded).
|
||||
|
||||
Returns:
|
||||
A mapping with any new auth chain links we need to add, keyed by
|
||||
event ID.
|
||||
"""
|
||||
|
||||
# Map from event ID to chain ID/sequence number.
|
||||
|
@ -708,11 +793,11 @@ class PersistEventsStore:
|
|||
room_id = event_to_room_id.get(event_id)
|
||||
if room_id:
|
||||
e_type, state_key = event_to_types[event_id]
|
||||
db_pool.simple_insert_txn(
|
||||
db_pool.simple_upsert_txn(
|
||||
txn,
|
||||
table="event_auth_chain_to_calculate",
|
||||
keyvalues={"event_id": event_id},
|
||||
values={
|
||||
"event_id": event_id,
|
||||
"room_id": room_id,
|
||||
"type": e_type,
|
||||
"state_key": state_key,
|
||||
|
@ -724,7 +809,7 @@ class PersistEventsStore:
|
|||
break
|
||||
|
||||
if not events_to_calc_chain_id_for:
|
||||
return
|
||||
return {}
|
||||
|
||||
# Allocate chain ID/sequence numbers to each new event.
|
||||
new_chain_tuples = cls._allocate_chain_ids(
|
||||
|
@ -739,23 +824,10 @@ class PersistEventsStore:
|
|||
)
|
||||
chain_map.update(new_chain_tuples)
|
||||
|
||||
db_pool.simple_insert_many_txn(
|
||||
txn,
|
||||
table="event_auth_chains",
|
||||
keys=("event_id", "chain_id", "sequence_number"),
|
||||
values=[
|
||||
(event_id, c_id, seq)
|
||||
for event_id, (c_id, seq) in new_chain_tuples.items()
|
||||
],
|
||||
)
|
||||
|
||||
db_pool.simple_delete_many_txn(
|
||||
txn,
|
||||
table="event_auth_chain_to_calculate",
|
||||
keyvalues={},
|
||||
column="event_id",
|
||||
values=new_chain_tuples,
|
||||
)
|
||||
to_return = {
|
||||
event_id: NewEventChainLinks(chain_id, sequence_number)
|
||||
for event_id, (chain_id, sequence_number) in new_chain_tuples.items()
|
||||
}
|
||||
|
||||
# Now we need to calculate any new links between chains caused by
|
||||
# the new events.
|
||||
|
@ -825,10 +897,38 @@ class PersistEventsStore:
|
|||
auth_chain_id, auth_sequence_number = chain_map[auth_id]
|
||||
|
||||
# Step 2a, add link between the event and auth event
|
||||
to_return[event_id].links.append((auth_chain_id, auth_sequence_number))
|
||||
chain_links.add_link(
|
||||
(chain_id, sequence_number), (auth_chain_id, auth_sequence_number)
|
||||
)
|
||||
|
||||
return to_return
|
||||
|
||||
@classmethod
|
||||
def _persist_chain_cover_index(
|
||||
cls,
|
||||
txn: LoggingTransaction,
|
||||
db_pool: DatabasePool,
|
||||
new_event_links: Dict[str, NewEventChainLinks],
|
||||
) -> None:
|
||||
db_pool.simple_insert_many_txn(
|
||||
txn,
|
||||
table="event_auth_chains",
|
||||
keys=("event_id", "chain_id", "sequence_number"),
|
||||
values=[
|
||||
(event_id, new_links.chain_id, new_links.sequence_number)
|
||||
for event_id, new_links in new_event_links.items()
|
||||
],
|
||||
)
|
||||
|
||||
db_pool.simple_delete_many_txn(
|
||||
txn,
|
||||
table="event_auth_chain_to_calculate",
|
||||
keyvalues={},
|
||||
column="event_id",
|
||||
values=new_event_links,
|
||||
)
|
||||
|
||||
db_pool.simple_insert_many_txn(
|
||||
txn,
|
||||
table="event_auth_chain_links",
|
||||
|
@ -838,7 +938,16 @@ class PersistEventsStore:
|
|||
"target_chain_id",
|
||||
"target_sequence_number",
|
||||
),
|
||||
values=list(chain_links.get_additions()),
|
||||
values=[
|
||||
(
|
||||
new_links.chain_id,
|
||||
new_links.sequence_number,
|
||||
target_chain_id,
|
||||
target_sequence_number,
|
||||
)
|
||||
for new_links in new_event_links.values()
|
||||
for (target_chain_id, target_sequence_number) in new_links.links
|
||||
],
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
|
|
474
synapse/storage/databases/main/futures.py
Normal file
474
synapse/storage/databases/main/futures.py
Normal file
|
@ -0,0 +1,474 @@
|
|||
#
|
||||
# This file is licensed under the Affero General Public License (AGPL) version 3.
|
||||
#
|
||||
# Copyright (C) 2024 New Vector, Ltd
|
||||
#
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Affero General Public License as
|
||||
# published by the Free Software Foundation, either version 3 of the
|
||||
# License, or (at your option) any later version.
|
||||
#
|
||||
# See the GNU Affero General Public License for more details:
|
||||
# <https://www.gnu.org/licenses/agpl-3.0.html>.
|
||||
#
|
||||
# Originally licensed under the Apache License, Version 2.0:
|
||||
# <http://www.apache.org/licenses/LICENSE-2.0>.
|
||||
#
|
||||
#
|
||||
|
||||
from binascii import crc32
|
||||
from enum import Enum
|
||||
from typing import (
|
||||
TYPE_CHECKING,
|
||||
Any,
|
||||
Callable,
|
||||
Dict,
|
||||
List,
|
||||
NewType,
|
||||
Optional,
|
||||
Tuple,
|
||||
TypeVar,
|
||||
)
|
||||
|
||||
from synapse.api.errors import NotFoundError, StoreError, SynapseError
|
||||
from synapse.storage._base import SQLBaseStore, db_to_json
|
||||
from synapse.storage.database import (
|
||||
DatabasePool,
|
||||
LoggingDatabaseConnection,
|
||||
LoggingTransaction,
|
||||
)
|
||||
from synapse.types import JsonDict, RoomID, UserID
|
||||
from synapse.util import json_encoder, stringutils as stringutils
|
||||
from synapse.util.stringutils import base62_encode
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from synapse.server import HomeServer
|
||||
|
||||
|
||||
class FutureTokenType(Enum):
|
||||
SEND = 1
|
||||
CANCEL = 2
|
||||
REFRESH = 4
|
||||
|
||||
def __str__(self) -> str:
|
||||
if self == FutureTokenType.SEND:
|
||||
return "is_send"
|
||||
elif self == FutureTokenType.CANCEL:
|
||||
return "is_cancel"
|
||||
else:
|
||||
return "is_refresh"
|
||||
|
||||
|
||||
FutureToken = NewType("FutureToken", str)
|
||||
GroupID = NewType("GroupID", str)
|
||||
EventType = NewType("EventType", str)
|
||||
StateKey = NewType("StateKey", str)
|
||||
|
||||
FutureID = NewType("FutureID", int)
|
||||
Timeout = NewType("Timeout", int)
|
||||
Timestamp = NewType("Timestamp", int)
|
||||
|
||||
AddFutureReturn = Tuple[
|
||||
FutureID,
|
||||
GroupID,
|
||||
FutureToken,
|
||||
FutureToken,
|
||||
Optional[FutureToken],
|
||||
]
|
||||
|
||||
|
||||
# TODO: Try to support workers
|
||||
class FuturesStore(SQLBaseStore):
|
||||
def __init__(
|
||||
self,
|
||||
database: DatabasePool,
|
||||
db_conn: LoggingDatabaseConnection,
|
||||
hs: "HomeServer",
|
||||
):
|
||||
super().__init__(database, db_conn, hs)
|
||||
|
||||
async def add_future(
|
||||
self,
|
||||
*,
|
||||
user_id: UserID,
|
||||
group_id: Optional[str],
|
||||
room_id: RoomID,
|
||||
event_type: str,
|
||||
state_key: Optional[str],
|
||||
origin_server_ts: Optional[int],
|
||||
content: JsonDict,
|
||||
timeout: Optional[int],
|
||||
timestamp: Optional[int],
|
||||
) -> AddFutureReturn:
|
||||
"""Inserts a new future in the DB."""
|
||||
user_localpart = user_id.localpart
|
||||
|
||||
def add_future_txn(txn: LoggingTransaction) -> AddFutureReturn:
|
||||
T = TypeVar("T", bound=str)
|
||||
|
||||
def insert_and_get_unique_colval(
|
||||
table: str,
|
||||
values: Dict[str, Any],
|
||||
column: str,
|
||||
colval_generator: Callable[[], T],
|
||||
) -> T:
|
||||
attempts_remaining = 10
|
||||
while True:
|
||||
colval = colval_generator()
|
||||
try:
|
||||
self.db_pool.simple_insert_txn(
|
||||
txn,
|
||||
table,
|
||||
values={
|
||||
**values,
|
||||
column: colval,
|
||||
},
|
||||
)
|
||||
return colval
|
||||
# TODO: Handle only the error type for DB key collisions
|
||||
except Exception:
|
||||
if attempts_remaining > 0:
|
||||
attempts_remaining -= 1
|
||||
else:
|
||||
raise SynapseError(
|
||||
500,
|
||||
f"Couldn't generate a unique value for column {column} in table {table}",
|
||||
)
|
||||
|
||||
if group_id is None:
|
||||
group_id_final = insert_and_get_unique_colval(
|
||||
"future_groups",
|
||||
{
|
||||
"user_localpart": user_localpart,
|
||||
},
|
||||
"group_id",
|
||||
_generate_group_id,
|
||||
)
|
||||
else:
|
||||
txn.execute(
|
||||
"""
|
||||
INSERT INTO future_groups (user_localpart, group_id)
|
||||
VALUES (?, ?)
|
||||
ON CONFLICT DO NOTHING
|
||||
""",
|
||||
(user_localpart, group_id),
|
||||
)
|
||||
group_id_final = GroupID(group_id)
|
||||
|
||||
txn.execute(
|
||||
"""
|
||||
INSERT INTO futures (
|
||||
user_localpart, group_id,
|
||||
room_id, event_type, state_key, origin_server_ts,
|
||||
content
|
||||
) VALUES (
|
||||
?, ?,
|
||||
?, ?, ?, ?,
|
||||
?
|
||||
)
|
||||
RETURNING future_id
|
||||
""",
|
||||
(
|
||||
user_localpart,
|
||||
group_id_final,
|
||||
room_id.to_string(),
|
||||
event_type,
|
||||
state_key,
|
||||
origin_server_ts,
|
||||
json_encoder.encode(content),
|
||||
),
|
||||
)
|
||||
row = txn.fetchone()
|
||||
assert row is not None
|
||||
future_id = FutureID(row[0])
|
||||
|
||||
def insert_and_get_future_token(token_type: FutureTokenType) -> FutureToken:
|
||||
return insert_and_get_unique_colval(
|
||||
"future_tokens",
|
||||
{
|
||||
"future_id": future_id,
|
||||
str(token_type): True,
|
||||
},
|
||||
"token",
|
||||
_generate_future_token,
|
||||
)
|
||||
|
||||
send_token = insert_and_get_future_token(FutureTokenType.SEND)
|
||||
cancel_token = insert_and_get_future_token(FutureTokenType.CANCEL)
|
||||
if timeout is not None:
|
||||
self.db_pool.simple_insert_txn(
|
||||
txn,
|
||||
table="future_timeouts",
|
||||
values={
|
||||
"future_id": future_id,
|
||||
"timeout": timeout,
|
||||
"timestamp": timestamp,
|
||||
},
|
||||
)
|
||||
refresh_token = insert_and_get_future_token(FutureTokenType.REFRESH)
|
||||
else:
|
||||
refresh_token = None
|
||||
|
||||
return (
|
||||
future_id,
|
||||
group_id_final,
|
||||
send_token,
|
||||
cancel_token,
|
||||
refresh_token,
|
||||
)
|
||||
|
||||
return await self.db_pool.runInteraction("add_future", add_future_txn)
|
||||
|
||||
async def update_future_timestamp(
|
||||
self,
|
||||
future_id: FutureID,
|
||||
current_ts: Timestamp,
|
||||
) -> Timeout:
|
||||
"""Updates the timestamp of the timeout future for the given future_id.
|
||||
|
||||
Params:
|
||||
future_id: The ID of the future timeout to update.
|
||||
current_ts: The current time to which the future's timestamp will be set relative to.
|
||||
|
||||
Returns: The timeout value for the future.
|
||||
|
||||
Raises:
|
||||
NotFoundError if there is no timeout future for the given future_id.
|
||||
"""
|
||||
rows = await self.db_pool.execute(
|
||||
"update_future_timestamp",
|
||||
"""
|
||||
UPDATE future_timeouts
|
||||
SET timestamp = ? + timeout
|
||||
WHERE future_id = ?
|
||||
RETURNING timeout
|
||||
""",
|
||||
current_ts,
|
||||
future_id,
|
||||
)
|
||||
if len(rows) == 0 or len(rows[0]) == 0:
|
||||
raise NotFoundError
|
||||
return Timeout(rows[0][0])
|
||||
|
||||
async def has_group_id(
|
||||
self,
|
||||
user_id: UserID,
|
||||
group_id: str,
|
||||
) -> bool:
|
||||
"""Returns whether a future group exists for the given group_id."""
|
||||
count: int = await self.db_pool.simple_select_one_onecol(
|
||||
table="future_groups",
|
||||
keyvalues={"user_localpart": user_id.localpart, "group_id": group_id},
|
||||
retcol="COUNT(1)",
|
||||
desc="has_group_id",
|
||||
)
|
||||
return count > 0
|
||||
|
||||
async def get_future_by_token(
|
||||
self,
|
||||
token: str,
|
||||
) -> Tuple[FutureID, FutureTokenType]:
|
||||
"""Returns the future ID for the given token, and what type of token it is.
|
||||
|
||||
Raises:
|
||||
NotFoundError if there is no future for the given token.
|
||||
"""
|
||||
row = await self.db_pool.simple_select_one(
|
||||
table="future_tokens",
|
||||
keyvalues={"token": token},
|
||||
retcols=("is_send", "is_cancel", "is_refresh", "future_id"),
|
||||
allow_none=True,
|
||||
desc="get_future_by_token",
|
||||
)
|
||||
if row is None:
|
||||
raise NotFoundError
|
||||
|
||||
return FutureID(row[3]), FutureTokenType(sum(row[i] * 2**i for i in range(3)))
|
||||
|
||||
async def get_all_futures_for_user(
|
||||
self,
|
||||
user_id: UserID,
|
||||
) -> List[JsonDict]:
|
||||
"""Returns all pending futures that were requested by the given user."""
|
||||
rows = await self.db_pool.execute(
|
||||
"_get_all_futures_for_user_txn",
|
||||
"""
|
||||
SELECT
|
||||
group_id, timeout,
|
||||
room_id, event_type, state_key, content,
|
||||
send_token, cancel_token, refresh_token
|
||||
FROM futures
|
||||
LEFT JOIN (
|
||||
SELECT future_id, token AS send_token
|
||||
FROM future_tokens
|
||||
WHERE is_send
|
||||
) USING (future_id)
|
||||
LEFT JOIN (
|
||||
SELECT future_id, token AS cancel_token
|
||||
FROM future_tokens
|
||||
WHERE is_cancel
|
||||
) USING (future_id)
|
||||
LEFT JOIN (
|
||||
SELECT future_id, token AS refresh_token
|
||||
FROM future_tokens
|
||||
WHERE is_refresh
|
||||
) USING (future_id)
|
||||
LEFT JOIN future_timeouts USING (future_id)
|
||||
WHERE user_localpart = ?
|
||||
""",
|
||||
user_id.localpart,
|
||||
)
|
||||
return [
|
||||
{
|
||||
"future_group_id": str(row[0]),
|
||||
**({"future_timeout": int(row[1])} if row[1] is not None else {}),
|
||||
"room_id": str(row[2]),
|
||||
"type": str(row[3]),
|
||||
**({"state_key": str(row[4])} if row[4] is not None else {}),
|
||||
# TODO: Verify contents?
|
||||
"content": db_to_json(row[5]),
|
||||
# TODO: If suppressing send/cancel tokens is allowed, omit them if None
|
||||
"send_token": str(row[6]),
|
||||
"cancel_token": str(row[7]),
|
||||
**({"refresh_token": str(row[8])} if row[8] is not None else {}),
|
||||
}
|
||||
for row in rows
|
||||
]
|
||||
|
||||
async def get_all_future_timestamps(
|
||||
self,
|
||||
) -> List[Tuple[FutureID, Timestamp]]:
|
||||
"""Returns all timeout futures' IDs and when they will be sent if not refreshed."""
|
||||
return await self.db_pool.simple_select_list(
|
||||
table="future_timeouts",
|
||||
keyvalues=None,
|
||||
retcols=("future_id", "timestamp"),
|
||||
desc="get_all_future_timestamps",
|
||||
)
|
||||
|
||||
async def pop_future_event(
|
||||
self,
|
||||
future_id: FutureID,
|
||||
) -> Tuple[
|
||||
str, RoomID, EventType, Optional[StateKey], Optional[Timestamp], JsonDict
|
||||
]:
|
||||
"""Get the partial event of the future with the specified future_id,
|
||||
and remove all futures in its group from the DB.
|
||||
"""
|
||||
|
||||
def pop_future_event_txn(txn: LoggingTransaction) -> Tuple[Any, ...]:
|
||||
try:
|
||||
row = self.db_pool.simple_select_one_txn(
|
||||
txn,
|
||||
table="futures",
|
||||
keyvalues={"future_id": future_id},
|
||||
retcols=(
|
||||
"user_localpart",
|
||||
"group_id",
|
||||
"room_id",
|
||||
"event_type",
|
||||
"state_key",
|
||||
"origin_server_ts",
|
||||
"content",
|
||||
),
|
||||
)
|
||||
assert row is not None
|
||||
except StoreError:
|
||||
raise NotFoundError
|
||||
|
||||
self.db_pool.simple_delete_txn(
|
||||
txn,
|
||||
table="future_groups",
|
||||
keyvalues={
|
||||
"user_localpart": str(row[0]),
|
||||
"group_id": GroupID(row[1]),
|
||||
},
|
||||
)
|
||||
return (row[0], *row[2:])
|
||||
|
||||
row = await self.db_pool.runInteraction(
|
||||
"pop_future_event", pop_future_event_txn
|
||||
)
|
||||
room_id = RoomID.from_string(row[1])
|
||||
# TODO: verify contents?
|
||||
content: JsonDict = db_to_json(row[5])
|
||||
return (row[0], room_id, *row[2:5], content)
|
||||
|
||||
async def remove_future(
|
||||
self,
|
||||
future_id: FutureID,
|
||||
) -> None:
|
||||
"""Removes the future for the given future_id from the DB.
|
||||
If the future is the only timeout future in its group, removes the whole group.
|
||||
"""
|
||||
|
||||
def remove_future_txn(txn: LoggingTransaction) -> None:
|
||||
txn.execute(
|
||||
"""
|
||||
WITH futures_with_timeout AS (
|
||||
SELECT future_id, user_localpart, group_id
|
||||
FROM futures
|
||||
JOIN future_timeouts USING (future_id)
|
||||
), timeout_group AS (
|
||||
SELECT user_localpart, group_id
|
||||
FROM futures_with_timeout
|
||||
WHERE future_id = ?
|
||||
)
|
||||
SELECT DISTINCT COUNT(1), user_localpart, group_id
|
||||
FROM futures_with_timeout
|
||||
JOIN timeout_group USING (user_localpart, group_id)
|
||||
GROUP BY user_localpart, group_id
|
||||
""",
|
||||
(future_id,),
|
||||
)
|
||||
row = txn.fetchone()
|
||||
|
||||
if row is not None and row[0] == 1:
|
||||
self.db_pool.simple_delete_one_txn(
|
||||
txn,
|
||||
"future_groups",
|
||||
keyvalues={
|
||||
"user_localpart": row[1],
|
||||
"group_id": row[2],
|
||||
},
|
||||
)
|
||||
else:
|
||||
self.db_pool.simple_delete_one_txn(
|
||||
txn,
|
||||
"futures",
|
||||
keyvalues={
|
||||
"future_id": future_id,
|
||||
},
|
||||
)
|
||||
|
||||
await self.db_pool.runInteraction("remove_future", remove_future_txn)
|
||||
|
||||
|
||||
def _generate_future_token() -> FutureToken:
|
||||
"""Generates an opaque string, for use as a future token"""
|
||||
|
||||
# We use the following format for future tokens:
|
||||
# syf_<random string>_<base62 crc check>
|
||||
# They are NOT scoped to user localparts so that any delegate given the token
|
||||
# won't necessarily know which user created the future.
|
||||
|
||||
random_string = stringutils.random_string(20)
|
||||
base = f"syf_{random_string}"
|
||||
|
||||
crc = base62_encode(crc32(base.encode("ascii")), minwidth=6)
|
||||
return FutureToken(f"{base}_{crc}")
|
||||
|
||||
|
||||
def _generate_group_id() -> GroupID:
|
||||
"""Generates an opaque string, for use as a future group ID"""
|
||||
|
||||
# We use the following format for future tokens:
|
||||
# syf_<random string>_<base62 crc check>
|
||||
# They are scoped to user localparts, but that CANNOT be relied on
|
||||
# to keep them globally unique, as users may set their own group_id.
|
||||
|
||||
random_string = stringutils.random_string(20)
|
||||
base = f"syg_{random_string}"
|
||||
|
||||
crc = base62_encode(crc32(base.encode("ascii")), minwidth=6)
|
||||
return GroupID(f"{base}_{crc}")
|
|
@ -606,6 +606,8 @@ class RoomWorkerStore(CacheInvalidationWorkerStore):
|
|||
order_by: str,
|
||||
reverse_order: bool,
|
||||
search_term: Optional[str],
|
||||
public_rooms: Optional[bool],
|
||||
empty_rooms: Optional[bool],
|
||||
) -> Tuple[List[Dict[str, Any]], int]:
|
||||
"""Function to retrieve a paginated list of rooms as json.
|
||||
|
||||
|
@ -617,30 +619,49 @@ class RoomWorkerStore(CacheInvalidationWorkerStore):
|
|||
search_term: a string to filter room names,
|
||||
canonical alias and room ids by.
|
||||
Room ID must match exactly. Canonical alias must match a substring of the local part.
|
||||
public_rooms: Optional flag to filter public and non-public rooms. If true, public rooms are queried.
|
||||
if false, public rooms are excluded from the query. When it is
|
||||
none (the default), both public rooms and none-public-rooms are queried.
|
||||
empty_rooms: Optional flag to filter empty and non-empty rooms.
|
||||
A room is empty if joined_members is zero.
|
||||
If true, empty rooms are queried.
|
||||
if false, empty rooms are excluded from the query. When it is
|
||||
none (the default), both empty rooms and none-empty rooms are queried.
|
||||
Returns:
|
||||
A list of room dicts and an integer representing the total number of
|
||||
rooms that exist given this query
|
||||
"""
|
||||
# Filter room names by a string
|
||||
where_statement = ""
|
||||
search_pattern: List[object] = []
|
||||
filter_ = []
|
||||
where_args = []
|
||||
if search_term:
|
||||
where_statement = """
|
||||
WHERE LOWER(state.name) LIKE ?
|
||||
OR LOWER(state.canonical_alias) LIKE ?
|
||||
OR state.room_id = ?
|
||||
"""
|
||||
filter_ = [
|
||||
"LOWER(state.name) LIKE ? OR "
|
||||
"LOWER(state.canonical_alias) LIKE ? OR "
|
||||
"state.room_id = ?"
|
||||
]
|
||||
|
||||
# Our postgres db driver converts ? -> %s in SQL strings as that's the
|
||||
# placeholder for postgres.
|
||||
# HOWEVER, if you put a % into your SQL then everything goes wibbly.
|
||||
# To get around this, we're going to surround search_term with %'s
|
||||
# before giving it to the database in python instead
|
||||
search_pattern = [
|
||||
"%" + search_term.lower() + "%",
|
||||
"#%" + search_term.lower() + "%:%",
|
||||
where_args = [
|
||||
f"%{search_term.lower()}%",
|
||||
f"#%{search_term.lower()}%:%",
|
||||
search_term,
|
||||
]
|
||||
if public_rooms is not None:
|
||||
filter_arg = "1" if public_rooms else "0"
|
||||
filter_.append(f"rooms.is_public = '{filter_arg}'")
|
||||
|
||||
if empty_rooms is not None:
|
||||
if empty_rooms:
|
||||
filter_.append("curr.joined_members = 0")
|
||||
else:
|
||||
filter_.append("curr.joined_members <> 0")
|
||||
|
||||
where_clause = "WHERE " + " AND ".join(filter_) if len(filter_) > 0 else ""
|
||||
|
||||
# Set ordering
|
||||
if RoomSortOrder(order_by) == RoomSortOrder.SIZE:
|
||||
|
@ -717,7 +738,7 @@ class RoomWorkerStore(CacheInvalidationWorkerStore):
|
|||
LIMIT ?
|
||||
OFFSET ?
|
||||
""".format(
|
||||
where=where_statement,
|
||||
where=where_clause,
|
||||
order_by=order_by_column,
|
||||
direction="ASC" if order_by_asc else "DESC",
|
||||
)
|
||||
|
@ -726,10 +747,12 @@ class RoomWorkerStore(CacheInvalidationWorkerStore):
|
|||
count_sql = """
|
||||
SELECT count(*) FROM (
|
||||
SELECT room_id FROM room_stats_state state
|
||||
INNER JOIN room_stats_current curr USING (room_id)
|
||||
INNER JOIN rooms USING (room_id)
|
||||
{where}
|
||||
) AS get_room_ids
|
||||
""".format(
|
||||
where=where_statement,
|
||||
where=where_clause,
|
||||
)
|
||||
|
||||
def _get_rooms_paginate_txn(
|
||||
|
@ -737,7 +760,7 @@ class RoomWorkerStore(CacheInvalidationWorkerStore):
|
|||
) -> Tuple[List[Dict[str, Any]], int]:
|
||||
# Add the search term into the WHERE clause
|
||||
# and execute the data query
|
||||
txn.execute(info_sql, search_pattern + [limit, start])
|
||||
txn.execute(info_sql, where_args + [limit, start])
|
||||
|
||||
# Refactor room query data into a structured dictionary
|
||||
rooms = []
|
||||
|
@ -767,7 +790,7 @@ class RoomWorkerStore(CacheInvalidationWorkerStore):
|
|||
# Execute the count query
|
||||
|
||||
# Add the search term into the WHERE clause if present
|
||||
txn.execute(count_sql, search_pattern)
|
||||
txn.execute(count_sql, where_args)
|
||||
|
||||
room_count = cast(Tuple[int], txn.fetchone())
|
||||
return rooms, room_count[0]
|
||||
|
|
|
@ -19,7 +19,7 @@
|
|||
#
|
||||
#
|
||||
|
||||
SCHEMA_VERSION = 85 # remember to update the list below when updating
|
||||
SCHEMA_VERSION = 86 # remember to update the list below when updating
|
||||
"""Represents the expectations made by the codebase about the database schema
|
||||
|
||||
This should be incremented whenever the codebase changes its requirements on the
|
||||
|
@ -139,6 +139,10 @@ Changes in SCHEMA_VERSION = 84
|
|||
|
||||
Changes in SCHEMA_VERSION = 85
|
||||
- Add a column `suspended` to the `users` table
|
||||
|
||||
Changes in SCHEMA_VERSION = 86:
|
||||
- MSC4140: Add `futures` table that keeps track of events that are to
|
||||
be posted in response to a refreshable timeout or an on-demand action.
|
||||
"""
|
||||
|
||||
|
||||
|
|
58
synapse/storage/schema/main/delta/86/01_add_futures.sql
Normal file
58
synapse/storage/schema/main/delta/86/01_add_futures.sql
Normal file
|
@ -0,0 +1,58 @@
|
|||
--
|
||||
-- This file is licensed under the Affero General Public License (AGPL) version 3.
|
||||
--
|
||||
-- Copyright (C) 2024 New Vector, Ltd
|
||||
--
|
||||
-- This program is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as
|
||||
-- published by the Free Software Foundation, either version 3 of the
|
||||
-- License, or (at your option) any later version.
|
||||
--
|
||||
-- See the GNU Affero General Public License for more details:
|
||||
-- <https://www.gnu.org/licenses/agpl-3.0.html>.
|
||||
|
||||
CREATE TABLE future_groups (
|
||||
user_localpart TEXT NOT NULL,
|
||||
group_id TEXT NOT NULL,
|
||||
PRIMARY KEY (user_localpart, group_id)
|
||||
);
|
||||
|
||||
CREATE TABLE futures (
|
||||
future_id INTEGER PRIMARY KEY, -- An alias of rowid in SQLite
|
||||
user_localpart TEXT NOT NULL,
|
||||
group_id TEXT NOT NULL,
|
||||
room_id TEXT NOT NULL,
|
||||
event_type TEXT NOT NULL,
|
||||
state_key TEXT,
|
||||
origin_server_ts BIGINT,
|
||||
content bytea NOT NULL,
|
||||
FOREIGN KEY (user_localpart, group_id)
|
||||
REFERENCES future_groups (user_localpart, group_id) ON DELETE CASCADE
|
||||
);
|
||||
-- TODO: Consider a trigger/constraint to disallow adding an action future to an empty group
|
||||
|
||||
CREATE INDEX future_group_idx ON futures (user_localpart, group_id);
|
||||
CREATE INDEX room_state_event_idx ON futures (room_id, event_type, state_key) WHERE state_key IS NOT NULL;
|
||||
|
||||
CREATE TABLE future_timeouts (
|
||||
future_id INTEGER PRIMARY KEY
|
||||
REFERENCES futures (future_id) ON DELETE CASCADE,
|
||||
timeout BIGINT NOT NULL,
|
||||
timestamp BIGINT NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE future_tokens (
|
||||
token TEXT PRIMARY KEY,
|
||||
future_id INTEGER NOT NULL
|
||||
REFERENCES futures (future_id) ON DELETE CASCADE,
|
||||
is_send BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
is_cancel BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
is_refresh BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
CHECK (
|
||||
+ CAST(is_send AS INTEGER)
|
||||
+ CAST(is_cancel AS INTEGER)
|
||||
+ CAST(is_refresh AS INTEGER)
|
||||
= 1),
|
||||
UNIQUE (future_id, is_send, is_cancel, is_refresh)
|
||||
);
|
||||
-- TODO: Consider a trigger/constraint to disallow refresh tokens for futures without a timeout
|
|
@ -0,0 +1,14 @@
|
|||
--
|
||||
-- This file is licensed under the Affero General Public License (AGPL) version 3.
|
||||
--
|
||||
-- Copyright (C) 2024 New Vector, Ltd
|
||||
--
|
||||
-- This program is free software: you can redistribute it and/or modify
|
||||
-- it under the terms of the GNU Affero General Public License as
|
||||
-- published by the Free Software Foundation, either version 3 of the
|
||||
-- License, or (at your option) any later version.
|
||||
--
|
||||
-- See the GNU Affero General Public License for more details:
|
||||
-- <https://www.gnu.org/licenses/agpl-3.0.html>.
|
||||
|
||||
ALTER TABLE futures ALTER COLUMN future_id ADD GENERATED ALWAYS AS IDENTITY;
|
|
@ -75,9 +75,6 @@ class PaginationConfig:
|
|||
raise SynapseError(400, "'to' parameter is invalid")
|
||||
|
||||
limit = parse_integer(request, "limit", default=default_limit)
|
||||
if limit < 0:
|
||||
raise SynapseError(400, "Limit must be 0 or above")
|
||||
|
||||
limit = min(limit, MAX_LIMIT)
|
||||
|
||||
try:
|
||||
|
|
173
tests/federation/test_federation_media.py
Normal file
173
tests/federation/test_federation_media.py
Normal file
|
@ -0,0 +1,173 @@
|
|||
#
|
||||
# This file is licensed under the Affero General Public License (AGPL) version 3.
|
||||
#
|
||||
# Copyright (C) 2024 New Vector, Ltd
|
||||
#
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU Affero General Public License as
|
||||
# published by the Free Software Foundation, either version 3 of the
|
||||
# License, or (at your option) any later version.
|
||||
#
|
||||
# See the GNU Affero General Public License for more details:
|
||||
# <https://www.gnu.org/licenses/agpl-3.0.html>.
|
||||
#
|
||||
# Originally licensed under the Apache License, Version 2.0:
|
||||
# <http://www.apache.org/licenses/LICENSE-2.0>.
|
||||
#
|
||||
# [This file includes modifications made by New Vector Limited]
|
||||
#
|
||||
#
|
||||
import io
|
||||
import os
|
||||
import shutil
|
||||
import tempfile
|
||||
|
||||
from twisted.test.proto_helpers import MemoryReactor
|
||||
|
||||
from synapse.media.filepath import MediaFilePaths
|
||||
from synapse.media.media_storage import MediaStorage
|
||||
from synapse.media.storage_provider import (
|
||||
FileStorageProviderBackend,
|
||||
StorageProviderWrapper,
|
||||
)
|
||||
from synapse.server import HomeServer
|
||||
from synapse.types import UserID
|
||||
from synapse.util import Clock
|
||||
|
||||
from tests import unittest
|
||||
from tests.test_utils import SMALL_PNG
|
||||
from tests.unittest import override_config
|
||||
|
||||
|
||||
class FederationUnstableMediaDownloadsTest(unittest.FederatingHomeserverTestCase):
|
||||
|
||||
def prepare(self, reactor: MemoryReactor, clock: Clock, hs: HomeServer) -> None:
|
||||
super().prepare(reactor, clock, hs)
|
||||
self.test_dir = tempfile.mkdtemp(prefix="synapse-tests-")
|
||||
self.addCleanup(shutil.rmtree, self.test_dir)
|
||||
self.primary_base_path = os.path.join(self.test_dir, "primary")
|
||||
self.secondary_base_path = os.path.join(self.test_dir, "secondary")
|
||||
|
||||
hs.config.media.media_store_path = self.primary_base_path
|
||||
|
||||
storage_providers = [
|
||||
StorageProviderWrapper(
|
||||
FileStorageProviderBackend(hs, self.secondary_base_path),
|
||||
store_local=True,
|
||||
store_remote=False,
|
||||
store_synchronous=True,
|
||||
)
|
||||
]
|
||||
|
||||
self.filepaths = MediaFilePaths(self.primary_base_path)
|
||||
self.media_storage = MediaStorage(
|
||||
hs, self.primary_base_path, self.filepaths, storage_providers
|
||||
)
|
||||
self.media_repo = hs.get_media_repository()
|
||||
|
||||
@override_config(
|
||||
{"experimental_features": {"msc3916_authenticated_media_enabled": True}}
|
||||
)
|
||||
def test_file_download(self) -> None:
|
||||
content = io.BytesIO(b"file_to_stream")
|
||||
content_uri = self.get_success(
|
||||
self.media_repo.create_content(
|
||||
"text/plain",
|
||||
"test_upload",
|
||||
content,
|
||||
46,
|
||||
UserID.from_string("@user_id:whatever.org"),
|
||||
)
|
||||
)
|
||||
# test with a text file
|
||||
channel = self.make_signed_federation_request(
|
||||
"GET",
|
||||
f"/_matrix/federation/unstable/org.matrix.msc3916/media/download/{content_uri.media_id}",
|
||||
)
|
||||
self.pump()
|
||||
self.assertEqual(200, channel.code)
|
||||
|
||||
content_type = channel.headers.getRawHeaders("content-type")
|
||||
assert content_type is not None
|
||||
assert "multipart/mixed" in content_type[0]
|
||||
assert "boundary" in content_type[0]
|
||||
|
||||
# extract boundary
|
||||
boundary = content_type[0].split("boundary=")[1]
|
||||
# split on boundary and check that json field and expected value exist
|
||||
stripped = channel.text_body.split("\r\n" + "--" + boundary)
|
||||
# TODO: the json object expected will change once MSC3911 is implemented, currently
|
||||
# {} is returned for all requests as a placeholder (per MSC3196)
|
||||
found_json = any(
|
||||
"\r\nContent-Type: application/json\r\n\r\n{}" in field
|
||||
for field in stripped
|
||||
)
|
||||
self.assertTrue(found_json)
|
||||
|
||||
# check that the text file and expected value exist
|
||||
found_file = any(
|
||||
"\r\nContent-Type: text/plain\r\n\r\nfile_to_stream" in field
|
||||
for field in stripped
|
||||
)
|
||||
self.assertTrue(found_file)
|
||||
|
||||
content = io.BytesIO(SMALL_PNG)
|
||||
content_uri = self.get_success(
|
||||
self.media_repo.create_content(
|
||||
"image/png",
|
||||
"test_png_upload",
|
||||
content,
|
||||
67,
|
||||
UserID.from_string("@user_id:whatever.org"),
|
||||
)
|
||||
)
|
||||
# test with an image file
|
||||
channel = self.make_signed_federation_request(
|
||||
"GET",
|
||||
f"/_matrix/federation/unstable/org.matrix.msc3916/media/download/{content_uri.media_id}",
|
||||
)
|
||||
self.pump()
|
||||
self.assertEqual(200, channel.code)
|
||||
|
||||
content_type = channel.headers.getRawHeaders("content-type")
|
||||
assert content_type is not None
|
||||
assert "multipart/mixed" in content_type[0]
|
||||
assert "boundary" in content_type[0]
|
||||
|
||||
# extract boundary
|
||||
boundary = content_type[0].split("boundary=")[1]
|
||||
# split on boundary and check that json field and expected value exist
|
||||
body = channel.result.get("body")
|
||||
assert body is not None
|
||||
stripped_bytes = body.split(b"\r\n" + b"--" + boundary.encode("utf-8"))
|
||||
found_json = any(
|
||||
b"\r\nContent-Type: application/json\r\n\r\n{}" in field
|
||||
for field in stripped_bytes
|
||||
)
|
||||
self.assertTrue(found_json)
|
||||
|
||||
# check that the png file exists and matches what was uploaded
|
||||
found_file = any(SMALL_PNG in field for field in stripped_bytes)
|
||||
self.assertTrue(found_file)
|
||||
|
||||
@override_config(
|
||||
{"experimental_features": {"msc3916_authenticated_media_enabled": False}}
|
||||
)
|
||||
def test_disable_config(self) -> None:
|
||||
content = io.BytesIO(b"file_to_stream")
|
||||
content_uri = self.get_success(
|
||||
self.media_repo.create_content(
|
||||
"text/plain",
|
||||
"test_upload",
|
||||
content,
|
||||
46,
|
||||
UserID.from_string("@user_id:whatever.org"),
|
||||
)
|
||||
)
|
||||
channel = self.make_signed_federation_request(
|
||||
"GET",
|
||||
f"/_matrix/federation/unstable/org.matrix.msc3916/media/download/{content_uri.media_id}",
|
||||
)
|
||||
self.pump()
|
||||
self.assertEqual(404, channel.code)
|
||||
self.assertEqual(channel.json_body.get("errcode"), "M_UNRECOGNIZED")
|
|
@ -1200,11 +1200,7 @@ class FilterRoomsTestCase(HomeserverTestCase):
|
|||
user2_tok = self.login(user2_id, "pass")
|
||||
|
||||
# Create a normal room
|
||||
room_id = self.helper.create_room_as(
|
||||
user1_id,
|
||||
is_public=False,
|
||||
tok=user1_tok,
|
||||
)
|
||||
room_id = self.helper.create_room_as(user1_id, tok=user1_tok)
|
||||
|
||||
# Create a DM room
|
||||
dm_room_id = self._create_dm_room(
|
||||
|
@ -1261,18 +1257,10 @@ class FilterRoomsTestCase(HomeserverTestCase):
|
|||
user1_tok = self.login(user1_id, "pass")
|
||||
|
||||
# Create a normal room
|
||||
room_id = self.helper.create_room_as(
|
||||
user1_id,
|
||||
is_public=False,
|
||||
tok=user1_tok,
|
||||
)
|
||||
room_id = self.helper.create_room_as(user1_id, tok=user1_tok)
|
||||
|
||||
# Create an encrypted room
|
||||
encrypted_room_id = self.helper.create_room_as(
|
||||
user1_id,
|
||||
is_public=False,
|
||||
tok=user1_tok,
|
||||
)
|
||||
encrypted_room_id = self.helper.create_room_as(user1_id, tok=user1_tok)
|
||||
self.helper.send_state(
|
||||
encrypted_room_id,
|
||||
EventTypes.RoomEncryption,
|
||||
|
@ -1319,6 +1307,62 @@ class FilterRoomsTestCase(HomeserverTestCase):
|
|||
|
||||
self.assertEqual(falsy_filtered_room_map.keys(), {room_id})
|
||||
|
||||
def test_filter_invite_rooms(self) -> None:
|
||||
"""
|
||||
Test `filter.is_invite` for rooms that the user has been invited to
|
||||
"""
|
||||
user1_id = self.register_user("user1", "pass")
|
||||
user1_tok = self.login(user1_id, "pass")
|
||||
user2_id = self.register_user("user2", "pass")
|
||||
user2_tok = self.login(user2_id, "pass")
|
||||
|
||||
# Create a normal room
|
||||
room_id = self.helper.create_room_as(user2_id, tok=user2_tok)
|
||||
self.helper.join(room_id, user1_id, tok=user1_tok)
|
||||
|
||||
# Create a room that user1 is invited to
|
||||
invite_room_id = self.helper.create_room_as(user2_id, tok=user2_tok)
|
||||
self.helper.invite(invite_room_id, src=user2_id, targ=user1_id, tok=user2_tok)
|
||||
|
||||
after_rooms_token = self.event_sources.get_current_token()
|
||||
|
||||
# Get the rooms the user should be syncing with
|
||||
sync_room_map = self.get_success(
|
||||
self.sliding_sync_handler.get_sync_room_ids_for_user(
|
||||
UserID.from_string(user1_id),
|
||||
from_token=None,
|
||||
to_token=after_rooms_token,
|
||||
)
|
||||
)
|
||||
|
||||
# Try with `is_invite=True`
|
||||
truthy_filtered_room_map = self.get_success(
|
||||
self.sliding_sync_handler.filter_rooms(
|
||||
UserID.from_string(user1_id),
|
||||
sync_room_map,
|
||||
SlidingSyncConfig.SlidingSyncList.Filters(
|
||||
is_invite=True,
|
||||
),
|
||||
after_rooms_token,
|
||||
)
|
||||
)
|
||||
|
||||
self.assertEqual(truthy_filtered_room_map.keys(), {invite_room_id})
|
||||
|
||||
# Try with `is_invite=False`
|
||||
falsy_filtered_room_map = self.get_success(
|
||||
self.sliding_sync_handler.filter_rooms(
|
||||
UserID.from_string(user1_id),
|
||||
sync_room_map,
|
||||
SlidingSyncConfig.SlidingSyncList.Filters(
|
||||
is_invite=False,
|
||||
),
|
||||
after_rooms_token,
|
||||
)
|
||||
)
|
||||
|
||||
self.assertEqual(falsy_filtered_room_map.keys(), {room_id})
|
||||
|
||||
|
||||
class SortRoomsTestCase(HomeserverTestCase):
|
||||
"""
|
||||
|
|
|
@ -688,7 +688,7 @@ class ModuleApiTestCase(BaseModuleApiTestCase):
|
|||
|
||||
channel = self.make_request(
|
||||
"GET",
|
||||
"/notifications?from=",
|
||||
"/notifications",
|
||||
access_token=tok,
|
||||
)
|
||||
self.assertEqual(channel.code, 200, channel.result)
|
||||
|
|
|
@ -1795,6 +1795,83 @@ class RoomTestCase(unittest.HomeserverTestCase):
|
|||
self.assertEqual(room_id, channel.json_body["rooms"][0].get("room_id"))
|
||||
self.assertEqual("ж", channel.json_body["rooms"][0].get("name"))
|
||||
|
||||
def test_filter_public_rooms(self) -> None:
|
||||
self.helper.create_room_as(
|
||||
self.admin_user, tok=self.admin_user_tok, is_public=True
|
||||
)
|
||||
self.helper.create_room_as(
|
||||
self.admin_user, tok=self.admin_user_tok, is_public=True
|
||||
)
|
||||
self.helper.create_room_as(
|
||||
self.admin_user, tok=self.admin_user_tok, is_public=False
|
||||
)
|
||||
|
||||
response = self.make_request(
|
||||
"GET",
|
||||
"/_synapse/admin/v1/rooms",
|
||||
access_token=self.admin_user_tok,
|
||||
)
|
||||
self.assertEqual(200, response.code, msg=response.json_body)
|
||||
self.assertEqual(3, response.json_body["total_rooms"])
|
||||
self.assertEqual(3, len(response.json_body["rooms"]))
|
||||
|
||||
response = self.make_request(
|
||||
"GET",
|
||||
"/_synapse/admin/v1/rooms?public_rooms=true",
|
||||
access_token=self.admin_user_tok,
|
||||
)
|
||||
self.assertEqual(200, response.code, msg=response.json_body)
|
||||
self.assertEqual(2, response.json_body["total_rooms"])
|
||||
self.assertEqual(2, len(response.json_body["rooms"]))
|
||||
|
||||
response = self.make_request(
|
||||
"GET",
|
||||
"/_synapse/admin/v1/rooms?public_rooms=false",
|
||||
access_token=self.admin_user_tok,
|
||||
)
|
||||
self.assertEqual(200, response.code, msg=response.json_body)
|
||||
self.assertEqual(1, response.json_body["total_rooms"])
|
||||
self.assertEqual(1, len(response.json_body["rooms"]))
|
||||
|
||||
def test_filter_empty_rooms(self) -> None:
|
||||
self.helper.create_room_as(
|
||||
self.admin_user, tok=self.admin_user_tok, is_public=True
|
||||
)
|
||||
self.helper.create_room_as(
|
||||
self.admin_user, tok=self.admin_user_tok, is_public=True
|
||||
)
|
||||
room_id = self.helper.create_room_as(
|
||||
self.admin_user, tok=self.admin_user_tok, is_public=False
|
||||
)
|
||||
self.helper.leave(room_id, self.admin_user, tok=self.admin_user_tok)
|
||||
|
||||
response = self.make_request(
|
||||
"GET",
|
||||
"/_synapse/admin/v1/rooms",
|
||||
access_token=self.admin_user_tok,
|
||||
)
|
||||
self.assertEqual(200, response.code, msg=response.json_body)
|
||||
self.assertEqual(3, response.json_body["total_rooms"])
|
||||
self.assertEqual(3, len(response.json_body["rooms"]))
|
||||
|
||||
response = self.make_request(
|
||||
"GET",
|
||||
"/_synapse/admin/v1/rooms?empty_rooms=false",
|
||||
access_token=self.admin_user_tok,
|
||||
)
|
||||
self.assertEqual(200, response.code, msg=response.json_body)
|
||||
self.assertEqual(2, response.json_body["total_rooms"])
|
||||
self.assertEqual(2, len(response.json_body["rooms"]))
|
||||
|
||||
response = self.make_request(
|
||||
"GET",
|
||||
"/_synapse/admin/v1/rooms?empty_rooms=true",
|
||||
access_token=self.admin_user_tok,
|
||||
)
|
||||
self.assertEqual(200, response.code, msg=response.json_body)
|
||||
self.assertEqual(1, response.json_body["total_rooms"])
|
||||
self.assertEqual(1, len(response.json_body["rooms"]))
|
||||
|
||||
def test_single_room(self) -> None:
|
||||
"""Test that a single room can be requested correctly"""
|
||||
# Create two test rooms
|
||||
|
|
|
@ -37,6 +37,7 @@ from synapse.api.constants import ApprovalNoticeMedium, LoginType, UserTypes
|
|||
from synapse.api.errors import Codes, HttpResponseException, ResourceLimitError
|
||||
from synapse.api.room_versions import RoomVersions
|
||||
from synapse.media.filepath import MediaFilePaths
|
||||
from synapse.rest import admin
|
||||
from synapse.rest.client import (
|
||||
devices,
|
||||
login,
|
||||
|
@ -5005,3 +5006,86 @@ class AllowCrossSigningReplacementTestCase(unittest.HomeserverTestCase):
|
|||
)
|
||||
assert timestamp is not None
|
||||
self.assertGreater(timestamp, self.clock.time_msec())
|
||||
|
||||
|
||||
class UserSuspensionTestCase(unittest.HomeserverTestCase):
|
||||
servlets = [
|
||||
synapse.rest.admin.register_servlets,
|
||||
login.register_servlets,
|
||||
admin.register_servlets,
|
||||
]
|
||||
|
||||
def prepare(self, reactor: MemoryReactor, clock: Clock, hs: HomeServer) -> None:
|
||||
self.admin = self.register_user("thomas", "hackme", True)
|
||||
self.admin_tok = self.login("thomas", "hackme")
|
||||
|
||||
self.bad_user = self.register_user("teresa", "hackme")
|
||||
self.bad_user_tok = self.login("teresa", "hackme")
|
||||
|
||||
self.store = hs.get_datastores().main
|
||||
|
||||
@override_config({"experimental_features": {"msc3823_account_suspension": True}})
|
||||
def test_suspend_user(self) -> None:
|
||||
# test that suspending user works
|
||||
channel = self.make_request(
|
||||
"PUT",
|
||||
f"/_synapse/admin/v1/suspend/{self.bad_user}",
|
||||
{"suspend": True},
|
||||
access_token=self.admin_tok,
|
||||
)
|
||||
self.assertEqual(channel.code, 200)
|
||||
self.assertEqual(channel.json_body, {f"user_{self.bad_user}_suspended": True})
|
||||
|
||||
res = self.get_success(self.store.get_user_suspended_status(self.bad_user))
|
||||
self.assertEqual(True, res)
|
||||
|
||||
# test that un-suspending user works
|
||||
channel2 = self.make_request(
|
||||
"PUT",
|
||||
f"/_synapse/admin/v1/suspend/{self.bad_user}",
|
||||
{"suspend": False},
|
||||
access_token=self.admin_tok,
|
||||
)
|
||||
self.assertEqual(channel2.code, 200)
|
||||
self.assertEqual(channel2.json_body, {f"user_{self.bad_user}_suspended": False})
|
||||
|
||||
res2 = self.get_success(self.store.get_user_suspended_status(self.bad_user))
|
||||
self.assertEqual(False, res2)
|
||||
|
||||
# test that trying to un-suspend user who isn't suspended doesn't cause problems
|
||||
channel3 = self.make_request(
|
||||
"PUT",
|
||||
f"/_synapse/admin/v1/suspend/{self.bad_user}",
|
||||
{"suspend": False},
|
||||
access_token=self.admin_tok,
|
||||
)
|
||||
self.assertEqual(channel3.code, 200)
|
||||
self.assertEqual(channel3.json_body, {f"user_{self.bad_user}_suspended": False})
|
||||
|
||||
res3 = self.get_success(self.store.get_user_suspended_status(self.bad_user))
|
||||
self.assertEqual(False, res3)
|
||||
|
||||
# test that trying to suspend user who is already suspended doesn't cause problems
|
||||
channel4 = self.make_request(
|
||||
"PUT",
|
||||
f"/_synapse/admin/v1/suspend/{self.bad_user}",
|
||||
{"suspend": True},
|
||||
access_token=self.admin_tok,
|
||||
)
|
||||
self.assertEqual(channel4.code, 200)
|
||||
self.assertEqual(channel4.json_body, {f"user_{self.bad_user}_suspended": True})
|
||||
|
||||
res4 = self.get_success(self.store.get_user_suspended_status(self.bad_user))
|
||||
self.assertEqual(True, res4)
|
||||
|
||||
channel5 = self.make_request(
|
||||
"PUT",
|
||||
f"/_synapse/admin/v1/suspend/{self.bad_user}",
|
||||
{"suspend": True},
|
||||
access_token=self.admin_tok,
|
||||
)
|
||||
self.assertEqual(channel5.code, 200)
|
||||
self.assertEqual(channel5.json_body, {f"user_{self.bad_user}_suspended": True})
|
||||
|
||||
res5 = self.get_success(self.store.get_user_suspended_status(self.bad_user))
|
||||
self.assertEqual(True, res5)
|
||||
|
|
|
@ -18,6 +18,7 @@
|
|||
# [This file includes modifications made by New Vector Limited]
|
||||
#
|
||||
#
|
||||
from typing import List, Optional, Tuple
|
||||
from unittest.mock import AsyncMock, Mock
|
||||
|
||||
from twisted.test.proto_helpers import MemoryReactor
|
||||
|
@ -48,6 +49,14 @@ class HTTPPusherTests(HomeserverTestCase):
|
|||
self.sync_handler = homeserver.get_sync_handler()
|
||||
self.auth_handler = homeserver.get_auth_handler()
|
||||
|
||||
self.user_id = self.register_user("user", "pass")
|
||||
self.access_token = self.login("user", "pass")
|
||||
self.other_user_id = self.register_user("otheruser", "pass")
|
||||
self.other_access_token = self.login("otheruser", "pass")
|
||||
|
||||
# Create a room
|
||||
self.room_id = self.helper.create_room_as(self.user_id, tok=self.access_token)
|
||||
|
||||
def make_homeserver(self, reactor: MemoryReactor, clock: Clock) -> HomeServer:
|
||||
# Mock out the calls over federation.
|
||||
fed_transport_client = Mock(spec=["send_transaction"])
|
||||
|
@ -61,32 +70,22 @@ class HTTPPusherTests(HomeserverTestCase):
|
|||
"""
|
||||
Local users will get notified for invites
|
||||
"""
|
||||
|
||||
user_id = self.register_user("user", "pass")
|
||||
access_token = self.login("user", "pass")
|
||||
other_user_id = self.register_user("otheruser", "pass")
|
||||
other_access_token = self.login("otheruser", "pass")
|
||||
|
||||
# Create a room
|
||||
room = self.helper.create_room_as(user_id, tok=access_token)
|
||||
|
||||
# Check we start with no pushes
|
||||
channel = self.make_request(
|
||||
"GET",
|
||||
"/notifications",
|
||||
access_token=other_access_token,
|
||||
)
|
||||
self.assertEqual(channel.code, 200, channel.result)
|
||||
self.assertEqual(len(channel.json_body["notifications"]), 0, channel.json_body)
|
||||
self._request_notifications(from_token=None, limit=1, expected_count=0)
|
||||
|
||||
# Send an invite
|
||||
self.helper.invite(room=room, src=user_id, targ=other_user_id, tok=access_token)
|
||||
self.helper.invite(
|
||||
room=self.room_id,
|
||||
src=self.user_id,
|
||||
targ=self.other_user_id,
|
||||
tok=self.access_token,
|
||||
)
|
||||
|
||||
# We should have a notification now
|
||||
channel = self.make_request(
|
||||
"GET",
|
||||
"/notifications",
|
||||
access_token=other_access_token,
|
||||
access_token=self.other_access_token,
|
||||
)
|
||||
self.assertEqual(channel.code, 200)
|
||||
self.assertEqual(len(channel.json_body["notifications"]), 1, channel.json_body)
|
||||
|
@ -95,3 +94,139 @@ class HTTPPusherTests(HomeserverTestCase):
|
|||
"invite",
|
||||
channel.json_body,
|
||||
)
|
||||
|
||||
def test_pagination_of_notifications(self) -> None:
|
||||
"""
|
||||
Check that pagination of notifications works.
|
||||
"""
|
||||
# Check we start with no pushes
|
||||
self._request_notifications(from_token=None, limit=1, expected_count=0)
|
||||
|
||||
# Send an invite and have the other user join the room.
|
||||
self.helper.invite(
|
||||
room=self.room_id,
|
||||
src=self.user_id,
|
||||
targ=self.other_user_id,
|
||||
tok=self.access_token,
|
||||
)
|
||||
self.helper.join(self.room_id, self.other_user_id, tok=self.other_access_token)
|
||||
|
||||
# Send 5 messages in the room and note down their event IDs.
|
||||
sent_event_ids = []
|
||||
for _ in range(5):
|
||||
resp = self.helper.send_event(
|
||||
self.room_id,
|
||||
"m.room.message",
|
||||
{"body": "honk", "msgtype": "m.text"},
|
||||
tok=self.access_token,
|
||||
)
|
||||
sent_event_ids.append(resp["event_id"])
|
||||
|
||||
# We expect to get notifications for messages in reverse order.
|
||||
# So reverse this list of event IDs to make it easier to compare
|
||||
# against later.
|
||||
sent_event_ids.reverse()
|
||||
|
||||
# We should have a few notifications now. Let's try and fetch the first 2.
|
||||
notification_event_ids, _ = self._request_notifications(
|
||||
from_token=None, limit=2, expected_count=2
|
||||
)
|
||||
|
||||
# Check we got the expected event IDs back.
|
||||
self.assertEqual(notification_event_ids, sent_event_ids[:2])
|
||||
|
||||
# Try requesting again without a 'from' query parameter. We should get the
|
||||
# same two notifications back.
|
||||
notification_event_ids, next_token = self._request_notifications(
|
||||
from_token=None, limit=2, expected_count=2
|
||||
)
|
||||
self.assertEqual(notification_event_ids, sent_event_ids[:2])
|
||||
|
||||
# Ask for the next 5 notifications, though there should only be
|
||||
# 4 remaining; the next 3 messages and the invite.
|
||||
#
|
||||
# We need to use the "next_token" from the response as the "from"
|
||||
# query parameter in the next request in order to paginate.
|
||||
notification_event_ids, next_token = self._request_notifications(
|
||||
from_token=next_token, limit=5, expected_count=4
|
||||
)
|
||||
# Ensure we chop off the invite on the end.
|
||||
notification_event_ids = notification_event_ids[:-1]
|
||||
self.assertEqual(notification_event_ids, sent_event_ids[2:])
|
||||
|
||||
def _request_notifications(
|
||||
self, from_token: Optional[str], limit: int, expected_count: int
|
||||
) -> Tuple[List[str], str]:
|
||||
"""
|
||||
Make a request to /notifications to get the latest events to be notified about.
|
||||
|
||||
Only the event IDs are returned. The request is made by the "other user".
|
||||
|
||||
Args:
|
||||
from_token: An optional starting parameter.
|
||||
limit: The maximum number of results to return.
|
||||
expected_count: The number of events to expect in the response.
|
||||
|
||||
Returns:
|
||||
A list of event IDs that the client should be notified about.
|
||||
Events are returned newest-first.
|
||||
"""
|
||||
# Construct the request path.
|
||||
path = f"/notifications?limit={limit}"
|
||||
if from_token is not None:
|
||||
path += f"&from={from_token}"
|
||||
|
||||
channel = self.make_request(
|
||||
"GET",
|
||||
path,
|
||||
access_token=self.other_access_token,
|
||||
)
|
||||
|
||||
self.assertEqual(channel.code, 200)
|
||||
self.assertEqual(
|
||||
len(channel.json_body["notifications"]), expected_count, channel.json_body
|
||||
)
|
||||
|
||||
# Extract the necessary data from the response.
|
||||
next_token = channel.json_body["next_token"]
|
||||
event_ids = [
|
||||
event["event"]["event_id"] for event in channel.json_body["notifications"]
|
||||
]
|
||||
|
||||
return event_ids, next_token
|
||||
|
||||
def test_parameters(self) -> None:
|
||||
"""
|
||||
Test that appropriate errors are returned when query parameters are malformed.
|
||||
"""
|
||||
# Test that no parameters are required.
|
||||
channel = self.make_request(
|
||||
"GET",
|
||||
"/notifications",
|
||||
access_token=self.other_access_token,
|
||||
)
|
||||
self.assertEqual(channel.code, 200)
|
||||
|
||||
# Test that limit cannot be negative
|
||||
channel = self.make_request(
|
||||
"GET",
|
||||
"/notifications?limit=-1",
|
||||
access_token=self.other_access_token,
|
||||
)
|
||||
self.assertEqual(channel.code, 400)
|
||||
|
||||
# Test that the 'limit' parameter must be an integer.
|
||||
channel = self.make_request(
|
||||
"GET",
|
||||
"/notifications?limit=foobar",
|
||||
access_token=self.other_access_token,
|
||||
)
|
||||
self.assertEqual(channel.code, 400)
|
||||
|
||||
# Test that the 'from' parameter must be an integer.
|
||||
channel = self.make_request(
|
||||
"GET",
|
||||
"/notifications?from=osborne",
|
||||
access_token=self.other_access_token,
|
||||
)
|
||||
self.assertEqual(channel.code, 400)
|
||||
|
|
|
@ -3819,3 +3819,108 @@ class TimestampLookupTestCase(unittest.HomeserverTestCase):
|
|||
|
||||
# Make sure the outlier event is not returned
|
||||
self.assertNotEqual(channel.json_body["event_id"], outlier_event.event_id)
|
||||
|
||||
|
||||
class UserSuspensionTests(unittest.HomeserverTestCase):
|
||||
servlets = [
|
||||
admin.register_servlets,
|
||||
login.register_servlets,
|
||||
room.register_servlets,
|
||||
profile.register_servlets,
|
||||
]
|
||||
|
||||
def prepare(self, reactor: MemoryReactor, clock: Clock, hs: HomeServer) -> None:
|
||||
self.user1 = self.register_user("thomas", "hackme")
|
||||
self.tok1 = self.login("thomas", "hackme")
|
||||
|
||||
self.user2 = self.register_user("teresa", "hackme")
|
||||
self.tok2 = self.login("teresa", "hackme")
|
||||
|
||||
self.room1 = self.helper.create_room_as(room_creator=self.user1, tok=self.tok1)
|
||||
self.store = hs.get_datastores().main
|
||||
|
||||
def test_suspended_user_cannot_send_message_to_room(self) -> None:
|
||||
# set the user as suspended
|
||||
self.get_success(self.store.set_user_suspended_status(self.user1, True))
|
||||
|
||||
channel = self.make_request(
|
||||
"PUT",
|
||||
f"/rooms/{self.room1}/send/m.room.message/1",
|
||||
access_token=self.tok1,
|
||||
content={"body": "hello", "msgtype": "m.text"},
|
||||
)
|
||||
self.assertEqual(
|
||||
channel.json_body["errcode"], "ORG.MATRIX.MSC3823.USER_ACCOUNT_SUSPENDED"
|
||||
)
|
||||
|
||||
def test_suspended_user_cannot_change_profile_data(self) -> None:
|
||||
# set the user as suspended
|
||||
self.get_success(self.store.set_user_suspended_status(self.user1, True))
|
||||
|
||||
channel = self.make_request(
|
||||
"PUT",
|
||||
f"/_matrix/client/v3/profile/{self.user1}/avatar_url",
|
||||
access_token=self.tok1,
|
||||
content={"avatar_url": "mxc://matrix.org/wefh34uihSDRGhw34"},
|
||||
shorthand=False,
|
||||
)
|
||||
self.assertEqual(
|
||||
channel.json_body["errcode"], "ORG.MATRIX.MSC3823.USER_ACCOUNT_SUSPENDED"
|
||||
)
|
||||
|
||||
channel2 = self.make_request(
|
||||
"PUT",
|
||||
f"/_matrix/client/v3/profile/{self.user1}/displayname",
|
||||
access_token=self.tok1,
|
||||
content={"displayname": "something offensive"},
|
||||
shorthand=False,
|
||||
)
|
||||
self.assertEqual(
|
||||
channel2.json_body["errcode"], "ORG.MATRIX.MSC3823.USER_ACCOUNT_SUSPENDED"
|
||||
)
|
||||
|
||||
def test_suspended_user_cannot_redact_messages_other_than_their_own(self) -> None:
|
||||
# first user sends message
|
||||
self.make_request("POST", f"/rooms/{self.room1}/join", access_token=self.tok2)
|
||||
res = self.helper.send_event(
|
||||
self.room1,
|
||||
"m.room.message",
|
||||
{"body": "hello", "msgtype": "m.text"},
|
||||
tok=self.tok2,
|
||||
)
|
||||
event_id = res["event_id"]
|
||||
|
||||
# second user sends message
|
||||
self.make_request("POST", f"/rooms/{self.room1}/join", access_token=self.tok1)
|
||||
res2 = self.helper.send_event(
|
||||
self.room1,
|
||||
"m.room.message",
|
||||
{"body": "bad_message", "msgtype": "m.text"},
|
||||
tok=self.tok1,
|
||||
)
|
||||
event_id2 = res2["event_id"]
|
||||
|
||||
# set the second user as suspended
|
||||
self.get_success(self.store.set_user_suspended_status(self.user1, True))
|
||||
|
||||
# second user can't redact first user's message
|
||||
channel = self.make_request(
|
||||
"PUT",
|
||||
f"/_matrix/client/v3/rooms/{self.room1}/redact/{event_id}/1",
|
||||
access_token=self.tok1,
|
||||
content={"reason": "bogus"},
|
||||
shorthand=False,
|
||||
)
|
||||
self.assertEqual(
|
||||
channel.json_body["errcode"], "ORG.MATRIX.MSC3823.USER_ACCOUNT_SUSPENDED"
|
||||
)
|
||||
|
||||
# but can redact their own
|
||||
channel = self.make_request(
|
||||
"PUT",
|
||||
f"/_matrix/client/v3/rooms/{self.room1}/redact/{event_id2}/1",
|
||||
access_token=self.tok1,
|
||||
content={"reason": "bogus"},
|
||||
shorthand=False,
|
||||
)
|
||||
self.assertEqual(channel.code, 200)
|
||||
|
|
|
@ -19,7 +19,8 @@
|
|||
#
|
||||
#
|
||||
import json
|
||||
from typing import List
|
||||
import logging
|
||||
from typing import Dict, List
|
||||
|
||||
from parameterized import parameterized, parameterized_class
|
||||
|
||||
|
@ -44,6 +45,8 @@ from tests.federation.transport.test_knocking import (
|
|||
)
|
||||
from tests.server import TimedOutException
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class FilterTestCase(unittest.HomeserverTestCase):
|
||||
user_id = "@apple:test"
|
||||
|
@ -1228,16 +1231,64 @@ class SlidingSyncTestCase(unittest.HomeserverTestCase):
|
|||
|
||||
def prepare(self, reactor: MemoryReactor, clock: Clock, hs: HomeServer) -> None:
|
||||
self.store = hs.get_datastores().main
|
||||
self.sync_endpoint = "/_matrix/client/unstable/org.matrix.msc3575/sync"
|
||||
self.sync_endpoint = (
|
||||
"/_matrix/client/unstable/org.matrix.simplified_msc3575/sync"
|
||||
)
|
||||
self.store = hs.get_datastores().main
|
||||
self.event_sources = hs.get_event_sources()
|
||||
|
||||
def _add_new_dm_to_global_account_data(
|
||||
self, source_user_id: str, target_user_id: str, target_room_id: str
|
||||
) -> None:
|
||||
"""
|
||||
Helper to handle inserting a new DM for the source user into global account data
|
||||
(handles all of the list merging).
|
||||
|
||||
Args:
|
||||
source_user_id: The user ID of the DM mapping we're going to update
|
||||
target_user_id: User ID of the person the DM is with
|
||||
target_room_id: Room ID of the DM
|
||||
"""
|
||||
|
||||
# Get the current DM map
|
||||
existing_dm_map = self.get_success(
|
||||
self.store.get_global_account_data_by_type_for_user(
|
||||
source_user_id, AccountDataTypes.DIRECT
|
||||
)
|
||||
)
|
||||
# Scrutinize the account data since it has no concrete type. We're just copying
|
||||
# everything into a known type. It should be a mapping from user ID to a list of
|
||||
# room IDs. Ignore anything else.
|
||||
new_dm_map: Dict[str, List[str]] = {}
|
||||
if isinstance(existing_dm_map, dict):
|
||||
for user_id, room_ids in existing_dm_map.items():
|
||||
if isinstance(user_id, str) and isinstance(room_ids, list):
|
||||
for room_id in room_ids:
|
||||
if isinstance(room_id, str):
|
||||
new_dm_map[user_id] = new_dm_map.get(user_id, []) + [
|
||||
room_id
|
||||
]
|
||||
|
||||
# Add the new DM to the map
|
||||
new_dm_map[target_user_id] = new_dm_map.get(target_user_id, []) + [
|
||||
target_room_id
|
||||
]
|
||||
# Save the DM map to global account data
|
||||
self.get_success(
|
||||
self.store.add_account_data_for_user(
|
||||
source_user_id,
|
||||
AccountDataTypes.DIRECT,
|
||||
new_dm_map,
|
||||
)
|
||||
)
|
||||
|
||||
def _create_dm_room(
|
||||
self,
|
||||
inviter_user_id: str,
|
||||
inviter_tok: str,
|
||||
invitee_user_id: str,
|
||||
invitee_tok: str,
|
||||
should_join_room: bool = True,
|
||||
) -> str:
|
||||
"""
|
||||
Helper to create a DM room as the "inviter" and invite the "invitee" user to the
|
||||
|
@ -1258,24 +1309,17 @@ class SlidingSyncTestCase(unittest.HomeserverTestCase):
|
|||
tok=inviter_tok,
|
||||
extra_data={"is_direct": True},
|
||||
)
|
||||
# Person that was invited joins the room
|
||||
self.helper.join(room_id, invitee_user_id, tok=invitee_tok)
|
||||
if should_join_room:
|
||||
# Person that was invited joins the room
|
||||
self.helper.join(room_id, invitee_user_id, tok=invitee_tok)
|
||||
|
||||
# Mimic the client setting the room as a direct message in the global account
|
||||
# data
|
||||
self.get_success(
|
||||
self.store.add_account_data_for_user(
|
||||
invitee_user_id,
|
||||
AccountDataTypes.DIRECT,
|
||||
{inviter_user_id: [room_id]},
|
||||
)
|
||||
# data for both users.
|
||||
self._add_new_dm_to_global_account_data(
|
||||
invitee_user_id, inviter_user_id, room_id
|
||||
)
|
||||
self.get_success(
|
||||
self.store.add_account_data_for_user(
|
||||
inviter_user_id,
|
||||
AccountDataTypes.DIRECT,
|
||||
{invitee_user_id: [room_id]},
|
||||
)
|
||||
self._add_new_dm_to_global_account_data(
|
||||
inviter_user_id, invitee_user_id, room_id
|
||||
)
|
||||
|
||||
return room_id
|
||||
|
@ -1395,15 +1439,28 @@ class SlidingSyncTestCase(unittest.HomeserverTestCase):
|
|||
user2_tok = self.login(user2_id, "pass")
|
||||
|
||||
# Create a DM room
|
||||
dm_room_id = self._create_dm_room(
|
||||
joined_dm_room_id = self._create_dm_room(
|
||||
inviter_user_id=user1_id,
|
||||
inviter_tok=user1_tok,
|
||||
invitee_user_id=user2_id,
|
||||
invitee_tok=user2_tok,
|
||||
should_join_room=True,
|
||||
)
|
||||
invited_dm_room_id = self._create_dm_room(
|
||||
inviter_user_id=user1_id,
|
||||
inviter_tok=user1_tok,
|
||||
invitee_user_id=user2_id,
|
||||
invitee_tok=user2_tok,
|
||||
should_join_room=False,
|
||||
)
|
||||
|
||||
# Create a normal room
|
||||
room_id = self.helper.create_room_as(user1_id, tok=user1_tok, is_public=True)
|
||||
room_id = self.helper.create_room_as(user1_id, tok=user2_tok)
|
||||
self.helper.join(room_id, user1_id, tok=user1_tok)
|
||||
|
||||
# Create a room that user1 is invited to
|
||||
invite_room_id = self.helper.create_room_as(user1_id, tok=user2_tok)
|
||||
self.helper.invite(invite_room_id, src=user2_id, targ=user1_id, tok=user2_tok)
|
||||
|
||||
# Make the Sliding Sync request
|
||||
channel = self.make_request(
|
||||
|
@ -1411,18 +1468,34 @@ class SlidingSyncTestCase(unittest.HomeserverTestCase):
|
|||
self.sync_endpoint,
|
||||
{
|
||||
"lists": {
|
||||
# Absense of filters does not imply "False" values
|
||||
"all": {
|
||||
"ranges": [[0, 99]],
|
||||
"required_state": [],
|
||||
"timeline_limit": 1,
|
||||
"filters": {},
|
||||
},
|
||||
# Test single truthy filter
|
||||
"dms": {
|
||||
"ranges": [[0, 99]],
|
||||
"required_state": [],
|
||||
"timeline_limit": 1,
|
||||
"filters": {"is_dm": True},
|
||||
},
|
||||
"foo-list": {
|
||||
# Test single falsy filter
|
||||
"non-dms": {
|
||||
"ranges": [[0, 99]],
|
||||
"required_state": [],
|
||||
"timeline_limit": 1,
|
||||
"filters": {"is_dm": False},
|
||||
},
|
||||
# Test how multiple filters should stack (AND'd together)
|
||||
"room-invites": {
|
||||
"ranges": [[0, 99]],
|
||||
"required_state": [],
|
||||
"timeline_limit": 1,
|
||||
"filters": {"is_dm": False, "is_invite": True},
|
||||
},
|
||||
}
|
||||
},
|
||||
access_token=user1_tok,
|
||||
|
@ -1432,32 +1505,59 @@ class SlidingSyncTestCase(unittest.HomeserverTestCase):
|
|||
# Make sure it has the foo-list we requested
|
||||
self.assertListEqual(
|
||||
list(channel.json_body["lists"].keys()),
|
||||
["dms", "foo-list"],
|
||||
["all", "dms", "non-dms", "room-invites"],
|
||||
channel.json_body["lists"].keys(),
|
||||
)
|
||||
|
||||
# Make sure the list includes the room we are joined to
|
||||
# Make sure the lists have the correct rooms
|
||||
self.assertListEqual(
|
||||
list(channel.json_body["lists"]["all"]["ops"]),
|
||||
[
|
||||
{
|
||||
"op": "SYNC",
|
||||
"range": [0, 99],
|
||||
"room_ids": [
|
||||
invite_room_id,
|
||||
room_id,
|
||||
invited_dm_room_id,
|
||||
joined_dm_room_id,
|
||||
],
|
||||
}
|
||||
],
|
||||
list(channel.json_body["lists"]["all"]),
|
||||
)
|
||||
self.assertListEqual(
|
||||
list(channel.json_body["lists"]["dms"]["ops"]),
|
||||
[
|
||||
{
|
||||
"op": "SYNC",
|
||||
"range": [0, 99],
|
||||
"room_ids": [dm_room_id],
|
||||
"room_ids": [invited_dm_room_id, joined_dm_room_id],
|
||||
}
|
||||
],
|
||||
list(channel.json_body["lists"]["dms"]),
|
||||
)
|
||||
self.assertListEqual(
|
||||
list(channel.json_body["lists"]["foo-list"]["ops"]),
|
||||
list(channel.json_body["lists"]["non-dms"]["ops"]),
|
||||
[
|
||||
{
|
||||
"op": "SYNC",
|
||||
"range": [0, 99],
|
||||
"room_ids": [room_id],
|
||||
"room_ids": [invite_room_id, room_id],
|
||||
}
|
||||
],
|
||||
list(channel.json_body["lists"]["foo-list"]),
|
||||
list(channel.json_body["lists"]["non-dms"]),
|
||||
)
|
||||
self.assertListEqual(
|
||||
list(channel.json_body["lists"]["room-invites"]["ops"]),
|
||||
[
|
||||
{
|
||||
"op": "SYNC",
|
||||
"range": [0, 99],
|
||||
"room_ids": [invite_room_id],
|
||||
}
|
||||
],
|
||||
list(channel.json_body["lists"]["room-invites"]),
|
||||
)
|
||||
|
||||
def test_sort_list(self) -> None:
|
||||
|
|
|
@ -36,6 +36,14 @@ class DeviceStoreTestCase(HomeserverTestCase):
|
|||
def prepare(self, reactor: MemoryReactor, clock: Clock, hs: HomeServer) -> None:
|
||||
self.store = hs.get_datastores().main
|
||||
|
||||
def default_config(self) -> JsonDict:
|
||||
config = super().default_config()
|
||||
|
||||
# We 'enable' federation otherwise `get_device_updates_by_remote` will
|
||||
# throw an exception.
|
||||
config["federation_sender_instances"] = ["master"]
|
||||
return config
|
||||
|
||||
def add_device_change(self, user_id: str, device_ids: List[str], host: str) -> None:
|
||||
"""Add a device list change for the given device to
|
||||
`device_lists_outbound_pokes` table.
|
||||
|
|
|
@ -447,7 +447,14 @@ class EventChainStoreTestCase(HomeserverTestCase):
|
|||
)
|
||||
|
||||
# Actually call the function that calculates the auth chain stuff.
|
||||
persist_events_store._persist_event_auth_chain_txn(txn, events)
|
||||
new_event_links = (
|
||||
persist_events_store.calculate_chain_cover_index_for_events_txn(
|
||||
txn, events[0].room_id, [e for e in events if e.is_state()]
|
||||
)
|
||||
)
|
||||
persist_events_store._persist_event_auth_chain_txn(
|
||||
txn, events, new_event_links
|
||||
)
|
||||
|
||||
self.get_success(
|
||||
persist_events_store.db_pool.runInteraction(
|
||||
|
|
|
@ -365,12 +365,19 @@ class EventFederationWorkerStoreTestCase(tests.unittest.HomeserverTestCase):
|
|||
},
|
||||
)
|
||||
|
||||
events = [
|
||||
cast(EventBase, FakeEvent(event_id, room_id, AUTH_GRAPH[event_id]))
|
||||
for event_id in AUTH_GRAPH
|
||||
]
|
||||
new_event_links = (
|
||||
self.persist_events.calculate_chain_cover_index_for_events_txn(
|
||||
txn, room_id, [e for e in events if e.is_state()]
|
||||
)
|
||||
)
|
||||
self.persist_events._persist_event_auth_chain_txn(
|
||||
txn,
|
||||
[
|
||||
cast(EventBase, FakeEvent(event_id, room_id, AUTH_GRAPH[event_id]))
|
||||
for event_id in AUTH_GRAPH
|
||||
],
|
||||
events,
|
||||
new_event_links,
|
||||
)
|
||||
|
||||
self.get_success(
|
||||
|
@ -544,6 +551,9 @@ class EventFederationWorkerStoreTestCase(tests.unittest.HomeserverTestCase):
|
|||
rooms.
|
||||
"""
|
||||
|
||||
# We allow partial covers for this test
|
||||
self.hs.get_datastores().main.tests_allow_no_chain_cover_index = True
|
||||
|
||||
room_id = "@ROOM:local"
|
||||
|
||||
# The silly auth graph we use to test the auth difference algorithm,
|
||||
|
@ -628,13 +638,20 @@ class EventFederationWorkerStoreTestCase(tests.unittest.HomeserverTestCase):
|
|||
)
|
||||
|
||||
# Insert all events apart from 'B'
|
||||
events = [
|
||||
cast(EventBase, FakeEvent(event_id, room_id, auth_graph[event_id]))
|
||||
for event_id in auth_graph
|
||||
if event_id != "b"
|
||||
]
|
||||
new_event_links = (
|
||||
self.persist_events.calculate_chain_cover_index_for_events_txn(
|
||||
txn, room_id, [e for e in events if e.is_state()]
|
||||
)
|
||||
)
|
||||
self.persist_events._persist_event_auth_chain_txn(
|
||||
txn,
|
||||
[
|
||||
cast(EventBase, FakeEvent(event_id, room_id, auth_graph[event_id]))
|
||||
for event_id in auth_graph
|
||||
if event_id != "b"
|
||||
],
|
||||
events,
|
||||
new_event_links,
|
||||
)
|
||||
|
||||
# Now we insert the event 'B' without a chain cover, by temporarily
|
||||
|
@ -647,9 +664,14 @@ class EventFederationWorkerStoreTestCase(tests.unittest.HomeserverTestCase):
|
|||
updatevalues={"has_auth_chain_index": False},
|
||||
)
|
||||
|
||||
events = [cast(EventBase, FakeEvent("b", room_id, auth_graph["b"]))]
|
||||
new_event_links = (
|
||||
self.persist_events.calculate_chain_cover_index_for_events_txn(
|
||||
txn, room_id, [e for e in events if e.is_state()]
|
||||
)
|
||||
)
|
||||
self.persist_events._persist_event_auth_chain_txn(
|
||||
txn,
|
||||
[cast(EventBase, FakeEvent("b", room_id, auth_graph["b"]))],
|
||||
txn, events, new_event_links
|
||||
)
|
||||
|
||||
self.store.db_pool.simple_update_txn(
|
||||
|
|
|
@ -344,6 +344,8 @@ class HomeserverTestCase(TestCase):
|
|||
self._hs_args = {"clock": self.clock, "reactor": self.reactor}
|
||||
self.hs = self.make_homeserver(self.reactor, self.clock)
|
||||
|
||||
self.hs.get_datastores().main.tests_allow_no_chain_cover_index = False
|
||||
|
||||
# Honour the `use_frozen_dicts` config option. We have to do this
|
||||
# manually because this is taken care of in the app `start` code, which
|
||||
# we don't run. Plus we want to reset it on tearDown.
|
||||
|
|
Loading…
Reference in a new issue