Commit graph

3653 commits

Author SHA1 Message Date
Eric Eastwood 44deafca7a Add is_encrypted filtering to Sliding Sync /sync
Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
2024-05-30 16:34:48 -05:00
Eric Eastwood 813cda92df Add test to make sure only joined spaces are taken into account 2024-05-30 16:21:14 -05:00
Eric Eastwood c069522881 Fix lints 2024-05-30 15:51:46 -05:00
Eric Eastwood 2cac70182a Update test description 2024-05-30 15:48:16 -05:00
Eric Eastwood d477a63df6 Add spaces filtering to Sliding Sync /sync
Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
2024-05-30 15:46:16 -05:00
Eric Eastwood 6b1eba4fee Add rest test to make sure filters apply 2024-05-30 13:25:07 -05:00
Eric Eastwood 92ea28674f Merge branch 'madlittlemods/msc3575-sliding-sync-0.0.1' into madlittlemods/msc3575-sliding-sync-filtering 2024-05-30 09:22:21 -05:00
Eric Eastwood 49998e053e Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-0.0.1 2024-05-30 09:21:58 -05:00
Erik Johnston 4e3868dc46
Fix deduplicating of membership events to not create unused state groups. (#17164)
We try and deduplicate in two places: 1) really early on, and 2) just
before we persist the event. The first case was broken due to it
occuring before the profile information was added, and so it thought the
event contents were different.

The second case did catch it and handle it correctly, however doing so
creates a redundant state group leading to bloat.

Fixes #3791
2024-05-30 11:33:48 +00:00
Erik Johnston d16910ca02
Replaces all usages of StreamIdGenerator with MultiWriterIdGenerator (#17229)
Replaces all usages of `StreamIdGenerator` with `MultiWriterIdGenerator`, which is safer.
2024-05-30 11:07:32 +00:00
Eric Eastwood f74cc3f166 Merge branch 'madlittlemods/msc3575-sliding-sync-0.0.1' into madlittlemods/msc3575-sliding-sync-filtering 2024-05-29 22:48:14 -05:00
Eric Eastwood 34d67fdcd1 Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-0.0.1 2024-05-29 17:37:31 -05:00
Eric Eastwood 8bf5a623d7 Add rest test 2024-05-29 17:05:53 -05:00
Eric Eastwood 3b670046a9 Rename _create_dm_room 2024-05-29 16:37:21 -05:00
Eric Eastwood 1dd04d2896 Add tests for DM filter 2024-05-29 16:35:27 -05:00
Eric Eastwood d65c694a71 Filter DM from account data 2024-05-29 15:32:06 -05:00
Erik Johnston 466f344547
Move towards using MultiWriterIdGenerator everywhere (#17226)
There is a problem with `StreamIdGenerator` where it can go backwards
over restarts when a stream ID is requested but then not inserted into
the DB. This is problematic if we want to land #17215, and is generally
a potential cause for all sorts of nastiness.

Instead of trying to fix `StreamIdGenerator`, we may as well move to
`MultiWriterIdGenerator` that does not suffer from this problem (the
latest positions are stored in `stream_positions` table). This involves
adding SQLite support to the class.

This only changes id generators that were already using
`MultiWriterIdGenerator` under postgres, a separate PR will move the
rest of the uses of `StreamIdGenerator` over.
2024-05-29 12:19:10 +00:00
Erik Johnston bb5a692946
Fix slipped logging context when media rejected (#17239)
When a module rejects a piece of media we end up trying to close the
same logging context twice.

Instead of fixing the existing code we refactor to use an async context
manager, which is easier to write correctly.
2024-05-29 11:14:42 +01:00
Eric Eastwood 950fd70948 Tweak comments 2024-05-28 15:51:05 -05:00
Eric Eastwood b12fee5f21 Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-0.0.1 2024-05-28 09:51:04 -05:00
Shay 9edb725ebc
Support MSC3916 by adding unstable media endpoints to _matrix/client (#17213)
[MSC3916](https://github.com/matrix-org/matrix-spec-proposals/blob/rav/authentication-for-media/proposals/3916-authentication-for-media.md)
adds new media endpoints under `_matrix/client`. This PR adds the
`/preview_url`, `/config`, and `/thumbnail` endpoints. `/download` will
be added in a follow-up PR once the work for the federation `/download`
endpoint is complete (see
https://github.com/element-hq/synapse/pull/17172).

Should be reviewable commit-by-commit.
2024-05-24 09:47:37 +01:00
Eric Eastwood 37af87a563 Add test to make sure we don't confuse multiple rooms 2024-05-23 14:23:49 -05:00
Eric Eastwood 04eeee648e Merge branch 'madlittlemods/msc3575-sliding-sync-e2ee' into madlittlemods/msc3575-sliding-sync-0.0.1 2024-05-23 12:06:56 -05:00
Eric Eastwood c97251d5ba
Add Sliding Sync /sync/e2ee endpoint for To-Device messages (#17167)
This is being introduced as part of Sliding Sync but doesn't have any sliding window component. It's just a way to get E2EE events without having to sit through a big initial sync  (`/sync` v2). And we can avoid encryption events being backed up by the main sync response or vice-versa.

Part of some Sliding Sync simplification/experimentation. See [this discussion](https://github.com/element-hq/synapse/pull/17167#discussion_r1610495866) for why it may not be as useful as we thought.

Based on:

 - https://github.com/matrix-org/matrix-spec-proposals/pull/3575
 - https://github.com/matrix-org/matrix-spec-proposals/pull/3885
 - https://github.com/matrix-org/matrix-spec-proposals/pull/3884
2024-05-23 12:06:16 -05:00
Eric Eastwood a482545119 Fix test after removing type ignore 2024-05-23 09:46:39 -05:00
Eric Eastwood ab0b844ce1 Add actual typing for params (not just docstrings)
See https://github.com/element-hq/synapse/pull/17167#discussion_r1611301044
2024-05-23 09:31:02 -05:00
Eric Eastwood 6606ac1d07 Add docstring for parametized attributes
See https://github.com/element-hq/synapse/pull/17167#discussion_r1611301044
2024-05-23 09:23:02 -05:00
Eric Eastwood 89db5663a1 Merge branch 'madlittlemods/msc3575-sliding-sync-e2ee' into madlittlemods/msc3575-sliding-sync-0.0.1 2024-05-22 15:07:36 -05:00
Eric Eastwood d4b41aaf43 Fix lints 2024-05-22 15:01:06 -05:00
Eric Eastwood 3da6bc1902 Use @parameterized_class
As suggested in https://github.com/element-hq/synapse/pull/17167#discussion_r1610255726
2024-05-22 14:50:35 -05:00
Eric Eastwood 06ac1da6ec Restore copyright header
See https://github.com/element-hq/synapse/pull/17167#discussion_r1609876335
2024-05-22 14:08:42 -05:00
Eric Eastwood 514aba5810 Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-e2ee
Conflicts:
	synapse/handlers/sync.py
2024-05-22 13:23:47 -05:00
Eric Eastwood c82a084006 Update comments and test docstrings 2024-05-22 13:18:13 -05:00
Eric Eastwood fd355f6b62 WIP 2024-05-21 17:37:46 -05:00
devonh 6a9a641fb8
Bring auto-accept invite logic into Synapse (#17147)
This PR ports the logic from the
[synapse_auto_accept_invite](https://github.com/matrix-org/synapse-auto-accept-invite)
module into synapse.

I went with the naive approach of injecting the "module" next to where
third party modules are currently loaded. If there is a better/preferred
way to handle this, I'm all ears. It wasn't obvious to me if there was a
better location to add this logic that would cleanly apply to all
incoming invite events.

Relies on https://github.com/element-hq/synapse/pull/17166 to fix linter
errors.
2024-05-21 20:09:17 +00:00
Eric Eastwood fe48188f7d Handle more edge cases 2024-05-21 15:05:19 -05:00
Eric Eastwood c826550524 Add some tests 2024-05-21 14:24:03 -05:00
Eric Eastwood 02cecfa626 Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-e2ee 2024-05-20 13:02:49 -05:00
Erik Johnston 52af16c561
Add a short sleep if the request is rate-limited (#17210)
This helps prevent clients from "tight-looping" retrying their request.
2024-05-18 12:03:30 +01:00
Eric Eastwood 4ad7a8b755 No need to change this formatting from develop 2024-05-16 17:24:28 -05:00
Eric Eastwood b23abca9e7 Fix test inheritance
See https://github.com/element-hq/synapse/pull/17167#discussion_r1594517041
2024-05-16 17:04:26 -05:00
Eric Eastwood 7331401e89 Lint 2024-05-16 13:36:34 -05:00
Eric Eastwood 9bdfa16b3e Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-e2ee
Conflicts:
	synapse/handlers/sync.py
	tests/handlers/test_sync.py
2024-05-16 13:10:58 -05:00
Eric Eastwood 28a948f04f
Removed request_key from the SyncConfig (moved outside as its own function parameter) (#17201)
Removed `request_key` from the `SyncConfig` (moved outside as its own function parameter) so it doesn't have to flow into `_generate_sync_entry_for_xxx` methods. This way we can separate the concerns of caching from generating the response and reuse the `_generate_sync_entry_for_xxx` functions as we see fit. Plus caching doesn't really have anything to do with the config of sync.

Split from https://github.com/element-hq/synapse/pull/17167

Spawning from https://github.com/element-hq/synapse/pull/17167#discussion_r1601497279
2024-05-16 11:54:46 -05:00
Erik Johnston 5e892671a7
Fix bug where push rules would be empty in /sync (#17142)
Fixes #16987

Some old accounts seem to have an entry in global account data table for
push rules, which we should ignore
2024-05-16 15:04:14 +01:00
Eric Eastwood d2d48cce85
Refactor Sync handler to be able to return different sync responses (SyncVersion) (#17200)
Refactor Sync handler to be able to be able to return different sync
responses (`SyncVersion`). Preparation to be able support sync v2 and a
new Sliding Sync `/sync/e2ee` endpoint which returns a subset of sync
v2.

Split upon request:
https://github.com/element-hq/synapse/pull/17167#discussion_r1601497279

Split from https://github.com/element-hq/synapse/pull/17167 where we
will add `SyncVersion.E2EE_SYNC` and a new type of sync response.
2024-05-16 11:36:54 +01:00
Erik Johnston 038b9ec59a
An federation whitelist query endpoint extension (#16848)
This is to allow clients to query the configured federation whitelist.
Disabled by default.

---------

Co-authored-by: Devon Hudson <devonhudson@librem.one>
Co-authored-by: devonh <devon.dmytro@gmail.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2024-05-13 19:38:45 +00:00
Eric Eastwood b9e5379836 Describe test 2024-05-09 15:15:40 -05:00
Eric Eastwood 6b7cfd7037 Add tests for device_unused_fallback_key_types in /sync 2024-05-09 15:11:27 -05:00
Eric Eastwood f09835556e Add device_one_time_keys_count tests 2024-05-09 14:41:04 -05:00