Commit graph

3667 commits

Author SHA1 Message Date
Eric Eastwood
935b98c474 All get_current_state_delta_membership_changes_for_user(...) tests passing 2024-06-27 00:49:51 -05:00
Eric Eastwood
7eb1806ee3 Fix lints 2024-06-26 21:06:05 -05:00
Eric Eastwood
b7914e7676 Add skipped test for state resets 2024-06-26 21:01:54 -05:00
Eric Eastwood
8df39d1baf Remove redundant instance_name column 2024-06-26 20:22:03 -05:00
Eric Eastwood
2a944ffcef Add state of the db in each situation 2024-06-26 20:20:17 -05:00
Eric Eastwood
48d0acfbcd Actually test get_current_state_delta_membership_changes_for_user(...) in remote join test 2024-06-26 19:52:12 -05:00
Eric Eastwood
0b9a903ca1 Add test that remotely joins room 2024-06-26 19:35:18 -05:00
Eric Eastwood
ec2d8dc1e3 Create events using helper 2024-06-26 18:52:24 -05:00
Eric Eastwood
2bf39231ed Add some tests for get_current_state_delta_membership_changes_for_user(...) 2024-06-26 18:40:36 -05:00
Eric Eastwood
7395e10420 Fix builtins.SyntaxError: EOL while scanning string literal (test_sync.py, line 1885)
See https://github.com/element-hq/synapse/actions/runs/9675073109/job/26692003103?pr=17320#step:9:5552

Worked fine locally but failed in CI with Python 3.8
2024-06-26 02:19:31 -05:00
Eric Eastwood
5c213158d7 Merge branch 'develop' into madlittlemods/sliding-sync-room-data
Conflicts:
	tests/rest/client/test_sync.py
2024-06-26 01:36:23 -05:00
Eric Eastwood
62c6a4e860 Add newly_joined support to get_sync_room_ids_for_user(...) 2024-06-26 01:10:00 -05:00
Eric Eastwood
fbd92e1c9d Add get_current_state_delta_membership_changes_for_user(...) (using current_state_delta_stream)
(still need to add newly_left rooms back)
2024-06-25 20:16:50 -05:00
Eric Eastwood
4c22131442 Start testing for the correct room membership (failing) 2024-06-25 15:27:43 -05:00
Eric Eastwood
e3e431fab4 Finish up stripped_state for invite rooms
See https://github.com/element-hq/synapse/pull/17320#discussion_r1646581077
2024-06-25 12:35:50 -05:00
Eric Eastwood
7aea406c22 Just stripped_state for invite rooms 2024-06-25 11:18:27 -05:00
Shay
a023538822
Re-introduce federation /download endpoint (#17350) 2024-06-25 14:35:37 +00:00
Erik Johnston
554a92601a
Reintroduce "Reduce device lists replication traffic."" (#17361)
Reintroduces https://github.com/element-hq/synapse/pull/17333


Turns out the reason for revert was down two master instances running
2024-06-25 10:34:34 +01:00
Erik Johnston
a98cb87bee
Revert "Reduce device lists replication traffic." (#17360)
Reverts element-hq/synapse#17333

It looks like master was still sending out replication RDATA with the
old format... somehow
2024-06-25 09:57:34 +01:00
Eric Eastwood
6e8af83193
Add is_invite filtering to Sliding Sync /sync (#17335)
Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
2024-06-24 19:07:56 -05:00
Eric Eastwood
b1b4231ccb Merge branch 'develop' into madlittlemods/sliding-sync-room-data 2024-06-24 10:12:05 -05:00
Erik Johnston
930a64b6c1
Reintroduce #17291. (#17338)
This is #17291 (which got reverted), with some added fixups, and change
so that tests actually pick up the error.

The problem was that we were not calculating any new chain IDs due to a
missing `not` in a condition.
2024-06-24 14:40:28 +00:00
Shay
7a11c0ac4f
Add support for MSC3823 - Account Suspension Part 2 (#17255) 2024-06-24 14:21:22 +01:00
Erik Johnston
cf711ac03c
Reduce device lists replication traffic. (#17333)
Reduce the replication traffic of device lists, by not sending every
destination that needs to be sent the device list update over
replication. Instead a "hosts to send to have been calculated"
notification over replication, and then federation senders read the
destinations from the DB.

For non federation senders this should heavily reduce the impact of a
user in many large rooms changing a device.
2024-06-24 14:15:13 +01:00
Eric Eastwood
884b448012 Update some wording 2024-06-19 13:50:28 -05:00
Eric Eastwood
6942b6476a Merge branch 'develop' into madlittlemods/sliding-sync-room-data 2024-06-19 13:24:12 -05:00
Eric Eastwood
d801db0d96 Fix lints 2024-06-19 13:24:01 -05:00
Eric Eastwood
c81f3006a5 Add better support for leave/ban 2024-06-19 12:54:39 -05:00
Erik Johnston
4243c1f074
Revert "Handle large chain calc better (#17291)" (#17334)
This reverts commit bdf82efea5  (#17291)

This seems to have stopped persisting auth chains for new events, and so
is causing state res to fall back to the slow methods
2024-06-19 17:39:33 +01:00
Erik Johnston
3239b7459c
Register sliding sync under a different path (#17331)
As the API is slightly incompatible.
2024-06-19 16:18:45 +00:00
Alexander Fechler
9104a9f0d0
Filter added to Admin-API GET /rooms (#17276) 2024-06-19 11:45:48 +01:00
Erik Johnston
bdf82efea5
Handle large chain calc better (#17291)
We calculate the auth chain links outside of the main persist event
transaction to ensure that we do not block other event sending during
the calculation.
2024-06-19 10:33:53 +01:00
Andrew Morgan
afaf2d9388
Require the 'from' parameter for /notifications be an integer (#17283)
Co-authored-by: Erik Johnston <erikj@element.io>
2024-06-19 10:05:39 +01:00
Eric Eastwood
57ba0336bd Merge branch 'develop' into madlittlemods/sliding-sync-room-data 2024-06-18 22:31:27 -05:00
Eric Eastwood
1c06153a0d Determine limited before filtering 2024-06-18 22:30:40 -05:00
Eric Eastwood
39b4f10533 Update comments 2024-06-18 19:55:12 -05:00
Eric Eastwood
9791209a3d Add more tests 2024-06-18 18:10:17 -05:00
Eric Eastwood
81d36f36c1 Add tests for limited 2024-06-18 15:28:23 -05:00
Andrew Morgan
199223062a
Revert "Support MSC3916 by adding a federation /download endpoint" (#17325) 2024-06-18 16:54:19 +01:00
Eric Eastwood
a5485437cf
Add is_encrypted filtering to Sliding Sync /sync (#17281)
Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
2024-06-17 12:06:18 -05:00
Eric Eastwood
e5b8a3e37f
Add stream_ordering sort to Sliding Sync /sync (#17293)
Sort is no longer configurable and we always sort rooms by the `stream_ordering` of the last event in the room or the point where the user can see up to in cases of leave/ban/invite/knock.
2024-06-17 11:27:14 -05:00
Quentin Gliech
e88332b5f4
Merge branch 'release-v1.109' into develop 2024-06-17 15:51:16 +02:00
Quentin Gliech
f983a77ab0
Set our own stream position from the current sequence value on startup (#17309) 2024-06-17 11:50:00 +00:00
Erik Johnston
a3cb244755
Automatically apply SQL for inconsistent sequence (#17305)
Rather than forcing the server operator to apply the SQL manually.

This should be safe, as there should be only one writer for these
sequences.
2024-06-14 16:40:29 +01:00
Richard van der Hoff
3aae60f17b
Enable cross-signing key upload without UIA (#17284)
Per MSC3967, which is now stable, we should not require UIA when
uploading cross-signing keys for the first time.

Fixes: #17227
2024-06-14 11:14:56 +01:00
Richard van der Hoff
2c36a679ae
Include user membership on events (#17282)
MSC4115 has now completed FCP, so we can enable it by default and switch
to the stable identifier.
2024-06-13 21:45:54 +00:00
Eric Eastwood
c12ee0d5ba
Add is_dm filtering to Sliding Sync /sync (#17277)
Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
2024-06-13 13:56:58 -05:00
Eric Eastwood
8aaff851b1
Fix newly_left rooms not appearing if we returned early (Sliding Sync) (#17301)
Fix `newly_left` rooms not appearing if we returned early when `membership_snapshot_token.is_before_or_eq(to_token.room_key)`. 

Introduced in https://github.com/element-hq/synapse/pull/17187 (part of Sliding Sync)

The tests didn't catch it because they had a small typo in it `room_id1` vs `room_id2`.

Found while working on https://github.com/element-hq/synapse/pull/17293
2024-06-13 11:36:57 -05:00
Eric Eastwood
8c58eb7f17
Add event.internal_metadata.instance_name (#17300)
Add `event.internal_metadata.instance_name` (the worker instance that persisted the event) to go alongside the existing `event.internal_metadata.stream_ordering`.

`instance_name` is useful to properly compare and query for events with a token since you need to compare both the `stream_ordering` and `instance_name` against the vector clock/`instance_map` in the `RoomStreamToken`.

This is pre-requisite work and may be used in https://github.com/element-hq/synapse/pull/17293

Adding `event.internal_metadata.instance_name` was first mentioned in the initial Sliding Sync PR while pairing with @erikjohnston, see 09609cb0db (diff-5cd773fb307aa754bd3948871ba118b1ef0303f4d72d42a2d21e38242bf4e096R405-R410)
2024-06-13 11:32:50 -05:00
Eric Eastwood
ebdce69f6a
Fix get_last_event_in_room_before_stream_ordering(...) finding the wrong last event (#17295)
PR where this was introduced: https://github.com/matrix-org/synapse/pull/14817

### What does this affect?

`get_last_event_in_room_before_stream_ordering(...)` is used in Sync v2 in a lot of different state calculations.

`get_last_event_in_room_before_stream_ordering(...)`  is also used in `/rooms/{roomId}/members`
2024-06-13 11:00:52 -05:00