Commit graph

23953 commits

Author SHA1 Message Date
Eric Eastwood c7d1fc33a1 Fix separator label 2024-06-05 14:20:02 -05:00
Eric Eastwood 703cdc9c3b Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-0.0.1 2024-06-05 11:55:14 -05:00
Eric Eastwood 567830769b Add validation for membership 2024-06-05 11:42:34 -05:00
Eric Eastwood 278ba63953 No need to check from/to token relationship
See https://github.com/element-hq/synapse/pull/17187#discussion_r1627771338

`get_membership_changes_for_user(from_key=xxx, to_key=xxx)` will handle
getting out what we need and filter the results based on the tokens
(even in cases where the from_key is ahead of the to_key).
2024-06-05 11:22:51 -05:00
Eric Eastwood 6a6cdc61f3 Use Set because Tuple doesn't allow - operations 2024-06-05 11:03:06 -05:00
Eric Eastwood 1fc1b58ba2 Remove assert since we no longer need that information
See https://github.com/element-hq/synapse/pull/17187#discussion_r1627794551
2024-06-05 10:57:53 -05:00
Eric Eastwood 7bbe2ed8d8 More clear way to express what membership we want to display
See https://github.com/element-hq/synapse/pull/17187#discussion_r1627754354
2024-06-05 10:56:48 -05:00
Eric Eastwood 2af467d4a8
Remove extra for-loop
Co-authored-by: Erik Johnston <erikj@element.io>
2024-06-05 10:52:22 -05:00
Shay fcbc79bb87
Ratelimiting of remote media downloads (#17256) 2024-06-05 13:43:36 +01:00
Erik Johnston aabf577166
Handle hyphens in user dir search porperly (#17254)
c.f. #16675
2024-06-05 10:40:34 +01:00
Eric Eastwood 2864837b65 Allow new get_sync_room_ids_for_user implementation to work with multiple event persisters
Before, the problem scenario would get caught in one of the assertions because
we expect the to_token <= membership_snapshot_token or vice-versa but it's
possible the tokens are intertwined and neither is ahead of each other.
Especially since the `instance_map` in `membership_snapshot_token` is made up
from the `stream_ordering` of membership events at various stream positions
and processed on different instances (not current stream positions).

We get into trouble when stream positions are lagging between workers and our
now/`to_token` doesn't cleanly compare to `membership_snapshot_token`.

What we really want to assert is that the `to_token` <= the stream positions
at the time we asked for the room membership snapshot. Since
`get_rooms_for_local_user_where_membership_is()` doesn't return that
information, the closest we can get is to get the stream positions before we
ask for the room membership snapshot and consider that good enough to compare
against.
2024-06-04 19:50:12 -05:00
Eric Eastwood 3ce08925e3 Add test with multiple event persisters that fails the previous get_sync_room_ids_for_user implementation
The new implementation catches the problem with an assert
but I think it's possible to make it work as well.

```
SYNAPSE_POSTGRES=1 SYNAPSE_POSTGRES_USER=postgres SYNAPSE_TEST_LOG_LEVEL=INFO poetry run trial tests.handlers.test_sliding_sync.GetSyncRoomIdsForUserEventShardTestCase
```
2024-06-04 19:27:28 -05:00
Eric Eastwood dfee21a1f4 Switch fixup order to fix edge case with newly_left rooms
Previously, we added back newly_left rooms and our second
fixup was removing them. We can just switch the order
of the fixups to solve this.
2024-06-04 18:29:14 -05:00
Eric Eastwood d3ce27b957 Balance parenthesis 2024-06-04 13:28:23 -05:00
Eric Eastwood 07f57a4066 Give a summary of what rooms we're looking for 2024-06-04 13:27:35 -05:00
Eric Eastwood f6a5905b6c Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-0.0.1
Conflicts:
	synapse/storage/databases/main/roommember.py
2024-06-04 12:59:33 -05:00
Eric Eastwood 7d8f0ef351
Use fully-qualified PersistedEventPosition when returning RoomsForUser (#17265)
Use fully-qualified `PersistedEventPosition` (`instance_name` and `stream_ordering`) when returning `RoomsForUser` to facilitate proper comparisons and `RoomStreamToken` generation.

Spawning from https://github.com/element-hq/synapse/pull/17187 where we want to utilize this change
2024-06-04 12:58:03 -05:00
Eric Eastwood 54dbc278d2 Add None defaults 2024-06-04 12:40:39 -05:00
Andrew Morgan eab0b548e4 Merge branch 'release-v1.109' into develop 2024-06-04 12:37:07 +01:00
dependabot[bot] 81cef38d4b
Bump sentry-sdk from 2.1.1 to 2.3.1 (#17263) 2024-06-04 11:58:27 +01:00
Andrew Morgan e2f8476044 Fix typo in CHANGES.md 2024-06-04 09:47:28 +01:00
Andrew Morgan 18c1196893 1.109.0rc1 2024-06-04 09:46:09 +01:00
Eric Eastwood 9e46b2a53b Fix typo 2024-06-04 00:52:40 -05:00
Eric Eastwood 03dd87ab3c Add test for notifier.wait_for_stream_token(from_token) 2024-06-03 23:46:47 -05:00
Eric Eastwood 8bb357a35e Note the extras 2024-06-03 23:24:23 -05:00
Eric Eastwood 64df6fbefa Revert change that should be separated and is failing 2024-06-03 23:17:16 -05:00
Eric Eastwood 970a0c623a Adjust wording 2024-06-03 23:08:51 -05:00
Eric Eastwood 3514aa0ff2 Add licensing headers 2024-06-03 22:49:27 -05:00
Eric Eastwood 35db057982 Add support for kicks 2024-06-03 22:47:01 -05:00
Eric Eastwood e4c66b8ac9 Avoid serializing response that will never be heard 2024-06-03 20:13:00 -05:00
Eric Eastwood 1268a5413f Properly compare tokens and event positions
Avoid flawed raw stream comparison.
2024-06-03 20:09:35 -05:00
Eric Eastwood 9c6ec25aca Create membership_snapshot_token with instance_map 2024-06-03 19:23:31 -05:00
Eric Eastwood 8dca8f548c Merge branch 'madlittlemods/rooms-for-user-event-pos' into madlittlemods/msc3575-sliding-sync-0.0.1
Conflicts:
	synapse/handlers/initial_sync.py
	tests/replication/storage/test_events.py
2024-06-03 18:08:56 -05:00
Eric Eastwood 09638ac31d Add changelog 2024-06-03 17:51:03 -05:00
Eric Eastwood 7b41f412c6 Fix random lints 2024-06-03 16:46:34 -05:00
Eric Eastwood 73c20d961f Use method to get instance name in tests 2024-06-03 16:45:08 -05:00
Eric Eastwood 939695dbb5 Update usage 2024-06-03 16:41:19 -05:00
Eric Eastwood 4155e18d76 Fix circular imports when running specific tests
Before:
```
$ SYNAPSE_POSTGRES=1 SYNAPSE_POSTGRES_USER=postgres SYNAPSE_TEST_LOG_LEVEL=INFO poetry run trial tests.replication.storage.test_events
[...]
Traceback (most recent call last):
  File "pypoetry/virtualenvs/matrix-synapse-xCtC9ulO-py3.12/lib/python3.12/site-packages/twisted/trial/runner.py", line 711, in loadByName
    return self.suiteFactory([self.findByName(name, recurse=recurse)])
  File "pypoetry/virtualenvs/matrix-synapse-xCtC9ulO-py3.12/lib/python3.12/site-packages/twisted/trial/runner.py", line 474, in findByName
    obj = reflect.namedModule(searchName)
  File "pypoetry/virtualenvs/matrix-synapse-xCtC9ulO-py3.12/lib/python3.12/site-packages/twisted/python/reflect.py", line 156, in namedModule
    topLevel = __import__(name)
  File "synapse/tests/replication/storage/test_events.py", line 33, in <module>
    from synapse.handlers.room import RoomEventSource
  File "synapse/synapse/handlers/room.py", line 74, in <module>
    from synapse.rest.admin._base import assert_user_is_admin
  File "synapse/synapse/rest/__init__.py", line 24, in <module>
    from synapse.rest import admin
  File "synapse/synapse/rest/admin/__init__.py", line 41, in <module>
    from synapse.handlers.pagination import PURGE_HISTORY_ACTION_NAME
  File "synapse/synapse/handlers/pagination.py", line 30, in <module>
    from synapse.handlers.room import ShutdownRoomParams, ShutdownRoomResponse
builtins.ImportError: cannot import name 'ShutdownRoomParams' from partially initialized module 'synapse.handlers.room' (most likely due to a circular import) (synapse/synapse/handlers/room.py)
```
2024-06-03 16:20:16 -05:00
Eric Eastwood 271a196121 Use fully-qualified PersistedEventPosition when returning membership for user
Spawning from https://github.com/element-hq/synapse/pull/17187
2024-06-03 16:07:56 -05:00
Eric Eastwood a0c042ef12 Re-arrange how this list will be returned 2024-06-03 16:05:18 -05:00
Eric Eastwood 803fbbe95e Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-0.0.1 2024-06-03 16:02:06 -05:00
Eric Eastwood 8f09313d7d Add instance name alongside stream_ordering (RoomsForUser.event_pos) 2024-06-03 16:01:11 -05:00
Eric Eastwood 09609cb0db WIP: TODO comments after pairing with Erik 2024-06-03 14:28:45 -05:00
dependabot[bot] 8a3270075b
Bump types-pyopenssl from 24.0.0.20240311 to 24.1.0.20240425 (#17260)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-03 14:26:41 +01:00
dependabot[bot] f458dff16d
Bump mypy-zope from 1.0.3 to 1.0.4 (#17262)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-03 14:23:29 +01:00
dependabot[bot] 6b709c512d
Bump lxml from 5.2.1 to 5.2.2 (#17261)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-03 14:21:20 +01:00
Erik Johnston 5c2a837e3c
Fix bug where typing replication breaks (#17252)
This can happen on restarts of the service, due to old rooms being
pruned.
2024-05-31 16:07:05 +01:00
Erik Johnston 64f5a4a353
Fix logging errors when receiving invalid User ID for key querys (#17250) 2024-05-31 11:27:56 +01:00
Erik Johnston 7dd14fadb1
Fix sentry default tags (#17251)
This was broken by the sentry 2.0 upgrade

Broke in v1.108.0
2024-05-31 11:27:47 +01:00
Eric Eastwood 49998e053e Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-0.0.1 2024-05-30 09:21:58 -05:00