Erik Johnston
5624c8b961
In sync wait for worker to catch up since token ( #17215 )
...
Otherwise things will get confused.
An alternative would be to make sure that for lagging stream we don't
return anything (and make sure the returned next_batch token doesn't go
backwards). But that is a faff.
2024-05-30 14:03:49 +01:00
Erik Johnston
4e3868dc46
Fix deduplicating of membership events to not create unused state groups. ( #17164 )
...
We try and deduplicate in two places: 1) really early on, and 2) just
before we persist the event. The first case was broken due to it
occuring before the profile information was added, and so it thought the
event contents were different.
The second case did catch it and handle it correctly, however doing so
creates a redundant state group leading to bloat.
Fixes #3791
2024-05-30 11:33:48 +00:00
Erik Johnston
d16910ca02
Replaces all usages of StreamIdGenerator
with MultiWriterIdGenerator
( #17229 )
...
Replaces all usages of `StreamIdGenerator` with `MultiWriterIdGenerator`, which is safer.
2024-05-30 11:07:32 +00:00
Erik Johnston
225f378ffa
Clean out invalid destinations from outbox ( #17242 )
...
We started ensuring we only insert valid destinations:
https://github.com/element-hq/synapse/pull/17240
2024-05-30 11:25:24 +01:00
Erik Johnston
8bd9ff0783
Ensure we delete media if we reject due to spam check ( #17246 )
...
Fixes up #17239
We need to keep the spam check within the `try/except` block. Also makes
it so that we don't enter the top span twice.
Also also ensures that we get the right thumbnail length.
2024-05-30 11:22:19 +01:00
Eric Eastwood
34d67fdcd1
Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-0.0.1
2024-05-29 17:37:31 -05:00
Eric Eastwood
8bf5a623d7
Add rest test
2024-05-29 17:05:53 -05:00
Erik Johnston
466f344547
Move towards using MultiWriterIdGenerator
everywhere ( #17226 )
...
There is a problem with `StreamIdGenerator` where it can go backwards
over restarts when a stream ID is requested but then not inserted into
the DB. This is problematic if we want to land #17215 , and is generally
a potential cause for all sorts of nastiness.
Instead of trying to fix `StreamIdGenerator`, we may as well move to
`MultiWriterIdGenerator` that does not suffer from this problem (the
latest positions are stored in `stream_positions` table). This involves
adding SQLite support to the class.
This only changes id generators that were already using
`MultiWriterIdGenerator` under postgres, a separate PR will move the
rest of the uses of `StreamIdGenerator` over.
2024-05-29 12:19:10 +00:00
Erik Johnston
726006cdf2
Don't invalidate all get_relations_for_event
on history purge ( #17083 )
...
This is a tree cache already, so may as well move the room ID to the
front and use that
2024-05-29 12:57:10 +01:00
Erik Johnston
967b6948b0
Change allow_unsafe_locale to also apply on new databases ( #17238 )
...
We relax this as there are use cases where this is safe, though it is
still highly recommended that people avoid using it.
2024-05-29 12:04:13 +01:00
Erik Johnston
d7198dfb95
Ignore attempts to send to-device messages to bad users ( #17240 )
...
Currently sending a to-device message to a user ID with a dodgy
destination is accepted, but then ends up spamming the logs when we try
and send to the destination.
An alternative would be to reject the request, but I'm slightly nervous
that could break things.
2024-05-29 11:52:48 +01:00
Erik Johnston
94ef2f4f5d
Handle duplicate OTK uploads racing ( #17241 )
...
Currently this causes one of then to 500.
2024-05-29 11:16:00 +01:00
Erik Johnston
bb5a692946
Fix slipped logging context when media rejected ( #17239 )
...
When a module rejects a piece of media we end up trying to close the
same logging context twice.
Instead of fixing the existing code we refactor to use an async context
manager, which is easier to write correctly.
2024-05-29 11:14:42 +01:00
Eric Eastwood
950fd70948
Tweak comments
2024-05-28 15:51:05 -05:00
Eric Eastwood
a28569f79d
Add understanding of this skip
2024-05-28 15:41:32 -05:00
Eric Eastwood
abf139a3b7
Fill out docstring todo
2024-05-28 15:41:11 -05:00
Eric Eastwood
b632cbb46a
Add better comments
2024-05-28 15:05:35 -05:00
Eric Eastwood
44e9a92f01
Fill in rest docstring
2024-05-28 14:29:41 -05:00
Eric Eastwood
b12fee5f21
Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-0.0.1
2024-05-28 09:51:04 -05:00
Olivier 'reivilibre
ad179b0136
Merge branch 'master' into develop
2024-05-28 13:34:19 +01:00
dependabot[bot]
5147ce294a
Bump phonenumbers from 8.13.35 to 8.13.37 ( #17235 )
2024-05-28 13:26:37 +01:00
Olivier 'reivilibre
f35bc08d39
1.108.0
2024-05-28 11:54:28 +01:00
dependabot[bot]
f2616edb73
Bump pyicu from 2.13 to 2.13.1 ( #17236 )
2024-05-28 11:28:58 +01:00
dependabot[bot]
86a2a0258f
Bump pyopenssl from 24.0.0 to 24.1.0 ( #17234 )
2024-05-28 11:28:32 +01:00
dependabot[bot]
0893ee9af8
Bump prometheus-client from 0.19.0 to 0.20.0 ( #17233 )
2024-05-28 11:28:16 +01:00
dependabot[bot]
887f773472
Bump serde from 1.0.202 to 1.0.203 ( #17232 )
2024-05-28 11:27:51 +01:00
Shay
9edb725ebc
Support MSC3916 by adding unstable media endpoints to _matrix/client
( #17213 )
...
[MSC3916](https://github.com/matrix-org/matrix-spec-proposals/blob/rav/authentication-for-media/proposals/3916-authentication-for-media.md )
adds new media endpoints under `_matrix/client`. This PR adds the
`/preview_url`, `/config`, and `/thumbnail` endpoints. `/download` will
be added in a follow-up PR once the work for the federation `/download`
endpoint is complete (see
https://github.com/element-hq/synapse/pull/17172 ).
Should be reviewable commit-by-commit.
2024-05-24 09:47:37 +01:00
Eric Eastwood
adc0e2f5e8
Fix unserialize type
2024-05-23 17:24:19 -05:00
Eric Eastwood
65d9b7968d
Fix lints
2024-05-23 17:22:27 -05:00
Eric Eastwood
b5b3e77e7e
Fix Pydantic conint
/constr
usage with mypy
...
See https://github.com/pydantic/pydantic/issues/156#issuecomment-1130883884
2024-05-23 16:56:48 -05:00
Eric Eastwood
d1bd02d91d
Add TODO to handle partial stated rooms
2024-05-23 16:16:26 -05:00
Eric Eastwood
f9fa683750
Fix another leaking loop variable
...
See https://github.com/element-hq/synapse/pull/17187#discussion_r1612211662
2024-05-23 15:18:53 -05:00
Eric Eastwood
a822a05bec
Revert as TODO says
2024-05-23 14:25:17 -05:00
Eric Eastwood
37af87a563
Add test to make sure we don't confuse multiple rooms
2024-05-23 14:23:49 -05:00
Eric Eastwood
8c3de846d4
Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-0.0.1
...
Conflicts:
synapse/rest/client/sync.py
2024-05-23 12:07:51 -05:00
Eric Eastwood
04eeee648e
Merge branch 'madlittlemods/msc3575-sliding-sync-e2ee' into madlittlemods/msc3575-sliding-sync-0.0.1
2024-05-23 12:06:56 -05:00
Eric Eastwood
c97251d5ba
Add Sliding Sync /sync/e2ee
endpoint for To-Device messages ( #17167 )
...
This is being introduced as part of Sliding Sync but doesn't have any sliding window component. It's just a way to get E2EE events without having to sit through a big initial sync (`/sync` v2). And we can avoid encryption events being backed up by the main sync response or vice-versa.
Part of some Sliding Sync simplification/experimentation. See [this discussion](https://github.com/element-hq/synapse/pull/17167#discussion_r1610495866 ) for why it may not be as useful as we thought.
Based on:
- https://github.com/matrix-org/matrix-spec-proposals/pull/3575
- https://github.com/matrix-org/matrix-spec-proposals/pull/3885
- https://github.com/matrix-org/matrix-spec-proposals/pull/3884
2024-05-23 12:06:16 -05:00
Eric Eastwood
a482545119
Fix test after removing type ignore
2024-05-23 09:46:39 -05:00
Eric Eastwood
ab0b844ce1
Add actual typing for params (not just docstrings)
...
See https://github.com/element-hq/synapse/pull/17167#discussion_r1611301044
2024-05-23 09:31:02 -05:00
Eric Eastwood
6606ac1d07
Add docstring for parametized attributes
...
See https://github.com/element-hq/synapse/pull/17167#discussion_r1611301044
2024-05-23 09:23:02 -05:00
Eric Eastwood
4c7d7e6365
Encode response
2024-05-22 17:08:34 -05:00
Eric Eastwood
c7f7ae4ec0
Start assembling lists
2024-05-22 16:28:44 -05:00
Eric Eastwood
13d61469b5
Fill out sliding window response types
2024-05-22 15:54:15 -05:00
Eric Eastwood
a7c64761e6
Use client_patterns()
2024-05-22 15:37:34 -05:00
Eric Eastwood
c7b8743454
Add changelog
2024-05-22 15:13:23 -05:00
Eric Eastwood
89db5663a1
Merge branch 'madlittlemods/msc3575-sliding-sync-e2ee' into madlittlemods/msc3575-sliding-sync-0.0.1
2024-05-22 15:07:36 -05:00
Eric Eastwood
d4b41aaf43
Fix lints
2024-05-22 15:01:06 -05:00
Eric Eastwood
3da6bc1902
Use @parameterized_class
...
As suggested in https://github.com/element-hq/synapse/pull/17167#discussion_r1610255726
2024-05-22 14:50:35 -05:00
Eric Eastwood
06ac1da6ec
Restore copyright header
...
See https://github.com/element-hq/synapse/pull/17167#discussion_r1609876335
2024-05-22 14:08:42 -05:00
Eric Eastwood
97497955ea
Update filter to be more precise and avoid more work
...
- Added `room.account_data` and `room.presence` to avoid extra work in `_generate_sync_entry_for_rooms()`
- Added a comment to the top-level `account_data` and `presence` filters that `(This is just here for good measure)`
See https://github.com/element-hq/synapse/pull/17167#discussion_r1610517164
2024-05-22 14:07:35 -05:00