Eric Eastwood
a917edad8c
No more sort option
...
We always just sort by stream_ordering
2024-06-13 15:47:39 -05:00
Eric Eastwood
35808b38db
Fix filtering
2024-06-13 14:53:23 -05:00
Eric Eastwood
8244b25be8
Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-sort
...
Conflicts:
synapse/handlers/sliding_sync.py
tests/handlers/test_sliding_sync.py
2024-06-13 14:17:47 -05:00
Eric Eastwood
c12ee0d5ba
Add is_dm
filtering to Sliding Sync /sync
( #17277 )
...
Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575 ): Sliding Sync
2024-06-13 13:56:58 -05:00
Eric Eastwood
185e0b5693
Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-sort
...
Conflicts:
synapse/storage/databases/main/stream.py
tests/storage/test_stream.py
2024-06-13 13:00:10 -05:00
Eric Eastwood
8aaff851b1
Fix newly_left
rooms not appearing if we returned early (Sliding Sync) ( #17301 )
...
Fix `newly_left` rooms not appearing if we returned early when `membership_snapshot_token.is_before_or_eq(to_token.room_key)`.
Introduced in https://github.com/element-hq/synapse/pull/17187 (part of Sliding Sync)
The tests didn't catch it because they had a small typo in it `room_id1` vs `room_id2`.
Found while working on https://github.com/element-hq/synapse/pull/17293
2024-06-13 11:36:57 -05:00
Eric Eastwood
8c58eb7f17
Add event.internal_metadata.instance_name
( #17300 )
...
Add `event.internal_metadata.instance_name` (the worker instance that persisted the event) to go alongside the existing `event.internal_metadata.stream_ordering`.
`instance_name` is useful to properly compare and query for events with a token since you need to compare both the `stream_ordering` and `instance_name` against the vector clock/`instance_map` in the `RoomStreamToken`.
This is pre-requisite work and may be used in https://github.com/element-hq/synapse/pull/17293
Adding `event.internal_metadata.instance_name` was first mentioned in the initial Sliding Sync PR while pairing with @erikjohnston, see 09609cb0db (diff-5cd773fb307aa754bd3948871ba118b1ef0303f4d72d42a2d21e38242bf4e096R405-R410)
2024-06-13 11:32:50 -05:00
Eric Eastwood
ebdce69f6a
Fix get_last_event_in_room_before_stream_ordering(...)
finding the wrong last event ( #17295 )
...
PR where this was introduced: https://github.com/matrix-org/synapse/pull/14817
### What does this affect?
`get_last_event_in_room_before_stream_ordering(...)` is used in Sync v2 in a lot of different state calculations.
`get_last_event_in_room_before_stream_ordering(...)` is also used in `/rooms/{roomId}/members`
2024-06-13 11:00:52 -05:00
Andrew Morgan
c6eb99c878
Bump mypy
from 1.8.0 to 1.9.0 ( #17297 )
...
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-13 15:50:10 +01:00
Quentin Gliech
5db3eec5bc
Clarify that MSC4151 is enabled on matrix.org ( #17296 )
...
This clarifies in the comments that the MSC is being used in matrix.org
See #17270
2024-06-13 13:49:00 +00:00
Eric Eastwood
93aa4ffb94
Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-sort
2024-06-12 18:31:02 -05:00
Eric Eastwood
8935c6c81a
Fix lints
2024-06-12 18:29:39 -05:00
Eric Eastwood
d5929f15af
Adjust wording
2024-06-12 18:25:20 -05:00
Eric Eastwood
5243a30e00
Fix ban test case
2024-06-12 18:20:47 -05:00
Eric Eastwood
5060588012
Fix newly_left not being added back if we returned early (when membership_snapshot_token.is_before_or_eq(to_token.room_key)
)
...
Wasn't caught in the tests because the test was wrong
2024-06-12 18:18:20 -05:00
Eric Eastwood
bd49c3415b
Add some tests
2024-06-12 17:53:24 -05:00
Eric Eastwood
af60f7b508
First pass on sort_rooms
and refactor to include room membership alongside the sync rooms
2024-06-12 15:46:37 -05:00
Eric Eastwood
afb6627b6f
Add rust changes for event.internal_metadata.instance_name
2024-06-12 15:11:29 -05:00
Eric Eastwood
c94550d542
Add event.internal_metadata.instance_name
and event position to get_last_event_in_room_before_stream_ordering(...)
2024-06-12 14:53:36 -05:00
Travis Ralston
f1c4dfb08b
Add report room API (MSC4151) ( #17270 )
...
https://github.com/matrix-org/matrix-spec-proposals/pull/4151
This is intended to be enabled by default for immediate use. When FCP is
complete, the unstable endpoint will be dropped and stable endpoint
supported instead - no backwards compatibility is expected for the
unstable endpoint.
2024-06-12 12:27:46 +02:00
dependabot[bot]
0edf1cacf7
Bump types-jsonschema from 4.21.0.20240311 to 4.22.0.20240610 ( #17288 )
2024-06-12 10:52:26 +01:00
dependabot[bot]
d0f90bd04e
Bump jinja2 from 3.1.3 to 3.1.4 ( #17287 )
2024-06-12 10:52:14 +01:00
Eric Eastwood
03547b09e3
Merge branch 'madlittlemods/fix-and-tests-for-get_last_event_in_room_before_stream_ordering' into madlittlemods/msc3575-sliding-sync-sort
...
Conflicts:
synapse/storage/databases/main/stream.py
tests/storage/test_stream.py
2024-06-11 22:40:08 -05:00
Eric Eastwood
42f24de8fd
Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-sort
2024-06-11 22:39:04 -05:00
Eric Eastwood
4d585b6886
Merge branch 'develop' into madlittlemods/fix-and-tests-for-get_last_event_in_room_before_stream_ordering
2024-06-11 22:37:24 -05:00
Eric Eastwood
54bdc0ca92
Fix invalid syntax
2024-06-11 22:36:15 -05:00
Eric Eastwood
3f317a9929
We're actually using sub-query syntax so we can ORDER each query
2024-06-11 22:05:27 -05:00
Eric Eastwood
a8056ae67d
Add changelog
2024-06-11 22:04:48 -05:00
Eric Eastwood
d7f40aedf7
Try to better explain why
...
See https://github.com/element-hq/synapse/pull/17293#discussion_r1633904606
2024-06-11 20:50:31 -05:00
Eric Eastwood
431b31e0f2
Add actual guranteed order for UNION
...
We use `union all` because we don't need any of the deduplication logic
(`union` is really a union + distinct). `UNION ALL`` does preserve the
ordering of the operand queries but there is no actual gurantee that it
has this behavior in all scenarios so we need the extra `ORDER BY` at the
bottom.
See https://dba.stackexchange.com/questions/316818/are-results-from-union-all-clauses-always-appended-in-order/316835#316835
2024-06-11 20:49:20 -05:00
Eric Eastwood
87ad458d77
Fix get_last_event_in_room_before_stream_ordering(...)
not finding the last event
...
Previously, it would use the event with the lowest `stream_ordering` that was
fetched (we want the highest `stream_ordering. `union` does not work how you
might think at first and does not preserve the ordering of the operand queries.
`union all` also doesn't necessarily have that gurantee but it does behave
that way. See https://dba.stackexchange.com/questions/316818/are-results-from-union-all-clauses-always-appended-in-order/316835#316835
2024-06-11 20:49:14 -05:00
Eric Eastwood
901ce62d07
Try to better explain why
...
See https://github.com/element-hq/synapse/pull/17293#discussion_r1633904606
2024-06-11 20:48:02 -05:00
Eric Eastwood
b1af992113
Some clean-up
2024-06-11 20:30:32 -05:00
Eric Eastwood
2e1d142892
Add actual guranteed order for UNION
...
We use `union all` because we don't need any of the deduplication logic
(`union` is really a union + distinct). `UNION ALL`` does preserve the
ordering of the operand queries but there is no actual gurantee that it
has this behavior in all scenarios so we need the extra `ORDER BY` at the
bottom.
See https://dba.stackexchange.com/questions/316818/are-results-from-union-all-clauses-always-appended-in-order/316835#316835
2024-06-11 20:29:40 -05:00
Eric Eastwood
2a82ac0cbe
Fix get_last_event_in_room_before_stream_ordering(...)
not finding the last event
...
Previously, it would use the event with the lowest `stream_ordering` that was
fetched (we want the highest `stream_ordering. `union` does not work how you
might think at first and does not preserve the ordering of the operand queries.
`union all` also doesn't necessarily have that gurantee but it does behave
that way. See https://dba.stackexchange.com/questions/316818/are-results-from-union-all-clauses-always-appended-in-order/316835#316835
2024-06-11 20:14:01 -05:00
Eric Eastwood
c8a240f59d
Prefer ? < a AND a <= ?
...
See https://github.com/element-hq/synapse/pull/17293#discussion_r1634863810
> FWIW I mildly prefer the styling `? < a AND a <= ?` as then my brain can more easily convert that to `a in (?...?]`
2024-06-11 16:00:02 -05:00
Quentin Gliech
0248ed70a9
Merge branch 'release-v1.109' into develop
2024-06-11 16:25:26 +02:00
Quentin Gliech
e6816babf6
CHANGES.md: s/OTKs/one-time-keys/
2024-06-11 15:39:30 +02:00
Quentin Gliech
a8069e9739
1.109.0rc2
2024-06-11 15:22:21 +02:00
dependabot[bot]
863578bfcf
Bump regex from 1.10.4 to 1.10.5 ( #17290 )
2024-06-11 10:50:13 +01:00
dependabot[bot]
9e59d18022
Bump dawidd6/action-download-artifact from 3.1.4 to 5 ( #17289 )
2024-06-11 10:50:03 +01:00
dependabot[bot]
491365f199
Bump types-pillow from 10.2.0.20240423 to 10.2.0.20240520 ( #17285 )
2024-06-11 10:47:03 +01:00
Eric Eastwood
75b701f0ee
Add changelog
2024-06-10 17:21:09 -05:00
Eric Eastwood
ce45cc1d44
Sliding sync sort stub
2024-06-10 17:18:48 -05:00
Eric Eastwood
dad1559721
Reorganize Pydantic models and types used in handlers ( #17279 )
...
Spawning from https://github.com/element-hq/synapse/pull/17187#discussion_r1619492779 around wanting to put `SlidingSyncBody` (parse the request in the rest layer), `SlidingSyncConfig` (from the rest layer, pass to the handler), `SlidingSyncResponse` (pass the response from the handler back to the rest layer to respond) somewhere that doesn't contaminate the imports and cause circular import issues.
- Moved Pydantic parsing models to `synapse/types/rest`
- Moved handler types to `synapse/types/handlers`
2024-06-10 15:03:50 -05:00
Erik Johnston
8c4937b216
Fix bug where device lists would break sync ( #17292 )
...
If the stream ID in the unconverted table is ahead of the device lists
ID gen, then it can break all /sync requests that had an ID from ahead
of the table.
The fix is to make sure we add the unconverted table to the list of
tables we check at start up.
Broke in https://github.com/element-hq/synapse/pull/17229
2024-06-10 15:56:57 +01:00
Erik Johnston
b84e31375b
Update changelog
2024-06-10 15:55:42 +01:00
Erik Johnston
06953bc193
Always return OTK counts ( #17275 )
...
Broke in https://github.com/element-hq/synapse/pull/17215
2024-06-10 15:53:45 +01:00
Eric Eastwood
265ee88f34
Wrong retention policy being used when filtering events (lint ControlVarUsedAfterBlockViolation
WPS441
) ( #17272 )
...
Fix loop var being used outside block.
Before this change, we were always using the last room_id's retention policy for all events being filtered.
I found this bug with the [new lint rule, `ControlVarUsedAfterBlockViolation` `WPS441`](https://github.com/astral-sh/ruff/pull/11769 ), that I re-implemented in `ruff`. Shout-out to @reivilibre for all the help in the beginning!
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog ).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html ) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters ))
2024-06-10 09:48:38 -05:00
Shay
ab94bce02c
Support MSC3916 by adding a federation /download
endpoint ( #17172 )
2024-06-07 13:54:28 +01:00