Commit graph

57 commits

Author SHA1 Message Date
Erik Johnston
ba47fea528
Allow multiple workers to write to receipts stream. (#16432)
Fixes #16417
2023-10-25 16:16:19 +01:00
Patrick Cloke
3ab861ab9e
Fix type hint errors from Twisted trunk (#16526) 2023-10-23 14:28:05 -04:00
Erik Johnston
009b47badf
Factor out MultiWriter token from RoomStreamToken (#16427) 2023-10-05 10:46:28 +01:00
Erik Johnston
80ec81dcc5
Some refactors around receipts stream (#16426) 2023-10-04 16:28:40 +01:00
Will Hunt
1e571cd664
Fix appservices being unable to handle to_device messages for multiple users (#16251) 2023-09-05 15:46:57 -04:00
Patrick Cloke
9ec3da06da
Bump mypy-zope & mypy. (#16188) 2023-08-29 10:38:56 -04:00
Patrick Cloke
a8a46b1336
Replace simple_async_mock with AsyncMock (#16180)
Python 3.8 has a native AsyncMock, use it instead of a custom
implementation.
2023-08-25 09:27:21 -04:00
Patrick Cloke
daf11e26ef
Replace make_awaitable with AsyncMock (#16179)
Python 3.8 provides a native AsyncMock, we can replace the
homegrown version we have.
2023-08-24 19:38:46 -04:00
Patrick Cloke
42aea0d8af
Add final type hint to tests.unittest. (#15072)
Adds a return type to HomeServerTestCase.make_homeserver and deal
with any variables which are no longer Any.
2023-02-14 14:03:35 -05:00
Patrick Cloke
652d1669c5
Add missing type hints to tests.handlers. (#14680)
And do not allow untyped defs in tests.handlers.
2022-12-16 11:53:01 +00:00
Richard van der Hoff
cb59e08062
Improve logging and opentracing for to-device message handling (#14598)
A batch of changes intended to make it easier to trace to-device messages through the system.

The intention here is that a client can set a property org.matrix.msgid in any to-device message it sends. That ID is then included in any tracing or logging related to the message. (Suggestions as to where this field should be documented welcome. I'm not enthusiastic about speccing it - it's very much an optional extra to help with debugging.)

I've also generally improved the data we send to opentracing for these messages.
2022-12-06 09:52:55 +00:00
Andrew Ferrazzutti
1183c372fa
Use device_one_time_keys_count to match MSC3202 (#14565)
* Use `device_one_time_keys_count` to match MSC3202

Rename the `device_one_time_key_counts` key in responses to
`device_one_time_keys_count` to match the name specified by MSC3202.

Also change related variable/class names for consistency.

Signed-off-by: Andrew Ferrazzutti <andrewf@element.io>

* Update changelog.d/14565.misc

* Revert name change for `one_time_key_counts` key

as this is a different key altogether from `device_one_time_keys_count`,
which is used for `/sync` instead of appservice transactions.

Signed-off-by: Andrew Ferrazzutti <andrewf@element.io>
2022-11-28 16:17:29 +00:00
Eric Eastwood
aa70556699
Check appservice user interest against the local users instead of all users (get_users_in_room mis-use) (#13958) 2022-10-27 18:29:23 +00:00
Shay
a86b2f6837
Fix a bug where redactions were not being sent over federation if we did not have the original event. (#13813) 2022-10-11 11:18:45 -07:00
Patrick Cloke
efd108b45d
Accept & store thread IDs for receipts (implement MSC3771). (#13782)
Updates the `/receipts` endpoint and receipt EDU handler to parse a
`thread_id` from the body and insert it in the database.
2022-09-23 14:33:28 +00:00
Nick Mills-Barrett
21eeacc995
Federation Sender & Appservice Pusher Stream Optimisations (#13251)
* Replace `get_new_events_for_appservice` with `get_all_new_events_stream`

The functions were near identical and this brings the AS worker closer
to the way federation senders work which can allow for multiple workers
to handle AS traffic.

* Pull received TS alongside events when processing the stream

This avoids an extra query -per event- when both federation sender
and appservice pusher process events.
2022-07-15 09:36:56 +01:00
Patrick Cloke
7bc08f3201
Remove remaining bits of groups code. (#12936)
* Update worker docs to remove group endpoints.
* Removes an unused parameter to `ApplicationService`.
* Break dependency between media repo and groups.
* Avoid copying `m.room.related_groups` state events during room upgrades.
2022-06-01 09:41:25 -04:00
Patrick Cloke
c52abc1cfd
Additional constants for EDU types. (#12884)
Instead of hard-coding strings in many places.
2022-05-27 07:14:36 -04:00
Richard van der Hoff
24b590de32
Remove code which updates application_services_state.last_txn (#12680)
This column is unused as of #12209, so let's stop writing to it.
2022-05-17 11:07:18 +01:00
Nick Mills-Barrett
63ba9ba38b
Bound ephemeral events by key (#12544)
Co-authored-by: Brad Murray <bradtgmurray@gmail.com>
Co-authored-by: Andrew Morgan <andrewm@element.io>
2022-04-26 20:14:21 +01:00
Andrew Morgan
d8d0271977
Send device list updates to application services (MSC3202) - part 1 (#11881)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-03-30 14:39:27 +01:00
Andrew Morgan
fb0ffa9676
Rename various ApplicationServices interested methods (#11915) 2022-03-03 18:14:09 +00:00
Patrick Cloke
02d708568b
Replace assertEquals and friends with non-deprecated versions. (#12092) 2022-02-28 07:12:29 -05:00
reivilibre
2cc5ea933d
Add support for MSC3202: sending one-time key counts and fallback key usage states to Application Services. (#11617)
Co-authored-by: Erik Johnston <erik@matrix.org>
2022-02-24 17:55:45 +00:00
Richard van der Hoff
e24ff8ebe3
Remove HomeServer.get_datastore() (#12031)
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.

Part of #11733
2022-02-23 11:04:02 +00:00
Andrew Morgan
64ec45fc1b
Send to-device messages to application services (#11215)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-02-01 14:13:38 +00:00
Andrew Morgan
a026695083
Clarifications and small fixes to to-device related code (#11247)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-11-09 14:31:15 +00:00
Nick Barrett
a271e233e9
Add a linearizer on (appservice, stream) when handling ephemeral events. (#11207)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-11-03 16:51:00 +00:00
Will Hunt
a8a27b2b8b
Only return an appservice protocol if it has a service providing it. (#10532)
If there are no services providing a protocol, omit it completely
instead of returning an empty dictionary.

This fixes a long-standing spec compliance bug.
2021-08-05 08:22:14 -04:00
Marcus
8070b893db
update black to 21.6b0 (#10197)
Reformat all files with the new version.

Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>
2021-06-17 15:20:06 +01:00
14mRh4X0r
8942e23a69
Always update AS last_pos, even on no events (#10107)
Fixes #1834.

`get_new_events_for_appservice` internally calls `get_events_as_list`, which will filter out any rejected events. If all returned events are filtered out, `_notify_interested_services` will return without updating the last handled stream position. If there are 100 consecutive such events, processing will halt altogether.

Breaking the loop is now done by checking whether we're up-to-date with `current_max` in the loop condition, instead of relying on an empty `events` list.


Signed-off-by: Willem Mulder <14mRh4X0r@gmail.com>
2021-06-07 15:42:05 +01:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Patrick Cloke
0b3112123d
Use mock from the stdlib. (#9772) 2021-04-09 13:44:38 -04:00
Patrick Cloke
74af356baf
Convert additional test-cases to homeserver test case. (#9396)
And convert some inlineDeferreds to async-friendly functions.
2021-02-16 08:04:15 -05:00
Erik Johnston
2b7c180879
Start fewer opentracing spans (#8640)
#8567 started a span for every background process. This is good as it means all Synapse code that gets run should be in a span (unless in the sentinel logging context), but it means we generate about 15x the number of spans as we did previously.

This PR attempts to reduce that number by a) not starting one for send commands to Redis, and b) deferring starting background processes until after we're sure they're necessary.

I don't really know how much this will help.
2020-10-26 09:30:19 +00:00
Erik Johnston
921a3f8a59
Fix not sending events over federation when using sharded event persisters (#8536)
* Fix outbound federaion with multiple event persisters.

We incorrectly notified federation senders that the minimum persisted
stream position had advanced when we got an `RDATA` from an event
persister.

Notifying of federation senders already correctly happens in the
notifier, so we just delete the offending line.

* Change some interfaces to use RoomStreamToken.

By enforcing use of `RoomStreamTokens` we make it less likely that
people pass in random ints that they got from somewhere random.
2020-10-14 13:27:51 +01:00
Patrick Cloke
7f837959ea
Convert directory, e2e_room_keys, end_to_end_keys, monthly_active_users database to async (#8042) 2020-08-07 13:36:29 -04:00
Patrick Cloke
4cce8ef74e
Convert appservice to async. (#7973) 2020-07-30 07:27:39 -04:00
Patrick Cloke
6d687ebba1
Convert the appservice handler to async/await. (#7775) 2020-07-06 07:40:35 -04:00
black
8b3d9b6b19 Run black. 2018-08-10 23:54:09 +10:00
Amber Brown
49af402019 run isort 2018-07-09 16:09:20 +10:00
Erik Johnston
e939f3bca6 Fix tests 2018-04-11 14:37:11 +01:00
Ilya Zhuravlev
8a4a0ddea6 Fix appservice tests to account for new behavior of notify_interested_services 2017-11-02 23:19:57 +03:00
Erik Johnston
9da84a9a1e Make AppserviceHandler stream events from database
This is for two reasons:

1. Suppresses duplicates correctly, as the notifier doesn't do any
   duplicate suppression.
2. Makes it easier to connect the AppserviceHandler to the replication
   stream.
2016-08-18 11:54:41 +01:00
Erik Johnston
320dfe523c Make notify_interested_services faster 2016-08-17 17:20:50 +01:00
Erik Johnston
b835ebcc79 Update unit tests 2016-08-17 11:22:11 +01:00
Mark Haines
c626fc576a Move the AS handler out of the Handlers object.
Access it directly from the homeserver itself. It already wasn't
inheriting from BaseHandler storing it on the Handlers object was
already somewhat dubious.
2016-05-31 13:53:48 +01:00
Mark Haines
700487a7c7 Fix flake8 warnings for tests 2016-02-19 15:34:38 +00:00
Matthew Hodgson
6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
Kegan Dougal
d88e20cdb9 Fix bug where synapse was sending AS user queries incorrectly.
Bug introduced in 92b20713d7
which reversed the comparison when checking if a user existed
in the users table. Added UTs to prevent this happening again.
2015-06-17 17:26:03 +01:00