Commit graph

21371 commits

Author SHA1 Message Date
Sean Quah
51c01d450a
Add some miscellaneous comments around sync (#13474)
Add some miscellaneous comments to document sync, especially around
`compute_state_delta`.

Signed-off-by: Sean Quah <seanq@matrix.org>

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-08-10 12:58:20 +01:00
Eric Eastwood
1b09b0832e
Allow use of both @trace and @tag_args stacked on the same function (#13453)
```py
@trace
@tag_args
async def get_oldest_event_ids_with_depth_in_room(...)
  ...
```

Before this PR, you would see a warning in the logs and the span was not exported:
```
2022-08-03 19:11:59,383 - synapse.logging.opentracing - 835 - ERROR - GET-0 - @trace may not have wrapped EventFederationWorkerStore.get_oldest_event_ids_with_depth_in_room correctly! The function is not async but returned a coroutine.
```
2022-08-09 14:32:33 -05:00
Dirk Klimpel
1595052b26
Use literals in place of HTTPStatus constants in tests (#13479)
Replace
- `HTTPStatus.NOT_FOUND`
- `HTTPStatus.FORBIDDEN`
- `HTTPStatus.UNAUTHORIZED`
- `HTTPStatus.CONFLICT`
- `HTTPStatus.CREATED`

Signed-off-by: Dirk Klimpel <dirk@klimpel.org>
2022-08-09 14:56:43 +01:00
Olivier Wilkinson (reivilibre)
54fb517c28 Merge branch 'release-v1.65' into develop 2022-08-09 13:45:11 +01:00
Olivier Wilkinson (reivilibre)
3d1b860f90 Fix changelog mistake 2022-08-09 13:13:22 +01:00
Patrick Cloke
5ce2887653
Strengthen tests about deleted old push actions. (#13471) 2022-08-09 07:56:16 -04:00
Olivier Wilkinson (reivilibre)
c962f87d6f Tweak the changelog 2022-08-09 12:54:59 +01:00
Olivier Wilkinson (reivilibre)
827f0669bf 1.65.0rc1 2022-08-09 11:43:30 +01:00
Dirk Klimpel
70d3e70009
Make the configuration for the cache clearer (#13481) 2022-08-09 11:30:22 +01:00
Shay
0f954466c4
Update matrix-synapse-ldap3 version in lockfile to v0.2.2. (#13470) 2022-08-08 13:44:43 -07:00
Dirk Klimpel
c97042f7ee
Use literals in place of HTTPStatus constants in tests (#13469) 2022-08-08 13:21:27 -07:00
David Robertson
7a19995120
Correct a misnamed argument in state res v2 (#13467)
In state res v2, we apply two passes of iterative auth checks. The first
pass replays power events and events in their auth chains, but only
those belonging to the full conflicted set. The source code as written
suggests that we want only those belonging to the auth difference (which
is a smaller set of events).

At runtime we were doing the correct thing anyway, because the only
callsite of `_reverse_topological_power_sort` passes in the
`full_conflicted_set`. So this really is just a rename.
2022-08-08 16:59:56 +01:00
Šimon Brandner
ab18441573
Support stable identifiers for MSC2285: private read receipts. (#13273)
This adds support for the stable identifiers of MSC2285 while
continuing to support the unstable identifiers behind the configuration
flag. These will be removed in a future version.
2022-08-05 11:09:33 -04:00
Dirk Klimpel
e2ed1b7155
Use literals in place of HTTPStatus constants in tests (#13463) 2022-08-05 16:59:09 +02:00
Julian-Samuel Gebühr
3d2cabf966
Mark token-authenticaticated-registration API as not-experimental (#11897) 2022-08-05 11:15:35 +00:00
Matt C
026ac4486c
Update module API "update room membership" method to allow for remote joins (#13441)
Co-authored-by: MattC <buffless-matt@users.noreply.github.com>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-08-05 09:37:58 +00:00
Erik Johnston
b6a6bb4027
Add comments about how event push actions are stored. (#13445) 2022-08-04 19:38:08 +00:00
Eric Eastwood
860fdd9098
Fix @tag_args being off-by-one (ahead) (#13452)
Fix @tag_args being off-by-one (ahead)

Example:

```
argspec.args=[
  'self',
  'room_id'
]

args=(
  <synapse.storage.databases.main.DataStore object at 0x10d0b8d00>,
  '!HBehERstyQBxyJDLfR:my.synapse.server'
)
```

---

The previous logic was also flawed and we can end up in a situation like this:

```
argspec.args=['self', 'dest', 'room_id', 'limit', 'extremities']

args=(<synapse.federation.federation_client.FederationClient object at 0x7f1651c18160>, 'hs1', '!jAEHKIubyIfuLOdfpY:hs1')
```

From this source:
```py
async def backfill(
    self, dest: str, room_id: str, limit: int, extremities: Collection[str]
) -> Optional[List[EventBase]]:
```

And this usage:
```py
events = await self._federation_client.backfill(
    dest, room_id, limit=limit, extremities=extremities
)
```

which would previously cause this error:
```
synapse_main | 2022-08-04 06:13:12,051 - synapse.handlers.federation - 424 - ERROR - GET-5 - Failed to backfill from hs1 because tuple index out of range
synapse_main | Traceback (most recent call last):
synapse_main |   File "/usr/local/lib/python3.9/site-packages/synapse/handlers/federation.py", line 392, in try_backfill
synapse_main |     await self._federation_event_handler.backfill(
synapse_main |   File "/usr/local/lib/python3.9/site-packages/synapse/logging/tracing.py", line 828, in _wrapper
synapse_main |     return await func(*args, **kwargs)
synapse_main |   File "/usr/local/lib/python3.9/site-packages/synapse/handlers/federation_event.py", line 593, in backfill
synapse_main |     events = await self._federation_client.backfill(
synapse_main |   File "/usr/local/lib/python3.9/site-packages/synapse/logging/tracing.py", line 828, in _wrapper
synapse_main |     return await func(*args, **kwargs)
synapse_main |   File "/usr/local/lib/python3.9/site-packages/synapse/logging/tracing.py", line 827, in _wrapper
synapse_main |     with wrapping_logic(func, *args, **kwargs):
synapse_main |   File "/usr/local/lib/python3.9/contextlib.py", line 119, in __enter__
synapse_main |     return next(self.gen)
synapse_main |   File "/usr/local/lib/python3.9/site-packages/synapse/logging/tracing.py", line 922, in _wrapping_logic
synapse_main |     set_attribute("ARG_" + arg, str(args[i + 1]))  # type: ignore[index]
synapse_main | IndexError: tuple index out of range
```
2022-08-04 14:29:41 -05:00
Patrick Cloke
ec24813220
Improve comments (& avoid a duplicate query) in push actions processing. (#13455)
* Adds docstrings and inline comments.
* Formats SQL queries using triple quoted strings.
* Minor formatting changes.
* Avoid fetching `event_push_summary_stream_ordering` multiple times
  in the same transactions.
2022-08-04 19:24:44 +00:00
Richard van der Hoff
96d92156d0
Update type of EventContext.rejected (#13460) 2022-08-04 17:45:01 +01:00
reivilibre
e9e6aacfbe
Faster Room Joins: prevent Synapse from answering federated join requests for a room which it has not fully joined yet. (#13416) 2022-08-04 16:27:04 +01:00
Nick Mills-Barrett
41320a0554
Optimise async get event lookups (#13435)
Still maintains local in memory lookup optimisation, but does any external
lookup as part of the deferred that prevents duplicate lookups for the same
event at once. This makes the assumption that fetching from an external
cache is a non-zero load operation.
2022-08-04 15:49:55 +01:00
Dirk Klimpel
6dd7fa12dc
Update some outdated information on sso_mapping_providers.md (#13449) 2022-08-04 13:06:02 +01:00
Dirk Klimpel
afbdbe0634
Fix return value in example on password_auth_provider_callbacks.md (#13450)
Fixes: #12534

Signed-off-by: Dirk Klimpel <dirk@klimpel.org>
2022-08-04 13:03:36 +01:00
Richard van der Hoff
166fafdf8d
synapse-workers docker: copy nginx and redis in from base images (#13447)
Part of my continuing quest to make the docker images build quicker: copy nginx and redis in from base docker images, rather than apt installing each time.
2022-08-04 12:59:27 +01:00
Matt C
a91078200d
Add module API method to create a room (#13429)
Co-authored-by: MattC <buffless-matt@users.noreply.github.com>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-08-04 09:34:05 +00:00
Brendan Abolivier
845732be45
Fix rooms not being properly excluded from incremental sync (#13408) 2022-08-04 11:02:29 +02:00
Shay
a648a06d52
Add some tracing spans to give insight into local joins (#13439) 2022-08-03 10:19:34 -07:00
Eric Eastwood
92d21faf12
Instrument /messages for understandable traces in Jaeger (#13368)
In Jaeger:

 - Before: huge list of uncategorized database calls
 - After: nice and collapsible into units of work
2022-08-03 10:57:38 -05:00
andrew do
78a3111c41
Return 404 or member list when getting joined_members after leaving (#13374)
Signed-off-by: Andrew Doh <andrewddo@gmail.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Andrew Morgan <andrewm@element.io>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-08-03 14:26:31 +02:00
Jasper Spaans
503a95804e
Install cryptography build dependencies in requirements image. (#13372) 2022-08-03 11:16:32 +01:00
jejo86
668597214f
Improve documentation on becoming server admin (#13230)
* Improved section regarding server admin

Added steps describing how to elevate an existing user to administrator by manipulating a `postgres` database.

Signed-off-by: jejo86 28619134+jejo86@users.noreply.github.com

* Improved section regarding server admin

* Reference database settings

Add instructions to check database settings to find out the database name, instead of listing all available PostgreSQL databases.

* Add suggestions from PR conversation

Replace config filename `homeserver.yaml`. with "config file".
Remove instructions to switch to `postgres` user.
Add instructions how to connect to SQLite database.

* Update changelog.d/13230.doc

Co-authored-by: reivilibre <olivier@librepush.net>
2022-08-03 11:15:23 +01:00
Dirk Klimpel
fb7a2cc4cc
Update doc for setting macaroon_secret_key (#13443)
* Update doc for setting `macaroon_secret_key`

* newsfile
2022-08-03 10:41:19 +01:00
Dirk Klimpel
d6e94ad9d9
Rename RateLimitConfig to RatelimitSettings (#13442) 2022-08-03 10:40:20 +01:00
Matt C
570bf32bbb
Add module API method to resolve a room alias to a room ID (#13428)
Co-authored-by: MattC <buffless-matt@users.noreply.github.com>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-08-03 09:25:36 +00:00
Dirk Klimpel
5eccfdfafd
Remove 'Contents' section from the Configuration Manual (#13438)
Fixes: #13053
2022-08-03 09:19:20 +00:00
Dirk Klimpel
ec6758d472
Fix wrong headline for url_preview_accept_language in docs (#13437)
Fixes: #13433
2022-08-03 09:41:57 +01:00
reivilibre
1c910e2216
Add a merge-back command to the release script, which automates merging the correct branches after a release. (#13393) 2022-08-02 15:56:28 +00:00
Sean Quah
8d317f6da5
Fix error when out of servers to sync partial state with (#13432)
so that we raise the intended error instead.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-08-02 12:12:44 +01:00
Olivier Wilkinson (reivilibre)
a2a867b521 Merge branch 'master' into develop 2022-08-02 11:56:02 +01:00
Olivier Wilkinson (reivilibre)
c2f4871226 Mention specific version in rc2 notes 2022-08-02 11:19:32 +01:00
Olivier Wilkinson (reivilibre)
cb209638ea Add upgrade notes 2022-08-02 11:10:26 +01:00
Olivier Wilkinson (reivilibre)
4e80ca2243 1.64.0 2022-08-02 11:04:08 +01:00
reivilibre
e17e5c97e0
Faster Room Joins: don't leave a stuck room partial state flag if the join fails. (#13403) 2022-08-01 16:45:39 +00:00
Patrick Cloke
f8e7a9418a
Fix missing import in federation_event handler. (#13431)
#13404 removed an import of `Optional` which was still needed
due to #13413 added more usages.
2022-08-01 14:14:29 +00:00
Sean Quah
224d792dd7
Refactor _resolve_state_at_missing_prevs to return an EventContext (#13404)
Previously, `_resolve_state_at_missing_prevs` returned the resolved
state before an event and a partial state flag. These were unwieldy to
carry around would only ever be used to build an event context. Build
the event context directly instead.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-08-01 13:53:56 +01:00
reivilibre
05aeeb3a80
Enable Complement CI tests in the 'latest deps' test run. (#13213)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-08-01 10:55:31 +00:00
reivilibre
b817574be7
Re-enable running Complement tests against Synapse with workers. (#13420) 2022-08-01 11:51:44 +01:00
Richard van der Hoff
23768ccb4d
Faster joins: fix rejected events becoming un-rejected during resync (#13413)
Make sure that we re-check the auth rules during state resync, otherwise
rejected events get un-rejected.
2022-08-01 11:20:05 +01:00
Richard van der Hoff
d548d8f18d Synapse 1.64.0rc2 (2022-07-29)
==============================
 
 This RC reintroduces support for `account_threepid_delegates.email`, which was removed in 1.64.0rc1. It remains deprecated and will be removed altogether in a future release. ([\#13406](https://github.com/matrix-org/synapse/issues/13406))
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEv27Axt/F4vrTL/8QOSor00I9eP8FAmLj4a8ACgkQOSor00I9
 eP8biQf/c8yY2mbeRZcBKtp6yoQCRYQvboSMEXyi+dLe1hNqdhSZwRQcAoFuAFwE
 WdScDvoTaElUxv0v6eCI1x9CoXnZ6xpDShvK39j5Yhzv+1tNsm5Uq9imyG3jK5i6
 U/3Gt6CrCsS01VkGslQ3B5I6MFtbC6ZZK9O48yg+GD8Oqw2HH/gllr5swyVbKdbc
 GGhRBHvgXn+w6d/KnKt8uRxJqIpDt9JMga+WdB8CwFR5WnWbGdw24KsyxmBuOLC3
 caQRiluJL/X4jApUpfsJMBBd/jrDod5wWDFO/4P+v0+2d3Ts+hKezZbt5h1VIYSw
 szZXbzxn5RNDkNiJDpOOOMYQ5DXGmA==
 =3/nK
 -----END PGP SIGNATURE-----

Merge tag 'v1.64.0rc2' into develop

Synapse 1.64.0rc2 (2022-07-29)
==============================

This RC reintroduces support for `account_threepid_delegates.email`, which was removed in 1.64.0rc1. It remains deprecated and will be removed altogether in a future release. ([\#13406](https://github.com/matrix-org/synapse/issues/13406))
2022-07-29 15:15:21 +01:00