Commit graph

3039 commits

Author SHA1 Message Date
Oliver Kurz
4e118742ca
Allow to use higher versions of prometheus_client (#7780)
Fixes https://github.com/matrix-org/synapse/issues/7641

The package was pinned to <0.8.0 without an obvious reasoning with
7ad1d7635
in https://github.com/matrix-org/synapse/pull/5636
while the version selection looks to just try to exclude an arbitrary
next minor version number that might introduce API breaking changes.
Selecting the next minor number might be a good conservative selection.

Downstream distributions already reported success patching out the version
requirements.

This also fixes the integration of upgraded packages into openSUSE packages,
e.g. for openSUSE Tumbleweed which already ships prometheus_client >= 0.8 .

Signed-off-by: Oliver Kurz <okurz@suse.de>

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-07-06 10:21:41 +01:00
Will Hunt
62b1ce8539
isort 5 compatibility (#7786)
The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
2020-07-05 16:32:02 +01:00
Erik Johnston
5cdca53aa0
Merge different Resource implementation classes (#7732) 2020-07-03 19:02:19 +01:00
Dirk Klimpel
21a212f8e5
Fix inconsistent handling of upper and lower cases of email addresses. (#7021)
fixes #7016
2020-07-03 14:03:13 +01:00
Alex Kotov
8097659f6e
Allow YAML config file to contain None (#7779)
Useful when config file is fully commented

Signed-off-by: Alex Kotov <kotovalexarian@gmail.com>
2020-07-03 13:19:03 +01:00
Patrick Cloke
f3e0f16240 Synapse 1.16.0rc2 (2020-07-02)
==============================
 
 Synapse 1.16.0rc2 includes the security fixes released with Synapse 1.15.2.
 Please see [below](https://github.com/matrix-org/synapse/blob/master/CHANGES.md#synapse-1152-2020-07-02) for more details.
 
 Improved Documentation
 ----------------------
 
 - Update postgres image in example `docker-compose.yaml` to tag `12-alpine`. ([\#7696](https://github.com/matrix-org/synapse/issues/7696))
 
 Internal Changes
 ----------------
 
 - Add some metrics for inbound and outbound federation latencies: `synapse_federation_server_pdu_process_time` and `synapse_event_processing_lag_by_event`. ([\#7771](https://github.com/matrix-org/synapse/issues/7771))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl79+qgACgkQM/xY9qcR
 MEhcaRAAjWLW3ojN1F0DUfE85jziZK2VdnMQC3g+uEOLX6QRbfqFNaNNMjLdK+vl
 K/+2ZoHkRsg6g8noSPhPmI1z1+hb5xDJaxjltzHxonIipW8XSU8o2PQMkf8O/BAy
 VS58y3GyLkhEgzWC+/hcII+LBgcqXpLuNM0xrKTHmxclIjdewlwe1v+hxkP+6wsX
 9Whhn1f4sNHrCtyFVK9uzMFcVyzcQaiWZRjEDMj2uR7rWT6UbCUifN/G4fWmtGbY
 xWoNoC4Qv8xiqXOG4U7juPp9T3bRyWMKyjBFM5PWO6Ec2zfafDyFzhBxJhlQhODG
 g21tS4PowX/dM/pBpJFEOPh1BVrPZzzTD+YMmTcd3NO79HeaQGqEX/+tzFCFUyPp
 0daJK3Y85+l5w/M09WU8DDN8CiR3PFJyGDIZp+nweMsiJZkbEbLOkh1tx6TL+5/6
 zwewU6cq8nTVGrn53Tn58l8C7Sj4w+Qk+1XDzymAoidyoWqAKW9Y/fw53PaViUSx
 voDu0rpsEUXR1OzCBG8SAPQCFy9gdEWV04OvIpzHuq2uojkz66f7NAXy+Wz+Occ9
 AYb/s6Ei80bGCLgRd5jg+myqavwRbzCyv+LIC6dxpopxZJ3AzrFuD11eXKtrIxOC
 FZYf3U4KeBk4Q9TV5IFV1xcGFrq5aK36LdmP6WOsEl3PXVT9p/Q=
 =YaJn
 -----END PGP SIGNATURE-----

Merge tag 'v1.16.0rc2' into develop

Synapse 1.16.0rc2 (2020-07-02)
==============================

Synapse 1.16.0rc2 includes the security fixes released with Synapse 1.15.2.
Please see [below](https://github.com/matrix-org/synapse/blob/master/CHANGES.md#synapse-1152-2020-07-02) for more details.

Improved Documentation
----------------------

- Update postgres image in example `docker-compose.yaml` to tag `12-alpine`. ([\#7696](https://github.com/matrix-org/synapse/issues/7696))

Internal Changes
----------------

- Add some metrics for inbound and outbound federation latencies: `synapse_federation_server_pdu_process_time` and `synapse_event_processing_lag_by_event`. ([\#7771](https://github.com/matrix-org/synapse/issues/7771))
2020-07-02 11:25:56 -04:00
Patrick Cloke
1319e53251 1.16.0rc2 2020-07-02 11:06:35 -04:00
Patrick Cloke
f2bcc6ecbf Merge branch 'master' into release-v1.16.0 2020-07-02 11:02:42 -04:00
Patrick Cloke
4d978d7db4 Merge branch 'master' into develop 2020-07-02 10:55:41 -04:00
reivilibre
e5808c4cfb
Hack to add push priority to push notifications (#7765)
* Remove obsolete comment about ancient temporary code

Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>

* Implement hack to set push priority

based on whether the tweaks indicate the event might cause
effects.

* Changelog for 7765

Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>

* Antilint

* Add tests for push priority

Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>

* Update synapse/push/httppusher.py

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

* Antilint

* Remove needless invites from tests.

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2020-07-01 17:02:31 +01:00
Richard van der Hoff
e866512367
Add early returns to _check_for_soft_fail (#7769)
my editor was complaining about unset variables, so let's add some early
returns to fix that and reduce indentation/cognitive load.
2020-07-01 16:41:19 +01:00
Richard van der Hoff
f01e2ca039
Use symbolic names for replication stream names (#7768)
This makes it much easier to find where streams are referenced.
2020-07-01 16:35:40 +01:00
Richard van der Hoff
a6eae69ffe
Type checking for FederationHandler (#7770)
fix a few things to make this pass mypy.
2020-07-01 16:21:02 +01:00
Erik Johnston
1e03513f9a
Fix new metric where we used ms instead of seconds (#7771)
Introduced in #7755, not yet released.
2020-07-01 15:23:58 +01:00
Richard van der Hoff
244dbb04f7
Fix incorrect error message when database CTYPE was set incorrectly. (#7760) 2020-07-01 13:56:16 +01:00
Andrew Morgan
dc80a0762d 1.16.0rc1 2020-07-01 11:26:58 +01:00
Brendan Abolivier
74d3e177f0
Back out MSC2625 implementation (#7761) 2020-07-01 11:08:25 +01:00
Patrick Cloke
71cccf1593
Additional configuration options for auto-join rooms (#7763) 2020-06-30 15:41:36 -04:00
Erik Johnston
a99658074d
Add some metrics for inbound and outbound federation processing times (#7755) 2020-06-30 16:58:06 +01:00
Andrew Morgan
2f6afdd8b4
Explain the purpose of the "tests" conditional dependency requirement (#7751) 2020-06-30 10:11:36 +01:00
Erik Johnston
831b31e563
Add another yield point to state res v2 (#7746) 2020-06-26 10:44:52 +01:00
Andrew Morgan
177b2d0c19
Move flake8 to end. Don't exit script on failure (#7738) 2020-06-25 17:58:55 +01:00
Dagfinn Ilmari Mannsåker
b099ef07d6
Make tox actions work on Debian 10 (#7703)
- Remove the requirement for a specific version of Python
- Move dep comment to a separate line, Tox 3.7.0 like trailing ones

Signed-off-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
2020-06-25 17:45:35 +01:00
Erik Johnston
0e0a2817a2
Yield during large v2 state res. (#7735)
State res v2 across large data sets can be very CPU intensive, and if
all the relevant events are in the cache the algorithm will run from
start to finish within a single reactor tick. This can result in
blocking the reactor tick for several seconds, which can have major
repercussions on other requests.

To fix this we simply add the occaisonal `sleep(0)` during iterations to
yield execution until the next reactor tick. The aim is to only do this
for large data sets so that we don't impact otherwise quick resolutions.=
2020-06-24 18:48:18 +01:00
Sorunome
6920e58136
add org.matrix.login.jwt so that m.login.jwt can be deprecated (#7675) 2020-06-24 10:23:55 +01:00
Christian Svensson
8bbe87f42d
Set Content-Length for Metrics requests (#7730)
HTTP requires the response to contain a Content-Length header unless chunked encoding is being used.
Prometheus metrics endpoint did not set this, causing software such as prometheus-proxy to not be able to scrape synapse for metrics.

Signed-off-by: Christian Svensson <blue@cmd.nu>
2020-06-23 18:06:01 +01:00
Patrick Cloke
24110255cd
Sync ignored table names in synapse_port_db to current database schema (#7717) 2020-06-23 07:33:25 -04:00
Patrick Cloke
95e41f368b
Allow local media to be marked as safe from being quarantined. (#7718) 2020-06-22 08:04:14 -04:00
Patrick Cloke
e060bf4462
Convert directory handler to async/await (#7727) 2020-06-22 07:18:00 -04:00
Erik Johnston
91e886d615
Speed up state res v2 across large state differences. (#7725) 2020-06-19 13:56:35 +01:00
Jesse Riddle
1b1489ff18
Fixed typo by adding a 'g' to PostgreSQL (#7724) 2020-06-19 07:19:21 -04:00
Brendan Abolivier
e35d44c01d
Merge pull request #7716 from matrix-org/babolivier/unread_fix
Fix unread counts in sync
2020-06-17 15:44:15 +01:00
Patrick Cloke
3630825612
Convert the typing handler to async/await. (#7679) 2020-06-17 10:37:59 -04:00
Oleg Girko
96bc110a68
Require parameterized package version to be at least 0.7.0. (#7680)
Older versions of `parameterized` package have no `parameterized_class` decorator. This decorator is used in tests.

Signed-off-by: Oleg Girko <ol@infoserver.lv>
2020-06-17 15:31:40 +01:00
Brendan Abolivier
5a5cf6460e
Fix unread counts in sync
* Always return an unread_count in get_unread_event_push_actions_by_room_for_user
* Don't always expect unread_count to be there so we don't take out sync entirely if something goes wrong
2020-06-17 15:10:44 +01:00
Patrick Cloke
6418b0379f
Ignore the UI Auth sessions when porting from sqlite to postgresql (#7711) 2020-06-17 10:01:18 -04:00
Erik Johnston
e07a8caf58
Add support for using rust-python-jaeger-reporter (#7697) 2020-06-17 14:13:41 +01:00
Erik Johnston
b44bdd7f7b
Support running multiple media repos. (#7706)
This requires a new config option to specify which media repo should be
responsible for running background jobs to e.g. clear out expired URL
preview caches.
2020-06-17 14:13:30 +01:00
Patrick Cloke
434716e1d3
Fetch from the r0 media path instead of the unspecced v1. (#7714) 2020-06-17 08:36:46 -04:00
lub
890c0c041d
Update postgres in the Docker compose example to 12-alpine. (#7696) 2020-06-17 08:29:08 -04:00
Brendan Abolivier
46613aaf79
Implement unread counter (MSC2625) (#7673)
Implementation of https://github.com/matrix-org/matrix-doc/pull/2625
2020-06-17 10:58:32 +01:00
Richard van der Hoff
e452973fd2
fix broken link in sample config (#7712) 2020-06-16 19:50:16 +01:00
Erik Johnston
f6f7511a4c
Refactor getting replication updates from database. (#7636)
The aim here is to make it easier to reason about when streams are limited and when they're not, by moving the logic into the database functions themselves. This should mean we can kill of `db_query_to_update_function` function.
2020-06-16 17:10:28 +01:00
Patrick Cloke
231252516c
Fix "argument of type 'ObservableDeferred' is not iterable" error (#7708) 2020-06-16 12:01:18 -04:00
hungrymonkey
5c5516f80e
Add instructions for authing with Keycloak via OpenID (#7659) 2020-06-16 11:28:21 -04:00
Patrick Cloke
ac51bd581a
Include a user agent in federation requests. (#7677) 2020-06-16 10:43:29 -04:00
Dagfinn Ilmari Mannsåker
a3f11567d9
Replace all remaining six usage with native Python 3 equivalents (#7704) 2020-06-16 08:51:47 -04:00
Patrick Cloke
98c4e35e3c
Convert the device message and pagination handlers to async/await. (#7678) 2020-06-16 08:06:17 -04:00
Richard van der Hoff
03619324fc
Create a ListenerConfig object (#7681)
This ended up being a bit more invasive than I'd hoped for (not helped by
generic_worker duplicating some of the code from homeserver), but hopefully
it's an improvement.

The idea is that, rather than storing unstructured `dict`s in the config for
the listener configurations, we instead parse it into a structured
`ListenerConfig` object.
2020-06-16 12:44:07 +01:00
Brendan Abolivier
65eb078498
1.15.1 2020-06-16 10:28:58 +01:00