Matrix/synapse
Matrix/synapse
1
0
Fork 0
mirror of https://github.com/element-hq/synapse synced 2024-07-08 14:04:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
12741 commits 890 branches 702 tags 485 MiB
Commit graph

683 commits

Author SHA1 Message Date
Amber Brown 0dce9e1379
Write some tests for the email pusher (#4095) 2018-10-30 23:55:43 +11:00
David Baker 4eacf0f200 news fragment 2018-10-30 10:05:51 +00:00
Erik Johnston 169851b412
Merge pull request #4109 from matrix-org/erikj/repl_devices 
A couple of replication fixes for device lists
 2018-10-29 18:16:48 +00:00
Erik Johnston 4f0fa7a120 Newsfile 2018-10-29 18:15:42 +00:00
Erik Johnston 39f419868f Newsfile 2018-10-29 17:38:09 +00:00
Amber Brown 4cd1c9f2ff
Delete the disused & unspecced identicon functionality (#4106) 2018-10-29 23:57:24 +11:00
Richard van der Hoff 7fbfea062e
Merge pull request #4100 from matrix-org/rav/room_upgrade_avatar 
Remember to copy the avatar on room upgrades
 2018-10-29 12:49:21 +00:00
Richard van der Hoff 56ca578f77
Merge pull request #4099 from matrix-org/rav/upgrade_odd_pls 
Better handling of odd PLs during room upgrades
 2018-10-29 12:48:51 +00:00
Richard van der Hoff bf33eed609
Merge pull request #4091 from matrix-org/rav/room_version_upgrades 
Room version upgrade support
 2018-10-29 12:47:20 +00:00
Amber Brown c4b3698a80
Make the replication logger quieter (#4108) 2018-10-29 22:59:44 +11:00
Amber Brown 3b0a85fc8e changelog 2018-10-29 21:54:01 +11:00
Richard van der Hoff db24d7f15e Better handling of odd PLs during room upgrades 
Fixes handling of rooms where we have permission to send the tombstone, but not
other state. We need to (a) fail more gracefully when we can't send the PLs in
the old room, and (b) not set the PLs in the new room until we are done with
the other stuff.
 2018-10-27 00:54:26 +01:00
Richard van der Hoff 5caf79b312 Remember to copy the avatar on room upgrades 2018-10-26 23:56:40 +01:00
Erik Johnston 03e634dad4 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups 2018-10-26 16:22:45 +01:00
Amber Brown 77d70a7646
Port register_new_matrix_user to Python 3 and add tests (#4085) 2018-10-26 22:05:22 +11:00
Richard van der Hoff 6cb2e2448a
Merge pull request #4089 from dekonnection/master 
Make Docker image listen on ipv6 as well as ipv4
 2018-10-25 19:23:37 +01:00
Richard van der Hoff 68c0ce62d8 changelog 2018-10-25 19:18:25 +01:00
Cédric Laudrel 379376e5e6 Make Docker image listening on ipv6 as well as ipv4 
Signed-off-by: Cédric Laudrel <dek@iono.me>
 2018-10-25 20:03:47 +02:00
Erik Johnston cb53ce9d64
Refactor state group lookup to reduce DB hits (#4011) 
Currently when fetching state groups from the data store we make two
hits two the database: once for members and once for non-members (unless
request is filtered to one or the other). This adds needless load to the
datbase, so this PR refactors the lookup to make only a single database
hit.
 2018-10-25 17:49:55 +01:00
Richard van der Hoff e5da60d75d Merge remote-tracking branch 'origin/master' into develop 2018-10-25 17:08:09 +01:00
Erik Johnston c85e063302
Merge pull request #4051 from matrix-org/erikj/alias_disallow_list 
Add config option to control alias creation
 2018-10-25 17:04:59 +01:00
Neil Johnson 95ad128851
Merge pull request #4081 from matrix-org/neilj/fix_mau_init 
fix race condiftion in calling initialise_reserved_users
 2018-10-25 16:33:40 +01:00
Erik Johnston b94a43d5b5 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/alias_disallow_list 2018-10-25 15:25:31 +01:00
Neil Johnson f7f487e14c Merge branch 'develop' of github.com:matrix-org/synapse into matthew/autocreate_autojoin 2018-10-25 14:40:06 +01:00
Travis Ralston 54def42c19 Merge branch 'develop' into travis/login-terms 2018-10-24 13:22:59 -06:00
Neil Johnson ea69a84bbb fix style inconsistencies 2018-10-24 17:18:08 +01:00
Neil Johnson 07126e43a4 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_mau_init 2018-10-24 16:25:39 +01:00
Richard van der Hoff 9f72c209ee
Update changelog.d/3975.feature 
Co-Authored-By: neilisfragile <neil@matrix.org>
 2018-10-24 14:37:36 +01:00
Richard van der Hoff 3ad359e5be Merge remote-tracking branch 'origin/develop' into rav/fix_event_filter_validation 2018-10-24 11:23:49 +01:00
Richard van der Hoff 7328039117
Merge pull request #4082 from matrix-org/rav/fix_pep8 
Fix a number of flake8 errors
 2018-10-24 11:23:35 +01:00
Erik Johnston 3904cbf307
Merge pull request #4040 from matrix-org/erikj/states_res_v2_rebase 
Add v2 state resolution algorithm
 2018-10-24 11:12:12 +01:00
Richard van der Hoff 7e07d25ed6 Allow backslashes in event field filters 
Fixes a bug introduced in https://github.com/matrix-org/synapse/pull/1783 which
meant that single backslashes were not allowed in event field filters.

The intention here is to allow single-backslashes, but disallow
double-backslashes.
 2018-10-24 11:11:24 +01:00
Richard van der Hoff ef771cc4c2 Fix a number of flake8 errors 
Broadly three things here:

* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
  them
* use `r""` for strings which include backslashes

Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.
 2018-10-24 10:39:03 +01:00
Richard van der Hoff e0b9d5f0af
Merge pull request #4075 from matrix-org/rav/fix_pusher_logcontexts 
Clean up the way logcontexts and threads work in the pushers
 2018-10-24 09:53:57 +01:00
Erik Johnston cb23aa4c42
Merge pull request #4063 from matrix-org/erikj/cleaup_alias_creation 
Clean up room alias creation
 2018-10-24 09:40:27 +01:00
Neil Johnson a67d8ace9b remove errant exception and style 2018-10-23 17:44:39 +01:00
Travis Ralston 43c3f0b02f
Merge pull request #3969 from turt2live/travis/fix-federated-group-requests 
Handle HttpResponseException more safely for federated groups
 2018-10-23 10:41:04 -06:00
Neil Johnson 329d18b39c remove white space 2018-10-23 15:27:20 +01:00
Neil Johnson 6105c6101f fix race condiftion in calling initialise_reserved_users 2018-10-23 15:24:58 +01:00
Richard van der Hoff b3f6dddad2
Give some more things logcontexts (#4077) 2018-10-23 14:29:17 +01:00
Richard van der Hoff 5c445114d3
Correctly account for cpu usage by background threads (#4074) 
Wrap calls to deferToThread() in a thing which uses a child logcontext to
attribute CPU usage to the right request.

While we're in the area, remove the logcontext_tracer stuff, which is never
used, and afaik doesn't work.

Fixes #4064
 2018-10-23 13:12:32 +01:00
Richard van der Hoff 1fe6bbb555
Merge pull request #3698 from spantaleev/add-matrix-docker-ansible-deploy 
Add information about the matrix-docker-ansible-deploy playbook
 2018-10-23 10:11:58 +01:00
Richard van der Hoff abd9914683 Changelog 2018-10-22 16:12:11 +01:00
Richard van der Hoff 7aea00069c
Merge pull request #4076 from matrix-org/rav/fix_init_logcontexts 
Run MAU queries as background processes
 2018-10-22 14:46:59 +01:00
Richard van der Hoff 911db96658
Merge pull request #4073 from matrix-org/rav/require_psutil 
Make psutil an explicit dependency
 2018-10-22 12:33:21 +01:00
Richard van der Hoff a6f421e812 Run MAU queries as background processes 
Fixes #3820
 2018-10-20 02:14:35 +01:00
Amber Brown e1728dfcbe
Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase on py3) (#4068) 2018-10-20 11:16:55 +11:00
steamport b85fe45f46
Add CL 2018-10-19 21:55:38 +00:00
Richard van der Hoff e5b52d0f94 Make psutil an explicit dependency 
As of #4027, we require psutil to be installed, so it should be in our
dependency list. We can also remove some of the conditional import code
introduced by #992.

Fixes #4062.
 2018-10-19 21:51:15 +01:00
Richard van der Hoff f62c597d14
Merge pull request #4069 from matrix-org/rav/fix_email_templates_4065 
Calculate absolute path for email templates
 2018-10-19 16:33:54 +01:00
Erik Johnston 056f099126 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups 2018-10-19 15:48:59 +01:00
Richard van der Hoff cc325c7069 Calculate absolute path for email templates 2018-10-19 14:01:59 +01:00
Amber Brown e404ba9aac
Fix manhole on py3 (pt 2) (#4067) 2018-10-19 22:26:00 +11:00
Amber Brown b69216f768
Make the metrics less racy (#4061) 2018-10-19 21:45:45 +11:00
Erik Johnston f9d6c677ea Newsfile 2018-10-19 10:22:45 +01:00
Erik Johnston 0d31109ed5 Newsfile 2018-10-19 10:14:29 +01:00
Amber Brown 1d17fc52ae changelog 2018-10-19 09:27:10 +11:00
Richard van der Hoff c69026a758 Use the right python when starting workers 
We should use the same python to start the workers as we do for the main
synapse (ie, the same one used to run synctl.)
 2018-10-18 21:06:30 +01:00
Travis Ralston 49a044aa5f Merge branch 'develop' into travis/login-terms 2018-10-18 09:57:58 -06:00
Richard van der Hoff 03287c350e remove redundant changelog file 
this change has been released
 2018-10-18 15:09:34 +01:00
Richard van der Hoff c632bc8654 Merge branch 'master' into develop 2018-10-18 15:07:03 +01:00
Erik Johnston e77f24d80a
Merge pull request #4049 from matrix-org/erikj/synctl_colour 
Only colourise synctl output when attached to tty
 2018-10-18 10:51:14 +01:00
Richard van der Hoff 52e3d3813b prep changelog 2018-10-17 17:40:01 +01:00
Richard van der Hoff 0fd2321629 Fix incorrect truncation in get_missing_events 
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.

We also shouldn't really be relying on depth, and should be checking the
room_id.
 2018-10-17 17:35:26 +01:00
Richard van der Hoff 1519572961 Ship the email templates as package_data 
move the example email templates into the synapse package so that they can be
used as package data, which should mean that all of the packaging mechanisms
(pip, docker, debian, arch, etc) should now come with the example templates.

In order to grandfather in people who relied on the templates being in the old
place, check for that situation and fall back to using the defaults if the
templates directory does not exist.
 2018-10-17 16:46:02 +01:00
Erik Johnston 3a5d8d5891 Newsfile 2018-10-17 16:12:45 +01:00
Erik Johnston 1af16acd4c Newsfile 2018-10-17 13:48:09 +01:00
Will Hunt d6a7797dd1 Fix roomlist since tokens on Python 3 (#4046) 
Thanks @Half-Shot !!!
 2018-10-17 23:04:55 +11:00
Richard van der Hoff 6ec9d8ba0a
Merge pull request #4045 from matrix-org/rav/fix_get_missing_events 
Fix incorrect truncation in get_missing_events
 2018-10-17 11:40:03 +01:00
Neil Johnson c6584f4b5f clean up config error logic and imports 2018-10-17 11:36:41 +01:00
Richard van der Hoff fc0f13dd03 Fix incorrect truncation in get_missing_events 
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.

We also shouldn't really be relying on depth, and should be checking the
room_id.
 2018-10-16 21:10:04 +01:00
Richard van der Hoff 017eb9d17a changelog 2018-10-16 16:31:47 +01:00
Erik Johnston fc954960e9 Newsfile 2018-10-16 16:28:42 +01:00
Richard van der Hoff b8a5b0097c
Various cleanups in the federation client code (#4031) 
- Improve logging: log things in the right order, include destination and txids
  in all log lines, don't log successful responses twice

- Fix the docstring on TransportLayerClient.send_transaction

- Don't use treq.request, which is overcomplicated for our purposes: just use a
  twisted.web.client.Agent.

- simplify the logic for setting up the bodyProducer

- fix bytes/str confusions
 2018-10-16 10:44:49 +01:00
Travis Ralston f293d124b6 Merge branch 'develop' into travis/login-terms 2018-10-15 14:44:32 -06:00
Amber Brown 24bc15eab4 update changelog 2018-10-15 22:23:13 +11:00
Amber Brown 4e50fe3edb update changelog 2018-10-15 22:22:49 +11:00
David Baker 03c11032c3
Merge pull request #4019 from matrix-org/dbkr/e2e_backups 
E2E backups
 2018-10-15 10:19:43 +01:00
Slavi Pantaleev c187638ee9 Add information about the matrix-docker-ansible-deploy playbook 
Signed-off-by: Slavi Pantaleev <slavi@devture.com>
 2018-10-14 21:50:18 +03:00
Ivan Shapovalov 06bc8d2fe5 synapse/app: frontend_proxy.py: actually make workers work on py3 2018-10-14 20:08:39 +03:00
Ivan Shapovalov fb216a22db synapse/visibility.py: fix SyntaxError on py3.7 2018-10-14 20:08:17 +03:00
Travis Ralston 7ede650956 Merge branch 'develop' into travis/login-terms 2018-10-12 16:24:07 -06:00
Travis Ralston e3586f7c06 Merge branch 'develop' into travis/fix-federated-group-requests 2018-10-12 14:49:58 -06:00
Neil Johnson a2bfb778c8 improve auto room join logic, comments and tests 2018-10-12 18:17:36 +01:00
Amber Brown 381d2cfdf0
Make workers work on Py3 (#4027) 2018-10-13 00:14:08 +11:00
Erik Johnston 49840f5ab2 Update newsfile 2018-10-10 11:31:02 +01:00
Erik Johnston 20733857ab Newsfile 2018-10-09 14:18:16 +01:00
David Baker d34657e1f2 Add changelog 2018-10-09 11:15:54 +01:00
Erik Johnston 9eb1a79100
Merge pull request #4008 from matrix-org/erikj/log_looping_exceptions 
Log looping call exceptions
 2018-10-09 10:12:10 +01:00
Richard van der Hoff 495975e231 Optimisation for filter_events_for_server 
We're better off hashing just the event_id than the whole ((type, state_key),
event_id) tuple - so use a dict instead of a set.

Also, iteritems > items.
 2018-10-08 13:46:52 +01:00
Erik Johnston 8164f6daf3 Newsfile 2018-10-05 11:25:09 +01:00
Neil Johnson ed82043efb
Merge branch 'develop' into matthew/autocreate_autojoin 2018-10-04 17:26:59 +01:00
Richard van der Hoff c6dbd216e6
Merge pull request #3995 from matrix-org/rav/no_deextrem_outliers 
Fix bug in forward_extremity update logic
 2018-10-04 16:27:05 +01:00
Erik Johnston d9f3db5081 Newsfile 2018-10-04 16:03:08 +01:00
Amber Brown d86794325f Merge branch 'master' into develop 2018-10-04 22:41:52 +10:00
Amber Brown 92faeb2a3f changelog 2018-10-04 22:38:10 +10:00
Travis Ralston 158d6c75b6 Changelog 2018-10-03 17:54:08 -06:00
Richard van der Hoff a59d899668 Pin to prometheus_client<0.4 to avoid renaming all of our metrics 2018-10-03 17:20:15 +01:00
Erik Johnston 8935ec5a93
Merge pull request #3999 from matrix-org/erikj/fix_3pid_invite_rejetion 
Fix handling of rejected threepid invites
 2018-10-03 14:47:41 +01:00
Erik Johnston 81e2813948
Merge pull request #3996 from matrix-org/erikj/fix_bg_iteration 
Fix exception in background metrics collection
 2018-10-03 14:14:38 +01:00
Erik Johnston 93a8603904 Newsfile 2018-10-03 11:59:05 +01:00
Erik Johnston 6e0c66f651 Newsfile 2018-10-03 11:40:02 +01:00
Erik Johnston c69faf8c4a Newsfile 2018-10-03 11:29:44 +01:00
Richard van der Hoff 2a4ea3baa8 fix newsfile name 2018-10-03 07:09:25 +01:00
Richard van der Hoff ae61ade891 Fix bug in forward_extremity update logic 
An event does not stop being a forward_extremity just because an outlier or
rejected event refers to it.
 2018-10-02 23:36:08 +01:00
Amber Brown ccce6f26a6 changelog 2018-10-03 02:16:21 +10:00
Erik Johnston 25baf3b2ac
Merge pull request #3991 from matrix-org/erikj/fix_pop_retry_cache 
Fix bug when invalidating destination retry timings
 2018-10-02 16:42:28 +01:00
Amber Brown 058a4c665e
Remove Jenkins & other old dev junk (#3988) 2018-10-03 00:59:11 +10:00
Erik Johnston 8f614f842d Newsfile 2018-10-02 15:54:31 +01:00
Richard van der Hoff 2b8d28b095
Merge pull request #3960 from matrix-org/rav/fix_missing_create_event_error 
Fix error handling for missing auth_event
 2018-10-02 13:57:54 +01:00
Amber Brown 7232917f12
Disable frozen dicts by default (#3987) 2018-10-02 22:53:47 +10:00
Erik Johnston eeb755d17b Newsfile 2018-10-02 12:06:25 +01:00
Richard van der Hoff 8c41b0ca66
Merge pull request #3989 from matrix-org/rav/better_stacktraces 
Avoid reraise, to improve stacktraces
 2018-10-02 10:47:38 +01:00
Erik Johnston bc29946809
Merge pull request #3986 from matrix-org/erikj/fix_sync_with_redacted_state 
Fix lazy loaded sync with rejected state events
 2018-10-02 11:38:35 +02:00
Richard van der Hoff 8174c6725b Avoid reraise, to improve stacktraces 2018-10-01 18:50:34 +01:00
Richard van der Hoff 5908a8f6f5 Merge branch 'develop' into rav/fix_missing_create_event_error 2018-10-01 17:16:20 +01:00
Amber Brown abdc141c2b
Update instructions to point to pip install (#3985) 2018-10-02 01:08:38 +10:00
Richard van der Hoff b5b93f45d5
Merge pull request #3968 from matrix-org/rav/fix_federation_errors 
Fix exceptions when handling incoming transactions
 2018-10-01 15:54:24 +01:00
Amber Brown 6e05fd032c
Fix userconsent on Python 3 (#3938) 2018-10-02 00:11:58 +10:00
Erik Johnston 5fa27eac78 Newsfile 2018-10-01 14:24:41 +01:00
Erik Johnston 8f5c23d0cd
Merge pull request #3933 from matrix-org/erikj/destination_retry_cache 
Add a five minute cache to get_destination_retry_timings
 2018-10-01 14:20:30 +02:00
Richard van der Hoff 53c5fa4e6c
Further reduce the size of the docker image (#3972) 
Rewrite the dockerfile as a multistage build: this means we can get rid of a whole load of cruft which we don't need.
 2018-10-01 12:29:17 +01:00
Matthew Hodgson faa462ef79 changelog 2018-09-29 02:21:01 +01:00
Bruno Windels 9a8bbc9a59 add --no-admin flag to registration script (#3836) 2018-09-28 13:36:56 +01:00
Richard van der Hoff 3deaad2fb4
Merge pull request #3964 from matrix-org/rav/remove_localhost_checks 
remove spurious federation checks on localhost
 2018-09-28 13:35:47 +01:00
Richard van der Hoff 26557c9db4
Merge pull request #3980 from matrix-org/rav/remove_broken_cache_call 
Remove redundant call to start_get_pdu_cache
 2018-09-28 13:13:19 +01:00
Jan Christian Grünhage 5304449738 build python 3 docker images on circle CI (#3976) 2018-09-28 12:36:35 +01:00
Richard van der Hoff 19475cf337 Remove redundant call to start_get_pdu_cache 
I think this got forgotten in #3932. We were getting away with it because it
was the last call in this function.
 2018-09-28 12:01:23 +01:00
Richard van der Hoff 9c8cec5dab Merge remote-tracking branch 'origin/develop' into erikj/destination_retry_cache 2018-09-28 10:51:09 +01:00
Richard van der Hoff f094f715cf Merge remote-tracking branch 'origin/develop' into rav/fix_federation_errors 2018-09-27 15:18:21 +01:00
Richard van der Hoff 36c62a67c4
Merge pull request #3794 from matrix-org/erikj/faster_typing 
Improve performance of getting typing updates for replication
 2018-09-27 15:14:26 +01:00
Richard van der Hoff b5c976347b
Merge pull request #3946 from matrix-org/michaelkaye/automate_docker_hub_upload 
Build and push docker image to hub automatically
 2018-09-27 15:07:41 +01:00
Richard van der Hoff e1e3e77bfd
Merge pull request #3967 from matrix-org/rav/federation_handler_cleanups 
Clarifications in FederationHandler
 2018-09-27 15:06:59 +01:00
Amber Brown 6de3884e5e
Merge pull request #3965 from matrix-org/rav/notify_app_services_bg_process 
Run notify_app_services as a bg process
 2018-09-27 23:40:30 +10:00
Amber Brown a512e637ac
Merge pull request #3970 from schnuffle/develop-py3 
Replaced all occurences of e.message with str(e)
 2018-09-27 23:39:45 +10:00
Amber Brown b3064532d0
Run our oldest supported configuration in CI (#3952) 2018-09-27 23:21:54 +10:00
Amber Brown 861c063ebc
Update 3970.bugfix 2018-09-27 22:47:01 +10:00
Schnuffle a873896096 Added changelog fragment 2018-09-27 14:01:44 +02:00
Richard van der Hoff 51d33d5178
Merge pull request #3961 from matrix-org/neilj/lock_mau_upserts 
fix #3854 MAU transaction errors
 2018-09-27 12:31:37 +01:00
Michael Kaye 74bbdd0412 Do the changelog.d dance 2018-09-27 12:01:43 +01:00
Richard van der Hoff 948a6d8776 changelog 2018-09-27 11:37:39 +01:00
Richard van der Hoff ad8e137062 changelog 2018-09-27 11:31:51 +01:00
Richard van der Hoff 92abd3d6d6
Merge pull request #3966 from matrix-org/rav/rx_txn_logging_2 
Logging improvements
 2018-09-27 11:27:46 +01:00
Richard van der Hoff 4a15a3e4d5
Include eventid in log lines when processing incoming federation transactions (#3959) 
when processing incoming transactions, it can be hard to see what's going on,
because we process a bunch of stuff in parallel, and because we may end up
recursively working our way through a chain of three or four events.

This commit creates a way to use logcontexts to add the relevant event ids to
the log lines.
 2018-09-27 11:25:34 +01:00
Richard van der Hoff ae6ad4cf41
docstrings and unittests for storage.state (#3958) 
I spent ages trying to figure out how I was going mad...
 2018-09-27 11:22:25 +01:00
Travis Ralston 2a7b3439de Changelog 2018-09-26 13:51:34 -06:00
Richard van der Hoff e70b4ce069 Logging improvements 
Some logging tweaks to help with debugging incoming federation transactions
 2018-09-26 17:36:14 +01:00
Richard van der Hoff 3c37c7e45b changelog 2018-09-26 17:01:30 +01:00
Richard van der Hoff 0c4a99ea2d changelog 2018-09-26 16:54:54 +01:00
Richard van der Hoff ab59f3d8da changelog 2018-09-26 16:52:24 +01:00