Commit graph

600 commits

Author SHA1 Message Date
Andrew Morgan
409c62b27b
Add config linting script that checks for bool casing (#6203)
Add a linting script that enforces all boolean values in the default config be lowercase.

This has annoyed me for a while so I decided to fix it.
2019-10-23 13:22:54 +01:00
Adrien Luxey
0327a00a37 Update postgres.md (#6234)
Added database owner authentication with `sudo` when `su` does not work
2019-10-22 13:48:02 +02:00
Neil Johnson
82c8799ec7 Set room version default to 5 2019-10-19 09:06:15 +01:00
Valérian Rousset
be9b55e0d2 cas: support setting display name (#6114)
Now, the CAS server can return an attribute stating what's the desired displayname, instead of using the username directly.
2019-10-11 12:33:12 +01:00
Brendan Abolivier
baf12bc02a Merge branch 'master' into develop 2019-10-02 14:41:02 +01:00
Erik Johnston
3423633d50 Fix 'redaction_retention_period' sampel config to match guidelines 2019-09-26 16:43:52 +01:00
Neil Johnson
8b8f8c7b3c Explicitly log when a homeserver does not have a trusted key server configured (#6090) 2019-09-26 12:57:01 +01:00
Erik Johnston
4fb3c129aa Merge branch 'develop' of github.com:matrix-org/synapse into erikj/cleanup_user_ips_2 2019-09-25 17:53:13 +01:00
Erik Johnston
39b50ad42a Review comments 2019-09-25 17:22:33 +01:00
Erik Johnston
242017db8b Prune rows in user_ips older than configured period
Defaults to pruning everything older than 28d.
2019-09-24 15:53:17 +01:00
Richard van der Hoff
ed8b92f0d2 Merge remote-tracking branch 'origin/develop' into rav/saml_mapping_work 2019-09-24 12:57:32 +01:00
Andrew Morgan
50776261e1 Add submit_url response parameter to msisdn /requestToken (#6079)
Second part of solving #6076
Fixes #6076

We return a submit_url parameter on calls to POST */msisdn/requestToken so that clients know where to submit token information to.
2019-09-23 21:21:03 +01:00
Richard van der Hoff
78e8ec368e
Merge pull request #6064 from matrix-org/rav/saml_config_cleanup
Make the sample saml config closer to our standards
2019-09-23 20:36:51 +01:00
Andrew Morgan
e08ea43463 Use the federation blacklist for requests to untrusted Identity Servers (#6000)
Uses a SimpleHttpClient instance equipped with the federation_ip_range_blacklist list for requests to identity servers provided by user input. Does not use a blacklist when contacting identity servers specified by account_threepid_delegates. The homeserver trusts the latter and we don't want to prevent homeserver admins from specifying delegates that are on internal IP addresses.

Fixes #5935
2019-09-23 20:23:20 +01:00
Andrew Morgan
df3401a71d
Allow HS to send emails when adding an email to the HS (#6042) 2019-09-20 15:21:30 +01:00
Richard van der Hoff
b65327ff66 Merge branch 'develop' into rav/saml_mapping_work 2019-09-19 18:13:31 +01:00
Richard van der Hoff
b789c7eb03 Merge branch 'develop' into rav/saml_config_cleanup 2019-09-19 15:05:31 +01:00
Jorik Schellekens
38fd1f8e3f Fix typo in account_threepid_delegates config (#6028) 2019-09-18 22:30:44 +01:00
Richard van der Hoff
7100b5cc9d fix sample config
this was apparently broken by #6040.
2019-09-18 10:16:00 +01:00
dstipp
379d2a8c39 (#5849) Convert rst to markdown (#6040)
Converting some of the rst documentation to markdown.  Attempted to
preserve whitespace and line breaks to minimize cosmetic change.
2019-09-17 12:55:29 +01:00
Richard van der Hoff
a8ac40445c Record mappings from saml users in an external table
We want to assign unique mxids to saml users based on an incrementing
suffix. For that to work, we need to record the allocated mxid in a separate
table.
2019-09-13 16:01:46 +01:00
Richard van der Hoff
785cbd3999 Make the sample saml config closer to our standards
It' still not great, thanks to the nested dictionaries, but it's better.
2019-09-13 12:07:03 +01:00
Travis Ralston
c755955f33 Add developer docs for using SAML without a server (#6032) 2019-09-13 08:58:18 +01:00
axel simon
1c7df13e7b add explanations on how to actually include an access_token (#6031) 2019-09-13 08:50:17 +01:00
Sorunome
dd2e5b0038 add report_stats_endpoint config option (#6012)
This PR adds the optional `report_stats_endpoint` to configure where stats are reported to, if enabled.
2019-09-12 11:24:57 +01:00
Erik Johnston
66ace43546 Update sample config 2019-09-11 14:50:40 +01:00
Erik Johnston
8df88b5ff3 Update sample config 2019-09-11 10:58:26 +01:00
Erik Johnston
470dc621ae
Merge pull request #5934 from matrix-org/erikj/censor_redactions
Censor redactions in DB after a month
2019-09-09 15:29:39 +01:00
Erik Johnston
8b9ade8c78 Default to censoring redactions after seven days 2019-09-09 13:55:28 +01:00
Amber Brown
55d5b3af88
Servers-known-about statistic (#5981) 2019-09-07 01:45:51 +10:00
Andrew Morgan
0c0b82b6d1
Allow Synapse to send registration emails + choose Synapse or an external server to handle 3pid validation (#5987)
This is a combination of a few different PRs, finally all being merged into `develop`:

* #5875 
* #5876 
* #5868 (This one added the `/versions` flag but the flag itself was actually [backed out](891afb57cb (diff-e591d42d30690ffb79f63bb726200891)) in #5969. What's left is just giving /versions access to the config file, which could be useful in the future)
* #5835 
* #5969 
* #5940

Clients should not actually use the new registration functionality until https://github.com/matrix-org/synapse/pull/5972 is merged.

UPGRADE.rst, changelog entries and config file changes should all be reviewed closely before this PR is merged.
2019-09-06 11:35:28 +01:00
Erik Johnston
591d82f06b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/censor_redactions 2019-09-05 17:27:46 +01:00
Erik Johnston
3ff0422d2d Make redaction retention period configurable 2019-09-05 17:16:45 +01:00
Erik Johnston
6e834e94fc
Fix and refactor room and user stats (#5971)
Previously the stats were not being correctly populated.
2019-09-04 13:04:27 +01:00
Jorik Schellekens
92c1550f4a Add a link to python's logging config schema (#5926) 2019-08-28 19:08:32 +01:00
Jorik Schellekens
6d97843793
Config templating (#5900)
Template config files

* Imagine a system composed entirely of x, y, z etc and the basic operations..

Wait George, why XOR? Why not just neq?

George: Eh, I didn't think of that..

Co-Authored-By: Erik Johnston <erik@matrix.org>
2019-08-28 13:12:22 +01:00
Amber Brown
7dc398586c
Implement a structured logging output system. (#5680) 2019-08-28 21:18:53 +10:00
reivilibre
a3f0635686
Merge pull request #5914 from matrix-org/rei/admin_getadmin
Add GET method to admin API /users/@user:dom/admin
2019-08-28 09:44:22 +01:00
Olivier Wilkinson (reivilibre)
1b959b6977 Document GET method for retrieving admin bit of user in admin API
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2019-08-27 13:19:19 +01:00
Erik Johnston
f5b50d0871
Merge pull request #5895 from matrix-org/erikj/notary_key
Add config option to sign remote key query responses with a separate key.
2019-08-27 11:51:37 +01:00
reivilibre
1a7e6eb633
Add Admin API capability to set adminship of a user (#5878)
Admin API: Set adminship of a user
2019-08-27 10:14:00 +01:00
Erik Johnston
7af5a63063 Fixup review comments 2019-08-23 15:36:28 +01:00
Jorik Schellekens
8767b63a82
Propagate opentracing contexts through EDUs (#5852)
Propagate opentracing contexts through EDUs
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-08-22 18:21:10 +01:00
Richard van der Hoff
119aa31b10
Servlet to purge old rooms (#5845) 2019-08-22 10:42:59 +01:00
Erik Johnston
5906be8589 Add config option for keys to use to sign keys
This allows servers to separate keys that are used to sign remote keys
when acting as a notary server.
2019-08-21 10:44:58 +01:00
Amber Brown
0b6fbb28a8
Don't load the media repo when configured to use an external media repo (#5754) 2019-08-13 21:49:28 +10:00
Brendan Abolivier
f25f638c35
Lint 2019-08-01 12:19:08 +02:00
Brendan Abolivier
3ff3dfe5a3
Sample config 2019-08-01 12:08:25 +02:00
Richard van der Hoff
8c97f6414c
Remove non-functional 'expire_access_token' setting (#5782)
The `expire_access_token` didn't do what it sounded like it should do. What it
actually did was make Synapse enforce the 'time' caveat on macaroons used as
access tokens, but since our access token macaroons never contained such a
caveat, it was always a no-op.

(The code to add 'time' caveats was removed back in v0.18.5, in #1656)
2019-07-30 08:25:02 +01:00
Amber Brown
865077f1d1
Room Complexity Client Implementation (#5783) 2019-07-30 02:47:27 +10:00
Jorik Schellekens
3641784e8c
Make Jaeger fully configurable (#5694)
* Allow Jaeger to be configured

* Update sample config
2019-07-23 15:46:04 +01:00
Amber Brown
4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Jorik Schellekens
0fd171770a Merge branch 'release-v1.2.0' into develop 2019-07-22 11:18:50 +01:00
Jorik Schellekens
826e6ec3bd
Opentracing Documentation (#5703)
* Opentracing survival guide

* Update decorator names in doc

* Doc cleanup

These are all alterations as a result of comments in #5703, it
includes mostly typos and clarifications. The most interesting
changes are:

- Split developer and user docs into two sections
- Add a high level description of OpenTracing

* newsfile

* Move contributer specific info to docstring.

* Sample config.

* Trailing whitespace.

* Update 5703.misc

* Apply suggestions from code review

Mostly just rewording parts of the docs for clarity.

Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-07-22 11:15:21 +01:00
Richard van der Hoff
b73ce4ba81
Update the coding style doc (#5719)
A few fixes and removal of duplicated stuff, but mostly a bunch of the words on the config file.
2019-07-19 11:55:14 +01:00
Richard van der Hoff
82345bc09a
Clean up opentracing configuration options (#5712)
Clean up config settings and dead code.

This is mostly about cleaning up the config format, to bring it into line with our conventions. In particular:
 * There should be a blank line after `## Section ##' headings
 * There should be a blank line between each config setting
 * There should be a `#`-only line between a comment and the setting it describes
 * We don't really do the `#  #` style commenting-out of whole sections if we can help it
 * rename `tracer_enabled` to `enabled`

While we're here, do more config parsing upfront, which makes it easier to use
later on.

Also removes redundant code from LogContextScopeManager.

Also changes the changelog fragment to a `feature` - it's exciting!
2019-07-18 15:06:54 +01:00
Amber Brown
7ad1d76356
Support Prometheus_client 0.4.0+ (#5636) 2019-07-18 23:57:15 +10:00
Richard van der Hoff
1def298119
Improve Depends specs in debian package. (#5675)
This is basically a contrived way of adding a `Recommends` on `libpq5`, to fix #5653.

The way this is supposed to happen in debhelper is to run
`dh_shlibdeps`, which in turn runs `dpkg-shlibdeps`, which spits things out
into `debian/<package>.substvars` whence they can later be included by
`control`.

Previously, we had disabled `dh_shlibdeps`, mostly because `dpkg-shlibdeps`
gets confused about PIL's interdependent objects, but that's not really the
right thing to do and there is another way to work around that.

Since we don't always use postgres, we don't necessarily want a hard Depends on
libpq5, so I've actually ended up adding an explicit invocation of
`dpkg-shlibdeps` for `psycopg2`.

I've also updated the build-depends list for the package, which was missing a
couple of entries.
2019-07-17 17:47:07 +01:00
Richard van der Hoff
5f158ec039
Implement access token expiry (#5660)
Record how long an access token is valid for, and raise a soft-logout once it
expires.
2019-07-12 17:26:02 +01:00
Ulrik Günther
d445b3ae57 Update reverse_proxy.rst (#5397)
Updates reverse_proxy.rst with information about nginx' URI normalisation.
2019-07-12 11:46:18 +01:00
Lrizika
39e9839a04 Improved docs on setting up Postgresql (#5661)
Added that synapse_user needs a database to access before it can auth
Noted you'll need to enable password auth, linked to pg_hba.conf docs
2019-07-11 14:31:36 +01:00
Jorik Schellekens
38a6d3eea7
Add basic opentracing support (#5544)
* Configure and initialise tracer

Includes config options for the tracer and sets up JaegerClient.

* Scope manager using LogContexts

We piggy-back our tracer scopes by using log context.
The current log context gives us the current scope. If new scope is
created we create a stack of scopes in the context.

* jaeger is a dependency now

* Carrier inject and extraction for Twisted Headers

* Trace federation requests on the way in and out.

The span is created in _started_processing and closed in
_finished_processing because we need a meaningful log context.

* Create logcontext for new scope.

Instead of having a stack of scopes in a logcontext we create a new
context for a new scope if the current logcontext already has a scope.

* Remove scope from logcontext if logcontext is top level

* Disable tracer if not configured

* typo

* Remove dependence on jaeger internals

* bools

* Set service name

* :Explicitely state that the tracer is disabled

* Black is the new black

* Newsfile

* Code style

* Use the new config setup.

* Generate config.

* Copyright

* Rename config to opentracing

* Remove user whitelisting

* Empty whitelist by default

* User ConfigError instead of RuntimeError

* Use isinstance

* Use tag constants for opentracing.

* Remove debug comment and no need to explicitely record error

* Two errors a "s(c)entry"

* Docstrings!

* Remove debugging brainslip

* Homeserver Whitlisting

* Better opentracing config comment

* linting

* Inclue worker name in service_name

* Make opentracing an optional dependency

* Neater config retreival

* Clean up dummy tags

* Instantiate tracing as object instead of global class

* Inlcude opentracing as a homeserver member.

* Thread opentracing to the request level

* Reference opetnracing through hs

* Instantiate dummy opentracin g for tests.

* About to revert, just keeping the unfinished changes just in case

* Revert back to global state, commit number:

9ce4a3d9067bf9889b86c360c05ac88618b85c4f

* Use class level methods in tracerutils

* Start and stop requests spans in a place where we
have access to the authenticated entity

* Seen it, isort it

* Make sure to close the active span.

* I'm getting black and blue from this.

* Logger formatting

Co-Authored-By: Erik Johnston <erik@matrix.org>

* Outdated comment

* Import opentracing at the top

* Return a contextmanager

* Start tracing client requests from the servlet

* Return noop context manager if not tracing

* Explicitely say that these are federation requests

* Include servlet name in client requests

* Use context manager

* Move opentracing to logging/

* Seen it, isort it again!

* Ignore twisted return exceptions on context exit

* Escape the scope

* Scopes should be entered to make them useful.

* Nicer decorator names

* Just one init, init?

* Don't need to close something that isn't open

* Docs make you smarter
2019-07-11 10:36:03 +01:00
Amber Brown
463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ (#5606) 2019-07-04 00:07:04 +10:00
Richard van der Hoff
b4fd86a9b4 Merge branch 'develop' into rav/saml2_client 2019-07-01 14:21:03 +01:00
Amber Brown
be3b901ccd
Update the TLS cipher string and provide configurability for TLS on outgoing federation (#5550) 2019-06-28 18:19:09 +10:00
Daniel Hoffend
9646a593ac Added possibilty to disable local password authentication (#5092)
Signed-off-by: Daniel Hoffend <dh@dotlan.net>
2019-06-27 18:37:29 +01:00
Andrew Morgan
c548dbc4b1
Make it clearer that the template dir is relative to synapse's root dir (#5543)
Helps address #5444
2019-06-27 18:20:17 +01:00
Richard van der Hoff
47fa836abb
Merge pull request #5313 from twrist/patch-1
Update HAProxy example rules
2019-06-27 00:53:48 +01:00
Richard van der Hoff
dde4118341 update sample config 2019-06-27 00:41:04 +01:00
Richard van der Hoff
a0acfcc73e update sample config 2019-06-26 23:56:28 +01:00
Andrew Morgan
3eb8c7b0eb Merge branch 'master' into develop
* master:
  Fix broken link in MSC1711 FAQ
  Update changelog to better expain password reset change (#5545)
2019-06-25 18:08:56 +01:00
Richard van der Hoff
c8cb186260
Fix broken link in MSC1711 FAQ 2019-06-25 12:27:56 +01:00
Andrew Morgan
28604ab03d
Add info about black to code_style.rst (#5537)
Fixes #5533

Adds information about how to install and run black on the codebase.
2019-06-24 17:48:05 +01:00
Richard van der Hoff
4ac7ef4b67
Merge pull request #5524 from matrix-org/rav/new_cmdline_options
Add --data-dir and --open-private-ports options.
2019-06-24 17:25:57 +01:00
Brendan Abolivier
deb4fe6ef3
Merge pull request #5534 from matrix-org/babolivier/federation-publicrooms
Split public rooms directory auth config in two
2019-06-24 16:08:02 +01:00
Brendan Abolivier
bfe84e051e Split public rooms directory auth config in two 2019-06-24 15:42:31 +01:00
Richard van der Hoff
3f8a252dd8 Add "--open-private-ports" cmdline option
This is helpful when generating a config file for running synapse under docker.
2019-06-24 14:15:34 +01:00
Richard van der Hoff
edea4bb5be Allow configuration of the path used for ACME account keys.
Because sticking it in the same place as the config isn't necessarily the right
thing to do.
2019-06-24 13:51:22 +01:00
Richard van der Hoff
6cda36777b Drop support for cpu_affinity (#5525)
This has no useful purpose on python3, and is generally a source of confusion.
2019-06-22 11:01:55 +10:00
Erik Johnston
60b912cf0d
Update docs/workers.rst
E_TOO_MANY_NEGATIVES

Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-06-21 11:54:03 +01:00
Erik Johnston
f3ab533374 Support pagination API in client_reader worker 2019-06-21 10:43:12 +01:00
Amber Brown
32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Amber Brown
eba7caf09f
Remove Postgres 9.4 support (#5448) 2019-06-18 00:59:00 +10:00
Brendan Abolivier
f12e1f029c
Merge pull request #5440 from matrix-org/babolivier/third_party_event_rules
Allow server admins to define implementations of extra rules for allowing or denying incoming events
2019-06-14 19:37:59 +01:00
Brendan Abolivier
f874b16b2e Add plugin APIs for implementations of custom event rules. 2019-06-14 18:16:03 +01:00
Neil Johnson
426218323b
Neilj/improve federation docs (#5419)
Add FAQ questions to federate.md. Add a health warning making it clear that the 1711 upgrade FAQ is now out of date.
2019-06-11 12:17:43 +01:00
Neil Johnson
a11865016e
Set default room version to v4. (#5379)
Set default room version to v4.
2019-06-06 20:13:47 +01:00
Andrew Morgan
3719680ee4
Add ability to perform password reset via email without trusting the identity server (#5377)
Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option.

This PR is a culmination of 3 smaller PRs which have each been separately reviewed:

* #5308
* #5345
* #5368
2019-06-06 17:34:07 +01:00
Richard van der Hoff
9fbb20a531
Stop hardcoding trust of old matrix.org key (#5374)
There are a few changes going on here:

* We make checking the signature on a key server response optional: if no
  verify_keys are specified, we trust to TLS to validate the connection.

* We change the default config so that it does not require responses to be
  signed by the old key.

* We replace the old 'perspectives' config with 'trusted_key_servers', which
  is also formatted slightly differently.

* We emit a warning to the logs every time we trust a key server response
  signed by the old key.
2019-06-06 17:33:11 +01:00
Neil Johnson
833c406b9b
Neilj/1.0 upgrade notes (#5371)
1.0 upgrade/install notes
2019-06-06 17:23:02 +01:00
Richard van der Hoff
7603a706eb Merge branch 'rav/fix_custom_ca' into rav/enable_tls_verification 2019-06-05 16:32:35 +01:00
Richard van der Hoff
b4f1cd31f4 Update sample config 2019-06-05 15:30:10 +01:00
Richard van der Hoff
95ab2eb4a1
Fix notes about well-known and acme (#5357)
fixes #4951
2019-06-05 15:12:33 +01:00
Neil Johnson
26713515de
Neilj/mau tracking config explainer (#5284)
Improve documentation of monthly active user blocking and mau_trial_days
2019-06-05 13:16:23 +01:00
Ike Johnson
145f57897d
Update HAProxy example rules
These new rules allow a user to instead route only matrix traffic, allowing them to run matrix on the domain without affecting their existing websites
2019-06-02 23:10:27 +08:00
Erik Johnston
58cce39f3a
Merge pull request #5276 from matrix-org/babolivier/account_validity_job_delta
Allow configuring a range for the account validity startup job
2019-05-31 12:11:56 +01:00
Brendan Abolivier
e975b15101 Sample config 2019-05-31 11:14:21 +01:00
Brendan Abolivier
6bfc5ad3a1 Sample config 2019-05-31 09:56:57 +01:00
Travis Ralston
3e1af5109c Clarify that the admin change password endpoint logs them out (#5303) 2019-05-31 09:45:46 +01:00
Erik Johnston
8541db741a
Merge pull request #5283 from aaronraimist/captcha-docs
Specify the type of reCAPTCHA key to use (#5013)
2019-05-29 19:02:27 +01:00