Commit graph

124 commits

Author SHA1 Message Date
Patrick Cloke
7d90d6ce9b
Run complement with Synapse workers manually. (#10039)
Adds an option to complement.sh to run Synapse in worker
mode (instead of the default monolith mode).
2021-05-24 15:32:45 -04:00
Jerin J Titus
057ce7b754
Remove tls_fingerprints option (#9280)
Signed-off-by: Jerin J Titus <72017981+jerinjtitus@users.noreply.github.com>
2021-05-24 17:43:30 +01:00
Richard van der Hoff
6482075c95
Run black on the scripts (#9981)
Turns out these scripts weren't getting linted.
2021-05-14 11:46:35 +01:00
Richard van der Hoff
7562d887e1
Change the format of access tokens away from macaroons (#5588) 2021-05-12 15:04:51 +01:00
Richard van der Hoff
765473567c
Fix make_full_schema to create the db with the right options and user (#9931) 2021-05-07 14:01:57 +01:00
Dan Callahan
56c4b47df3
Build Debian packages for Ubuntu 21.04 Hirsute (#9909)
Signed-off-by: Dan Callahan <danc@element.io>
2021-04-30 15:36:05 +01:00
Jonathan de Jong
495b214f4f
Fix (final) Bugbear violations (#9838) 2021-04-20 11:50:49 +01:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Erik Johnston
c1dbe84c3d
Add release helper script (#9713)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-04-13 11:51:10 +01:00
Dan Callahan
3efde8b69a
Add option to skip unit tests when building debs (#9793)
Signed-off-by: Dan Callahan <danc@element.io>
2021-04-12 15:27:05 +01:00
Patrick Cloke
abc814dcbf
Enable complement tests for MSC2946. (#9771)
By providing the additional build tag for `msc2946`.
2021-04-09 08:11:51 -04:00
Richard van der Hoff
9e167d9c53 Merge remote-tracking branch 'origin/develop' into rav/drop_py35 2021-04-08 18:30:38 +01:00
Richard van der Hoff
9278eb701e drop support for stretch and xenial 2021-04-08 16:42:32 +01:00
Patrick Cloke
452991527a
MSC3083: Check for space membership during a local join of restricted rooms. (#9735)
When joining a room with join rules set to 'restricted', check if the
user is a member of the spaces defined in the 'allow' key of the join rules.

This only applies to an experimental room version, as defined in MSC3083.
2021-04-08 08:28:32 -04:00
Patrick Cloke
e32294f54b Merge branch 'release-v1.31.0' into develop 2021-03-31 14:19:14 -04:00
Patrick Cloke
5fe38e07e7
Revert "Use 'dmypy run' in lint.sh instead of 'mypy' (#9701)" (#9720) 2021-03-31 14:17:52 -04:00
Andrew Morgan
ac99774dac
Rewrite complement.sh (#9685)
This PR rewrites the original complement.sh script with a number of improvements:

* We can now use a local checkout of Complement (configurable with `COMPLEMENT_DIR`), though the default behaviour still downloads the master branch.
* You can now specify a regex of test names to run, or just run all tests.
* We now use the Synapse test blacklist tag (so all tests will pass).
2021-03-31 11:58:12 +01:00
Andrew Morgan
f380bb77d1
Use 'dmypy run' in lint.sh instead of 'mypy' (#9701)
For it's obvious performance benefits. `dmypy` support landed in #9692.
2021-03-30 10:30:43 +01:00
Quentin Gliech
d4c4798a25
Use interpreter from $PATH instead of absolute paths in various scripts using /usr/bin/env (#9689)
On NixOS, `bash` isn't under `/bin/bash` but rather in some directory in `$PATH`. Locally, I've been patching those scripts to make them work.

`/usr/bin/env` seems to be the only [portable way](https://unix.stackexchange.com/questions/29608/why-is-it-better-to-use-usr-bin-env-name-instead-of-path-to-name-as-my) to use binaries from the PATH as interpreters.

Signed-off-by: Quentin Gliech <quentingliech@gmail.com>
2021-03-25 16:53:54 +00:00
Richard van der Hoff
0e35584734
federation_client: handle inline signing_keys in hs.yaml (#9647) 2021-03-18 21:12:07 +00:00
Richard van der Hoff
201178db1a
federation_client: stop adding URL prefix (#9645) 2021-03-18 20:31:47 +00:00
Andrew Morgan
22db45bd4d
Prevent the config-lint script erroring out on any sample_config changes (#9562)
I noticed that I'd occasionally have `scripts-dev/lint.sh` fail when messing about with config options in my PR. The script calls `scripts-dev/config-lint.sh`, which attempts some validation on the sample config.

 It does this by using `sed` to edit the sample_config, and then seeing if the file changed using `git diff`.

The problem is: if you changed the sample_config as part of your commit, this script will error regardless.

This PR attempts to change the check so that existing, unstaged changes to the sample_config will not cause the script to report an invalid file.
2021-03-09 11:11:42 +00:00
Eric Eastwood
0a00b7ff14
Update black, and run auto formatting over the codebase (#9381)
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Eric Eastwood
80d6dc9783
Remove conflicting sqlite tables that are "reserved" (shadow fts4 tables) (#9003)
Remove conflicting sqlite tables that throw sqlite3.OperationalError: object name reserved for internal use: event_search_content when running the twisted unit tests.

Fix #8996
2021-02-10 20:12:57 +00:00
Andrew Morgan
31c5382d7a
Align the directories linted in CI with the defaults in scripts-dev/lint.sh (#9191)
The lists of source directories to lint between `tox.ini` and `lint.sh` became out of sync. This PR tightens them up and adds some comments reminding any future readers to keep the list in sync.
2021-01-21 18:26:52 +00:00
Patrick Cloke
5d34f40d49
Add type hints to the push module. (#8901) 2020-12-11 11:43:53 -05:00
Richard van der Hoff
97b35ee259
Add a script to sign arbitrary json objects. (#8772) 2020-11-24 12:53:00 +00:00
Erik Johnston
22eeb6bc54
Fix cache call signature to accept on_invalidate. (#8684)
Cached functions accept an `on_invalidate` function, which we failed to add to the type signature. It's rarely used in the files that we have typed, which is why we haven't noticed it before.
2020-10-29 15:18:17 +00:00
Patrick Cloke
00b24aa545
Support generating structured logs in addition to standard logs. (#8607)
This modifies the configuration of structured logging to be usable from
the standard Python logging configuration.

This also separates the formatting of logs from the transport allowing
JSON logs to files or standard logs to sockets.
2020-10-29 07:27:37 -04:00
kleph
29ce6d43b5
Run mypy as part of the lint.sh script. (#8633) 2020-10-28 08:49:08 -04:00
Andrew Morgan
654e239b25
Add option to scripts-dev/lint.sh to only lint files changed since the last git commit (#8472)
This PR makes several changes to the `./scripts-dev/lint.sh` script, which lints the codebase with a number of tools:

* Adds usage information, with `-h` flag to show it. Otherwise it will show when providing an unknown flag.
* Adds option `-d` which will check both staged and unstaged files that have changed since the last commit and add them to the list of files to lint.
  - Note that only files without an extension, or with a `.py` extension will be allowed. This prevents editing bash scripts causing the linters to break on non-python files.
* Improves the print-out of which files/directories are being linted.
2020-10-15 15:45:13 +01:00
Erik Johnston
1baa895310 Merge remote-tracking branch 'origin/release-v1.21.0' into develop 2020-10-07 17:20:24 +01:00
Patrick Cloke
d9b55bd830
Add Ubuntu 20.10 (Groovy Gorilla) to build scripts. (#8475) 2020-10-07 08:48:54 -04:00
Andrew Morgan
01f82bfe32
Remove docs/sphinx and related references (#8480)
https://github.com/matrix-org/synapse/tree/develop/docs/sphinx doesn't seem to really be utilised or changed recently since the initial commit. I like the idea of exportable documentation of the codebase, but at the moment after running through the build instructions the generated website wasn't very useful...
2020-10-07 11:45:31 +01:00
Andrew Morgan
12f0d18611
Add support for running Complement against the local checkout (#8317)
This PR adds a script that:

* Builds the local Synapse checkout using our existing `docker/Dockerfile` image.
* Downloads [Complement](https://github.com/matrix-org/complement/)'s source code.
* Builds the [Synapse.Dockerfile](https://github.com/matrix-org/complement/blob/master/dockerfiles/Synapse.Dockerfile) using the above dockerfile as a base.
* Builds and runs Complement against it.

This set up differs slightly from [that of the dendrite repo](https://github.com/matrix-org/dendrite/blob/master/build/scripts/complement.sh) (`complement.sh`, `Complement.Dockerfile`), which instead stores a separate, but slightly modified, dockerfile in Dendrite's repo rather than running the one stored in Complement's repo. That synapse equivalent to that dockerfile (`Synapse.Dockerfile`) in Complement's repo is just based on top of `matrixdotorg/synapse:latest`, which we opt to build here locally.

Thus copying over the files from Complement's repo wouldn't change any functionality, and would result in two instances of the same files. So just using the dockerfile in Complement's repo was decided upon instead.
2020-09-29 13:47:47 +01:00
Patrick Cloke
8a4a4186de
Simplify super() calls to Python 3 syntax. (#8344)
This converts calls like super(Foo, self) -> super().

Generated with:

    sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
Jonathan de Jong
837293c314
Remove obsolete __future__ imports (#8337) 2020-09-17 08:37:01 -04:00
Patrick Cloke
c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Erik Johnston
208e1d3eb3
Fix typing for @cached wrapped functions (#8240)
This requires adding a mypy plugin to fiddle with the type signatures a bit.
2020-09-03 15:38:32 +01:00
Richard van der Hoff
88b9807ba4
Update the test federation client to handle streaming responses (#8130)
Now that the server supports streaming back JSON responses, it would be nice to
show the response as it is streamed, in the test tool.
2020-08-26 14:11:38 +01:00
Erik Johnston
a7bdf98d01
Rename database classes to make some sense (#8033) 2020-08-05 21:38:57 +01:00
Andrew Morgan
7cac9006d6
Spruce up the check-newsfragment CI output (#8024)
This PR:

* Reduces the amount of noise in the `check-newsfragment` CI output by hiding the dependency installation output by default.
* Prints a link to the changelog/debian changelog section of the contributing guide if an error is found.
2020-08-04 22:10:23 +01:00
Richard van der Hoff
8078dec3be
Fix exit code for check_line_terminators.sh (#7970)
If there are *no* files with CRLF line endings, then the xargs exits with a
non-zero exit code (as expected), but then, since that is the last thing to
happen in the script, the script as a whole exits non-zero, making the whole
thing fail.

using `if/then/fi` instead of `&& (...)` means that the script exits with a
zero exit code.
2020-07-28 08:52:25 -04:00
Andrew Morgan
c4ce0da6fe
Add script for finding files with unix line terminators (#7965)
This PRs adds a script to check for unix-line terminators in the repo. It will be used to address https://github.com/matrix-org/synapse/issues/7943 by adding the check to CI.

I've changed the original script slightly as proposed in https://github.com/matrix-org/pipelines/pull/81#discussion_r460580664
2020-07-28 01:26:50 +01:00
Andrew Morgan
b7ddece2a6
Lint the contrib/ directory in CI and linting scripts, add synctl to linting script (#7914)
Run `isort`, `flake8` and `black` over the `contrib/` directory and `synctl` script. The latter was already being done in CI, but now the linting script does it too.

Fixes https://github.com/matrix-org/synapse/issues/7910
2020-07-20 21:43:49 +01:00
Gary Kim
f2af3e4fc5
Remove Ubuntu Eoan that is now EOL (#7888) 2020-07-17 15:38:41 -04:00
Will Hunt
62b1ce8539
isort 5 compatibility (#7786)
The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
2020-07-05 16:32:02 +01:00
Andrew Morgan
177b2d0c19
Move flake8 to end. Don't exit script on failure (#7738) 2020-06-25 17:58:55 +01:00
Dagfinn Ilmari Mannsåker
a3f11567d9
Replace all remaining six usage with native Python 3 equivalents (#7704) 2020-06-16 08:51:47 -04:00
Richard van der Hoff
86d814cdde
Check the changelog number in check-newsfragment (#7623) 2020-06-03 17:01:43 +01:00