Commit graph

11193 commits

Author SHA1 Message Date
Matthew Hodgson
12ec58301f shift to using an explicit deleted flag on m.device_list_update EDUs
and generally make it work.
2018-07-12 11:39:43 +01:00
Matthew Hodgson
5797f5542b WIP to announce deleted devices over federation
Previously we queued up the poke correctly when the device was deleted,
but then the actual EDU wouldn't get sent, as the device was no longer known.
Instead, we now send EDUs for deleted devices too if there's a poke for them.
2018-07-12 01:32:39 +01:00
Amber Brown
129ffd7b88
Merge pull request #3498 from OlegGirko/fix_attrs_syntax
* Use more portable syntax using attrs package.

Newer syntax

    attr.ib(factory=dict)

is just a syntactic sugar for

    attr.ib(default=attr.Factory(dict))

It was introduced in newest version of attrs package (18.1.0)
and doesn't work with older versions.

We should either require minimum version of attrs to be 18.1.0,
or use older (slightly more verbose) syntax.
Requiring newest version is not a good solution because
Linux distributions may have older version of attrs (17.4.0 in Fedora 28),
and requiring to build (and package)
newer version just to use newer syntactic sugar in only one test
is just too much.
It's much better to fix that test to use older syntax.

Signed-off-by: Oleg Girko <ol@infoserver.lv>
2018-07-11 04:22:46 +10:00
Amber Brown
85354bb18e changelog entry 2018-07-11 03:27:03 +10:00
Matthew Hodgson
ea752bdd99 s/becuase/because/g 2018-07-10 17:58:18 +01:00
Richard van der Hoff
c3c29aa196
Attempt to include db threads in cpu usage stats (#3496)
Let's try to include time spent in the DB threads in the per-request/block cpu
usage metrics.
2018-07-10 16:12:36 +01:00
Richard van der Hoff
55370331da
Refactor logcontext resource usage tracking (#3501)
Factor out the resource usage tracking out to a separate object, which can be
passed around and copied independently of the logcontext itself.
2018-07-10 13:56:07 +01:00
Matthew Hodgson
16b10666e7 another typo 2018-07-10 12:28:42 +01:00
Matthew Hodgson
4ea391a6ae typo (i think) 2018-07-10 12:08:09 +01:00
Richard van der Hoff
b1fe697b3c
Merge pull request #3497 from matrix-org/rav/measure_fetch_event_loop
Add CPU metrics for _fetch_event_list
2018-07-10 10:00:24 +01:00
Oleg Girko
6c1ec5a1bd Use more portable syntax using attrs package.
Newer syntax

    attr.ib(factory=dict)

is just a syntactic sugar for

    attr.ib(default=attr.Factory(dict))

It was introduced in newest version of attrs package (18.1.0)
and doesn't work with older versions.

We should either require minimum version of attrs to be 18.1.0,
or use older (slightly more verbose) syntax.
Requiring newest version is not a good solution because
Linux distributions may have older version of attrs (17.4.0 in Fedora 28),
and requiring to build (and package)
newer version just to use newer syntactic sugar in only one test
is just too much.
It's much better to fix that test to use older syntax.

Signed-off-by: Oleg Girko <ol@infoserver.lv>
2018-07-10 00:38:49 +01:00
Richard van der Hoff
f3b3b9dd8f changelog 2018-07-09 18:16:52 +01:00
Richard van der Hoff
e31e5dee38 Add CPU metrics for _fetch_event_list
add a Measure block on _fetch_event_list, in the hope that we can better
measure CPU usage here.
2018-07-09 18:15:54 +01:00
Richard van der Hoff
395fa8d1fd
Merge pull request #3464 from matrix-org/hawkowl/isort-run
Run isort on Synapse
2018-07-09 10:24:43 +01:00
Amber Brown
09477bd884 changelog 2018-07-09 16:09:37 +10:00
Amber Brown
49af402019 run isort 2018-07-09 16:09:20 +10:00
Amber Brown
2ee9f1bd1a
Add an isort configuration (#3463) 2018-07-09 16:05:21 +10:00
Amber Brown
1241156c82 changelog 2018-07-07 10:48:30 +10:00
Amber Brown
3060bcc8e9 version 2018-07-07 10:48:06 +10:00
Amber Brown
e845fd41c2
Correct attrs package name in requirements (#3492) 2018-07-07 10:46:59 +10:00
Richard van der Hoff
20ff89d6e1 Synapse 0.32.1 (2018-07-06)
===========================
 
 Bugfixes
 --------
 
 - Add explicit dependency on netaddr ([#3488](https://github.com/matrix-org/synapse/issues/3488))
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQEcBAABAgAGBQJbP5AkAAoJEIofk9V1tejV7a0H/A2Xn3IMCA/tUMi8J2WFXnR/
 gpyprut24fAvU+NruDDKwYs81vr5pvYHpfc6TTEhKnrna5V26S+hXb15ZwVUm5VK
 drjCt1iexe2sZ0foeK1kmUmMMKbg8QXh+qUO2vXFRyRiOs9ly5JPehuNr7AgJron
 htjlGakD5Kmc5lnQHMK6nnR6nEzs6gQnIUHkshcPNxKRZ8lwUm/0PT2zGsspvjjJ
 oI/pKp7PCwTBZAauX+eT2h+AGLgvZxurht8N/9c4ls8JyoL50/g1c3eb58ieOOcm
 I8cbkLGFPe1YXKRm+lyIMJGvbk3c8uDOl7IeXi+qyekluOqdjy0c2ftLzlY98rc=
 =mORK
 -----END PGP SIGNATURE-----

Merge tag 'v0.32.1'

Synapse 0.32.1 (2018-07-06)
===========================

Bugfixes
--------

- Add explicit dependency on netaddr ([#3488](https://github.com/matrix-org/synapse/issues/3488))
2018-07-06 16:56:23 +01:00
Richard van der Hoff
1cfc2c4790 Prepare 0.32.1 release 2018-07-06 16:50:52 +01:00
Richard van der Hoff
2087d5d046
Merge pull request #3488 from matrix-org/rav/fix-netaddr-dep
Add explicit dependency on netaddr
2018-07-06 16:39:56 +01:00
Richard van der Hoff
1464a0578a Add explicit dependency on netaddr
the dependencies file, causing failures on upgrade (and presumably for new
installs).
2018-07-06 16:27:17 +01:00
Neil Johnson
277c561766 0.32.0 version bump, update changelog 2018-07-06 15:07:29 +01:00
Amber Brown
89690aaaeb changelog 2018-07-05 20:46:40 +10:00
Amber Brown
be8b32dbc2 ACL changelog 2018-07-05 20:45:12 +10:00
Amber Brown
d196fe42a9 bump version to 0.32.0rc1 2018-07-05 20:22:35 +10:00
Neil Johnson
feef8461d1 Merge remote-tracking branch 'hera/rav/server_acls' into develop 2018-07-05 11:04:01 +01:00
Erik Johnston
1a88640677
Merge pull request #3483 from matrix-org/rav/more_server_name_validation
More server_name validation
2018-07-05 10:04:20 +01:00
Richard van der Hoff
3cf3e08a97 Implementation of server_acls
... as described at
https://docs.google.com/document/d/1EttUVzjc2DWe2ciw4XPtNpUpIl9lWXGEsy2ewDS7rtw.
2018-07-04 19:06:20 +01:00
Richard van der Hoff
546bc9e28b More server_name validation
We need to do a bit more validation when we get a server name, but don't want
to be re-doing it all over the shop, so factor out a separate
parse_and_validate_server_name, and do the extra validation.

Also, use it to verify the server name in the config file.
2018-07-04 18:59:51 +01:00
Richard van der Hoff
f192a93875
Merge pull request #3481 from matrix-org/rav/fix_cachedescriptor_test
Reinstate lost run_on_reactor in unit test
2018-07-04 18:55:33 +01:00
Erik Johnston
13f7adf84b
Merge pull request #3473 from matrix-org/erikj/thread_cache
Invalidate cache on correct thread
2018-07-04 10:11:38 +01:00
Erik Johnston
40252d13d1
Merge pull request #3474 from matrix-org/erikj/py3_auth
Fix up auth check
2018-07-04 09:41:33 +01:00
Richard van der Hoff
ea555d5633 Reinstate lost run_on_reactor in unit test
a61738b removed a call to run_on_reactor from a unit test, but that call was
doing something useful, in making the function in question asynchronous.

Reinstate the call and add a check that we are testing what we wanted to be
testing.
2018-07-04 09:40:01 +01:00
Richard van der Hoff
508196e08a
Reject invalid server names (#3480)
Make sure that server_names used in auth headers are sane, and reject them with
a sensible error code, before they disappear off into the depths of the system.
2018-07-03 14:36:14 +01:00
Richard van der Hoff
f741630847
Merge pull request #3470 from matrix-org/matthew/fix-utf8-logging
don't mix unicode strings with utf8-in-byte-strings
2018-07-02 15:25:36 +01:00
Matthew Hodgson
6ec3aa2f72 news snippet 2018-07-02 13:43:34 +01:00
Erik Johnston
abb183438c Correct newsfile 2018-07-02 13:12:38 +01:00
Erik Johnston
f88dea577d Newsfile 2018-07-02 11:46:32 +01:00
Erik Johnston
cea4662b13 Newsfile 2018-07-02 11:46:20 +01:00
Erik Johnston
2c33b55738 Avoid relying on int vs None comparison
Python 3 doesn't support comparing None to ints
2018-07-02 11:40:32 +01:00
Erik Johnston
cbf82dddf1 Ensure that we define sender_domain 2018-07-02 11:37:57 +01:00
Erik Johnston
3905c693c5 Invalidate cache on correct thread 2018-07-02 11:36:44 +01:00
Matthew Hodgson
fc4f8f33be replace invalid utf8 with \ufffd 2018-07-02 11:33:02 +01:00
Matthew Hodgson
1c867f5391 a fix which doesn't NPE everywhere 2018-07-01 11:56:33 +01:00
Matthew Hodgson
f131bf8d3e don't mix unicode strings with utf8-in-byte-strings
otherwise we explode with:

```
Traceback (most recent call last):
  File /usr/lib/python2.7/logging/handlers.py, line 78, in emit
    logging.FileHandler.emit(self, record)
  File /usr/lib/python2.7/logging/__init__.py, line 950, in emit
    StreamHandler.emit(self, record)
  File /usr/lib/python2.7/logging/__init__.py, line 887, in emit
    self.handleError(record)
  File /usr/lib/python2.7/logging/__init__.py, line 810, in handleError
    None, sys.stderr)
  File /usr/lib/python2.7/traceback.py, line 124, in print_exception
    _print(file, 'Traceback (most recent call last):')
  File /usr/lib/python2.7/traceback.py, line 13, in _print
    file.write(str+terminator)
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/logger/_io.py, line 170, in write
    self.log.emit(self.level, format=u{log_io}, log_io=line)
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/logger/_logger.py, line 144, in emit
    self.observer(event)
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/logger/_observer.py, line 136, in __call__
    errorLogger = self._errorLoggerForObserver(brokenObserver)
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/logger/_observer.py, line 156, in _errorLoggerForObserver
    if obs is not observer
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/logger/_observer.py, line 81, in __init__
    self.log = Logger(observer=self)
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/logger/_logger.py, line 64, in __init__
    namespace = self._namespaceFromCallingContext()
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/logger/_logger.py, line 42, in _namespaceFromCallingContext
    return currentframe(2).f_globals[__name__]
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/python/compat.py, line 93, in currentframe
    for x in range(n + 1):
RuntimeError: maximum recursion depth exceeded while calling a Python object
Logged from file site.py, line 129
  File /usr/lib/python2.7/logging/__init__.py, line 859, in emit
    msg = self.format(record)
  File /usr/lib/python2.7/logging/__init__.py, line 732, in format
    return fmt.format(record)
  File /usr/lib/python2.7/logging/__init__.py, line 471, in format
    record.message = record.getMessage()
  File /usr/lib/python2.7/logging/__init__.py, line 335, in getMessage
    msg = msg % self.args
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 4: ordinal not in range(128)
Logged from file site.py, line 129
```

...where the logger apparently recurses whilst trying to log the error, hitting the
maximum recursion depth and killing everything badly.
2018-07-01 05:08:58 +01:00
Matthew Hodgson
75d4986a8c
Merge pull request #3467 from matrix-org/hawkowl/contributor-requirements
Clarify "real name" in contributor requirements
2018-06-30 00:21:10 +01:00
Amber Brown
7c0cdd330f topfile 2018-06-29 14:13:15 +01:00