Erik Johnston
eefd9fee81
Fix up tests
2017-03-30 14:14:46 +01:00
Richard van der Hoff
f9b4bb05e0
Fix the logcontext handling in the cache wrappers ( #2077 )
...
The cache wrappers had a habit of leaking the logcontext into the reactor while
the lookup function was running, and then not restoring it correctly when the
lookup function had completed. It's all the fault of
`preserve_context_over_{fn,deferred}` which are basically a bit broken.
2017-03-30 13:22:24 +01:00
Erik Johnston
650f0e69f2
Compile the regex's used in ASes
2017-03-28 13:27:21 +01:00
Erik Johnston
57cfa513f5
Merge pull request #2054 from matrix-org/erikj/user_iter_cursor
...
Reduce some CPU work on DB threads
2017-03-24 11:40:08 +00:00
Richard van der Hoff
9397edb28b
Merge pull request #2050 from matrix-org/rav/federation_backoff
...
push federation retry limiter down to matrixfederationclient
2017-03-23 22:27:01 +00:00
Richard van der Hoff
06ce7335e9
Merge pull request #2052 from matrix-org/rav/time_bound_deferred
...
Fix time_bound_deferred to throw the right exception
2017-03-23 22:22:54 +00:00
Erik Johnston
00957d1aa4
User Cursor.__iter__ instead of fetchall
...
This prevents unnecessary construction of lists
2017-03-23 17:53:49 +00:00
pik
566641a0b5
use jsonschema.FormatChecker for RoomID and UserID strings
...
* use a valid filter in rest/client/v2_alpha test
Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23 11:42:41 -03:00
pik
acafcf1c5b
Add valid filter tests, flake8, fix typo
...
Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23 11:42:10 -03:00
pik
e56c79c114
check_valid_filter using JSONSchema
...
* add invalid filter tests
Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23 11:42:07 -03:00
Richard van der Hoff
0bfea9a2be
fix tests
2017-03-23 13:20:08 +00:00
Richard van der Hoff
b88a323ffb
Fix time_bound_deferred to throw the right exception
...
Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred
would throw a CancelledError when the deferred timed out, which was rather
confusing.
2017-03-23 12:07:11 +00:00
Richard van der Hoff
4bd597d9fc
push federation retry limiter down to matrixfederationclient
...
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
2017-03-23 09:28:46 +00:00
Richard van der Hoff
95f21c7a66
Fix caching of remote servers' signature keys
...
The `@cached` decorator on `KeyStore._get_server_verify_key` was missing
its `num_args` parameter, which meant that it was returning the wrong key for
any server which had more than one recorded key.
By way of a fix, change the default for `num_args` to be *all* arguments. To
implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.
2017-03-22 15:11:30 +00:00
Erik Johnston
37a187bfab
Merge pull request #2033 from matrix-org/erikj/repl_speed
...
Don't send the full event json over replication
2017-03-21 13:11:15 +00:00
Erik Johnston
737f283a07
Fix unit test
2017-03-20 14:03:58 +00:00
Richard van der Hoff
f40c2db05a
Stop preserve_fn leaking context into the reactor
...
Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the
reactor, and add a test for it.
Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do
the same thing but had its own, different, set of bugs.
2017-03-18 00:07:43 +00:00
Will Hunt
c144292373
Modify test_user_sync so it doesn't look at last_active_ts over last_user_sync_ts
2017-03-15 15:38:57 +00:00
Erik Johnston
095b45c165
Aggregate event push actions
2017-02-14 13:39:41 +00:00
Erik Johnston
51adaac953
Fix email push in pusher worker
...
This was broken when device list updates were implemented, as Mailer
could no longer instantiate an AuthHandler due to a dependency on
federation sending.
2017-02-02 10:53:36 +00:00
Erik Johnston
692daf6f54
Remote membership tests for replication
...
This is because it now relies of the caches stream, which only works on
postgres. We are trying to test with sqlite.
2017-01-31 16:10:16 +00:00
Erik Johnston
c7a26b7c32
Fix unit tests
2017-01-30 17:11:24 +00:00
Erik Johnston
b3e1f2aa7a
Fix unit tests
2017-01-26 17:16:24 +00:00
Erik Johnston
c974116f19
Implement device key caching over federation
2017-01-26 16:07:24 +00:00
Erik Johnston
2367c5568c
Add basic implementation of local device list changes
2017-01-25 14:27:27 +00:00
Erik Johnston
a55fa2047f
Insert delta of current_state_events to be more efficient
2017-01-20 17:10:18 +00:00
Erik Johnston
09eb08f910
Derive current_state_events from state groups
2017-01-20 11:52:51 +00:00
Erik Johnston
15f012032c
Merge pull request #1818 from matrix-org/erikj/state_auth_splitout_split
...
Optimise state resolution
2017-01-18 10:53:00 +00:00
Erik Johnston
5d6bad1b3c
Optimise state resolution
2017-01-17 13:22:19 +00:00
Erik Johnston
9e8e236d98
Tidy up test
2017-01-17 11:50:18 +00:00
Erik Johnston
f85b6ca494
Speed up cache size calculation
...
Instead of calculating the size of the cache repeatedly, which can take
a long time now that it can use a callback, instead cache the size and
update that on insertion and deletion.
This requires changing the cache descriptors to have two caches, one for
pending deferreds and the other for the actual values. There's no reason
to evict from the pending deferreds as they won't take up any more
memory.
2017-01-17 11:18:13 +00:00
Erik Johnston
f2f179dce2
Add ExpiringCache tests
2017-01-16 15:33:34 +00:00
Erik Johnston
2fae34bd2c
Optionally measure size of cache by sum of length of values
2017-01-13 17:46:17 +00:00
Erik Johnston
6823fe5241
Linearize updates to membership via PUT /state/
2017-01-09 18:25:13 +00:00
Marcin Bachry
24c16fc349
Fix crash in url preview when html tag has no text
...
Signed-off-by: Marcin Bachry <hegel666@gmail.com>
2016-12-14 22:38:18 +01:00
Erik Johnston
d53a80af25
Merge pull request #1620 from matrix-org/erikj/concurrent_room_access
...
Limit the number of events that can be created on a given room concurrently
2016-12-12 10:30:23 +00:00
Erik Johnston
8b34f71bea
Fix unit tests
2016-12-09 16:48:48 +00:00
Richard van der Hoff
1529c19675
Prevent user tokens being used as guest tokens ( #1675 )
...
Make sure that a user cannot pretend to be a guest by adding 'guest = True'
caveats.
2016-12-06 15:31:37 +00:00
Johannes Löthberg
6c9a0ba415
test_preview: Fix incorrect wrapping
...
The old test expected an incorrect wrapping due to the preview function
not using unicode properly, so it got the wrong length.
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-05 16:33:57 +01:00
Johannes Löthberg
0697bb2247
Make test_preview use unicode strings
...
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-05 16:33:57 +01:00
Richard van der Hoff
12f3b9000c
fix imports
2016-11-30 17:45:49 +00:00
Richard van der Hoff
aa09d6b8f0
Rip out more refresh_token code
...
We might as well treat all refresh_tokens as invalid. Just return a 403 from
/tokenrefresh, so that we don't have a load of dead, untestable code hanging
around.
Still TODO: removing the table from the schema.
2016-11-30 17:40:18 +00:00
Richard van der Hoff
dc4b23e1a1
Merge branch 'develop' into rav/no_more_refresh_tokens
2016-11-30 17:10:04 +00:00
Richard van der Hoff
1c4f05db41
Stop putting a time caveat on access tokens
...
The 'time' caveat on the access tokens was something of a lie, since we weren't
enforcing it; more pertinently its presence stops us ever adding useful time
caveats.
Let's move in the right direction by not lying in our caveats.
2016-11-29 16:49:41 +00:00
Richard van der Hoff
5c4edc83b5
Stop generating refresh tokens
...
Since we're not doing refresh tokens any more, we should start killing off the
dead code paths. /tokenrefresh itself is a bit of a thornier subject, since
there might be apps out there using it, but we can at least not generate
refresh tokens on new logins.
2016-11-28 10:13:01 +00:00
Erik Johnston
feec718265
Shuffle receipt handler around so that worker apps don't need to load it
2016-11-23 15:14:24 +00:00
Erik Johnston
ee5e8d71ac
Fix tests
2016-11-23 14:57:07 +00:00
Erik Johnston
b69f76c106
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_out_fed_txn
2016-11-23 11:31:53 +00:00
Erik Johnston
54fed21c04
Fix tests and flake8
2016-11-22 18:18:31 +00:00
Kegan Dougal
c3d963ac24
Review comments
2016-11-22 13:42:11 +00:00
Kegan Dougal
0a8b0eeca1
More tests
2016-11-22 09:59:27 +00:00
Kegan Dougal
53b27bbf06
Add remaining tests
2016-11-21 17:58:22 +00:00
Kegan Dougal
70a2157b64
Start adding some tests
2016-11-21 17:52:45 +00:00
Kegan Dougal
f97511a1f3
Move event_fields filtering to serialize_event
...
Also make it an inclusive not exclusive filter, as the spec demands.
2016-11-21 17:42:16 +00:00
Erik Johnston
524d61bf7e
Fix tests
2016-11-21 11:53:02 +00:00
Kegan Dougal
3991b4cbdb
Clean transactions based on time. Add HttpTransactionCache tests.
2016-11-14 11:19:24 +00:00
Erik Johnston
d073cb7ead
Add Limiter: limit concurrent access to resource
2016-11-10 16:29:51 +00:00
Mark Haines
177f104432
Merge pull request #1098 from matrix-org/markjh/bearer_token
...
Allow clients to supply access_tokens as headers
2016-10-25 17:33:15 +01:00
Erik Johnston
78c083f159
Merge pull request #1164 from pik/error-codes
...
Clarify Error codes for GET /filter/
2016-10-19 14:26:17 +01:00
pik
e8b1d2a452
Refactor test_filter to use real DataStore
...
* add tests for filter api errors
2016-10-18 12:17:38 -05:00
pik
d43b63818c
Fix MockHttpRequest always returning M_UNKNOWN errcode in testing
2016-10-14 15:46:54 -05:00
Alexander Maznev
d9350b0db8
Error codes for filters
...
* add tests
Signed-off-by: Alexander Maznev <alexander.maznev@gmail.com>
2016-10-14 10:18:28 -05:00
Erik Johnston
35e2cc8b52
Merge pull request #1155 from matrix-org/erikj/pluggable_pwd_auth
...
Implement pluggable password auth
2016-10-12 11:41:20 +01:00
Erik Johnston
3061dac53e
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/replication_noop
2016-10-11 14:08:29 +01:00
Patrik Oldsberg
7b5546d077
rest/client/v1/register: use the correct requester in createUser
...
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-10-06 22:12:32 +02:00
Patrik Oldsberg
9bfc617791
storage/appservice: make appservice methods only relying on the cache synchronous
2016-10-06 15:24:59 +02:00
Erik Johnston
850b103b36
Implement pluggable password auth
...
Allows delegating the password auth to an external module. This also
moves the LDAP auth to using this system, allowing it to be removed from
the synapse tree entirely in the future.
2016-10-03 10:36:40 +01:00
Erik Johnston
748d8fdc7b
Reduce DB hits for replication
...
Some streams will occaisonally advance their positions without actually
having any new rows to send over federation. Currently this means that
the token will not advance on the workers, leading to them repeatedly
sending a slightly out of date token. This in turns requires the master
to hit the DB to check if there are any new rows, rather than hitting
the no op logic where we check if the given token matches the current
token.
This commit changes the API to always return an entry if the position
for a stream has changed, allowing workers to advance their tokens
correctly.
2016-09-23 16:49:21 +01:00
Erik Johnston
22578545a0
Time out typing over federation
2016-09-23 14:00:52 +01:00
Mark Haines
ec609f8094
Fix unit tests
2016-09-12 10:46:02 +01:00
Mark Haines
74cbfdc7de
Fix unit tests
2016-09-06 18:30:03 +01:00
Erik Johnston
c10cb581c6
Correctly handle the difference between prev and current state
2016-08-31 14:26:22 +01:00
Erik Johnston
bc1a8b1f7a
Don't notify for online -> online transitions.
...
Specifically, if currently_active remains true then we should not notify
if only the last active time changes.
2016-08-30 15:05:32 +01:00
Erik Johnston
55fc17cf4b
Merge pull request #1049 from matrix-org/erikj/presence_users_in_room
...
Use state handler instead of get_users_in_room/get_joined_hosts
2016-08-30 10:50:37 +01:00
Erik Johnston
93b32d4515
Fix unit tests
2016-08-26 15:40:27 +01:00
Mark Haines
4bbef62124
Merge remote-tracking branch 'origin/develop' into markjh/direct_to_device
2016-08-26 14:35:31 +01:00
Erik Johnston
3f11953fcb
Fix tests
2016-08-26 10:15:52 +01:00
Mark Haines
3b8d0ceb22
More 0_0 in tests
2016-08-25 18:42:46 +01:00
Erik Johnston
a3dc1e9cbe
Replace context.current_state with context.current_state_ids
2016-08-25 17:32:22 +01:00
Erik Johnston
45fd2c8942
Ensure invalidation list does not grow unboundedly
2016-08-19 16:09:16 +01:00
Erik Johnston
c0d7d9d642
Rename to on_invalidate
2016-08-19 15:13:58 +01:00
Erik Johnston
dc76a3e909
Make cache_context an explicit option
2016-08-19 15:02:38 +01:00
Erik Johnston
ba214a5e32
Remove lru option
2016-08-19 14:17:11 +01:00
Erik Johnston
4161ff2fc4
Add concept of cache contexts
2016-08-19 14:17:07 +01:00
Erik Johnston
9da84a9a1e
Make AppserviceHandler stream events from database
...
This is for two reasons:
1. Suppresses duplicates correctly, as the notifier doesn't do any
duplicate suppression.
2. Makes it easier to connect the AppserviceHandler to the replication
stream.
2016-08-18 11:54:41 +01:00
Erik Johnston
320dfe523c
Make notify_interested_services faster
2016-08-17 17:20:50 +01:00
Erik Johnston
7321f45457
Clean up _ServiceQueuer
2016-08-17 12:03:04 +01:00
Erik Johnston
b835ebcc79
Update unit tests
2016-08-17 11:22:11 +01:00
Erik Johnston
48b5829aea
Fix up preview URL API. Add tests.
...
This includes:
- Splitting out methods of a class into stand alone functions, to make
them easier to test.
- Adding unit tests to split out functions, testing HTML -> preview.
- Handle the fact that elements in lxml may have tail text.
2016-08-16 14:53:24 +01:00
Richard van der Hoff
a8bcc7274d
PEP8
2016-08-08 17:20:38 +01:00
Richard van der Hoff
6fe6a6f029
Fix login with m.login.token
...
login with token (as used by CAS auth) was broken by 067596d
, such that it
always returned a 401.
2016-08-08 16:40:39 +01:00
Erik Johnston
e97648c4e2
Test summarization
2016-08-04 16:09:09 +01:00
Richard van der Hoff
98385888b8
PEP8
2016-08-03 15:42:08 +01:00
Richard van der Hoff
68264d7404
Include device name in /keys/query response
...
Add an 'unsigned' section which includes the device display name.
2016-08-03 15:42:08 +01:00
Richard van der Hoff
91fa69e029
keys/query: return all users which were asked for
...
In the situation where all of a user's devices get deleted, we want to
indicate this to a client, so we want to return an empty dictionary, rather
than nothing at all.
2016-08-03 15:41:44 +01:00
Mark Haines
0a7d3cd00f
Create separate methods for getting messages to push
...
for the email and http pushers rather than trying to make a single
method that will work with their conflicting requirements.
The http pusher needs to get the messages in ascending stream order, and
doesn't want to miss a message.
The email pusher needs to get the messages in descending timestamp order,
and doesn't mind if it misses messages.
2016-07-28 20:24:24 +01:00
Richard van der Hoff
eb359eced4
Add create_requester
function
...
Wrap the `Requester` constructor with a function which provides sensible
defaults, and use it throughout
2016-07-26 16:46:53 +01:00
Richard van der Hoff
8e02494166
Delete refresh tokens when deleting devices
2016-07-26 11:10:37 +01:00
Richard van der Hoff
012b4c1913
Implement updating devices
...
You can update the displayname of devices now.
2016-07-26 07:35:48 +01:00
Richard van der Hoff
436bffd15f
Implement deleting devices
2016-07-26 07:35:48 +01:00
Richard van der Hoff
bf3de7b90b
Merge pull request #945 from matrix-org/rav/background_reindex
...
Create index on user_ips in the background
2016-07-25 14:04:05 +01:00
Richard van der Hoff
42f4feb2b7
PEP8
2016-07-25 12:25:06 +01:00
Richard van der Hoff
f16f0e169d
Slightly saner logging for unittests
...
1. Give the handler used for logging in unit tests a formatter, so that the
output is slightly more meaningful
2. Log some synapse.storage stuff, because it's useful.
2016-07-25 12:12:47 +01:00
Richard van der Hoff
465117d7ca
Fix background_update tests
...
A bit of a cleanup for background_updates, and make sure that the real
background updates have run before we start the unit tests, so that they don't
interfere with the tests.
2016-07-25 12:10:42 +01:00
Richard van der Hoff
55abbe1850
make /devices return a list
...
Turns out I specced this to return a list of devices rather than a dict of them
2016-07-21 15:57:28 +01:00
Richard van der Hoff
1c3c202b96
Fix PEP8 errors
2016-07-21 13:15:15 +01:00
Richard van der Hoff
406f7aa0f6
Implement GET /device/{deviceId}
2016-07-21 12:00:29 +01:00
Richard van der Hoff
40a1c96617
Fix PEP8 errors
2016-07-20 18:06:28 +01:00
Richard van der Hoff
7314bf4682
Merge branch 'develop' into rav/get_devices_api
...
(pick up PR #938 in the hope of fixing the UTs)
2016-07-20 17:40:00 +01:00
Richard van der Hoff
e9e3eaa67d
Merge pull request #938 from matrix-org/rav/add_device_id_to_client_ips
...
Record device_id in client_ips
2016-07-20 17:38:45 +01:00
Richard van der Hoff
bc8f265f0a
GET /devices endpoint
...
implement a GET /devices endpoint which lists all of the user's devices.
It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
Richard van der Hoff
ec041b335e
Record device_id in client_ips
...
Record the device_id when we add a client ip; it's somewhat redundant as we
could get it via the access_token, but it will make querying rather easier.
2016-07-20 16:41:03 +01:00
Richard van der Hoff
b97a1356b1
Register a device_id in the /v2/register flow.
...
This doesn't cover *all* of the registration flows, but it does cover the most
common ones: in particular: shared_secret registration, appservice
registration, and normal user/pass registration.
Pull device_id from the registration parameters. Register the device in the
devices table. Associate the device with the returned access and refresh
tokens. Profit.
2016-07-20 16:38:27 +01:00
David Baker
e967bc86e7
Merge pull request #932 from matrix-org/rav/register_refactor
...
Further registration refactoring
2016-07-20 11:03:33 +01:00
Erik Johnston
1e2a7f18a1
Merge pull request #922 from matrix-org/erikj/file_api2
...
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
Richard van der Hoff
40cbffb2d2
Further registration refactoring
...
* `RegistrationHandler.appservice_register` no longer issues an access token:
instead it is left for the caller to do it. (There are two of these, one in
`synapse/rest/client/v1/register.py`, which now simply calls
`AuthHandler.issue_access_token`, and the other in
`synapse/rest/client/v2_alpha/register.py`, which is covered below).
* In `synapse/rest/client/v2_alpha/register.py`, move the generation of
access_tokens into `_create_registration_details`. This means that the normal
flow no longer needs to call `AuthHandler.issue_access_token`; the
shared-secret flow can tell `RegistrationHandler.register` not to generate a
token; and the appservice flow continues to work despite the above change.
2016-07-19 18:46:19 +01:00
David Baker
b9e997f561
Merge pull request #931 from matrix-org/rav/refactor_register
...
rest/client/v2_alpha/register.py: Refactor flow somewhat.
2016-07-19 16:13:45 +01:00
Richard van der Hoff
0da0d0a29d
rest/client/v2_alpha/register.py: Refactor flow somewhat.
...
This is meant to be an *almost* non-functional change, with the exception that
it fixes what looks a lot like a bug in that it only calls
`auth_handler.add_threepid` and `add_pusher` once instead of three times.
The idea is to move the generation of the `access_token` out of
`registration_handler.register`, because `access_token`s now require a
device_id, and we only want to generate a device_id once registration has been
successful.
2016-07-19 13:12:22 +01:00
Richard van der Hoff
f863a52cea
Add device_id support to /login
...
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.
Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.
Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
Erik Johnston
a98d215204
Add filter param to /messages API
2016-07-14 16:30:56 +01:00
Negar Fazeli
0136a522b1
Bug fix: expire invalid access tokens
2016-07-13 15:00:37 +02:00
Erik Johnston
7335f0adda
Add ReadWriteLock
2016-07-05 15:23:17 +01:00
David Baker
2455ad8468
Remove room name & alias test
...
as get_room_name_and_alias is now gone
2016-06-24 13:34:20 +01:00
Martin Weinelt
0a32208e5d
Rework ldap integration with ldap3
...
Use the pure-python ldap3 library, which eliminates the need for a
system dependency.
Offer both a `search` and `simple_bind` mode, for more sophisticated
ldap scenarios.
- `search` tries to find a matching DN within the `user_base` while
employing the `user_filter`, then tries the bind when a single
matching DN was found.
- `simple_bind` tries the bind against a specific DN by combining the
localpart and `user_base`
Offer support for STARTTLS on a plain connection.
The configuration was changed to reflect these new possibilities.
Signed-off-by: Martin Weinelt <hexa@darmstadt.ccc.de>
2016-06-22 17:51:59 +02:00
Erik Johnston
0113ad36ee
Enable use_frozen_events in tests
2016-06-17 15:13:13 +01:00
Mark Haines
7dbb473339
Add function to load config without generating it
...
Renames ``load_config`` to ``load_or_generate_config``
Adds a method called ``load_config`` that just loads the
config.
The main synapse.app.homeserver will continue to use
``load_or_generate_config`` to retain backwards compat.
However new worker processes can use ``load_config`` to
load the config avoiding some of the cruft needed to generate
the config.
As the new ``load_config`` method is expected to be used by new
configs it removes support for the legacy commandline overrides
that ``load_or_generate_config`` supports
2016-06-09 18:50:38 +01:00
Negar Fazeli
6e7dc7c7dd
Fix a bug caused by a change in auth_handler function
...
Fix the relevant unit test cases
2016-06-08 23:22:39 +02:00
Erik Johnston
84379062f9
Fix AS retries, but with correct ordering
2016-06-07 10:24:50 +01:00
Erik Johnston
310197bab5
Fix AS retries
2016-06-07 09:34:50 +01:00
Erik Johnston
05e01f21d7
Remove event fetching from DB threads
2016-06-03 17:22:13 +01:00
Erik Johnston
73c7112433
Change CacheMetrics to be quicker
...
We change it so that each cache has an individual CacheMetric, instead
of having one global CacheMetric. This means that when a cache tries to
increment a counter it does not need to go through so many indirections.
2016-06-03 11:26:52 +01:00
Mark Haines
56d15a0530
Store the typing users as user_id strings. ( #819 )
...
Rather than storing them as UserID objects.
2016-06-02 16:28:54 +01:00
Mark Haines
70599ce925
Allow external processes to mark a user as syncing. ( #812 )
...
* Add infrastructure to the presence handler to track sync requests in external processes
* Expire stale entries for dead external processes
* Add an http endpoint for making users as syncing
Add some docstrings and comments.
* Fixes
2016-06-02 15:20:15 +01:00
David Baker
4a10510cd5
Split out the auth handler
2016-06-02 13:31:45 +01:00
David Baker
195254cae8
Inject fake room list handler in tests
...
Otherwise it tries to start the remote public room list updating looping call which breaks.
2016-06-01 11:14:16 +01:00
Mark Haines
c626fc576a
Move the AS handler out of the Handlers object.
...
Access it directly from the homeserver itself. It already wasn't
inheriting from BaseHandler storing it on the Handlers object was
already somewhat dubious.
2016-05-31 13:53:48 +01:00
Negi Fazeli
6fe04ffef2
Fix set profile error with Requester.
...
Replace flush_user with delete access token due to function removal
Add a new test case for if the user is already registered
2016-05-23 19:50:28 +02:00
Mark Haines
0cb441fedd
Move typing handler out of the Handlers object
2016-05-17 15:58:46 +01:00
Mark Haines
425e6b4983
Merge branch 'develop' into markjh/member_cleanup
2016-05-17 10:13:16 +01:00
Mark Haines
816df9f267
get_room_members is unused now
2016-05-16 19:51:43 +01:00
Mark Haines
821306120a
Replaces calls to fetch_room_distributions_into with get_joined_hosts_for_room
2016-05-16 19:48:07 +01:00
Mark Haines
3b86ecfa79
Move the presence handler out of the Handlers object
2016-05-16 18:56:37 +01:00
Mark Haines
21018c2c13
Merge pull request #783 from matrix-org/markjh/slave_account_data
...
Add a slaved datastore for account data
2016-05-13 15:56:04 +01:00
Mark Haines
3abab26458
Add a slaved datastore for account data
2016-05-13 15:34:06 +01:00
Negi Fazeli
40aa6e8349
Create user with expiry
...
- Add unittests for client, api and handler
Signed-off-by: Negar Fazeli <negar.fazeli@ericsson.com>
2016-05-13 15:34:15 +02:00
Mark Haines
c0d8e0eb63
Replicate push actions
2016-04-21 15:25:58 +01:00
Mark Haines
f52dd35ac3
Merge pull request #738 from matrix-org/markjh/slaved_receipts
...
Add a slaved receipts store
2016-04-19 17:31:59 +01:00
Mark Haines
5bbd424ee0
Add a slaved receipts store
2016-04-19 17:14:08 +01:00
Mark Haines
e99365f601
Replicate get_invited_rooms_for_user
2016-04-19 15:22:14 +01:00
Erik Johnston
eb8619e256
Create log context in Measure if one doesn't exist
2016-04-18 16:08:32 +01:00
Erik Johnston
79fc4ff6f9
Merge pull request #677 from matrix-org/erikj/dns_cache
...
Read from DNS cache if within TTL
2016-04-08 14:09:56 +01:00
Mark Haines
7e2f971c08
Remove some unused functions ( #711 )
...
* Remove some unused functions
* get_room_events_stream is only used in tests
* is_exclusive_room might actually be something we want
2016-04-08 14:01:56 +01:00
Mark Haines
ceb599e789
Add tests for redactions
2016-04-07 16:52:07 +01:00
Mark Haines
8c82b06904
Merge pull request #704 from matrix-org/markh/slaveIII
...
Add tests for get_latest_event_ids_in_room and get_current_state
2016-04-07 16:49:34 +01:00
Mark Haines
57fa1801c3
Add sensible __eq__ operators inside the tests.
...
Rather than adding them globally. This limits the changes to only
affect the tests.
2016-04-07 16:41:37 +01:00
Erik Johnston
af03ecf352
Deduplicate joins
2016-04-07 14:19:02 +01:00
Mark Haines
60ec9793fb
Add tests for get_latest_event_ids_in_room and get_current_state
2016-04-07 13:17:56 +01:00
Erik Johnston
a28d066732
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dns_cache
2016-04-07 11:11:17 +01:00
Mark Haines
6bfec56796
Test that room membership is replicated
2016-04-06 16:20:13 +01:00
Mark Haines
e815763b7f
Merge pull request #697 from matrix-org/markjh/slaveI
...
Add a slaved events store class
2016-04-06 16:19:25 +01:00
Mark Haines
75fb9ac1be
Add a slaved events store class
...
Add a test to check that get_room_names_and_aliases does the same
thing on both the master and on the slave data store.
2016-04-06 14:18:35 +01:00
Erik Johnston
8aab9d87fa
Don't require config to create database
2016-04-06 14:15:45 +01:00
Erik Johnston
11860637e1
Tests
2016-04-06 10:12:30 +01:00
Erik Johnston
c906f30661
Do checks for memberships before creating events
2016-04-01 16:17:32 +01:00
Erik Johnston
f699b8f997
Read from DNS cache if within TTL
2016-03-31 10:04:28 +01:00
Mark Haines
31a9eceda5
Add a replication stream for state groups
2016-03-30 16:01:58 +01:00
Mark Haines
673c96ce97
Remove dead code left over from presence changes
2016-03-17 11:03:47 +00:00
David Baker
ff7d3dc3a0
Fix tests
2016-03-16 14:25:14 +00:00
Mark Haines
b6e8420aee
Add replication stream for pushers
2016-03-15 17:33:10 +00:00
Mark Haines
e9c1cabac2
Use parse_json_object_from_request to parse JSON out of request bodies
2016-03-11 16:41:03 +00:00
blide
1be438f2a6
Flake8 fix
2016-03-10 12:13:35 +03:00
blide
40160e24ab
Register endpoint returns refresh_token
...
Guest registration still doesn't return refresh_token
2016-03-10 10:29:19 +03:00
David Baker
316c00936f
Fix tests
2016-03-07 17:32:36 +00:00
Mark Haines
3110c37d02
Fix unit tests
2016-03-04 14:48:17 +00:00
Mark Haines
ec7460b4f2
Merge branch 'develop' into markjh/pushrule_stream
2016-03-04 14:44:34 +00:00
Patrik Oldsberg
bb0e82fff1
tests/utils: added room_invite_state_types to test config
...
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-03-04 10:43:17 +01:00
Daniel Wagner-Hall
b4022cc487
Pass whole requester to ratelimiting
...
This will enable more detailed decisions
2016-03-03 16:43:42 +00:00
Daniel Wagner-Hall
95481e7ba7
Merge pull request #571 from matrix-org/daniel/asids
...
Mark AS users with their AS's ID
2016-03-03 13:56:28 +00:00
Mark Haines
2223204eba
Hook push rules up to the replication API
2016-03-02 17:26:20 +00:00
Erik Johnston
2a78dac60d
Merge pull request #612 from matrix-org/erikj/cache_size
...
Add environment variable SYNAPSE_CACHE_FACTOR, default it to 0.1
2016-03-02 14:36:10 +00:00
Mark Haines
60a0f81c7a
Add a /replication API for extracting the updates that happened on
...
synapse
This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.
This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.
Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.
Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.
The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
2016-03-01 14:49:41 +00:00
Erik Johnston
ce2cdced61
Move cache size fiddling to descriptors only. Fix tests
2016-03-01 13:21:46 +00:00
Mark Haines
5e2890bd49
Check that the disable_registration config key is handled correctly
2016-02-22 16:01:29 +00:00
Mark Haines
7641a90c34
Add a test for TreeCache.__contains__
2016-02-22 15:22:38 +00:00
Mark Haines
5c79ef9396
Test Filter.filter_rooms
...
Also check that the __repr__ method for FilterCollection does something
sensible.
2016-02-19 17:55:28 +00:00
Mark Haines
700487a7c7
Fix flake8 warnings for tests
2016-02-19 15:34:38 +00:00
Erik Johnston
929cb0ed7d
Don't set currently_active for remote presence
2016-02-19 10:58:27 +00:00
Erik Johnston
b4796a62ee
Add unit test
2016-02-18 11:52:33 +00:00
Erik Johnston
9da9826b85
Remove old tests.
2016-02-18 10:46:16 +00:00
Erik Johnston
e5999bfb1a
Initial cut
2016-02-17 15:40:50 +00:00
Erik Johnston
9e7900da1e
Add wheeltimer impl
2016-02-17 14:29:28 +00:00
Daniel Wagner-Hall
763360594d
Mark AS users with their AS's ID
2016-02-11 17:26:42 +00:00
Erik Johnston
e557dc80b8
Merge pull request #566 from matrix-org/erikj/logcontext
...
Don't bother copying records on parent context
2016-02-10 11:41:45 +00:00
Erik Johnston
78a5482267
Typo
2016-02-09 16:23:11 +00:00
Erik Johnston
e664e9737c
Fix test
2016-02-09 14:57:43 +00:00
Erik Johnston
13ba8d878c
Fix test
2016-02-09 14:55:21 +00:00
Daniel Wagner-Hall
6a9f1209df
Error if macaroon key is missing from config
...
Currently we store all access tokens in the DB, and fall back to that
check if we can't validate the macaroon, so our fallback works here, but
for guests, their macaroons don't get persisted, so we don't get to
find them in the database. Each restart, we generate a new ephemeral
key, so guests lose access after each server restart.
I tried to fix up the config stuff to be less insane, but gave up, so
instead I bolt on yet another piece of custom one-off insanity.
Also, add some basic tests for config generation and loading.
2016-02-05 01:58:23 +00:00
Daniel Wagner-Hall
66bb255fcd
Merge pull request #556 from matrix-org/daniel/config
...
Rename config field to reflect yaml name
2016-02-03 14:55:54 +00:00
Daniel Wagner-Hall
5054806ec1
Rename config field to reflect yaml name
2016-02-03 14:42:01 +00:00
Erik Johnston
f8aae79a72
Simplify get_rooms
2016-02-03 13:24:35 +00:00
Mark Haines
f2d5ff5bf2
Fix the mock homserver used in the tests
2016-01-29 14:53:14 +00:00
Erik Johnston
b18114e19e
Merge pull request #536 from matrix-org/erikj/sync
...
Make /sync "better".
2016-01-29 13:04:51 +00:00
Erik Johnston
02a9c3be6c
Merge pull request #538 from matrix-org/erikj/fix_lru_cache
...
Fix LruCache. Make TreeCache track its own size.
2016-01-29 11:53:55 +00:00
Erik Johnston
4fce59f274
Add tests
2016-01-29 11:33:11 +00:00
Erik Johnston
35981c8b71
Fix test
2016-01-28 17:20:05 +00:00
Erik Johnston
167d1df699
Merge pull request #534 from matrix-org/erikj/setup
...
Add a Homeserver.setup method
2016-01-28 14:45:36 +00:00
Erik Johnston
4e7948b47a
Allow paginating backwards from stream token
2016-01-28 11:52:34 +00:00
Erik Johnston
61eaa6ec64
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync
2016-01-28 10:50:57 +00:00
Richard van der Hoff
e26390ca46
Merge pull request #535 from matrix-org/rav/paginate_from_stream_token
...
Make it possible to paginate forwards from stream tokens
2016-01-28 09:49:46 +00:00
Richard van der Hoff
5cba88ea7c
Make it possible to paginate forwards from stream tokens
...
In order that we can fill the gap after a /sync, make it possible to paginate
forwards from a stream token.
2016-01-27 17:42:45 +00:00
Erik Johnston
0487c9441f
Fix tests
2016-01-27 17:33:13 +00:00
Erik Johnston
9959d9ece8
Remove redundated BaseHomeServer
2016-01-26 13:52:29 +00:00
Erik Johnston
8c6012a4af
Fix tests
2016-01-25 13:12:35 +00:00
Erik Johnston
88baa3865e
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync
2016-01-22 15:57:12 +00:00
David Baker
d552861346
Revert all the bits changing keys of eeverything that used LRUCaches to tuples
2016-01-22 12:18:14 +00:00
David Baker
10f76dc5da
Make LRU cache not default to treecache & add options to use it
2016-01-22 12:10:33 +00:00
David Baker
31a051b677
Test treecache directly
2016-01-22 11:22:00 +00:00
Erik Johnston
8f9c74e9f1
Fix tests
2016-01-22 10:48:27 +00:00
David Baker
4efcaa43c8
Add tests for treecache directly and test del_multi at the LruCache level too.
2016-01-22 10:37:37 +00:00
David Baker
f1f8122120
Change LRUCache to be tree-based so we can delete subtrees.
2016-01-21 19:16:25 +00:00
Erik Johnston
191070123d
Cache dns lookups, and use the cache if we fail to lookup servers later
2016-01-20 11:34:09 +00:00
Erik Johnston
2c176e02ae
Make unit tests work
2016-01-18 14:48:50 +00:00
Daniel Wagner-Hall
ac5a4477ad
Require unbanning before other membership changes
2016-01-15 16:27:26 +00:00
Daniel Wagner-Hall
2680043bc6
Require ID and as_token be unique for ASs
...
Defaults ID to as_token if not specified. This will change
when IDs are fully supported.
2016-01-14 14:34:01 +00:00
Mark Haines
c0a279e808
Delete the table objects from TransactionStore
2016-01-13 11:15:20 +00:00
Daniel Wagner-Hall
42aa1f3f33
Merge pull request #478 from matrix-org/daniel/userobject
...
Introduce a User object
I'm sick of passing around more and more things as tuple items around
the whole world, and needing to edit every call site every time there is
more information about a user. So pass them around together as an
object.
This object has incredibly poorly named fields because we have a
convention that `user` indicates a UserID object, and `user_id`
indicates a string. I tried to clean up the whole repo to fix this, but
gave up. So instead, I introduce a second convention. A user_object is a
User, and a user_id_object is a UserId. I may have cried a little bit.
2016-01-11 17:50:22 +00:00
Daniel Wagner-Hall
2110e35fd6
Introduce a Requester object
...
This tracks data about the entity which made the request. This is
instead of passing around a tuple, which requires call-site
modifications every time a new piece of optional context is passed
around.
I tried to introduce a User object. I gave up.
2016-01-11 17:48:45 +00:00
David Baker
c232780081
Merge pull request #456 from matrix-org/store_event_actions
...
Send unread notification counts
2016-01-08 14:47:15 +00:00
Matthew Hodgson
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
David Baker
0e48f7f245
fix tests
2016-01-06 16:46:41 +00:00
David Baker
eb03625626
Merge remote-tracking branch 'origin/develop' into store_event_actions
2016-01-05 18:39:50 +00:00
Daniel Wagner-Hall
cfd07aafff
Allow guests to upgrade their accounts
2016-01-05 18:01:18 +00:00
David Baker
92a1e74b20
fix tests
2016-01-04 14:17:35 +00:00
David Baker
928c575c6f
Merge remote-tracking branch 'origin/develop' into store_event_actions
2016-01-04 13:39:51 +00:00
Mark Haines
7fa71e3267
Add a unit test for the snapshot cache
2015-12-23 11:48:03 +00:00
David Baker
d79e90f078
Add mocks to make tests work again
2015-12-22 17:56:56 +00:00
Mark Haines
d547afeae0
Merge remote-tracking branch 'origin/master' into develop
2015-12-07 13:13:43 +00:00
Oleg Girko
4013216fcc
Fix mock import in tests.
...
For some reason, one test imports Mock class from mock.mock
rather than from mock.
This change fixes this error.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
2015-12-06 20:50:11 +00:00
Daniel Wagner-Hall
14d7acfad4
Host /unstable and /r0 versions of r0 APIs
2015-12-01 17:34:32 +00:00
Mark Haines
7b593af7e1
rename the method in the tests as well
2015-12-01 16:06:17 +00:00
Erik Johnston
bd3de8f39a
Update tests
2015-11-17 18:38:48 +00:00
Daniel Wagner-Hall
468a2ed4ec
Return non-room events from guest /events calls
2015-11-12 16:45:28 +00:00
Richard van der Hoff
78f6010207
Fix an issue with ignoring power_level changes on divergent graphs
...
Changes to m.room.power_levels events are supposed to be handled at a high
priority; however a typo meant that the relevant bit of code was never
executed, so they were handled just like any other state change - which meant
that a bad person could cause room state changes by forking the graph from a
point in history when they were allowed to do so.
2015-11-12 15:24:59 +00:00
Erik Johnston
e21cef9bb5
Merge pull request #359 from matrix-org/markjh/incremental_indexing
...
Incremental background updates for db indexes
2015-11-11 17:19:51 +00:00
Daniel Wagner-Hall
cf437900e0
Return world_readable and guest_can_join in /publicRooms
2015-11-10 17:10:27 +00:00
Mark Haines
36c58b18a3
Test for background updates
2015-11-10 15:51:40 +00:00
Richard van der Hoff
9107ed23b7
Add a couple of unit tests for room/<x>/messages
...
... merely because I was trying to figure out how it worked, and couldn't.
2015-11-09 16:16:43 +00:00
Daniel Wagner-Hall
ca2f90742d
Open up /events to anonymous users for room events only
...
Squash-merge of PR #345 from daniel/anonymousevents
2015-11-05 14:32:26 +00:00
Daniel Wagner-Hall
4e62ffdb21
Merge branch 'develop' of github.com:matrix-org/synapse into develop
2015-11-04 17:31:01 +00:00
Daniel Wagner-Hall
f522f50a08
Allow guests to register and call /events?room_id=
...
This follows the same flows-based flow as regular registration, but as
the only implemented flow has no requirements, it auto-succeeds. In the
future, other flows (e.g. captcha) may be required, so clients should
treat this like the regular registration flow choices.
2015-11-04 17:29:07 +00:00
Mark Haines
33b3e04049
Merge branch 'develop' into daniel/removesomelies
...
Conflicts:
synapse/notifier.py
2015-11-04 16:01:00 +00:00
Mark Haines
c452dabc3d
Remove the LockManager class because it wasn't being used
2015-11-04 14:08:15 +00:00
Daniel Wagner-Hall
771ca56c88
Remove more unused parameters
2015-11-02 15:31:57 +00:00
Mark Haines
0e36756383
Merge branch 'develop' into markjh/room_tags
2015-11-02 10:57:00 +00:00
Mark Haines
fb46937413
Support clients supplying older tokens, fix short poll test
2015-10-30 16:38:35 +00:00
Richard van der Hoff
f69a5c9134
Fix a 500 error resulting from empty room_ids
...
POST /_matrix/client/api/v1/rooms//send/a.b.c gave a 500 error, because we
assumed that rooms always had at least one character.
2015-10-26 18:44:03 +00:00
Erik Johnston
b0ac0a9438
Merge pull request #319 from matrix-org/erikj/filter_refactor
...
Refactor api.filtering to have a Filter API
2015-10-22 13:17:10 +01:00
Mark Haines
87da71bace
Merge pull request #314 from matrix-org/paul/event-redaction
...
Add some unit tests of prune_events()
2015-10-22 11:07:20 +01:00
Erik Johnston
45cd2b0233
Refactor api.filtering to have a Filter API
2015-10-20 15:33:25 +01:00
Daniel Wagner-Hall
f510586372
Merge branch 'develop' of github.com:matrix-org/synapse into develop
2015-10-20 12:00:22 +01:00
Daniel Wagner-Hall
137fafce4e
Allow rejecting invites
...
This is done by using the same /leave flow as you would use if you had
already accepted the invite and wanted to leave.
2015-10-20 11:58:58 +01:00
Paul "LeoNerd" Evans
9ed784098a
Invoke EventBuilder directly instead of going via the EventBuilderFactory
2015-10-19 17:42:34 +01:00
Paul "LeoNerd" Evans
531e3aa75e
Capture __init__.py
2015-10-19 17:37:35 +01:00
Paul "LeoNerd" Evans
a8795c9644
Use assertIn() instead of assertTrue on the 'in' operator
2015-10-19 15:24:49 +01:00
Paul "LeoNerd" Evans
07b58a431f
Another signing test vector using an 'm.room.message' with content, so that the implementation will have to redact it
2015-10-19 15:00:52 +01:00
Paul "LeoNerd" Evans
0aab34004b
Initial minimial hack at a test of event hashing and signing
2015-10-19 14:40:15 +01:00
Paul "LeoNerd" Evans
aff4d850bd
Add some unit tests of prune_events()
2015-10-16 19:56:46 +01:00
Mark Haines
8897781558
update filtering tests
2015-10-13 14:13:51 +01:00
Erik Johnston
17c80c8a3d
rename schema_prepare to prepare_database
2015-10-13 13:56:22 +01:00
Erik Johnston
ec398af41c
Expose error more nicely
2015-10-13 11:43:43 +01:00
Erik Johnston
d5e081c7ae
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/unfederatable
2015-10-02 10:33:49 +01:00
Mark Haines
1ee3d26432
synapse/storage/_base.py:_simple_selectupdate_one was unused
2015-09-23 10:30:03 +01:00
Mark Haines
bb4dddd6c4
Move NullSource out of synapse and into tests since it is only used by the tests
2015-09-22 18:33:34 +01:00
Daniel Wagner-Hall
7213588083
Implement configurable stats reporting
...
SYN-287
This requires that HS owners either opt in or out of stats reporting.
When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
2015-09-22 12:57:40 +01:00
Mark Haines
ee2d722f0f
Merge pull request #276 from matrix-org/markjh/history_for_rooms_that_have_been_left
...
SPEC-216: Allow users to view the history of rooms that they have left.
2015-09-21 14:38:13 +01:00
Daniel Wagner-Hall
728d07c8c1
Merge pull request #256 from matrix-org/auth
...
Attempt to validate macaroons
2015-09-14 18:09:33 +01:00
Mark Haines
e2054ce21a
Allow users to GET individual state events for rooms that they have left
2015-09-10 15:06:47 +01:00
Mark Haines
bc8b25eb56
Allow users that have left the room to view the member list from the point they left
2015-09-09 15:42:16 +01:00
Erik Johnston
187320b019
Merge branch 'erikj/check_room_exists' into erikj/unfederatable
2015-09-01 15:58:10 +01:00
Erik Johnston
b345853918
Check against sender rather than event_id
2015-09-01 15:57:35 +01:00
Erik Johnston
00149c063b
Fix tests
2015-09-01 15:42:03 +01:00
Daniel Wagner-Hall
e255c2c32f
s/user_id/user/g for consistency
2015-09-01 12:41:16 +01:00
Daniel Wagner-Hall
3063383547
Swap out bcrypt for md5 in tests
...
This reduces our ~8 second sequential test time down to ~7 seconds
2015-08-26 15:59:32 +01:00
Daniel Wagner-Hall
81450fded8
Turn TODO into thing which actually will fail
2015-08-26 13:56:01 +01:00
Daniel Wagner-Hall
6a4b650d8a
Attempt to validate macaroons
...
A couple of weird caveats:
* If we can't validate your macaroon, we fall back to checking that
your access token is in the DB, and ignoring the failure
* Even if we can validate your macaroon, we still have to hit the DB to
get the access token ID, which we pretend is a device ID all over the
codebase.
This mostly adds the interesting code, and points out the two pieces we
need to delete (and necessary conditions) in order to fix the above
caveats.
2015-08-26 13:22:23 +01:00
Daniel Wagner-Hall
a9d8bd95e7
Stop looking up "admin", which we never read
2015-08-25 16:29:39 +01:00
Daniel Wagner-Hall
a0b181bd17
Remove completely unused concepts from codebase
...
Removes device_id and ClientInfo
device_id is never actually written, and the matrix.org DB has no
non-null entries for it. Right now, it's just cluttering up code.
This doesn't remove the columns from the database, because that's
fiddly.
2015-08-25 16:23:06 +01:00
Daniel Wagner-Hall
e8cf77fa49
Merge branch 'develop' into refresh
...
Conflicts:
synapse/rest/client/v1/login.py
2015-08-20 16:25:40 +01:00
Daniel Wagner-Hall
cecbd636e9
/tokenrefresh POST endpoint
...
This allows refresh tokens to be exchanged for (access_token,
refresh_token).
It also starts issuing them on login, though no clients currently
interpret them.
2015-08-20 16:21:35 +01:00
Daniel Wagner-Hall
13a6517d89
s/by_token/by_access_token/g
...
We're about to have two kinds of token, access and refresh
2015-08-20 16:01:29 +01:00
Daniel Wagner-Hall
617501dd2a
Move token generation to auth handler
...
I prefer the auth handler to worry about all auth, and register to call
into it as needed, than to smatter auth logic between the two.
2015-08-20 11:35:56 +01:00
Mark Haines
a0b8e5f2fe
Merge pull request #211 from matrix-org/email_in_use
...
Changes for unique emails
2015-08-20 10:04:04 +01:00
Daniel Wagner-Hall
70e265e695
Re-add whitespace around caveat operators
2015-08-19 14:30:31 +01:00
Daniel Wagner-Hall
ce832c38d4
Remove padding space around caveat operators
2015-08-18 17:39:26 +01:00
Daniel Wagner-Hall
42e858daeb
Fix units in test
...
I made the non-test seconds instead of ms, but not the test
2015-08-18 17:38:37 +01:00
Daniel Wagner-Hall
3e6fdfda00
Fix some formatting to use tuples
2015-08-18 15:18:50 +01:00
Daniel Wagner-Hall
1469141023
Merge branch 'develop' into auth
2015-08-18 14:43:44 +01:00
Daniel Wagner-Hall
2d3462714e
Issue macaroons as opaque auth tokens
...
This just replaces random bytes with macaroons. The macaroons are not
inspected by the client or server.
In particular, they claim to have an expiry time, but nothing verifies
that they have not expired.
Follow-up commits will actually enforce the expiration, and allow for
token refresh.
See https://bit.ly/matrix-auth for more information
2015-08-18 14:22:02 +01:00
Erik Johnston
d7451e0f22
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-12 10:30:30 +01:00
Erik Johnston
2df8dd9b37
Move all the caches into their own package, synapse.util.caches
2015-08-11 18:00:59 +01:00
Daniel Wagner-Hall
4ff0228c25
Remove call to recently removed function in mock
2015-08-11 16:56:30 +01:00
Daniel Wagner-Hall
dcd5983fe4
Remove call to recently removed function in mock
2015-08-11 16:54:06 +01:00
Erik Johnston
20addfa358
Change Cache to not use *args in its interface
2015-08-07 18:32:47 +01:00
Erik Johnston
b8e386db59
Change Cache to not use *args in its interface
2015-08-07 11:52:21 +01:00
Erik Johnston
f596ff402e
Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache
2015-08-06 13:37:56 +01:00
Erik Johnston
2efb93af52
Merge branch 'erikj/cached_keyword_args' into erikj/cache_deferreds
2015-08-06 13:35:28 +01:00
Erik Johnston
7eea3e356f
Make @cached cache deferreds rather than the deferreds' values
2015-08-06 13:33:34 +01:00
Erik Johnston
07507643cb
Use dictionary cache to do group -> state fetching
2015-08-05 15:11:42 +01:00
Erik Johnston
c67ba143fa
Move DictionaryCache
2015-08-04 15:58:28 +01:00
Erik Johnston
e7768e77f5
Add basic dictionary cache
2015-08-04 15:56:56 +01:00
David Baker
07ad03d5df
Fix tests
2015-08-04 15:18:40 +01:00
Kegan Dougal
11b0a34074
Use the same reg paths as register v1 for ASes.
...
Namely this means using registration_handler.appservice_register.
2015-07-29 10:00:54 +01:00
Kegan Dougal
a4d62ba36a
Fix v2_alpha registration. Add unit tests.
...
V2 Registration forced everyone (including ASes) to create a password for a
user, when ASes should be able to omit passwords. Also unbreak AS registration
in general which checked too early if the given username was claimed by an AS;
it was checked before knowing if the AS was the one doing the registration! Add
unit tests for AS reg, user reg and disabled_registration flag.
2015-07-28 17:34:12 +01:00
Erik Johnston
f0dd6d4cbd
Fix test.
2015-07-07 16:18:36 +01:00
Erik Johnston
bd1236c0ee
Consolidate duplicate code in notifier
2015-07-02 11:46:05 +01:00
Erik Johnston
ddf7979531
Add receipts_key to StreamToken
2015-07-02 11:45:44 +01:00
Kegan Dougal
d88e20cdb9
Fix bug where synapse was sending AS user queries incorrectly.
...
Bug introduced in 92b20713d7
which reversed the comparison when checking if a user existed
in the users table. Added UTs to prevent this happening again.
2015-06-17 17:26:03 +01:00
Erik Johnston
261ccd7f5f
Fix tests
2015-06-12 17:17:29 +01:00
Erik Johnston
3bbd0d0e09
Merge pull request #180 from matrix-org/erikj/prev_state_context
...
Don't needlessly compute prev_state
2015-06-03 17:20:56 +01:00
Erik Johnston
9dda396baa
Merge pull request #179 from matrix-org/erikj/state_group_outliers
...
Don't compute EventContext for outliers.
2015-06-03 17:20:40 +01:00
Erik Johnston
55bf90b9e4
Don't needlessly compute prev_state
2015-06-03 16:44:24 +01:00
Erik Johnston
1c3d844e73
Don't needlessly compute context
2015-06-03 16:41:51 +01:00
Erik Johnston
d8866d7277
Caches should be bound to instances.
...
Before, caches were global and so different instances of the stores
would share caches. This caused problems in the unit tests.
2015-06-03 14:45:17 +01:00
Mark Haines
17167898c8
Fix the presence tests
2015-05-22 16:22:54 +01:00
Mark Haines
49a2c10279
Merge pull request #157 from matrix-org/markjh/presence_performance
...
Improve presence performance in loadtest
2015-05-22 16:04:40 +01:00
Mark Haines
f43544eecc
Make the appservice use 'users_in_room' rather than get_room_members since it is cached
2015-05-22 11:01:28 +01:00
Mark Haines
88f1ea36ce
Oops, get_rooms_for_user returns a namedtuple, not a room_id
2015-05-21 15:23:58 +01:00
Erik Johnston
a551c5dad7
Merge pull request #155 from matrix-org/erikj/perf
...
Bulk and batch retrieval of events.
2015-05-21 14:54:40 +01:00
Mark Haines
8eca5bd50a
Fix the presence tests
2015-05-20 13:22:18 +01:00
Mark Haines
0b0033c40b
Merge branch 'develop' into notifier_performance
2015-05-18 13:50:01 +01:00
Erik Johnston
131bdf9bb1
Merge branch 'erikj/events_move' of github.com:matrix-org/synapse into erikj/perf
2015-05-18 10:23:37 +01:00
Erik Johnston
a2c4f3f150
Fix daedlock
2015-05-15 10:54:04 +01:00
Mark Haines
c5d1b4986b
Remove unused arguments and doc PresenceHandler.push_update_to_clients
2015-05-14 14:59:31 +01:00
Mark Haines
f1b83d88a3
Discard unused NotifierUserStreams
2015-05-13 16:54:02 +01:00
Mark Haines
63878c0379
Don't bother checking for updates if the stream token hasn't advanced for a user
2015-05-13 13:42:21 +01:00
Erik Johnston
43c2e8deae
Add support for using executemany
2015-05-05 15:13:25 +01:00
Erik Johnston
327ca883ec
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-28 13:39:42 +01:00
Erik Johnston
6f8e2d517e
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-27 14:41:40 +01:00
David Baker
6532b6e607
Merge branch 'develop' into csauth
...
Conflicts:
synapse/http/server.py
2015-04-24 09:37:54 +01:00
Paul "LeoNerd" Evans
e1e5e53127
Remove users from the remote_offline_serials list (and clean up empty elements) when they go online again
2015-04-23 19:01:37 +01:00
Paul "LeoNerd" Evans
8a785c3006
Store a list of the presence serial number at which remote users went offline, so that when we delete them from the cachemap, we can still synthesize OFFLINE events for them (SYN-261)
2015-04-23 18:40:19 +01:00
David Baker
cb03fafdf1
Merge branch 'develop' into csauth
2015-04-17 13:51:10 +01:00
Erik Johnston
6ba2e3df4e
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-17 11:22:31 +01:00
Paul "LeoNerd" Evans
04c7f3576e
Various minor fixes to unit-test structure around typing notifications
2015-04-15 23:32:11 +01:00
Paul "LeoNerd" Evans
0268d40281
Have TypingNotificationEventSource.get_new_events_for_user() return a deferred, for consistency and extensibility
2015-04-15 23:09:35 +01:00
Erik Johnston
22d7a59306
Fix tests after commit 9a0579
2015-04-08 16:57:14 +01:00
Erik Johnston
4fe95094d1
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-07 18:05:39 +01:00
Erik Johnston
9236136f3a
Make work in both Maria and SQLite. Fix tests
2015-04-01 14:12:33 +01:00
Kegan Dougal
9fa8bda099
Merge branch 'develop' into application-services-registration-script
2015-04-01 10:19:17 +01:00
Kegan Dougal
09cbff174a
Fix thinko whereby events *for the AS specifically* were not passed on.
...
This was caused by not explicitly checking the service.sender field. This
has now been fixed and a regression test has been added.
2015-03-31 16:44:45 +01:00
Kegan Dougal
cf1fa59f4b
Use a sender localpart instead of a user ID.
...
Form the user ID at runtime instead, This gives less room for error in AS
config files since they cannot specify the domain of another HS.
2015-03-31 13:48:03 +01:00
Kegan Dougal
c217504949
Edit SQL schema to use string IDs not ints. Use token as ID. Update tests.
2015-03-31 12:07:56 +01:00
Kegan Dougal
b59aa74556
Fix tests and missing returns on deferreds.
2015-03-31 11:35:45 +01:00
Kegan Dougal
d33ae65efc
Remove more reg/unreg methods. Read config not database for cache.
2015-03-31 11:00:00 +01:00
Kegan Dougal
4edcbcee3b
Merge branch 'develop' into application-services-txn-reliability
...
Conflicts:
synapse/storage/__init__.py
2015-03-26 10:07:59 +00:00
Paul "LeoNerd" Evans
9ba6487b3f
Allow a choice of LRU behaviour for Cache() by using LruCache() or OrderedDict()
2015-03-25 19:05:34 +00:00
Paul "LeoNerd" Evans
7ab9f91a60
Unit-test that Cache() key eviction is ordered
2015-03-25 18:50:43 +00:00
David Baker
ce2766d19c
Fix tests
2015-03-24 18:56:51 +00:00
David Baker
438a21c87b
Don't test exact equality of the list: as long as it has the fields we expect, that's just fine. I added the user_id (as in database pkey) and it broke: no point testing what that comes out as: it's determined by the db.
2015-03-24 18:21:54 +00:00
Paul "LeoNerd" Evans
0f86312c4c
Pull out the cache logic from the @cached wrapper into its own class we can reuse
2015-03-20 18:25:42 +00:00
Erik Johnston
7e282a53a5
Tidy up _simple_... methods
2015-03-20 15:05:10 +00:00
Kegan Dougal
d04fa1f712
Implement ServiceQueuer with tests.
2015-03-16 14:03:16 +00:00
Kegan Dougal
6279285b2a
Replace EventGrouper for ServiceQueuer to move to push-based txns. Fix tests and add stub tests for ServiceQueuer.
2015-03-16 13:15:40 +00:00
Kegan Dougal
835e01fc70
Minor PR comment tweaks.
2015-03-16 10:16:59 +00:00
Kegan Dougal
f9232c7917
Merge branch 'develop' into application-services-txn-reliability
...
Conflicts:
synapse/storage/appservice.py
2015-03-16 10:09:15 +00:00
Paul "LeoNerd" Evans
c1cdd7954d
Add an .inc_by() method to CounterMetric; implement DistributionMetric a neater way
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
f1fbe3e09f
Rename TimerMetric to DistributionMetric; as it could count more than just time
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
cbc0406be8
Export CacheMetric as hits+total, rather than hits+misses, as it's easier to derive hit ratio from that
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
0e847540c3
Prometheus needs "escaped" label values
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
b3a0179d64
Bugfix to rendering output of vectored TimerMetrics
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
f9478e475b
Rename Metrics' "keys" to "labels"
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
72625f2f4d
Initial hack at a TimerMetric; for storing counts + duration accumulators
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
23ab0c68c2
Implement vector CallbackMetrics
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
8664599af7
Rename CacheCounterMetric to just CacheMetric; add a CallbackMetric component to give the size of the cache
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
d8caa5454d
Initial attempt at a scalar callback-based metric to give instantaneous snapshot gauges
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
ce8b5769f7
Create the concept of a cachecounter metric; generating two counters specific to caches
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
e7420a3bef
Initial tiny attempt at (vectorable) counter metrics
2015-03-12 16:24:50 +00:00
Kegan Dougal
b98cd03193
Use event IDs instead of dumping event content in the txns table.
2015-03-09 17:25:20 +00:00
Kegan Dougal
21fd84dcb8
Use seconds; start gluing in the AS scheduler into the AS handler.
2015-03-09 17:01:19 +00:00
Erik Johnston
6d74e46621
Fix tests
2015-03-09 17:01:11 +00:00
Kegan Dougal
0a60bbf4fa
Finish appservice txn storage impl and tests.
2015-03-09 15:53:03 +00:00
Kegan Dougal
1ead1caa18
Implement create_appservice_txn with tests.
2015-03-09 13:54:20 +00:00
Kegan Dougal
1c2dcf762a
Partially implement txn store methods with tests.
2015-03-09 13:10:31 +00:00
Kegan Dougal
10766f1e93
Update UTs
2015-03-06 16:17:01 +00:00
Kegan Dougal
2602ddc379
Apply clarity and docstrings
2015-03-06 16:16:14 +00:00
Kegan Dougal
0354659f9d
Finish synapse.appservice.scheduler implementation.
...
With tests to assert behaviour. Not hooked up yet. Stub datastore methods
not implemented yet.
2015-03-06 16:09:05 +00:00
Kegan Dougal
f260cb72cd
Flesh out more stub functions.
2015-03-06 15:12:24 +00:00
Kegan Dougal
0fbfe1b08a
Add more tests; fix bugs.
2015-03-06 14:36:52 +00:00
Kegan Dougal
192e228a98
Start adding some tests
2015-03-06 11:50:27 +00:00
Kegsay
8ad024ea80
Merge pull request #93 from matrix-org/application-services-exclusive
...
Application services exclusive flag support
2015-03-02 14:56:32 +00:00
Erik Johnston
0a036944bd
Merge branch 'develop' of github.com:matrix-org/synapse into batched_get_pdu
2015-03-02 13:53:30 +00:00
Kegan Dougal
de190e49d5
Add more unit tests for exclusive namespaces.
2015-02-27 11:51:06 +00:00
Kegan Dougal
127efeeb68
Update unit tests to use new format.
2015-02-27 11:10:48 +00:00
Kegan Dougal
0ebd632d39
Fix unit tests
2015-02-27 09:46:38 +00:00
Paul "LeoNerd" Evans
f53fcbce97
Use cache.pop() instead of a separate membership test + del []
2015-02-23 18:30:45 +00:00
Paul "LeoNerd" Evans
bd84755e64
Merge remote-tracking branch 'origin/develop' into performance-cache-improvements
2015-02-23 17:16:03 +00:00
Paul "LeoNerd" Evans
e76d485e29
Allow @cached-wrapped functions to have a prefill method for setting entries
2015-02-23 15:41:54 +00:00
Erik Johnston
852816befe
Fix presence tests
2015-02-23 15:14:09 +00:00
Paul "LeoNerd" Evans
ebc3db295b
Take named arguments to @cached() decorator, add a 'max_entries' limit
2015-02-19 18:36:02 +00:00
Erik Johnston
5f4c28d313
Update tests
2015-02-19 14:34:32 +00:00
Erik Johnston
5358966a87
Use git aware version string in User-Agent and Server headers
2015-02-18 16:52:04 +00:00
Mark Haines
0d872f5aa6
Merge pull request #50 from matrix-org/application-services
...
Application Services
2015-02-13 15:06:14 +00:00
Erik Johnston
021d93db11
Merge pull request #62 from matrix-org/state-chache
...
State chache
2015-02-11 17:10:28 +00:00
Kegan Dougal
cb43fbeeb4
Fix tests which broke when event caching was introduced.
2015-02-11 16:46:01 +00:00
Kegan Dougal
f2fdcb7c4b
Merge branch 'develop' into application-services
2015-02-11 16:43:26 +00:00
Mark Haines
f5a70e0d2e
Add a cache for get_event
2015-02-11 15:01:15 +00:00
Mark Haines
d8324d5a2b
Add a lru cache class
2015-02-11 14:52:23 +00:00
Mark Haines
896253e085
Factor out some of the common homeserver setup code into a
...
setup_test_homeserver function in utils.
2015-02-11 11:37:30 +00:00
Kegan Dougal
c7783d6fee
Notify ASes for events sent by other users in a room which an AS user is a part of.
2015-02-11 10:36:08 +00:00
Erik Johnston
164f6b9256
Fix tests
2015-02-09 14:23:57 +00:00
Kegan Dougal
ab3c897ce1
Remove unused imports.
2015-02-09 14:16:36 +00:00
Kegan Dougal
5a7dd05818
Modify auth.get_user_by_req for authing appservices directly.
...
Add logic to map the appservice token to the autogenned appservice user ID.
Add unit tests for all forms of get_user_by_req (user/appservice,
valid/bad/missing tokens)
2015-02-09 14:14:15 +00:00
Kegan Dougal
11e6b3d18b
Dependency inject ApplicationServiceApi when creating ApplicationServicesHandler.
2015-02-05 17:04:59 +00:00
Kegan Dougal
951690e54d
Merge branch 'develop' into application-services
2015-02-05 14:28:03 +00:00
Kegan Dougal
131e036402
Fix unit tests.
2015-02-05 13:22:20 +00:00
Kegan Dougal
bc658907f0
Add unit test for appservice_handler.query_room_alias_exists
2015-02-05 11:54:36 +00:00
Kegan Dougal
f0c730252f
Add unknown user ID check. Use store.get_aliases_for_room(room_id) when searching for services by alias.
2015-02-05 11:25:32 +00:00
Kegan Dougal
27091f146a
Add hs_token column and generate a different token f.e application service.
2015-02-05 10:08:12 +00:00
Kegan Dougal
89f2e8fbdf
Fix bug in store defer. Add more unit tests.
2015-02-04 15:21:03 +00:00
Kegan Dougal
525a218b2b
Begin to add unit tests for appservice glue and regex testing.
2015-02-04 12:24:20 +00:00
Erik Johnston
e7ca813dd4
Try to ensure we don't persist an event we have already persisted. In persist_event check if we already have the event, if so then update instead of replacing so that we don't cause a bump of the stream_ordering.
2015-02-03 10:39:41 +00:00
Erik Johnston
1bd540ef79
Merge branch 'develop' of github.com:matrix-org/synapse into rejections
...
Conflicts:
synapse/storage/schema/im.sql
2015-01-30 15:16:38 +00:00
Erik Johnston
471c47441d
Merge pull request #37 from matrix-org/client_v2_filter
...
Client v2 filter
2015-01-30 14:56:08 +00:00
Erik Johnston
2aaedab203
Merge branch 'develop' of github.com:matrix-org/synapse into new_state_resolution
2015-01-30 14:09:32 +00:00
Erik Johnston
2ebf795c0a
Merge branch 'develop' of github.com:matrix-org/synapse into rejections
...
Conflicts:
synapse/storage/__init__.py
synapse/storage/schema/delta/v12.sql
2015-01-30 11:10:37 +00:00
Mark Haines
93ed31dda2
Create a separate filter object to do the actual filtering, so that we can
...
split the storage and management of filters from the actual filter code
and don't have to load a filter from the db each time we filter an event
2015-01-29 17:45:07 +00:00
Erik Johnston
78015948a7
Initial implementation of auth conflict resolution
2015-01-29 16:52:33 +00:00
Mark Haines
33391db5f8
Merge in auth changes from develop
2015-01-29 15:54:54 +00:00
Mark Haines
9d8f798a3f
Merge changes from develop
2015-01-29 14:55:27 +00:00
Kegan Dougal
38b27bd2cb
Add filter_room_state unit tests.
2015-01-29 14:28:34 +00:00
Kegan Dougal
83172487b0
Add basic filtering public API unit tests. Use defers in the right places.
2015-01-29 12:20:59 +00:00
Kegan Dougal
5561a87920
Add more unit tests for the filter algorithm.
2015-01-29 12:06:16 +00:00
Kegan Dougal
777d9914b5
Implement filter algorithm. Add basic event type unit tests to assert it works.
2015-01-29 11:38:06 +00:00
Mark Haines
26c8fff19e
Merge pull request #36 from matrix-org/device_id_from_access_token
...
Extract the device id and token id from the access token when autheniticating users
2015-01-28 17:19:28 +00:00
Mark Haines
388581e087
Extract the id token of the token when authing users, include the token and device_id in the internal meta data for the event along with the transaction id when sending events
2015-01-28 16:58:23 +00:00
Kegan Dougal
c23e3db544
Add filter JSON sanity checks.
2015-01-28 16:45:18 +00:00
Mark Haines
c59bcabf0b
Return the device_id from get_auth_by_req
2015-01-28 15:43:41 +00:00
Paul "LeoNerd" Evans
06cc147012
Initial stab at real SQL storage implementation of user filter definitions
2015-01-27 18:46:03 +00:00
Paul "LeoNerd" Evans
0c14a699bb
More unit-testing of REST errors
2015-01-27 18:07:21 +00:00
Paul "LeoNerd" Evans
54e513b4e6
Move storage of user filters into real datastore layer; now have to mock it out in the REST-level tests
2015-01-27 17:48:13 +00:00
Paul "LeoNerd" Evans
059651efa1
Have the Filtering API return Deferreds, so we can do the Datastore implementation nicely
2015-01-27 16:17:56 +00:00
Paul "LeoNerd" Evans
b1503112ce
Initial trivial unittest of Filtering object
2015-01-27 15:56:14 +00:00
Paul "LeoNerd" Evans
f9958f3404
Use new V2AlphaRestTestCase
2015-01-27 13:17:25 +00:00
Paul "LeoNerd" Evans
0484d7f6e9
Merge branch 'develop' into client_v2_filter
2015-01-27 13:11:03 +00:00
Paul "LeoNerd" Evans
57d2bfca3f
Initial cut of a shared base class for REST unit tests
2015-01-27 13:09:57 +00:00
Paul "LeoNerd" Evans
39c1892b22
Minor changes to v2_alpha filter REST test to allow the setUp method to be shareable
2015-01-27 13:03:31 +00:00
Paul "LeoNerd" Evans
37b8a71f10
Initial trivial REST test of v2_alpha filter API
2015-01-26 15:27:40 +00:00
Paul "LeoNerd" Evans
d07dfe5392
Create (empty) v2_alpha REST tests directory
2015-01-26 14:32:17 +00:00
Erik Johnston
9b1e552b51
Merge branch 'develop' of github.com:matrix-org/synapse into rejections
2015-01-23 15:51:48 +00:00
Mark Haines
ada711504e
Replace hs.parse_roomalias with RoomAlias.from_string
2015-01-23 13:21:58 +00:00