synapse/tests/storage
Erik Johnston f112cfe5bb
Fix MultiWriteIdGenerator's handling of restarts. (#8374)
On startup `MultiWriteIdGenerator` fetches the maximum stream ID for
each instance from the table and uses that as its initial "current
position" for each writer. This is problematic as a) it involves either
a scan of events table or an index (neither of which is ideal), and b)
if rows are being persisted out of order elsewhere while the process
restarts then using the maximum stream ID is not correct. This could
theoretically lead to race conditions where e.g. events that are
persisted out of order are not sent down sync streams.

We fix this by creating a new table that tracks the current positions of
each writer to the stream, and update it each time we finish persisting
a new entry. This is a relatively small overhead when persisting events.
However for the cache invalidation stream this is a much bigger relative
overhead, so instead we note that for invalidation we don't actually
care about reliability over restarts (as there's no caches to
invalidate) and simply don't bother reading and writing to the new table
in that particular case.
2020-09-24 16:53:51 +01:00
..
__init__.py Reference Matrix Home Server 2014-08-12 15:10:52 +01:00
test__base.py Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
test_appservice.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
test_background_update.py Do not yield on awaitables in tests. (#8193) 2020-08-27 17:24:46 -04:00
test_base.py Convert additional database methods to async (select list, search, insert_many, delete_*) (#8168) 2020-08-27 07:41:01 -04:00
test_cleanup_extrems.py Reduce run-times of tests by advancing the reactor less (#7757) 2020-08-27 11:39:53 +01:00
test_client_ips.py Allow for make_awaitable's return value to be re-used. (#8261) 2020-09-08 07:26:55 -04:00
test_database.py bg update to clear out duplicate outbound_device_list_pokes (#7193) 2020-04-07 23:06:39 +01:00
test_devices.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
test_directory.py Convert simple_update* and simple_select* to async (#8173) 2020-08-27 07:08:38 -04:00
test_e2e_room_keys.py add etag and count to key backup endpoints (#5858) 2019-11-27 16:14:44 -05:00
test_end_to_end_keys.py Rename get_e2e_device_keys to better reflect its purpose (#8205) 2020-08-29 00:14:17 +01:00
test_event_federation.py Rename database classes to make some sense (#8033) 2020-08-05 21:38:57 +01:00
test_event_metrics.py Add a shadow-banned flag to users. (#8092) 2020-08-14 12:37:59 -04:00
test_event_push_actions.py Re-implement unread counts (again) (#8059) 2020-09-02 17:19:37 +01:00
test_id_generators.py Fix MultiWriteIdGenerator's handling of restarts. (#8374) 2020-09-24 16:53:51 +01:00
test_keys.py Type defintions for use in refactoring for redaction changes (#6803) 2020-01-30 11:25:59 +00:00
test_main.py Do not yield on awaitables in tests. (#8193) 2020-08-27 17:24:46 -04:00
test_monthly_active_users.py Allow for make_awaitable's return value to be re-used. (#8261) 2020-09-08 07:26:55 -04:00
test_profile.py Convert simple_select_one and simple_select_one_onecol to async (#8162) 2020-08-26 07:19:32 -04:00
test_purge.py Convert stream database to async/await. (#8074) 2020-08-17 07:24:46 -04:00
test_redaction.py Add an assertion on prev_events in create_new_client_event (#8041) 2020-08-10 12:29:47 +01:00
test_registration.py Do not yield on awaitables in tests. (#8193) 2020-08-27 17:24:46 -04:00
test_room.py Convert simple_select_one and simple_select_one_onecol to async (#8162) 2020-08-26 07:19:32 -04:00
test_roommember.py Reduce run-times of tests by advancing the reactor less (#7757) 2020-08-27 11:39:53 +01:00
test_state.py Convert some of the data store to async. (#7976) 2020-07-30 07:20:41 -04:00
test_transactions.py Fix errors storing large retry intervals. 2019-10-02 10:36:27 +01:00
test_user_directory.py Do not yield on awaitables in tests. (#8193) 2020-08-27 17:24:46 -04:00