synapse/tests/handlers
Eric Eastwood 40bb37eb27
Stop getting missing prev_events after we already know their signature is invalid (#13816)
While https://github.com/matrix-org/synapse/pull/13635 stops us from doing the slow thing after we've already done it once, this PR stops us from doing one of the slow things in the first place.

Related to
 - https://github.com/matrix-org/synapse/issues/13622
    - https://github.com/matrix-org/synapse/pull/13635
 - https://github.com/matrix-org/synapse/issues/13676

Part of https://github.com/matrix-org/synapse/issues/13356

Follow-up to https://github.com/matrix-org/synapse/pull/13815 which tracks event signature failures.

With this PR, we avoid the call to the costly `_get_state_ids_after_missing_prev_event` because the signature failure will count as an attempt before and we filter events based on the backoff before calling `_get_state_ids_after_missing_prev_event` now.

For example, this will save us 156s out of the 185s total that this `matrix.org` `/messages` request. If you want to see the full Jaeger trace of this, you can drag and drop this `trace.json` into your own Jaeger, https://gist.github.com/MadLittleMods/4b12d0d0afe88c2f65ffcc907306b761

To explain this exact scenario around `/messages` -> backfill, we call `/backfill` and first check the signatures of the 100 events. We see bad signature for `$luA4l7QHhf_jadH3mI-AyFqho0U2Q-IXXUbGSMq6h6M` and `$zuOn2Rd2vsC7SUia3Hp3r6JSkSFKcc5j3QTTqW_0jDw` (both member events). Then we process the 98 events remaining that have valid signatures but one of the events references `$luA4l7QHhf_jadH3mI-AyFqho0U2Q-IXXUbGSMq6h6M` as a `prev_event`. So we have to do the whole `_get_state_ids_after_missing_prev_event` rigmarole which pulls in those same events which fail again because the signatures are still invalid.

 - `backfill`
    - `outgoing-federation-request` `/backfill`
    - `_check_sigs_and_hash_and_fetch`
       - `_check_sigs_and_hash_and_fetch_one` for each event received over backfill
          -  `$luA4l7QHhf_jadH3mI-AyFqho0U2Q-IXXUbGSMq6h6M` fails with `Signature on retrieved event was invalid.`: `unable to verify signature for sender domain xxx: 401: Failed to find any key to satisfy: _FetchKeyRequest(...)`
          -  `$zuOn2Rd2vsC7SUia3Hp3r6JSkSFKcc5j3QTTqW_0jDw` fails with `Signature on retrieved event was invalid.`: `unable to verify signature for sender domain xxx: 401: Failed to find any key to satisfy: _FetchKeyRequest(...)`
   - `_process_pulled_events`
      - `_process_pulled_event` for each validated event
         -  Event `$Q0iMdqtz3IJYfZQU2Xk2WjB5NDF8Gg8cFSYYyKQgKJ0` references `$luA4l7QHhf_jadH3mI-AyFqho0U2Q-IXXUbGSMq6h6M` as a `prev_event` which is missing so we try to get it
            - `_get_state_ids_after_missing_prev_event`
               - `outgoing-federation-request` `/state_ids`
               -  `get_pdu` for `$luA4l7QHhf_jadH3mI-AyFqho0U2Q-IXXUbGSMq6h6M` which fails the signature check again
               -  `get_pdu` for `$zuOn2Rd2vsC7SUia3Hp3r6JSkSFKcc5j3QTTqW_0jDw` which fails the signature check
2022-10-15 00:36:49 -05:00
..
__init__.py Reference Matrix Home Server 2014-08-12 15:10:52 +01:00
oidc_test_key.p8 JWT OIDC secrets for Sign in with Apple (#9549) 2021-03-09 15:03:37 +00:00
oidc_test_key.pub.pem JWT OIDC secrets for Sign in with Apple (#9549) 2021-03-09 15:03:37 +00:00
test_admin.py Add some type hints to the tests.handlers module. (#12207) 2022-03-11 07:07:15 -05:00
test_appservice.py Fix a bug where redactions were not being sent over federation if we did not have the original event. (#13813) 2022-10-11 11:18:45 -07:00
test_auth.py Decouple synapse.api.auth_blocking.AuthBlocking from synapse.api.auth.Auth. (#13021) 2022-06-14 09:51:15 +01:00
test_cas.py Use getClientAddress instead of getClientIP. (#12599) 2022-05-04 14:11:21 -04:00
test_deactivate_account.py Port the push rule classes to Rust. (#13768) 2022-09-20 12:10:31 +01:00
test_device.py Consolidate the logic of delete_device/delete_devices. (#12970) 2022-06-07 07:43:35 -04:00
test_directory.py Add missing type hints for tests.unittest. (#13397) 2022-07-27 17:18:41 +00:00
test_e2e_keys.py Fix bug in device list caching when remote users leave rooms (#13749) 2022-09-14 10:42:57 +01:00
test_e2e_room_keys.py Remove redundant "coding: utf-8" lines (#9786) 2021-04-14 15:34:27 +01:00
test_federation.py Faster Room Joins: don't leave a stuck room partial state flag if the join fails. (#13403) 2022-08-01 16:45:39 +00:00
test_federation_event.py Stop getting missing prev_events after we already know their signature is invalid (#13816) 2022-10-15 00:36:49 -05:00
test_message.py Persist CreateRoom events to DB in a batch (#13800) 2022-09-28 10:11:48 +00:00
test_oidc.py Move the "email unsubscribe" resource, refactor the macaroon generator & simplify the access token verification logic. (#12986) 2022-06-14 09:12:08 -04:00
test_password_providers.py Register homeserver modules when creating test homeserver (#13558) 2022-08-19 16:52:20 +01:00
test_presence.py Prevent a sync request from removing a user's busy presence status (#12213) 2022-04-13 16:21:07 +01:00
test_profile.py Remove redundant get_success calls in test code (#12346) 2022-04-01 16:10:31 +01:00
test_receipts.py Remove support for unstable private read receipts (#13653) 2022-09-01 13:31:54 +01:00
test_register.py Persist CreateRoom events to DB in a batch (#13800) 2022-09-28 10:11:48 +00:00
test_room.py Pull out encrypted_by_default tests from user_directory tests (#10752) 2021-09-06 11:37:54 +01:00
test_room_member.py Fix that user cannot /forget rooms after the last member has left (#13546) 2022-08-30 09:58:38 +00:00
test_room_summary.py Stop depending on room_id to be returned for children state in the hierarchy response. (#12991) 2022-06-10 07:15:51 -04:00
test_saml.py Use getClientAddress instead of getClientIP. (#12599) 2022-05-04 14:11:21 -04:00
test_send_email.py Support Implicit TLS for sending emails (#13317) 2022-07-25 16:27:19 +01:00
test_stats.py Replace noop background updates with DELETE. (#12954) 2022-06-13 14:06:27 -04:00
test_sync.py Optimise get_rooms_for_user (drop with_stream_ordering) (#13787) 2022-09-29 13:55:12 +00:00
test_typing.py typing: check origin server of typing event against room's servers (#13830) 2022-09-26 17:33:32 +02:00
test_user_directory.py Remove remaining bits of groups code. (#12936) 2022-06-01 09:41:25 -04:00