diff --git a/develop/admin_api/purge_room.html b/develop/admin_api/purge_room.html deleted file mode 100644 index 26fba8ff11..0000000000 --- a/develop/admin_api/purge_room.html +++ /dev/null @@ -1,268 +0,0 @@ - - - - - - Purge Rooms - Synapse - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- -
- - - - - - - - - - - -
-
- -
- -
- -

Deprecated: Purge room API

-

The old Purge room API is deprecated and will be removed in a future release. -See the new Delete Room API for more details.

-

This API will remove all trace of a room from your database.

-

All local users must have left the room before it can be removed.

-

The API is:

-
POST /_synapse/admin/v1/purge_room
-
-{
-    "room_id": "!room:id"
-}
-
-

You must authenticate using the access token of an admin user.

- -
- - -
-
- - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/develop/admin_api/shutdown_room.html b/develop/admin_api/shutdown_room.html deleted file mode 100644 index ac030e876d..0000000000 --- a/develop/admin_api/shutdown_room.html +++ /dev/null @@ -1,343 +0,0 @@ - - - - - - Shutdown Room - Synapse - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- -
- - - - - - - - - - - -
-
- -
- -
- -

Deprecated: Shutdown room API

-

The old Shutdown room API is deprecated and will be removed in a future release. -See the new Delete Room API for more details.

-

Shuts down a room, preventing new joins and moves local users and room aliases automatically -to a new room. The new room will be created with the user specified by the -new_room_user_id parameter as room administrator and will contain a message -explaining what happened. Users invited to the new room will have power level --10 by default, and thus be unable to speak. The old room's power levels will be changed to -disallow any further invites or joins.

-

The local server will only have the power to move local user and room aliases to -the new room. Users on other servers will be unaffected.

-

API

-

You will need to authenticate with an access token for an admin user.

-

URL

-

POST /_synapse/admin/v1/shutdown_room/{room_id}

-

URL Parameters

-
    -
  • room_id - The ID of the room (e.g !someroom:example.com)
  • -
-

JSON Body Parameters

-
    -
  • new_room_user_id - Required. A string representing the user ID of the user that will admin -the new room that all users in the old room will be moved to.
  • -
  • room_name - Optional. A string representing the name of the room that new users will be -invited to.
  • -
  • message - Optional. A string containing the first message that will be sent as -new_room_user_id in the new room. Ideally this will clearly convey why the -original room was shut down.
  • -
-

If not specified, the default value of room_name is "Content Violation -Notification". The default value of message is "Sharing illegal content on -othis server is not permitted and rooms in violation will be blocked."

-

Response Parameters

-
    -
  • kicked_users - An integer number representing the number of users that -were kicked.
  • -
  • failed_to_kick_users - An integer number representing the number of users -that were not kicked.
  • -
  • local_aliases - An array of strings representing the local aliases that were migrated from -the old room to the new.
  • -
  • new_room_id - A string representing the room ID of the new room.
  • -
-

Example

-

Request:

-
POST /_synapse/admin/v1/shutdown_room/!somebadroom%3Aexample.com
-
-{
-    "new_room_user_id": "@someuser:example.com",
-    "room_name": "Content Violation Notification",
-    "message": "Bad Room has been shutdown due to content violations on this server. Please review our Terms of Service."
-}
-
-

Response:

-
{
-    "kicked_users": 5,
-    "failed_to_kick_users": 0,
-    "local_aliases": ["#badroom:example.com", "#evilsaloon:example.com],
-    "new_room_id": "!newroomid:example.com",
-},
-
-

Undoing room shutdowns

-

Note: This guide may be outdated by the time you read it. By nature of room shutdowns being performed at the database level, -the structure can and does change without notice.

-

First, it's important to understand that a room shutdown is very destructive. Undoing a shutdown is not as simple as pretending it -never happened - work has to be done to move forward instead of resetting the past. In fact, in some cases it might not be possible -to recover at all:

-
    -
  • If the room was invite-only, your users will need to be re-invited.
  • -
  • If the room no longer has any members at all, it'll be impossible to rejoin.
  • -
  • The first user to rejoin will have to do so via an alias on a different server.
  • -
-

With all that being said, if you still want to try and recover the room:

-
    -
  1. For safety reasons, shut down Synapse.
  2. -
  3. In the database, run DELETE FROM blocked_rooms WHERE room_id = '!example:example.org'; -
      -
    • For caution: it's recommended to run this in a transaction: BEGIN; DELETE ...;, verify you got 1 result, then COMMIT;.
    • -
    • The room ID is the same one supplied to the shutdown room API, not the Content Violation room.
    • -
    -
  4. -
  5. Restart Synapse.
  6. -
-

You will have to manually handle, if you so choose, the following:

-
    -
  • Aliases that would have been redirected to the Content Violation room.
  • -
  • Users that would have been booted from the room (and will have been force-joined to the Content Violation room).
  • -
  • Removal of the Content Violation room if desired.
  • -
- -
- - -
-
- - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/develop/dev/cas.html b/develop/dev/cas.html deleted file mode 100644 index f70e1081e9..0000000000 --- a/develop/dev/cas.html +++ /dev/null @@ -1,317 +0,0 @@ - - - - - - CAS - Synapse - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- -
- - - - - - - - - - - -
-
- -
- -
- -

How to test CAS as a developer without a server

-

The django-mama-cas project is an -easy to run CAS implementation built on top of Django.

-

Prerequisites

-
    -
  1. Create a new virtualenv: python3 -m venv <your virtualenv>
  2. -
  3. Activate your virtualenv: source /path/to/your/virtualenv/bin/activate
  4. -
  5. Install Django and django-mama-cas: -
    python -m pip install "django<3" "django-mama-cas==2.4.0"
    -
    -
  6. -
  7. Create a Django project in the current directory: -
    django-admin startproject cas_test .
    -
    -
  8. -
  9. Follow the install directions for django-mama-cas
  10. -
  11. Setup the SQLite database: python manage.py migrate
  12. -
  13. Create a user: -
    python manage.py createsuperuser
    -
    -
      -
    1. Use whatever you want as the username and password.
    2. -
    3. Leave the other fields blank.
    4. -
    -
  14. -
  15. Use the built-in Django test server to serve the CAS endpoints on port 8000: -
    python manage.py runserver
    -
    -
  16. -
-

You should now have a Django project configured to serve CAS authentication with -a single user created.

-

Configure Synapse (and Element) to use CAS

-
    -
  1. Modify your homeserver.yaml to enable CAS and point it to your locally -running Django test server: -
    cas_config:
    -  enabled: true
    -  server_url: "http://localhost:8000"
    -  service_url: "http://localhost:8081"
    -  #displayname_attribute: name
    -  #required_attributes:
    -  #    name: value
    -
    -
  2. -
  3. Restart Synapse.
  4. -
-

Note that the above configuration assumes the homeserver is running on port 8081 -and that the CAS server is on port 8000, both on localhost.

-

Testing the configuration

-

Then in Element:

-
    -
  1. Visit the login page with a Element pointing at your homeserver.
  2. -
  3. Click the Single Sign-On button.
  4. -
  5. Login using the credentials created with createsuperuser.
  6. -
  7. You should be logged in.
  8. -
-

If you want to repeat this process you'll need to manually logout first:

-
    -
  1. http://localhost:8000/admin/
  2. -
  3. Click "logout" in the top right.
  4. -
- -
- - -
-
- - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/develop/dev/git.html b/develop/dev/git.html deleted file mode 100644 index 8c3c5bf7bb..0000000000 --- a/develop/dev/git.html +++ /dev/null @@ -1,376 +0,0 @@ - - - - - - Git Usage - Synapse - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- -
- - - - - - - - - - - -
-
- -
- -
- -

Some notes on how we use git

-

On keeping the commit history clean

-

In an ideal world, our git commit history would be a linear progression of -commits each of which contains a single change building on what came -before. Here, by way of an arbitrary example, is the top of git log --graph b2dba0607:

-clean git graph -

Note how the commit comment explains clearly what is changing and why. Also -note the absence of merge commits, as well as the absence of commits called -things like (to pick a few culprits): -“pep8”, “fix broken -test”, -“oops”, -“typo”, or “Who's -the president?”.

-

There are a number of reasons why keeping a clean commit history is a good -thing:

-
    -
  • -

    From time to time, after a change lands, it turns out to be necessary to -revert it, or to backport it to a release branch. Those operations are -much easier when the change is contained in a single commit.

    -
  • -
  • -

    Similarly, it's much easier to answer questions like “is the fix for -/publicRooms on the release branch?” if that change consists of a single -commit.

    -
  • -
  • -

    Likewise: “what has changed on this branch in the last week?” is much -clearer without merges and “pep8” commits everywhere.

    -
  • -
  • -

    Sometimes we need to figure out where a bug got introduced, or some -behaviour changed. One way of doing that is with git bisect: pick an -arbitrary commit between the known good point and the known bad point, and -see how the code behaves. However, that strategy fails if the commit you -chose is the middle of someone's epic branch in which they broke the world -before putting it back together again.

    -
  • -
-

One counterargument is that it is sometimes useful to see how a PR evolved as -it went through review cycles. This is true, but that information is always -available via the GitHub UI (or via the little-known refs/pull -namespace).

-

Of course, in reality, things are more complicated than that. We have release -branches as well as develop and master, and we deliberately merge changes -between them. Bugs often slip through and have to be fixed later. That's all -fine: this not a cast-iron rule which must be obeyed, but an ideal to aim -towards.

-

Merges, squashes, rebases: wtf?

-

Ok, so that's what we'd like to achieve. How do we achieve it?

-

The TL;DR is: when you come to merge a pull request, you probably want to -“squash and merge”:

-

squash and merge.

-

(This applies whether you are merging your own PR, or that of another -contributor.)

-

“Squash and merge”1 takes all of the changes in the -PR, and bundles them into a single commit. GitHub gives you the opportunity to -edit the commit message before you confirm, and normally you should do so, -because the default will be useless (again: * woops typo is not a useful -thing to keep in the historical record).

-

The main problem with this approach comes when you have a series of pull -requests which build on top of one another: as soon as you squash-merge the -first PR, you'll end up with a stack of conflicts to resolve in all of the -others. In general, it's best to avoid this situation in the first place by -trying not to have multiple related PRs in flight at the same time. Still, -sometimes that's not possible and doing a regular merge is the lesser evil.

-

Another occasion in which a regular merge makes more sense is a PR where you've -deliberately created a series of commits each of which makes sense in its own -right. For example: a PR which gradually propagates a refactoring operation -through the codebase, or a -PR which is the culmination of several other -PRs. In this case the ability -to figure out when a particular change/bug was introduced could be very useful.

-

Ultimately: this is not a hard-and-fast-rule. If in doubt, ask yourself “do -each of the commits I am about to merge make sense in their own right”, but -remember that we're just doing our best to balance “keeping the commit history -clean” with other factors.

-

Git branching model

-

A lot -of -words have been -written in the past about git branching models (no really, a -lot). I tend to -think the whole thing is overblown. Fundamentally, it's not that -complicated. Here's how we do it.

-

Let's start with a picture:

-

branching model

-

It looks complicated, but it's really not. There's one basic rule: anyone is -free to merge from any more-stable branch to any less-stable branch at -any time2. (The principle behind this is that if a -change is good enough for the more-stable branch, then it's also good enough go -put in a less-stable branch.)

-

Meanwhile, merging (or squashing, as per the above) from a less-stable to a -more-stable branch is a deliberate action in which you want to publish a change -or a set of changes to (some subset of) the world: for example, this happens -when a PR is landed, or as part of our release process.

-

So, what counts as a more- or less-stable branch? A little reflection will show -that our active branches are ordered thus, from more-stable to less-stable:

-
    -
  • master (tracks our last release).
  • -
  • release-vX.Y (the branch where we prepare the next release)3.
  • -
  • PR branches which are targeting the release.
  • -
  • develop (our "mainline" branch containing our bleeding-edge).
  • -
  • regular PR branches.
  • -
-

The corollary is: if you have a bugfix that needs to land in both -release-vX.Y and develop, then you should base your PR on -release-vX.Y, get it merged there, and then merge from release-vX.Y to -develop. (If a fix lands in develop and we later need it in a -release-branch, we can of course cherry-pick it, but landing it in the release -branch first helps reduce the chance of annoying conflicts.)

-
-

[1]: “Squash and merge” is GitHub's term for this -operation. Given that there is no merge involved, I'm not convinced it's the -most intuitive name. ^

-

[2]: Well, anyone with commit access.^

-

[3]: Very, very occasionally (I think this has happened once in -the history of Synapse), we've had two releases in flight at once. Obviously, -release-v1.2 is more-stable than release-v1.3. ^

- -
- - -
-
- - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/develop/dev/git/branches.jpg b/develop/dev/git/branches.jpg deleted file mode 100644 index 715ecc8cd0..0000000000 Binary files a/develop/dev/git/branches.jpg and /dev/null differ diff --git a/develop/dev/git/clean.png b/develop/dev/git/clean.png deleted file mode 100644 index 3accd7ccef..0000000000 Binary files a/develop/dev/git/clean.png and /dev/null differ diff --git a/develop/dev/git/squash.png b/develop/dev/git/squash.png deleted file mode 100644 index 234caca3e4..0000000000 Binary files a/develop/dev/git/squash.png and /dev/null differ diff --git a/develop/dev/saml.html b/develop/dev/saml.html deleted file mode 100644 index 749b12817b..0000000000 --- a/develop/dev/saml.html +++ /dev/null @@ -1,294 +0,0 @@ - - - - - - SAML - Synapse - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- -
- - - - - - - - - - - -
-
- -
- -
- -

How to test SAML as a developer without a server

-

https://capriza.github.io/samling/samling.html (https://github.com/capriza/samling) is a great -resource for being able to tinker with the SAML options within Synapse without needing to -deploy and configure a complicated software stack.

-

To make Synapse (and therefore Riot) use it:

-
    -
  1. Use the samling.html URL above or deploy your own and visit the IdP Metadata tab.
  2. -
  3. Copy the XML to your clipboard.
  4. -
  5. On your Synapse server, create a new file samling.xml next to your homeserver.yaml with -the XML from step 2 as the contents.
  6. -
  7. Edit your homeserver.yaml to include: -
    saml2_config:
    -  sp_config:
    -    allow_unknown_attributes: true  # Works around a bug with AVA Hashes: https://github.com/IdentityPython/pysaml2/issues/388
    -    metadata:
    -      local: ["samling.xml"]   
    -
    -
  8. -
  9. Ensure that your homeserver.yaml has a setting for public_baseurl: -
    public_baseurl: http://localhost:8080/
    -
    -
  10. -
  11. Run apt-get install xmlsec1 and pip install --upgrade --force 'pysaml2>=4.5.0' to ensure -the dependencies are installed and ready to go.
  12. -
  13. Restart Synapse.
  14. -
-

Then in Riot:

-
    -
  1. Visit the login page with a Riot pointing at your homeserver.
  2. -
  3. Click the Single Sign-On button.
  4. -
  5. On the samling page, enter a Name Identifier and add a SAML Attribute for uid=your_localpart. -The response must also be signed.
  6. -
  7. Click "Next".
  8. -
  9. Click "Post Response" (change nothing).
  10. -
  11. You should be logged in.
  12. -
-

If you try and repeat this process, you may be automatically logged in using the information you -gave previously. To fix this, open your developer console (F12 or Ctrl+Shift+I) while on the -samling page and clear the site data. In Chrome, this will be a button on the Application tab.

- -
- - -
-
- - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/develop/modules.html b/develop/modules.html deleted file mode 100644 index 1f1de14915..0000000000 --- a/develop/modules.html +++ /dev/null @@ -1,555 +0,0 @@ - - - - - - Pluggable Modules - Synapse - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- -
- - - - - - - - - - - -
-
- -
- -
- -

Modules

-

Synapse supports extending its functionality by configuring external modules.

-

Using modules

-

To use a module on Synapse, add it to the modules section of the configuration file:

-
modules:
-  - module: my_super_module.MySuperClass
-    config:
-      do_thing: true
-  - module: my_other_super_module.SomeClass
-    config: {}
-
-

Each module is defined by a path to a Python class as well as a configuration. This -information for a given module should be available in the module's own documentation.

-

Note: When using third-party modules, you effectively allow someone else to run -custom code on your Synapse homeserver. Server admins are encouraged to verify the -provenance of the modules they use on their homeserver and make sure the modules aren't -running malicious code on their instance.

-

Also note that we are currently in the process of migrating module interfaces to this -system. While some interfaces might be compatible with it, others still require -configuring modules in another part of Synapse's configuration file. Currently, only the -spam checker interface is compatible with this new system.

-

Writing a module

-

A module is a Python class that uses Synapse's module API to interact with the -homeserver. It can register callbacks that Synapse will call on specific operations, as -well as web resources to attach to Synapse's web server.

-

When instantiated, a module is given its parsed configuration as well as an instance of -the synapse.module_api.ModuleApi class. The configuration is a dictionary, and is -either the output of the module's parse_config static method (see below), or the -configuration associated with the module in Synapse's configuration file.

-

See the documentation for the ModuleApi class -here.

-

Handling the module's configuration

-

A module can implement the following static method:

-
@staticmethod
-def parse_config(config: dict) -> dict
-
-

This method is given a dictionary resulting from parsing the YAML configuration for the -module. It may modify it (for example by parsing durations expressed as strings (e.g. -"5d") into milliseconds, etc.), and return the modified dictionary. It may also verify -that the configuration is correct, and raise an instance of -synapse.module_api.errors.ConfigError if not.

-

Registering a web resource

-

Modules can register web resources onto Synapse's web server using the following module -API method:

-
def ModuleApi.register_web_resource(path: str, resource: IResource) -> None
-
-

The path is the full absolute path to register the resource at. For example, if you -register a resource for the path /_synapse/client/my_super_module/say_hello, Synapse -will serve it at http(s)://[HS_URL]/_synapse/client/my_super_module/say_hello. Note -that Synapse does not allow registering resources for several sub-paths in the /_matrix -namespace (such as anything under /_matrix/client for example). It is strongly -recommended that modules register their web resources under the /_synapse/client -namespace.

-

The provided resource is a Python class that implements Twisted's IResource -interface (such as Resource).

-

Only one resource can be registered for a given path. If several modules attempt to -register a resource for the same path, the module that appears first in Synapse's -configuration file takes priority.

-

Modules must register their web resources in their __init__ method.

-

Registering a callback

-

Modules can use Synapse's module API to register callbacks. Callbacks are functions that -Synapse will call when performing specific actions. Callbacks must be asynchronous, and -are split in categories. A single module may implement callbacks from multiple categories, -and is under no obligation to implement all callbacks from the categories it registers -callbacks for.

-

Modules can register callbacks using one of the module API's register_[...]_callbacks -methods. The callback functions are passed to these methods as keyword arguments, with -the callback name as the argument name and the function as its value. This is demonstrated -in the example below. A register_[...]_callbacks method exists for each module type -documented in this section.

-

Spam checker callbacks

-

Spam checker callbacks allow module developers to implement spam mitigation actions for -Synapse instances. Spam checker callbacks can be registered using the module API's -register_spam_checker_callbacks method.

-

The available spam checker callbacks are:

-
async def check_event_for_spam(event: "synapse.events.EventBase") -> Union[bool, str]
-
-

Called when receiving an event from a client or via federation. The module can return -either a bool to indicate whether the event must be rejected because of spam, or a str -to indicate the event must be rejected because of spam and to give a rejection reason to -forward to clients.

-
async def user_may_invite(inviter: str, invitee: str, room_id: str) -> bool
-
-

Called when processing an invitation. The module must return a bool indicating whether -the inviter can invite the invitee to the given room. Both inviter and invitee are -represented by their Matrix user ID (e.g. @alice:example.com).

-
async def user_may_create_room(user: str) -> bool
-
-

Called when processing a room creation request. The module must return a bool indicating -whether the given user (represented by their Matrix user ID) is allowed to create a room.

-
async def user_may_create_room_alias(user: str, room_alias: "synapse.types.RoomAlias") -> bool
-
-

Called when trying to associate an alias with an existing room. The module must return a -bool indicating whether the given user (represented by their Matrix user ID) is allowed -to set the given alias.

-
async def user_may_publish_room(user: str, room_id: str) -> bool
-
-

Called when trying to publish a room to the homeserver's public rooms directory. The -module must return a bool indicating whether the given user (represented by their -Matrix user ID) is allowed to publish the given room.

-
async def check_username_for_spam(user_profile: Dict[str, str]) -> bool
-
-

Called when computing search results in the user directory. The module must return a -bool indicating whether the given user profile can appear in search results. The profile -is represented as a dictionary with the following keys:

-
    -
  • user_id: The Matrix ID for this user.
  • -
  • display_name: The user's display name.
  • -
  • avatar_url: The mxc:// URL to the user's avatar.
  • -
-

The module is given a copy of the original dictionary, so modifying it from within the -module cannot modify a user's profile when included in user directory search results.

-
async def check_registration_for_spam(
-    email_threepid: Optional[dict],
-    username: Optional[str],
-    request_info: Collection[Tuple[str, str]],
-    auth_provider_id: Optional[str] = None,
-) -> "synapse.spam_checker_api.RegistrationBehaviour"
-
-

Called when registering a new user. The module must return a RegistrationBehaviour -indicating whether the registration can go through or must be denied, or whether the user -may be allowed to register but will be shadow banned.

-

The arguments passed to this callback are:

-
    -
  • email_threepid: The email address used for registering, if any.
  • -
  • username: The username the user would like to register. Can be None, meaning that -Synapse will generate one later.
  • -
  • request_info: A collection of tuples, which first item is a user agent, and which -second item is an IP address. These user agents and IP addresses are the ones that were -used during the registration process.
  • -
  • auth_provider_id: The identifier of the SSO authentication provider, if any.
  • -
-
async def check_media_file_for_spam(
-    file_wrapper: "synapse.rest.media.v1.media_storage.ReadableFileWrapper",
-    file_info: "synapse.rest.media.v1._base.FileInfo",
-) -> bool
-
-

Called when storing a local or remote file. The module must return a boolean indicating -whether the given file can be stored in the homeserver's media store.

-

Account validity callbacks

-

Account validity callbacks allow module developers to add extra steps to verify the -validity on an account, i.e. see if a user can be granted access to their account on the -Synapse instance. Account validity callbacks can be registered using the module API's -register_account_validity_callbacks method.

-

The available account validity callbacks are:

-
async def is_user_expired(user: str) -> Optional[bool]
-
-

Called when processing any authenticated request (except for logout requests). The module -can return a bool to indicate whether the user has expired and should be locked out of -their account, or None if the module wasn't able to figure it out. The user is -represented by their Matrix user ID (e.g. @alice:example.com).

-

If the module returns True, the current request will be denied with the error code -ORG_MATRIX_EXPIRED_ACCOUNT and the HTTP status code 403. Note that this doesn't -invalidate the user's access token.

-
async def on_user_registration(user: str) -> None
-
-

Called after successfully registering a user, in case the module needs to perform extra -operations to keep track of them. (e.g. add them to a database table). The user is -represented by their Matrix user ID.

-

Third party rules callbacks

-

Third party rules callbacks allow module developers to add extra checks to verify the -validity of incoming events. Third party event rules callbacks can be registered using -the module API's register_third_party_rules_callbacks method.

-

The available third party rules callbacks are:

-
async def check_event_allowed(
-    event: "synapse.events.EventBase",
-    state_events: "synapse.types.StateMap",
-) -> Tuple[bool, Optional[dict]]
-
-

-This callback is very experimental and can and will break without notice. Module developers -are encouraged to implement check_event_for_spam from the spam checker category instead. -

-

Called when processing any incoming event, with the event and a StateMap -representing the current state of the room the event is being sent into. A StateMap is -a dictionary that maps tuples containing an event type and a state key to the -corresponding state event. For example retrieving the room's m.room.create event from -the state_events argument would look like this: state_events.get(("m.room.create", "")). -The module must return a boolean indicating whether the event can be allowed.

-

Note that this callback function processes incoming events coming via federation -traffic (on top of client traffic). This means denying an event might cause the local -copy of the room's history to diverge from that of remote servers. This may cause -federation issues in the room. It is strongly recommended to only deny events using this -callback function if the sender is a local user, or in a private federation in which all -servers are using the same module, with the same configuration.

-

If the boolean returned by the module is True, it may also tell Synapse to replace the -event with new data by returning the new event's data as a dictionary. In order to do -that, it is recommended the module calls event.get_dict() to get the current event as a -dictionary, and modify the returned dictionary accordingly.

-

Note that replacing the event only works for events sent by local users, not for events -received over federation.

-
async def on_create_room(
-    requester: "synapse.types.Requester",
-    request_content: dict,
-    is_requester_admin: bool,
-) -> None
-
-

Called when processing a room creation request, with the Requester object for the user -performing the request, a dictionary representing the room creation request's JSON body -(see the spec -for a list of possible parameters), and a boolean indicating whether the user performing -the request is a server admin.

-

Modules can modify the request_content (by e.g. adding events to its initial_state), -or deny the room's creation by raising a module_api.errors.SynapseError.

-

Presence router callbacks

-

Presence router callbacks allow module developers to specify additional users (local or remote) -to receive certain presence updates from local users. Presence router callbacks can be -registered using the module API's register_presence_router_callbacks method.

-

The available presence router callbacks are:

-
async def get_users_for_states(
-    self,
-    state_updates: Iterable["synapse.api.UserPresenceState"],
-) -> Dict[str, Set["synapse.api.UserPresenceState"]]:
-
-

Requires get_interested_users to also be registered

-

Called when processing updates to the presence state of one or more users. This callback can -be used to instruct the server to forward that presence state to specific users. The module -must return a dictionary that maps from Matrix user IDs (which can be local or remote) to the -UserPresenceState changes that they should be forwarded.

-

Synapse will then attempt to send the specified presence updates to each user when possible.

-
async def get_interested_users(
-        self, 
-        user_id: str
-) -> Union[Set[str], "synapse.module_api.PRESENCE_ALL_USERS"]
-
-

Requires get_users_for_states to also be registered

-

Called when determining which users someone should be able to see the presence state of. This -callback should return complementary results to get_users_for_state or the presence information -may not be properly forwarded.

-

The callback is given the Matrix user ID for a local user that is requesting presence data and -should return the Matrix user IDs of the users whose presence state they are allowed to -query. The returned users can be local or remote.

-

Alternatively the callback can return synapse.module_api.PRESENCE_ALL_USERS -to indicate that the user should receive updates from all known users.

-

For example, if the user @alice:example.org is passed to this method, and the Set -{"@bob:example.com", "@charlie:somewhere.org"} is returned, this signifies that Alice -should receive presence updates sent by Bob and Charlie, regardless of whether these users -share a room.

-

Porting an existing module that uses the old interface

-

In order to port a module that uses Synapse's old module interface, its author needs to:

-
    -
  • ensure the module's callbacks are all asynchronous.
  • -
  • register their callbacks using one or more of the register_[...]_callbacks methods -from the ModuleApi class in the module's __init__ method (see this section -for more info).
  • -
-

Additionally, if the module is packaged with an additional web resource, the module -should register this resource in its __init__ method using the register_web_resource -method from the ModuleApi class (see this section for -more info).

-

The module's author should also update any example in the module's configuration to only -use the new modules section in Synapse's configuration file (see this section -for more info).

-

Example

-

The example below is a module that implements the spam checker callback -user_may_create_room to deny room creation to user @evilguy:example.com, and registers -a web resource to the path /_synapse/client/demo/hello that returns a JSON object.

-
import json
-
-from twisted.web.resource import Resource
-from twisted.web.server import Request
-
-from synapse.module_api import ModuleApi
-
-
-class DemoResource(Resource):
-    def __init__(self, config):
-        super(DemoResource, self).__init__()
-        self.config = config
-
-    def render_GET(self, request: Request):
-        name = request.args.get(b"name")[0]
-        request.setHeader(b"Content-Type", b"application/json")
-        return json.dumps({"hello": name})
-
-
-class DemoModule:
-    def __init__(self, config: dict, api: ModuleApi):
-        self.config = config
-        self.api = api
-
-        self.api.register_web_resource(
-            path="/_synapse/client/demo/hello",
-            resource=DemoResource(self.config),
-        )
-
-        self.api.register_spam_checker_callbacks(
-            user_may_create_room=self.user_may_create_room,
-        )
-
-    @staticmethod
-    def parse_config(config):
-        return config
-
-    async def user_may_create_room(self, user: str) -> bool:
-        if user == "@evilguy:example.com":
-            return False
-
-        return True
-
- -
- - -
-
- - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/develop/presence_router_module.html b/develop/presence_router_module.html deleted file mode 100644 index 2206fd8b75..0000000000 --- a/develop/presence_router_module.html +++ /dev/null @@ -1,461 +0,0 @@ - - - - - - Presence Router - Synapse - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- -
- - - - - - - - - - - -
-
- -
- -
- -

-This page of the Synapse documentation is now deprecated. For up to date -documentation on setting up or writing a presence router module, please see -this page. -

-

Presence Router Module

-

Synapse supports configuring a module that can specify additional users -(local or remote) to should receive certain presence updates from local -users.

-

Note that routing presence via Application Service transactions is not -currently supported.

-

The presence routing module is implemented as a Python class, which will -be imported by the running Synapse.

-

Python Presence Router Class

-

The Python class is instantiated with two objects:

-
    -
  • A configuration object of some type (see below).
  • -
  • An instance of synapse.module_api.ModuleApi.
  • -
-

It then implements methods related to presence routing.

-

Note that one method of ModuleApi that may be useful is:

-
async def ModuleApi.send_local_online_presence_to(users: Iterable[str]) -> None
-
-

which can be given a list of local or remote MXIDs to broadcast known, online user -presence to (for those users that the receiving user is considered interested in). -It does not include state for users who are currently offline, and it can only be -called on workers that support sending federation. Additionally, this method must -only be called from the process that has been configured to write to the -the presence stream. -By default, this is the main process, but another worker can be configured to do -so.

-

Module structure

-

Below is a list of possible methods that can be implemented, and whether they are -required.

-

parse_config

-
def parse_config(config_dict: dict) -> Any
-
-

Required. A static method that is passed a dictionary of config options, and -should return a validated config object. This method is described further in -Configuration.

-

get_users_for_states

-
async def get_users_for_states(
-    self,
-    state_updates: Iterable[UserPresenceState],
-) -> Dict[str, Set[UserPresenceState]]:
-
-

Required. An asynchronous method that is passed an iterable of user presence -state. This method can determine whether a given presence update should be sent to certain -users. It does this by returning a dictionary with keys representing local or remote -Matrix User IDs, and values being a python set -of synapse.handlers.presence.UserPresenceState instances.

-

Synapse will then attempt to send the specified presence updates to each user when -possible.

-

get_interested_users

-
async def get_interested_users(self, user_id: str) -> Union[Set[str], str]
-
-

Required. An asynchronous method that is passed a single Matrix User ID. This -method is expected to return the users that the passed in user may be interested in the -presence of. Returned users may be local or remote. The presence routed as a result of -what this method returns is sent in addition to the updates already sent between users -that share a room together. Presence updates are deduplicated.

-

This method should return a python set of Matrix User IDs, or the object -synapse.events.presence_router.PresenceRouter.ALL_USERS to indicate that the passed -user should receive presence information for all known users.

-

For clarity, if the user @alice:example.org is passed to this method, and the Set -{"@bob:example.com", "@charlie:somewhere.org"} is returned, this signifies that Alice -should receive presence updates sent by Bob and Charlie, regardless of whether these -users share a room.

-

Example

-

Below is an example implementation of a presence router class.

-
from typing import Dict, Iterable, Set, Union
-from synapse.events.presence_router import PresenceRouter
-from synapse.handlers.presence import UserPresenceState
-from synapse.module_api import ModuleApi
-
-class PresenceRouterConfig:
-    def __init__(self):
-        # Config options with their defaults
-        # A list of users to always send all user presence updates to
-        self.always_send_to_users = []  # type: List[str]
-        
-        # A list of users to ignore presence updates for. Does not affect
-        # shared-room presence relationships
-        self.blacklisted_users = []  # type: List[str]
-
-class ExamplePresenceRouter:
-    """An example implementation of synapse.presence_router.PresenceRouter.
-    Supports routing all presence to a configured set of users, or a subset
-    of presence from certain users to members of certain rooms.
-
-    Args:
-        config: A configuration object.
-        module_api: An instance of Synapse's ModuleApi.
-    """
-    def __init__(self, config: PresenceRouterConfig, module_api: ModuleApi):
-        self._config = config
-        self._module_api = module_api
-
-    @staticmethod
-    def parse_config(config_dict: dict) -> PresenceRouterConfig:
-        """Parse a configuration dictionary from the homeserver config, do
-        some validation and return a typed PresenceRouterConfig.
-
-        Args:
-            config_dict: The configuration dictionary.
-
-        Returns:
-            A validated config object.
-        """
-        # Initialise a typed config object
-        config = PresenceRouterConfig()
-        always_send_to_users = config_dict.get("always_send_to_users")
-        blacklisted_users = config_dict.get("blacklisted_users")
-
-        # Do some validation of config options... otherwise raise a
-        # synapse.config.ConfigError.
-        config.always_send_to_users = always_send_to_users
-        config.blacklisted_users = blacklisted_users
-
-        return config
-
-    async def get_users_for_states(
-        self,
-        state_updates: Iterable[UserPresenceState],
-    ) -> Dict[str, Set[UserPresenceState]]:
-        """Given an iterable of user presence updates, determine where each one
-        needs to go. Returned results will not affect presence updates that are
-        sent between users who share a room.
-
-        Args:
-            state_updates: An iterable of user presence state updates.
-
-        Returns:
-          A dictionary of user_id -> set of UserPresenceState that the user should 
-          receive.
-        """
-        destination_users = {}  # type: Dict[str, Set[UserPresenceState]
-
-        # Ignore any updates for blacklisted users
-        desired_updates = set()
-        for update in state_updates:
-            if update.state_key not in self._config.blacklisted_users:
-                desired_updates.add(update)
-
-        # Send all presence updates to specific users
-        for user_id in self._config.always_send_to_users:
-            destination_users[user_id] = desired_updates
-
-        return destination_users
-
-    async def get_interested_users(
-        self,
-        user_id: str,
-    ) -> Union[Set[str], PresenceRouter.ALL_USERS]:
-        """
-        Retrieve a list of users that `user_id` is interested in receiving the
-        presence of. This will be in addition to those they share a room with.
-        Optionally, the object PresenceRouter.ALL_USERS can be returned to indicate
-        that this user should receive all incoming local and remote presence updates.
-
-        Note that this method will only be called for local users.
-
-        Args:
-          user_id: A user requesting presence updates.
-
-        Returns:
-          A set of user IDs to return additional presence updates for, or
-          PresenceRouter.ALL_USERS to return presence updates for all other users.
-        """
-        if user_id in self._config.always_send_to_users:
-            return PresenceRouter.ALL_USERS
-
-        return set()
-
-

A note on get_users_for_states and get_interested_users

-

Both of these methods are effectively two different sides of the same coin. The logic -regarding which users should receive updates for other users should be the same -between them.

-

get_users_for_states is called when presence updates come in from either federation -or local users, and is used to either direct local presence to remote users, or to -wake up the sync streams of local users to collect remote presence.

-

In contrast, get_interested_users is used to determine the users that presence should -be fetched for when a local user is syncing. This presence is then retrieved, before -being fed through get_users_for_states once again, with only the syncing user's -routing information pulled from the resulting dictionary.

-

Their routing logic should thus line up, else you may run into unintended behaviour.

-

Configuration

-

Once you've crafted your module and installed it into the same Python environment as -Synapse, amend your homeserver config file with the following.

-
presence:
-  enabled: true
-
-  presence_router:
-    module: my_module.ExamplePresenceRouter
-    config:
-      # Any configuration options for your module. The below is an example.
-      # of setting options for ExamplePresenceRouter.
-      always_send_to_users: ["@presence_gobbler:example.org"]
-      blacklisted_users:
-        - "@alice:example.com"
-        - "@bob:example.com"
-      ...
-
-

The contents of config will be passed as a Python dictionary to the static -parse_config method of your class. The object returned by this method will -then be passed to the __init__ method of your module as config.

- -
- - -
-
- - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/develop/spam_checker.html b/develop/spam_checker.html deleted file mode 100644 index 1186bdef1e..0000000000 --- a/develop/spam_checker.html +++ /dev/null @@ -1,360 +0,0 @@ - - - - - - Spam Checker - Synapse - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- -
- - - - - - - - - - - -
-
- -
- -
- -

-This page of the Synapse documentation is now deprecated. For up to date -documentation on setting up or writing a spam checker module, please see -this page. -

-

Handling spam in Synapse

-

Synapse has support to customize spam checking behavior. It can plug into a -variety of events and affect how they are presented to users on your homeserver.

-

The spam checking behavior is implemented as a Python class, which must be -able to be imported by the running Synapse.

-

Python spam checker class

-

The Python class is instantiated with two objects:

-
    -
  • Any configuration (see below).
  • -
  • An instance of synapse.module_api.ModuleApi.
  • -
-

It then implements methods which return a boolean to alter behavior in Synapse. -All the methods must be defined.

-

There's a generic method for checking every event (check_event_for_spam), as -well as some specific methods:

-
    -
  • user_may_invite
  • -
  • user_may_create_room
  • -
  • user_may_create_room_alias
  • -
  • user_may_publish_room
  • -
  • check_username_for_spam
  • -
  • check_registration_for_spam
  • -
  • check_media_file_for_spam
  • -
-

The details of each of these methods (as well as their inputs and outputs) -are documented in the synapse.events.spamcheck.SpamChecker class.

-

The ModuleApi class provides a way for the custom spam checker class to -call back into the homeserver internals.

-

Additionally, a parse_config method is mandatory and receives the plugin config -dictionary. After parsing, It must return an object which will be -passed to __init__ later.

-

Example

-
from synapse.spam_checker_api import RegistrationBehaviour
-
-class ExampleSpamChecker:
-    def __init__(self, config, api):
-        self.config = config
-        self.api = api
-
-    @staticmethod
-    def parse_config(config):
-        return config
-        
-    async def check_event_for_spam(self, foo):
-        return False  # allow all events
-
-    async def user_may_invite(self, inviter_userid, invitee_userid, room_id):
-        return True  # allow all invites
-
-    async def user_may_create_room(self, userid):
-        return True  # allow all room creations
-
-    async def user_may_create_room_alias(self, userid, room_alias):
-        return True  # allow all room aliases
-
-    async def user_may_publish_room(self, userid, room_id):
-        return True  # allow publishing of all rooms
-
-    async def check_username_for_spam(self, user_profile):
-        return False  # allow all usernames
-
-    async def check_registration_for_spam(
-        self,
-        email_threepid,
-        username,
-        request_info,
-        auth_provider_id,
-    ):
-        return RegistrationBehaviour.ALLOW  # allow all registrations
-
-    async def check_media_file_for_spam(self, file_wrapper, file_info):
-        return False  # allow all media
-
-

Configuration

-

Modify the spam_checker section of your homeserver.yaml in the following -manner:

-

Create a list entry with the keys module and config.

-
    -
  • -

    module should point to the fully qualified Python class that implements your -custom logic, e.g. my_module.ExampleSpamChecker.

    -
  • -
  • -

    config is a dictionary that gets passed to the spam checker class.

    -
  • -
-

Example

-

This section might look like:

-
spam_checker:
-  - module: my_module.ExampleSpamChecker
-    config:
-      # Enable or disable a specific option in ExampleSpamChecker.
-      my_custom_option: true
-
-

More spam checkers can be added in tandem by appending more items to the list. An -action is blocked when at least one of the configured spam checkers flags it.

-

Examples

-

The Mjolnir project is a full fledged -example using the Synapse spam checking API, including a bot for dynamic -configuration.

- -
- - -
-
- - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/develop/url_previews.html b/develop/url_previews.html deleted file mode 100644 index 0d6d5ab232..0000000000 --- a/develop/url_previews.html +++ /dev/null @@ -1,365 +0,0 @@ - - - - - - URL Previews - Synapse - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- -
- - - - - - - - - - - -
-
- -
- -
- -

URL Previews

-

Design notes on a URL previewing service for Matrix:

-

Options are:

-
    -
  1. Have an AS which listens for URLs, downloads them, and inserts an event that describes their metadata.
  2. -
-
    -
  • Pros: -
      -
    • Decouples the implementation entirely from Synapse.
    • -
    • Uses existing Matrix events & content repo to store the metadata.
    • -
    -
  • -
  • Cons: -
      -
    • Which AS should provide this service for a room, and why should you trust it?
    • -
    • Doesn't work well with E2E; you'd have to cut the AS into every room
    • -
    • the AS would end up subscribing to every room anyway.
    • -
    -
  • -
-
    -
  1. Have a generic preview API (nothing to do with Matrix) that provides a previewing service:
  2. -
-
    -
  • Pros: -
      -
    • Simple and flexible; can be used by any clients at any point
    • -
    -
  • -
  • Cons: -
      -
    • If each HS provides one of these independently, all the HSes in a room may needlessly DoS the target URI
    • -
    • We need somewhere to store the URL metadata rather than just using Matrix itself
    • -
    • We can't piggyback on matrix to distribute the metadata between HSes.
    • -
    -
  • -
-
    -
  1. Make the synapse of the sending user responsible for spidering the URL and inserting an event asynchronously which describes the metadata.
  2. -
-
    -
  • Pros: -
      -
    • Works transparently for all clients
    • -
    • Piggy-backs nicely on using Matrix for distributing the metadata.
    • -
    • No confusion as to which AS
    • -
    -
  • -
  • Cons: -
      -
    • Doesn't work with E2E
    • -
    • We might want to decouple the implementation of the spider from the HS, given spider behaviour can be quite complicated and evolve much more rapidly than the HS. It's more like a bot than a core part of the server.
    • -
    -
  • -
-
    -
  1. Make the sending client use the preview API and insert the event itself when successful.
  2. -
-
    -
  • Pros: -
      -
    • Works well with E2E
    • -
    • No custom server functionality
    • -
    • Lets the client customise the preview that they send (like on FB)
    • -
    -
  • -
  • Cons: -
      -
    • Entirely specific to the sending client, whereas it'd be nice if /any/ URL was correctly previewed if clients support it.
    • -
    -
  • -
-
    -
  1. Have the option of specifying a shared (centralised) previewing service used by a room, to avoid all the different HSes in the room DoSing the target.
  2. -
-

Best solution is probably a combination of both 2 and 4.

-
    -
  • Sending clients do their best to create and send a preview at the point of sending the message, perhaps delaying the message until the preview is computed? (This also lets the user validate the preview before sending)
  • -
  • Receiving clients have the option of going and creating their own preview if one doesn't arrive soon enough (or if the original sender didn't create one)
  • -
-

This is a bit magical though in that the preview could come from two entirely different sources - the sending HS or your local one. However, this can always be exposed to users: "Generate your own URL previews if none are available?"

-

This is tantamount also to senders calculating their own thumbnails for sending in advance of the main content - we are trusting the sender not to lie about the content in the thumbnail. Whereas currently thumbnails are calculated by the receiving homeserver to avoid this attack.

-

However, this kind of phishing attack does exist whether we let senders pick their thumbnails or not, in that a malicious sender can send normal text messages around the attachment claiming it to be legitimate. We could rely on (future) reputation/abuse management to punish users who phish (be it with bogus metadata or bogus descriptions). Bogus metadata is particularly bad though, especially if it's avoidable.

-

As a first cut, let's do #2 and have the receiver hit the API to calculate its own previews (as it does currently for image thumbnails). We can then extend/optimise this to option 4 as a special extra if needed.

-

API

-
GET /_matrix/media/r0/preview_url?url=http://wherever.com
-200 OK
-{
-    "og:type"        : "article"
-    "og:url"         : "https://twitter.com/matrixdotorg/status/684074366691356672"
-    "og:title"       : "Matrix on Twitter"
-    "og:image"       : "https://pbs.twimg.com/profile_images/500400952029888512/yI0qtFi7_400x400.png"
-    "og:description" : "“Synapse 0.12 is out! Lots of polishing, performance &amp;amp; bugfixes: /sync API, /r0 prefix, fulltext search, 3PID invites https://t.co/5alhXLLEGP”"
-    "og:site_name"   : "Twitter"
-}
-
-
    -
  • Downloads the URL -
      -
    • If HTML, just stores it in RAM and parses it for OG meta tags -
        -
      • Download any media OG meta tags to the media repo, and refer to them in the OG via mxc:// URIs.
      • -
      -
    • -
    • If a media filetype we know we can thumbnail: store it on disk, and hand it to the thumbnailer. Generate OG meta tags from the thumbnailer contents.
    • -
    • Otherwise, don't bother downloading further.
    • -
    -
  • -
- -
- - -
-
- - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file