Matrix/synapse
Matrix/synapse
1
0
Fork 0
mirror of https://github.com/element-hq/synapse synced 2024-07-07 07:53:34 +00:00
Code Issues Packages Projects Releases Wiki Activity

Redo architecture diagram. Reword parts of federation. Formatting fixes and tweaks.

Browse source
This commit is contained in:
Kegan Dougal 2014-08-29 10:30:14 +01:00
parent 089d1b1b78
commit d1bf659ed7
1 changed files with 97 additions and 81 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

178
docs/specification.rst
View file

@ -28,6 +28,9 @@ Architecture
- Client is an end-user (web app, mobile app) which uses C-S APIs to talk to the home server. - Client is an end-user (web app, mobile app) which uses C-S APIs to talk to the home server.
A given client is typically responsible for a single user. A given client is typically responsible for a single user.
- A single user is represented by a User ID, scoped to the home server which allocated the account.
User IDs MUST have @ prefix; looks like @foo:domain - domain indicates the user's home
server.
- Home server provides C-S APIs and has the ability to federate with other HSes. - Home server provides C-S APIs and has the ability to federate with other HSes.
Typically responsible for N clients. Typically responsible for N clients.
- Federation's purpose is to share content between interested HSes; no SPOF. - Federation's purpose is to share content between interested HSes; no SPOF.
@ -46,10 +49,16 @@ participants in that room will receive the message. Rooms are uniquely
identified via a room ID. There is exactly one room ID for each room. Each identified via a room ID. There is exactly one room ID for each room. Each
room can also have an alias. Each room can have many aliases. room can also have an alias. Each room can have many aliases.
:: - Room IDs MUST have ! prefix; looks like !foo:domain - domain is simply for namespacing,
the room does NOT reside on any one domain. NOT human readable.
How events flow in rooms - Room Aliases MUST have # prefix; looks like #foo:domain - domain indicates where this
======================== alias can be mapped to a room ID. Key point: human readable / friendly.
- Aliases can be queried on the domain they specify, which will return a room ID if a
mapping exists. These mappings can change.
::
{ @alice:matrix.org } { @bob:domain.com } { @alice:matrix.org } { @bob:domain.com }
| ^ | ^
@ -59,25 +68,24 @@ room can also have an alias. Each room can have many aliases.
Content: { JSON object } Content: { JSON object } Content: { JSON object } Content: { JSON object }
| | | |
V | V |
+------------------+ +------------------+ +------------------+ +------------------+
| Home Server | | Home Server | | Home Server | | Home Server |
| matrix.org |<-------Federation--------->| domain.com | | matrix.org |<-------Federation------->| domain.com |
+------------------+ +------------------+ +------------------+ +------------------+
Room ID: !qporfwt:matrix.org Room ID: !qporfwt:matrix.org | ................................. |
Servers: matrix.org, domain.com Servers: matrix.org, domain.com |______| Shared State |_______|
Members: Members: | Room ID: !qporfwt:matrix.org |
- @alice:matrix.org - @alice:matrix.org | Servers: matrix.org, domain.com |
- @bob:domain.com - @bob:domain.com | Members: |
| - @alice:matrix.org |
| - @bob:domain.com |
|.................................|
- Federation's goal is to maintain the shared state. Don't need FULL state in order
to be a part of a room.
- Introduce the DAG.
- Events are wrapped in PDUs.
- Room IDs MUST have ! prefix; looks like !foo:domain - domain is simply for namespacing,
the room does NOT reside on domain. NOT human readable.
- Room Aliases MUST have # prefix; looks like #foo:domain - domain indicates where this
alias can be mapped to a room ID. Key point: human readable / friendly.
- User IDs MUST have @ prefix; looks like @foo:domain - domain indicates the user's home
server.
- Aliases can be queried on the domain they specify, which will return a room ID if a
mapping exists. These mappings can change.
Identity Identity
-------- --------
@ -103,7 +111,7 @@ Receiving live updates on a client
Rooms Rooms
===== =====
- How are they created? - How are they created? PDU anchor point: "root of the tree".
- Adding / removing aliases. - Adding / removing aliases.
- Invite/join dance - Invite/join dance
- State and non-state data (+extensibility) - State and non-state data (+extensibility)
@ -167,15 +175,15 @@ below:
- ``body`` : "string" - The alt text of the image, or some kind of content - ``body`` : "string" - The alt text of the image, or some kind of content
description for accessibility e.g. "image attachment". description for accessibility e.g. "image attachment".
ImageInfo: ImageInfo:
Information about an image:: Information about an image::
{ {
"size" : integer (size of image in bytes), "size" : integer (size of image in bytes),
"w" : integer (width of image in pixels), "w" : integer (width of image in pixels),
"h" : integer (height of image in pixels), "h" : integer (height of image in pixels),
"mimetype" : "string (e.g. image/jpeg)", "mimetype" : "string (e.g. image/jpeg)",
} }
``m.audio`` ``m.audio``
Required keys: Required keys:
@ -186,15 +194,14 @@ ImageInfo:
- ``body`` : "string" - A description of the audio e.g. "Bee Gees - - ``body`` : "string" - A description of the audio e.g. "Bee Gees -
Stayin' Alive", or some kind of content description for accessibility e.g. Stayin' Alive", or some kind of content description for accessibility e.g.
"audio attachment". "audio attachment".
AudioInfo:
Information about a piece of audio::
AudioInfo: {
Information about a piece of audio:: "mimetype" : "string (e.g. audio/aac)",
"size" : integer (size of audio in bytes),
{ "duration" : integer (duration of audio in milliseconds),
"mimetype" : "string (e.g. audio/aac)", }
"size" : integer (size of audio in bytes),
"duration" : integer (duration of audio in milliseconds),
}
``m.video`` ``m.video``
Required keys: Required keys:
@ -205,18 +212,18 @@ AudioInfo:
- ``body`` : "string" - A description of the video e.g. "Gangnam style", - ``body`` : "string" - A description of the video e.g. "Gangnam style",
or some kind of content description for accessibility e.g. "video attachment". or some kind of content description for accessibility e.g. "video attachment".
VideoInfo: VideoInfo:
Information about a video:: Information about a video::
{ {
"mimetype" : "string (e.g. video/mp4)", "mimetype" : "string (e.g. video/mp4)",
"size" : integer (size of video in bytes), "size" : integer (size of video in bytes),
"duration" : integer (duration of video in milliseconds), "duration" : integer (duration of video in milliseconds),
"w" : integer (width of video in pixels), "w" : integer (width of video in pixels),
"h" : integer (height of video in pixels), "h" : integer (height of video in pixels),
"thumbnail_url" : "string (URL to image)", "thumbnail_url" : "string (URL to image)",
"thumbanil_info" : JSON object (ImageInfo) "thumbanil_info" : JSON object (ImageInfo)
} }
``m.location`` ``m.location``
Required keys: Required keys:
@ -624,61 +631,62 @@ can also be performed.
There are three main kinds of communication that occur between home servers: There are three main kinds of communication that occur between home servers:
- Queries :Queries:
These are single request/response interactions between a given pair of These are single request/response interactions between a given pair of
servers, initiated by one side sending an HTTP request to obtain some servers, initiated by one side sending an HTTP GET request to obtain some
information, and responded by the other. They are not persisted and contain information, and responded by the other. They are not persisted and contain
no long-term significant history. They simply request a snapshot state at the no long-term significant history. They simply request a snapshot state at the
instant the query is made. instant the query is made.
- EDUs - Ephemeral Data Units :Ephemeral Data Units (EDUs):
These are notifications of events that are pushed from one home server to These are notifications of events that are pushed from one home server to
another. They are not persisted and contain no long-term significant history, another. They are not persisted and contain no long-term significant history,
nor does the receiving home server have to reply to them. nor does the receiving home server have to reply to them.
- PDUs - Persisted Data Units :Persisted Data Units (PDUs):
These are notifications of events that are broadcast from one home server to These are notifications of events that are broadcast from one home server to
any others that are interested in the same "context" (namely, a Room ID). any others that are interested in the same "context" (namely, a Room ID).
They are persisted to long-term storage and form the record of history for They are persisted to long-term storage and form the record of history for
that context. that context.
Where Queries are presented directly across the HTTP connection as GET requests EDUs and PDUs are further wrapped in an envelope called a Transaction, which is
to specific URLs, EDUs and PDUs are further wrapped in an envelope called a transferred from the origin to the destination home server using an HTTP PUT request.
Transaction, which is transferred from the origin to the destination home server
using a PUT request.
Transactions and EDUs/PDUs Transactions
-------------------------- ------------
The transfer of EDUs and PDUs between home servers is performed by an exchange The transfer of EDUs and PDUs between home servers is performed by an exchange
of Transaction messages, which are encoded as JSON objects with a dict as the of Transaction messages, which are encoded as JSON objects, passed over an
top-level element, passed over an HTTP PUT request. A Transaction is meaningful HTTP PUT request. A Transaction is meaningful only to the pair of home servers that
only to the pair of home servers that exchanged it; they are not globally- exchanged it; they are not globally-meaningful.
meaningful.
Each transaction has an opaque ID and timestamp (UNIX epoch time in Each transaction has:
milliseconds) generated by its origin server, an origin and destination server - An opaque transaction ID.
name, a list of "previous IDs", and a list of PDUs - the actual message payload - A timestamp (UNIX epoch time in milliseconds) generated by its origin server.
that the Transaction carries. - An origin and destination server name.
- A list of "previous IDs".
- A list of PDUs and EDUs - the actual message payload that the Transaction carries.
:: ::
{"transaction_id":"916d630ea616342b42e98a3be0b74113", {
"transaction_id":"916d630ea616342b42e98a3be0b74113",
"ts":1404835423000, "ts":1404835423000,
"origin":"red", "origin":"red",
"destination":"blue", "destination":"blue",
"prev_ids":["e1da392e61898be4d2009b9fecce5325"], "prev_ids":["e1da392e61898be4d2009b9fecce5325"],
"pdus":[...], "pdus":[...],
"edus":[...]} "edus":[...]
}
The "previous IDs" field will contain a list of previous transaction IDs that The ``prev_ids`` field contains a list of previous transaction IDs that
the origin server has sent to this destination. Its purpose is to act as a the ``origin`` server has sent to this ``destination``. Its purpose is to act as a
sequence checking mechanism - the destination server can check whether it has sequence checking mechanism - the destination server can check whether it has
successfully received that Transaction, or ask for a retransmission if not. successfully received that Transaction, or ask for a retransmission if not.
The "pdus" field of a transaction is a list, containing zero or more PDUs.[*] The ``pdus`` field of a transaction is a list, containing zero or more PDUs.[*]
Each PDU is itself a dict containing a number of keys, the exact details of Each PDU is itself a JSON object containing a number of keys, the exact details of
which will vary depending on the type of PDU. Similarly, the "edus" field is which will vary depending on the type of PDU. Similarly, the ``edus`` field is
another list containing the EDUs. This key may be entirely absent if there are another list containing the EDUs. This key may be entirely absent if there are
no EDUs to transfer. no EDUs to transfer.
@ -687,27 +695,35 @@ receiving an "empty" transaction, as this is useful for informing peers of other
transaction IDs they should be aware of. This effectively acts as a push transaction IDs they should be aware of. This effectively acts as a push
mechanism to encourage peers to continue to replicate content.) mechanism to encourage peers to continue to replicate content.)
All PDUs have an ID, a context, a declaration of their type, a list of other PDU PDUs and EDUs
IDs that have been seen recently on that context (regardless of which origin -------------
sent them), and a nested content field containing the actual event content.
All PDUs have:
- An ID
- A context
- A declaration of their type
- A list of other PDU IDs that have been seen recently on that context (regardless of which origin
sent them)
[[TODO(paul): Update this structure so that 'pdu_id' is a two-element [[TODO(paul): Update this structure so that 'pdu_id' is a two-element
[origin,ref] pair like the prev_pdus are]] [origin,ref] pair like the prev_pdus are]]
:: ::
{"pdu_id":"a4ecee13e2accdadf56c1025af232176", {
"pdu_id":"a4ecee13e2accdadf56c1025af232176",
"context":"#example.green", "context":"#example.green",
"origin":"green", "origin":"green",
"ts":1404838188000, "ts":1404838188000,
"pdu_type":"m.text", "pdu_type":"m.text",
"prev_pdus":[["blue","99d16afbc857975916f1d73e49e52b65"]], "prev_pdus":[["blue","99d16afbc857975916f1d73e49e52b65"]],
"content":... "content":...
"is_state":false} "is_state":false
}
In contrast to the transaction layer, it is important to note that the prev_pdus In contrast to Transactions, it is important to note that the ``prev_pdus``
field of a PDU refers to PDUs that any origin server has sent, rather than field of a PDU refers to PDUs that any origin server has sent, rather than
previous IDs that this origin has sent. This list may refer to other PDUs sent previous IDs that this ``origin`` has sent. This list may refer to other PDUs sent
by the same origin as the current one, or other origins. by the same origin as the current one, or other origins.
Because of the distributed nature of participants in a Matrix conversation, it Because of the distributed nature of participants in a Matrix conversation, it