mirror of
https://github.com/element-hq/synapse
synced 2024-09-19 22:15:11 +00:00
Update docs
This commit is contained in:
parent
9d6fcf3ab9
commit
fcc42208ae
3 changed files with 83 additions and 88 deletions
|
@ -109,7 +109,7 @@ apply if you want your config file to be read properly. A few helpful things to
|
||||||
|
|
||||||
[Rooms](#rooms)
|
[Rooms](#rooms)
|
||||||
|
|
||||||
[Opentracing](#opentracing)
|
[Tracing](#tracing)
|
||||||
|
|
||||||
[Workers](#workers)
|
[Workers](#workers)
|
||||||
|
|
||||||
|
@ -3554,47 +3554,47 @@ default_power_level_content_override:
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
## Opentracing ##
|
## Tracing ##
|
||||||
Configuration options related to Opentracing support.
|
Configuration options related to tracing support.
|
||||||
|
|
||||||
---
|
---
|
||||||
### `opentracing`
|
### `tracing`
|
||||||
|
|
||||||
These settings enable and configure opentracing, which implements distributed tracing.
|
These settings enable and configure tracing. This allows you to observe the
|
||||||
This allows you to observe the causal chains of events across servers
|
causal chains of events across servers including requests, key lookups etc.,
|
||||||
including requests, key lookups etc., across any server running
|
across any server running synapse or any other services which support
|
||||||
synapse or any other services which support opentracing
|
OpenTelemetry.
|
||||||
(specifically those implemented with Jaeger).
|
|
||||||
|
|
||||||
Sub-options include:
|
Sub-options include:
|
||||||
* `enabled`: whether tracing is enabled. Set to true to enable. Disabled by default.
|
* `enabled`: whether tracing is enabled. Set to true to enable. Disabled by default.
|
||||||
* `homeserver_whitelist`: The list of homeservers we wish to send and receive span contexts and span baggage.
|
* `homeserver_whitelist`: The list of homeservers we wish to send and receive span contexts and span baggage.
|
||||||
See [here](../../opentracing.md) for more.
|
See [here](../../tracing.md) for more.
|
||||||
This is a list of regexes which are matched against the `server_name` of the homeserver.
|
This is a list of regexes which are matched against the `server_name` of the homeserver.
|
||||||
By default, it is empty, so no servers are matched.
|
By default, it is empty, so no servers are matched.
|
||||||
* `force_tracing_for_users`: # A list of the matrix IDs of users whose requests will always be traced,
|
* `sample_rate`: The probability that a given span and subsequent child spans in the trace will be
|
||||||
|
recorded. This controls the amount of spans that record and are exported from Synapse.
|
||||||
|
* `force_tracing_for_users`: A list of the matrix IDs of users whose requests will always be traced,
|
||||||
even if the tracing system would otherwise drop the traces due to probabilistic sampling.
|
even if the tracing system would otherwise drop the traces due to probabilistic sampling.
|
||||||
By default, the list is empty.
|
By default, the list is empty.
|
||||||
* `jaeger_config`: Jaeger can be configured to sample traces at different rates.
|
* `jaeger_exporter_config`: Configure authentication and where you Jaeger instance is located.
|
||||||
All configuration options provided by Jaeger can be set here. Jaeger's configuration is
|
Full options available in the [`JaegerExporter` API docs](https://opentelemetry-python.readthedocs.io/en/latest/exporter/jaeger/jaeger.html#opentelemetry.exporter.jaeger.thrift.JaegerExporter).
|
||||||
mostly related to trace sampling which is documented [here](https://www.jaegertracing.io/docs/latest/sampling/).
|
|
||||||
|
|
||||||
Example configuration:
|
Example configuration:
|
||||||
```yaml
|
```yaml
|
||||||
opentracing:
|
tracing:
|
||||||
enabled: true
|
enabled: true
|
||||||
homeserver_whitelist:
|
homeserver_whitelist:
|
||||||
- ".*"
|
- ".*"
|
||||||
|
|
||||||
|
sample_rate: 1
|
||||||
force_tracing_for_users:
|
force_tracing_for_users:
|
||||||
- "@user1:server_name"
|
- "@user1:server_name"
|
||||||
- "@user2:server_name"
|
- "@user2:server_name"
|
||||||
|
|
||||||
jaeger_config:
|
jaeger_exporter_config:
|
||||||
sampler:
|
agent_host_name: localhost
|
||||||
type: const
|
agent_port: 6831
|
||||||
param: 1
|
collector_endpoint: "http://localhost:14268/api/traces?format=jaeger.thrift"
|
||||||
logging:
|
|
||||||
false
|
|
||||||
```
|
```
|
||||||
---
|
---
|
||||||
## Workers ##
|
## Workers ##
|
||||||
|
|
|
@ -13,87 +13,88 @@
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
|
|
||||||
# NOTE
|
# NOTE This is a small wrapper around opentelemetry because tracing is optional
|
||||||
# This is a small wrapper around opentracing because opentracing is not currently
|
# and not always packaged downstream. Since opentelemetry instrumentation is
|
||||||
# packaged downstream (specifically debian). Since opentracing instrumentation is
|
# fairly invasive it was awkward to make it optional. As a result we opted to
|
||||||
# fairly invasive it was awkward to make it optional. As a result we opted to encapsulate
|
# encapsulate all opentelemetry state in these methods which effectively noop if
|
||||||
# all opentracing state in these methods which effectively noop if opentracing is
|
# opentelemetry is not present. We should strongly consider encouraging the
|
||||||
# not present. We should strongly consider encouraging the downstream distributers
|
# downstream distributers to package opentelemetry and making opentelemetry a
|
||||||
# to package opentracing and making opentracing a full dependency. In order to facilitate
|
# full dependency. In order to facilitate this move the methods have work very
|
||||||
# this move the methods have work very similarly to opentracing's and it should only
|
# similarly to opentelemetry's and it should only be a matter of few regexes to
|
||||||
# be a matter of few regexes to move over to opentracing's access patterns proper.
|
# move over to opentelemetry's access patterns proper.
|
||||||
|
|
||||||
"""
|
"""
|
||||||
============================
|
============================
|
||||||
Using OpenTracing in Synapse
|
Using OpenTelemetry in Synapse
|
||||||
============================
|
============================
|
||||||
|
|
||||||
Python-specific tracing concepts are at https://opentracing.io/guides/python/.
|
Python-specific tracing concepts are at
|
||||||
Note that Synapse wraps OpenTracing in a small module (this one) in order to make the
|
https://opentelemetry.io/docs/instrumentation/python/. Note that Synapse wraps
|
||||||
OpenTracing dependency optional. That means that the access patterns are
|
OpenTelemetry in a small module (this one) in order to make the OpenTelemetry
|
||||||
different to those demonstrated in the OpenTracing guides. However, it is
|
dependency optional. That means that some access patterns are different to those
|
||||||
still useful to know, especially if OpenTracing is included as a full dependency
|
demonstrated in the OpenTelemetry guides. However, it is still useful to know,
|
||||||
in the future or if you are modifying this module.
|
especially if OpenTelemetry is included as a full dependency in the future or if
|
||||||
|
you are modifying this module.
|
||||||
|
|
||||||
|
|
||||||
OpenTracing is encapsulated so that
|
OpenTelemetry is encapsulated so that no span objects from OpenTelemetry are
|
||||||
no span objects from OpenTracing are exposed in Synapse's code. This allows
|
exposed in Synapse's code. This allows OpenTelemetry to be easily disabled in
|
||||||
OpenTracing to be easily disabled in Synapse and thereby have OpenTracing as
|
Synapse and thereby have OpenTelemetry as an optional dependency. This does
|
||||||
an optional dependency. This does however limit the number of modifiable spans
|
however limit the number of modifiable spans at any point in the code to one.
|
||||||
at any point in the code to one. From here out references to `opentracing`
|
From here out references to `tracing` in the code snippets refer to the Synapses
|
||||||
in the code snippets refer to the Synapses module.
|
module. Most methods provided in the module have a direct correlation to those
|
||||||
Most methods provided in the module have a direct correlation to those provided
|
provided by OpenTelemetry. Refer to docs there for a more in-depth documentation
|
||||||
by opentracing. Refer to docs there for a more in-depth documentation on some of
|
on some of the args and methods.
|
||||||
the args and methods.
|
|
||||||
|
|
||||||
Tracing
|
Tracing
|
||||||
-------
|
-------
|
||||||
|
|
||||||
In Synapse it is not possible to start a non-active span. Spans can be started
|
In Synapse, it is not possible to start a non-active span. Spans can be started
|
||||||
using the ``start_active_span`` method. This returns a scope (see
|
using the ``start_active_span`` method. This returns a context manager that
|
||||||
OpenTracing docs) which is a context manager that needs to be entered and
|
needs to be entered and exited to expose the ``span``. This is usually done by
|
||||||
exited. This is usually done by using ``with``.
|
using a ``with`` statement.
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
from synapse.logging.opentracing import start_active_span
|
from synapse.logging.tracing import start_active_span
|
||||||
|
|
||||||
with start_active_span("operation name"):
|
with start_active_span("operation name"):
|
||||||
# Do something we want to tracer
|
# Do something we want to trace
|
||||||
|
|
||||||
Forgetting to enter or exit a scope will result in some mysterious and grievous log
|
Forgetting to enter or exit a scope will result in unstarted and unfinished
|
||||||
context errors.
|
spans that will not be reported (exported).
|
||||||
|
|
||||||
At anytime where there is an active span ``opentracing.set_attribute`` can be used to
|
At anytime where there is an active span ``set_attribute`` can be
|
||||||
set a tag on the current active span.
|
used to set a tag on the current active span.
|
||||||
|
|
||||||
Tracing functions
|
Tracing functions
|
||||||
-----------------
|
-----------------
|
||||||
|
|
||||||
Functions can be easily traced using decorators. The name of
|
Functions can be easily traced using decorators. The name of the function
|
||||||
the function becomes the operation name for the span.
|
becomes the operation name for the span.
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
from synapse.logging.opentracing import trace
|
from synapse.logging.tracing import trace
|
||||||
|
|
||||||
# Start a span using 'interesting_function' as the operation name
|
# Start a span using 'interesting_function' as the operation name
|
||||||
@trace
|
@trace
|
||||||
def interesting_function(*args, **kwargs):
|
def interesting_function(*args, **kwargs):
|
||||||
# Does all kinds of cool and expected things
|
# Does all kinds of cool and expected things return
|
||||||
return something_usual_and_useful
|
something_usual_and_useful
|
||||||
|
|
||||||
|
|
||||||
Operation names can be explicitly set for a function by using ``trace_with_opname``:
|
Operation names can be explicitly set for a function by using
|
||||||
|
``trace_with_opname``:
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
from synapse.logging.opentracing import trace_with_opname
|
from synapse.logging.tracing import trace_with_opname
|
||||||
|
|
||||||
@trace_with_opname("a_better_operation_name")
|
@trace_with_opname("a_better_operation_name")
|
||||||
def interesting_badly_named_function(*args, **kwargs):
|
def interesting_badly_named_function(*args, **kwargs):
|
||||||
# Does all kinds of cool and expected things
|
# Does all kinds of cool and expected things return
|
||||||
return something_usual_and_useful
|
something_usual_and_useful
|
||||||
|
|
||||||
Setting Tags
|
Setting Tags
|
||||||
------------
|
------------
|
||||||
|
@ -102,7 +103,7 @@ To set a tag on the active span do
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
from synapse.logging.opentracing import set_attribute
|
from synapse.logging.tracing import set_attribute
|
||||||
|
|
||||||
set_attribute(tag_name, tag_value)
|
set_attribute(tag_name, tag_value)
|
||||||
|
|
||||||
|
@ -111,13 +112,10 @@ inspection in order to use the formal parameter names prefixed with 'ARG_' as
|
||||||
tag names. It uses kwarg names as tag names without the prefix.
|
tag names. It uses kwarg names as tag names without the prefix.
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
from synapse.logging.tracing import tag_args
|
||||||
from synapse.logging.opentracing import tag_args
|
|
||||||
|
|
||||||
@tag_args
|
@tag_args
|
||||||
def set_fates(clotho, lachesis, atropos, father="Zues", mother="Themis"):
|
def set_fates(clotho, lachesis, atropos, father="Zues", mother="Themis"):
|
||||||
pass
|
pass
|
||||||
|
|
||||||
set_fates("the story", "the end", "the act")
|
set_fates("the story", "the end", "the act")
|
||||||
# This will have the following tags
|
# This will have the following tags
|
||||||
# - ARG_clotho: "the story"
|
# - ARG_clotho: "the story"
|
||||||
|
@ -130,25 +128,22 @@ Contexts and carriers
|
||||||
---------------------
|
---------------------
|
||||||
|
|
||||||
There are a selection of wrappers for injecting and extracting contexts from
|
There are a selection of wrappers for injecting and extracting contexts from
|
||||||
carriers provided. Unfortunately OpenTracing's three context injection
|
carriers provided. We use these to inject of OpenTelemetry Contexts into
|
||||||
techniques are not adequate for our inject of OpenTracing span-contexts into
|
Twisted's http headers, EDU contents and our database tables. Please refer to
|
||||||
Twisted's http headers, EDU contents and our database tables. Also note that
|
the end of ``logging/tracing.py`` for the available injection and extraction
|
||||||
the binary encoding format mandated by OpenTracing is not actually implemented
|
methods.
|
||||||
by jaeger_client v4.0.0 - it will silently noop.
|
|
||||||
Please refer to the end of ``logging/opentracing.py`` for the available
|
|
||||||
injection and extraction methods.
|
|
||||||
|
|
||||||
Homeserver whitelisting
|
Homeserver whitelisting
|
||||||
-----------------------
|
-----------------------
|
||||||
|
|
||||||
Most of the whitelist checks are encapsulated in the modules's injection
|
Most of the whitelist checks are encapsulated in the modules's injection and
|
||||||
and extraction method but be aware that using custom carriers or crossing
|
extraction method but be aware that using custom carriers or crossing
|
||||||
unchartered waters will require the enforcement of the whitelist.
|
unchartered waters will require the enforcement of the whitelist.
|
||||||
``logging/opentracing.py`` has a ``whitelisted_homeserver`` method which takes
|
``logging/tracing.py`` has a ``whitelisted_homeserver`` method which takes
|
||||||
in a destination and compares it to the whitelist.
|
in a destination and compares it to the whitelist.
|
||||||
|
|
||||||
Most injection methods take a 'destination' arg. The context will only be injected
|
Most injection methods take a 'destination' arg. The context will only be
|
||||||
if the destination matches the whitelist or the destination is None.
|
injected if the destination matches the whitelist or the destination is None.
|
||||||
|
|
||||||
=======
|
=======
|
||||||
Gotchas
|
Gotchas
|
||||||
|
@ -157,10 +152,10 @@ Gotchas
|
||||||
- Checking whitelists on span propagation
|
- Checking whitelists on span propagation
|
||||||
- Inserting pii
|
- Inserting pii
|
||||||
- Forgetting to enter or exit a scope
|
- Forgetting to enter or exit a scope
|
||||||
- Span source: make sure that the span you expect to be active across a
|
- Span source: make sure that the span you expect to be active across a function
|
||||||
function call really will be that one. Does the current function have more
|
call really will be that one. Does the current function have more than one
|
||||||
than one caller? Will all of those calling functions have be in a context
|
caller? Will all of those calling functions have be in a context with an
|
||||||
with an active span?
|
active span?
|
||||||
"""
|
"""
|
||||||
import contextlib
|
import contextlib
|
||||||
import inspect
|
import inspect
|
||||||
|
@ -901,7 +896,7 @@ def trace_servlet(
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
request
|
request
|
||||||
extract_context: Whether to attempt to extract the opentracing
|
extract_context: Whether to attempt to extract the tracing
|
||||||
context from the request the servlet is handling.
|
context from the request the servlet is handling.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
|
@ -208,7 +208,7 @@ def run_as_background_process(
|
||||||
Args:
|
Args:
|
||||||
desc: a description for this background process type
|
desc: a description for this background process type
|
||||||
func: a function, which may return a Deferred or a coroutine
|
func: a function, which may return a Deferred or a coroutine
|
||||||
bg_start_span: Whether to start an opentracing span. Defaults to True.
|
bg_start_span: Whether to start an tracing span. Defaults to True.
|
||||||
Should only be disabled for processes that will not log to or tag
|
Should only be disabled for processes that will not log to or tag
|
||||||
a span.
|
a span.
|
||||||
args: positional args for func
|
args: positional args for func
|
||||||
|
|
Loading…
Reference in a new issue