Clean-up demo scripts & documentation (#12143)

* Rewrites the demo documentation to be clearer, accurate, and moves it to our documentation tree.
* Improvements to the demo scripts:
	* `clean.sh` now runs `stop.sh` first to avoid zombie processes.
	* Uses more modern Synapse configuration (and removes some obsolete configuration).
	* Consistently use the HTTP ports for server name, etc.
	* Remove the `demo/etc` directory and place everything into the `demo/808x` directories.
This commit is contained in:
Patrick Cloke 2022-03-08 15:02:59 -05:00 committed by GitHub
parent 5627182788
commit 9a0172d49f
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
9 changed files with 88 additions and 70 deletions

View file

@ -312,6 +312,9 @@ We recommend using the demo which starts 3 federated instances running on ports
(to stop, you can use `./demo/stop.sh`) (to stop, you can use `./demo/stop.sh`)
See the [demo documentation](https://matrix-org.github.io/synapse/develop/development/demo.html)
for more information.
If you just want to start a single instance of the app and run it directly:: If you just want to start a single instance of the app and run it directly::
# Create the homeserver.yaml config once # Create the homeserver.yaml config once

1
changelog.d/12143.doc Normal file
View file

@ -0,0 +1 @@
Improve documentation for demo scripts.

11
demo/.gitignore vendored
View file

@ -1,7 +1,4 @@
*.db # Ignore all the temporary files from the demo servers.
*.log 8080/
*.log.* 8081/
*.pid 8082/
/media_store.*
/etc

View file

@ -1,26 +0,0 @@
DO NOT USE THESE DEMO SERVERS IN PRODUCTION
Requires you to have done:
python setup.py develop
The demo start.sh will start three synapse servers on ports 8080, 8081 and 8082, with host names localhost:$port. This can be easily changed to `hostname`:$port in start.sh if required.
To enable the servers to communicate untrusted ssl certs are used. In order to do this the servers do not check the certs
and are configured in a highly insecure way. Do not use these configuration files in production.
stop.sh will stop the synapse servers and the webclient.
clean.sh will delete the databases and log files.
To start a completely new set of servers, run:
./demo/stop.sh; ./demo/clean.sh && ./demo/start.sh
Logs and sqlitedb will be stored in demo/808{0,1,2}.{log,db}
Also note that when joining a public room on a different HS via "#foo:bar.net", then you are (in the current impl) joining a room with room_id "foo". This means that it won't work if your HS already has a room with that name.

View file

@ -4,6 +4,9 @@ set -e
DIR="$( cd "$( dirname "$0" )" && pwd )" DIR="$( cd "$( dirname "$0" )" && pwd )"
# Ensure that the servers are stopped.
$DIR/stop.sh
PID_FILE="$DIR/servers.pid" PID_FILE="$DIR/servers.pid"
if [ -f "$PID_FILE" ]; then if [ -f "$PID_FILE" ]; then

View file

@ -6,8 +6,6 @@ CWD=$(pwd)
cd "$DIR/.." || exit cd "$DIR/.." || exit
mkdir -p demo/etc
PYTHONPATH=$(readlink -f "$(pwd)") PYTHONPATH=$(readlink -f "$(pwd)")
export PYTHONPATH export PYTHONPATH
@ -21,22 +19,26 @@ for port in 8080 8081 8082; do
mkdir -p demo/$port mkdir -p demo/$port
pushd demo/$port || exit pushd demo/$port || exit
#rm $DIR/etc/$port.config # Generate the configuration for the homeserver at localhost:848x.
python3 -m synapse.app.homeserver \ python3 -m synapse.app.homeserver \
--generate-config \ --generate-config \
-H "localhost:$https_port" \ --server-name "localhost:$port" \
--config-path "$DIR/etc/$port.config" \ --config-path "$port.config" \
--report-stats no --report-stats no
if ! grep -F "Customisation made by demo/start.sh" -q "$DIR/etc/$port.config"; then if ! grep -F "Customisation made by demo/start.sh" -q "$port.config"; then
# Generate tls keys # Generate TLS keys.
openssl req -x509 -newkey rsa:4096 -keyout "$DIR/etc/localhost:$https_port.tls.key" -out "$DIR/etc/localhost:$https_port.tls.crt" -days 365 -nodes -subj "/O=matrix" openssl req -x509 -newkey rsa:4096 \
-keyout "localhost:$port.tls.key" \
-out "localhost:$port.tls.crt" \
-days 365 -nodes -subj "/O=matrix"
# Regenerate configuration # Add customisations to the configuration.
{ {
printf '\n\n# Customisation made by demo/start.sh\n' printf '\n\n# Customisation made by demo/start.sh\n\n'
echo "public_baseurl: http://localhost:$port/" echo "public_baseurl: http://localhost:$port/"
echo 'enable_registration: true' echo 'enable_registration: true'
echo ''
# Warning, this heredoc depends on the interaction of tabs and spaces. # Warning, this heredoc depends on the interaction of tabs and spaces.
# Please don't accidentaly bork me with your fancy settings. # Please don't accidentaly bork me with your fancy settings.
@ -63,38 +65,34 @@ for port in 8080 8081 8082; do
echo "${listeners}" echo "${listeners}"
# Disable tls for the servers # Disable TLS for the servers
printf '\n\n# Disable tls on the servers.' printf '\n\n# Disable TLS for the servers.'
echo '# DO NOT USE IN PRODUCTION' echo '# DO NOT USE IN PRODUCTION'
echo 'use_insecure_ssl_client_just_for_testing_do_not_use: true' echo 'use_insecure_ssl_client_just_for_testing_do_not_use: true'
echo 'federation_verify_certificates: false' echo 'federation_verify_certificates: false'
# Set tls paths # Set paths for the TLS certificates.
echo "tls_certificate_path: \"$DIR/etc/localhost:$https_port.tls.crt\"" echo "tls_certificate_path: \"$DIR/$port/localhost:$port.tls.crt\""
echo "tls_private_key_path: \"$DIR/etc/localhost:$https_port.tls.key\"" echo "tls_private_key_path: \"$DIR/$port/localhost:$port.tls.key\""
# Ignore keys from the trusted keys server # Ignore keys from the trusted keys server
echo '# Ignore keys from the trusted keys server' echo '# Ignore keys from the trusted keys server'
echo 'trusted_key_servers:' echo 'trusted_key_servers:'
echo ' - server_name: "matrix.org"' echo ' - server_name: "matrix.org"'
echo ' accept_keys_insecurely: true' echo ' accept_keys_insecurely: true'
echo ''
# Reduce the blacklist # Allow the servers to communicate over localhost.
blacklist=$(cat <<-BLACK allow_list=$(cat <<-ALLOW_LIST
# Set the blacklist so that it doesn't include 127.0.0.1, ::1 # Allow the servers to communicate over localhost.
federation_ip_range_blacklist: ip_range_whitelist:
- '10.0.0.0/8' - '127.0.0.1/8'
- '172.16.0.0/12' - '::1/128'
- '192.168.0.0/16' ALLOW_LIST
- '100.64.0.0/10'
- '169.254.0.0/16'
- 'fe80::/64'
- 'fc00::/7'
BLACK
) )
echo "${blacklist}" echo "${allow_list}"
} >> "$DIR/etc/$port.config" } >> "$port.config"
fi fi
# Check script parameters # Check script parameters
@ -141,19 +139,18 @@ for port in 8080 8081 8082; do
burst_count: 1000 burst_count: 1000
RC RC
) )
echo "${ratelimiting}" >> "$DIR/etc/$port.config" echo "${ratelimiting}" >> "$port.config"
fi fi
fi fi
if ! grep -F "full_twisted_stacktraces" -q "$DIR/etc/$port.config"; then # Always disable reporting of stats if the option is not there.
echo "full_twisted_stacktraces: true" >> "$DIR/etc/$port.config" if ! grep -F "report_stats" -q "$port.config" ; then
fi echo "report_stats: false" >> "$port.config"
if ! grep -F "report_stats" -q "$DIR/etc/$port.config" ; then
echo "report_stats: false" >> "$DIR/etc/$port.config"
fi fi
# Run the homeserver in the background.
python3 -m synapse.app.homeserver \ python3 -m synapse.app.homeserver \
--config-path "$DIR/etc/$port.config" \ --config-path "$port.config" \
-D \ -D \
popd || exit popd || exit

View file

@ -82,6 +82,7 @@
- [Release Cycle](development/releases.md) - [Release Cycle](development/releases.md)
- [Git Usage](development/git.md) - [Git Usage](development/git.md)
- [Testing]() - [Testing]()
- [Demo scripts](development/demo.md)
- [OpenTracing](opentracing.md) - [OpenTracing](opentracing.md)
- [Database Schemas](development/database_schema.md) - [Database Schemas](development/database_schema.md)
- [Experimental features](development/experimental_features.md) - [Experimental features](development/experimental_features.md)

41
docs/development/demo.md Normal file
View file

@ -0,0 +1,41 @@
# Synapse demo setup
**DO NOT USE THESE DEMO SERVERS IN PRODUCTION**
Requires you to have a [Synapse development environment setup](https://matrix-org.github.io/synapse/develop/development/contributing_guide.html#4-install-the-dependencies).
The demo setup allows running three federation Synapse servers, with server
names `localhost:8080`, `localhost:8081`, and `localhost:8082`.
You can access them via any Matrix client over HTTP at `localhost:8080`,
`localhost:8081`, and `localhost:8082` or over HTTPS at `localhost:8480`,
`localhost:8481`, and `localhost:8482`.
To enable the servers to communicate, self-signed SSL certificates are generated
and the servers are configured in a highly insecure way, including:
* Not checking certificates over federation.
* Not verifying keys.
The servers are configured to store their data under `demo/8080`, `demo/8081`, and
`demo/8082`. This includes configuration, logs, SQLite databases, and media.
Note that when joining a public room on a different HS via "#foo:bar.net", then
you are (in the current impl) joining a room with room_id "foo". This means that
it won't work if your HS already has a room with that name.
## Using the demo scripts
There's three main scripts with straightforward purposes:
* `start.sh` will start the Synapse servers, generating any missing configuration.
* This accepts a single parameter `--no-rate-limit` to "disable" rate limits
(they actually still exist, but are very high).
* `stop.sh` will stop the Synapse servers.
* `clean.sh` will delete the configuration, databases, log files, etc.
To start a completely new set of servers, run:
```sh
./demo/stop.sh; ./demo/clean.sh && ./demo/start.sh
```

View file

@ -63,4 +63,5 @@ release of Synapse.
If you want to get up and running quickly with a trio of homeservers in a If you want to get up and running quickly with a trio of homeservers in a
private federation, there is a script in the `demo` directory. This is mainly private federation, there is a script in the `demo` directory. This is mainly
useful just for development purposes. See [demo/README](https://github.com/matrix-org/synapse/tree/develop/demo/). useful just for development purposes. See
[demo scripts](https://matrix-org.github.io/synapse/develop/development/demo.html).