Add a way to run tests in PostgreSQL in Docker (#3699)

This commit is contained in:
Amber Brown 2018-09-20 18:12:45 +10:00 committed by GitHub
parent aeca5a5ed5
commit 741571cf22
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
8 changed files with 79 additions and 9 deletions

1
.gitignore vendored
View file

@ -1,6 +1,7 @@
*.pyc *.pyc
.*.swp .*.swp
*~ *~
*.lock
.DS_Store .DS_Store
_trial_temp/ _trial_temp/

View file

@ -30,12 +30,28 @@ use github's pull request workflow to review the contribution, and either ask
you to make any refinements needed or merge it and make them ourselves. The you to make any refinements needed or merge it and make them ourselves. The
changes will then land on master when we next do a release. changes will then land on master when we next do a release.
We use `Jenkins <http://matrix.org/jenkins>`_ and We use `CircleCI <https://circleci.com/gh/matrix-org>`_ and `Travis CI
`Travis <https://travis-ci.org/matrix-org/synapse>`_ for continuous <https://travis-ci.org/matrix-org/synapse>`_ for continuous integration. All
integration. All pull requests to synapse get automatically tested by Travis; pull requests to synapse get automatically tested by Travis and CircleCI.
the Jenkins builds require an adminstrator to start them. If your change If your change breaks the build, this will be shown in GitHub, so please
breaks the build, this will be shown in github, so please keep an eye on the keep an eye on the pull request for feedback.
pull request for feedback.
To run unit tests in a local development environment, you can use:
- ``tox -e py27`` (requires tox to be installed by ``pip install tox``) for
SQLite-backed Synapse on Python 2.7.
- ``tox -e py35`` for SQLite-backed Synapse on Python 3.5.
- ``tox -e py36`` for SQLite-backed Synapse on Python 3.6.
- ``tox -e py27-postgres`` for PostgreSQL-backed Synapse on Python 2.7
(requires a running local PostgreSQL with access to create databases).
- ``./test_postgresql.sh`` for PostgreSQL-backed Synapse on Python 2.7
(requires Docker). Entirely self-contained, recommended if you don't want to
set up PostgreSQL yourself.
Docker images are available for running the integration tests (SyTest) locally,
see the `documentation in the SyTest repo
<https://github.com/matrix-org/sytest/blob/develop/docker/README.md>`_ for more
information.
Code style Code style
~~~~~~~~~~ ~~~~~~~~~~
@ -77,7 +93,8 @@ AUTHORS.rst file for the project in question. Please feel free to include a
change to AUTHORS.rst in your pull request to list yourself and a short change to AUTHORS.rst in your pull request to list yourself and a short
description of the area(s) you've worked on. Also, we sometimes have swag to description of the area(s) you've worked on. Also, we sometimes have swag to
give away to contributors - if you feel that Matrix-branded apparel is missing give away to contributors - if you feel that Matrix-branded apparel is missing
from your life, please mail us your shipping address to matrix at matrix.org and we'll try to fix it :) from your life, please mail us your shipping address to matrix at matrix.org and
we'll try to fix it :)
Sign off Sign off
~~~~~~~~ ~~~~~~~~
@ -144,4 +161,9 @@ flag to ``git commit``, which uses the name and email set in your
Conclusion Conclusion
~~~~~~~~~~ ~~~~~~~~~~
That's it! Matrix is a very open and collaborative project as you might expect given our obsession with open communication. If we're going to successfully matrix together all the fragmented communication technologies out there we are reliant on contributions and collaboration from the community to do so. So please get involved - and we hope you have as much fun hacking on Matrix as we do! That's it! Matrix is a very open and collaborative project as you might expect
given our obsession with open communication. If we're going to successfully
matrix together all the fragmented communication technologies out there we are
reliant on contributions and collaboration from the community to do so. So
please get involved - and we hope you have as much fun hacking on Matrix as we
do!

View file

@ -28,6 +28,7 @@ exclude jenkins*.sh
exclude jenkins* exclude jenkins*
exclude Dockerfile exclude Dockerfile
exclude .dockerignore exclude .dockerignore
exclude test_postgresql.sh
recursive-exclude jenkins *.sh recursive-exclude jenkins *.sh
include pyproject.toml include pyproject.toml

2
changelog.d/3699.misc Normal file
View file

@ -0,0 +1,2 @@
Unit tests can now be run under PostgreSQL in Docker using
``test_postgresql.sh``.

12
docker/Dockerfile-pgtests Normal file
View file

@ -0,0 +1,12 @@
# Use the Sytest image that comes with a lot of the build dependencies
# pre-installed
FROM matrixdotorg/sytest:latest
# The Sytest image doesn't come with python, so install that
RUN apt-get -qq install -y python python-dev python-pip
# We need tox to run the tests in run_pg_tests.sh
RUN pip install tox
ADD run_pg_tests.sh /pg_tests.sh
ENTRYPOINT /pg_tests.sh

20
docker/run_pg_tests.sh Executable file
View file

@ -0,0 +1,20 @@
#!/bin/bash
# This script runs the PostgreSQL tests inside a Docker container. It expects
# the relevant source files to be mounted into /src (done automatically by the
# caller script). It will set up the database, run it, and then use the tox
# configuration to run the tests.
set -e
# Set PGUSER so Synapse's tests know what user to connect to the database with
export PGUSER=postgres
# Initialise & start the database
su -c '/usr/lib/postgresql/9.6/bin/initdb -D /var/lib/postgresql/data -E "UTF-8" --lc-collate="en_US.UTF-8" --lc-ctype="en_US.UTF-8" --username=postgres' postgres
su -c '/usr/lib/postgresql/9.6/bin/pg_ctl -w -D /var/lib/postgresql/data start' postgres
# Run the tests
cd /src
export TRIAL_FLAGS="-j 4"
tox --workdir=/tmp -e py27-postgres

12
test_postgresql.sh Executable file
View file

@ -0,0 +1,12 @@
#!/bin/bash
# This script builds the Docker image to run the PostgreSQL tests, and then runs
# the tests.
set -e
# Build, and tag
docker build docker/ -f docker/Dockerfile-pgtests -t synapsepgtests
# Run, mounting the current directory into /src
docker run --rm -it -v $(pwd)\:/src synapsepgtests

View file

@ -4,7 +4,7 @@ envlist = packaging, py27, py36, pep8, check_isort
[base] [base]
deps = deps =
coverage coverage
Twisted>=15.1 Twisted>=17.1
mock mock
python-subunit python-subunit
junitxml junitxml