Matrix/synapse
Matrix/synapse
1
0
Fork 0
mirror of https://github.com/element-hq/synapse synced 2024-10-02 11:32:41 +00:00
Code Issues Projects Releases Packages Wiki Activity

deploy: 79f48b2b4f

Browse source
This commit is contained in:
DMRobertson 2023-10-24 13:24:24 +00:00
parent d1694b49a0
commit 743d74598d
5 changed files with 102 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

49
latest/development/database_schema.html
View file

@ -272,6 +272,55 @@ def run_upgrade(
"""Called whenever an existing database is to be upgraded."""
...
</code></pre>
<h2 id="background-updates"><a class="header" href="#background-updates">Background updates</a></h2>
<p>It is sometimes appropriate to perform database migrations as part of a background
process (instead of blocking Synapse until the migration is done). In particular,
this is useful for migrating data when adding new columns or tables.</p>
<p>Pending background updates stored in the <code>background_updates</code> table and are denoted
by a unique name, the current status (stored in JSON), and some dependency information:</p>
<ul>
<li>Whether the update requires a previous update to be complete.</li>
<li>A rough ordering for which to complete updates.</li>
</ul>
<p>A new background updates needs to be added to the <code>background_updates</code> table:</p>
<pre><code class="language-sql">INSERT INTO background_updates (ordering, update_name, depends_on, progress_json) VALUES
(7706, 'my_background_update', 'a_previous_background_update' '{}');
</code></pre>
<p>And then needs an associated handler in the appropriate datastore:</p>
<pre><code class="language-python">self.db_pool.updates.register_background_update_handler(
&quot;my_background_update&quot;,
update_handler=self._my_background_update,
)
</code></pre>
<p>There are a few types of updates that can be performed, see the <code>BackgroundUpdater</code>:</p>
<ul>
<li><code>register_background_update_handler</code>: A generic handler for custom SQL</li>
<li><code>register_background_index_update</code>: Create an index in the background</li>
<li><code>register_background_validate_constraint</code>: Validate a constraint in the background
(PostgreSQL-only)</li>
<li><code>register_background_validate_constraint_and_delete_rows</code>: Similar to
<code>register_background_validate_constraint</code>, but deletes rows which don't fit
the constraint.</li>
</ul>
<p>For <code>register_background_update_handler</code>, the generic handler must track progress
and then finalize the background update:</p>
<pre><code class="language-python">async def _my_background_update(self, progress: JsonDict, batch_size: int) -&gt; int:
def _do_something(txn: LoggingTransaction) -&gt; int:
...
self.db_pool.updates._background_update_progress_txn(
txn, &quot;my_background_update&quot;, {&quot;last_processed&quot;: last_processed}
)
return last_processed - prev_last_processed
num_processed = await self.db_pool.runInteraction(&quot;_do_something&quot;, _do_something)
await self.db_pool.updates._end_background_update(&quot;my_background_update&quot;)
return num_processed
</code></pre>
<p>Synapse will attempt to rate-limit how often background updates are run via the
given batch-size and the returned number of processed entries (and how long the
function took to run). See
<a href="../modules/background_update_controller_callbacks.html">background update controller callbacks</a>.</p>
<h2 id="boolean-columns"><a class="header" href="#boolean-columns">Boolean columns</a></h2>
<p>Boolean columns require special treatment, since SQLite treats booleans the
same as integers.</p>

51
latest/print.html
View file

@ -15902,7 +15902,7 @@ WHERE room_stats_state.room_id = event_json.room_id&quot; | psql -d synapse -h l
rsc.joined_members, rsc.local_users_in_room, rss.join_rules
FROM room_stats_state rss
LEFT JOIN room_stats_current rsc USING (room_id)
WHERE room_id IN ( WHERE room_id IN (
WHERE room_id IN (
'!OGEhHVWSdvArJzumhm:matrix.org',
'!YTvKGNlinIzlkMTVRl:matrix.org'
);
@ -17172,6 +17172,55 @@ def run_upgrade(
&quot;&quot;&quot;Called whenever an existing database is to be upgraded.&quot;&quot;&quot;
...
</code></pre>
<h2 id="background-updates-1"><a class="header" href="#background-updates-1">Background updates</a></h2>
<p>It is sometimes appropriate to perform database migrations as part of a background
process (instead of blocking Synapse until the migration is done). In particular,
this is useful for migrating data when adding new columns or tables.</p>
<p>Pending background updates stored in the <code>background_updates</code> table and are denoted
by a unique name, the current status (stored in JSON), and some dependency information:</p>
<ul>
<li>Whether the update requires a previous update to be complete.</li>
<li>A rough ordering for which to complete updates.</li>
</ul>
<p>A new background updates needs to be added to the <code>background_updates</code> table:</p>
<pre><code class="language-sql">INSERT INTO background_updates (ordering, update_name, depends_on, progress_json) VALUES
(7706, 'my_background_update', 'a_previous_background_update' '{}');
</code></pre>
<p>And then needs an associated handler in the appropriate datastore:</p>
<pre><code class="language-python">self.db_pool.updates.register_background_update_handler(
&quot;my_background_update&quot;,
update_handler=self._my_background_update,
)
</code></pre>
<p>There are a few types of updates that can be performed, see the <code>BackgroundUpdater</code>:</p>
<ul>
<li><code>register_background_update_handler</code>: A generic handler for custom SQL</li>
<li><code>register_background_index_update</code>: Create an index in the background</li>
<li><code>register_background_validate_constraint</code>: Validate a constraint in the background
(PostgreSQL-only)</li>
<li><code>register_background_validate_constraint_and_delete_rows</code>: Similar to
<code>register_background_validate_constraint</code>, but deletes rows which don't fit
the constraint.</li>
</ul>
<p>For <code>register_background_update_handler</code>, the generic handler must track progress
and then finalize the background update:</p>
<pre><code class="language-python">async def _my_background_update(self, progress: JsonDict, batch_size: int) -&gt; int:
def _do_something(txn: LoggingTransaction) -&gt; int:
...
self.db_pool.updates._background_update_progress_txn(
txn, &quot;my_background_update&quot;, {&quot;last_processed&quot;: last_processed}
)
return last_processed - prev_last_processed
num_processed = await self.db_pool.runInteraction(&quot;_do_something&quot;, _do_something)
await self.db_pool.updates._end_background_update(&quot;my_background_update&quot;)
return num_processed
</code></pre>
<p>Synapse will attempt to rate-limit how often background updates are run via the
given batch-size and the returned number of processed entries (and how long the
function took to run). See
<a href="development/../modules/background_update_controller_callbacks.html">background update controller callbacks</a>.</p>
<h2 id="boolean-columns"><a class="header" href="#boolean-columns">Boolean columns</a></h2>
<p>Boolean columns require special treatment, since SQLite treats booleans the
same as integers.</p>

2
latest/searchindex.js
View file

File diff suppressed because one or more lines are too long

2
latest/searchindex.json
View file

File diff suppressed because one or more lines are too long

2
latest/usage/administration/useful_sql_for_admins.html
View file

@ -306,7 +306,7 @@ WHERE room_stats_state.room_id = event_json.room_id&quot; | psql -d synapse -h l
rsc.joined_members, rsc.local_users_in_room, rss.join_rules
FROM room_stats_state rss
LEFT JOIN room_stats_current rsc USING (room_id)
WHERE room_id IN ( WHERE room_id IN (
WHERE room_id IN (
'!OGEhHVWSdvArJzumhm:matrix.org',
'!YTvKGNlinIzlkMTVRl:matrix.org'
);