Merge branch 'develop' into shay/localpart_mig_2

matrix-org · May 4, 2023 · dbfb72f · dbfb72f
2 parents b82c0c8 + cc872ea
commit dbfb72f
Show file tree

Hide file tree

Showing 75 changed files with 787 additions and 455 deletions.
diff --git a/CHANGES.md b/CHANGES.md
@@ -1,3 +1,56 @@
+Synapse 1.83.0rc1 (2023-05-02)
+==============================
+
+Features
+--------
+
+- Experimental support to recursively provide relations per [MSC3981](https://github.com/matrix-org/matrix-spec-proposals/pull/3981). ([\#15315](https://github.com/matrix-org/synapse/issues/15315))
+- Experimental support for [MSC3970](https://github.com/matrix-org/matrix-spec-proposals/pull/3970): Scope transaction IDs to devices. ([\#15318](https://github.com/matrix-org/synapse/issues/15318))
+- Add an [admin API endpoint](https://matrix-org.github.io/synapse/v1.83/admin_api/experimental_features.html) to support per-user feature flags. ([\#15344](https://github.com/matrix-org/synapse/issues/15344))
+- Add a module API to send an HTTP push notification. ([\#15387](https://github.com/matrix-org/synapse/issues/15387))
+- Add an [admin API endpoint](https://matrix-org.github.io/synapse/v1.83/admin_api/statistics.html#get-largest-rooms-by-size-in-database) to query the largest rooms by disk space used in the database. ([\#15482](https://github.com/matrix-org/synapse/issues/15482))
+
+
+Bugfixes
+--------
+
+- Disable push rule evaluation for rooms excluded from sync. ([\#15361](https://github.com/matrix-org/synapse/issues/15361))
+- Fix a long-standing bug where cached server key results which were directly fetched would not be properly re-used. ([\#15417](https://github.com/matrix-org/synapse/issues/15417))
+- Fix a bug introduced in Synapse 1.73.0 where some experimental push rules were returned by default. ([\#15494](https://github.com/matrix-org/synapse/issues/15494))
+
+
+Improved Documentation
+----------------------
+
+- Add Nginx loadbalancing example with sticky mxid for workers. ([\#15411](https://github.com/matrix-org/synapse/issues/15411))
+- Update outdated development docs that mention restrictions in versions of SQLite that we no longer support. ([\#15498](https://github.com/matrix-org/synapse/issues/15498))
+
+
+Internal Changes
+----------------
+
+- Speedup tests by caching HomeServerConfig instances. ([\#15284](https://github.com/matrix-org/synapse/issues/15284))
+- Add denormalised event stream ordering column to membership state tables for future use. Contributed by Nick @ Beeper (@fizzadar). ([\#15356](https://github.com/matrix-org/synapse/issues/15356))
+- Always use multi-user device resync replication endpoints. ([\#15418](https://github.com/matrix-org/synapse/issues/15418))
+- Add column `full_user_id` to tables `profiles` and `user_filters`. ([\#15458](https://github.com/matrix-org/synapse/issues/15458))
+- Update support for [MSC3983](https://github.com/matrix-org/matrix-spec-proposals/pull/3983) to allow always returning fallback-keys in a `/keys/claim` request. ([\#15462](https://github.com/matrix-org/synapse/issues/15462))
+- Improve type hints. ([\#15465](https://github.com/matrix-org/synapse/issues/15465), [\#15496](https://github.com/matrix-org/synapse/issues/15496), [\#15497](https://github.com/matrix-org/synapse/issues/15497))
+- Support claiming more than one OTK at a time. ([\#15468](https://github.com/matrix-org/synapse/issues/15468))
+- Bump types-pyyaml from 6.0.12.8 to 6.0.12.9. ([\#15471](https://github.com/matrix-org/synapse/issues/15471))
+- Bump pyasn1-modules from 0.2.8 to 0.3.0. ([\#15473](https://github.com/matrix-org/synapse/issues/15473))
+- Bump cryptography from 40.0.1 to 40.0.2. ([\#15474](https://github.com/matrix-org/synapse/issues/15474))
+- Bump types-netaddr from 0.8.0.7 to 0.8.0.8. ([\#15475](https://github.com/matrix-org/synapse/issues/15475))
+- Bump types-jsonschema from 4.17.0.6 to 4.17.0.7. ([\#15476](https://github.com/matrix-org/synapse/issues/15476))
+- Ask bug reporters to provide logs as text. ([\#15479](https://github.com/matrix-org/synapse/issues/15479))
+- Add a Nix flake for use as a development environment. ([\#15495](https://github.com/matrix-org/synapse/issues/15495))
+- Bump anyhow from 1.0.70 to 1.0.71. ([\#15507](https://github.com/matrix-org/synapse/issues/15507))
+- Bump types-pillow from 9.4.0.19 to 9.5.0.2. ([\#15508](https://github.com/matrix-org/synapse/issues/15508))
+- Bump packaging from 23.0 to 23.1. ([\#15510](https://github.com/matrix-org/synapse/issues/15510))
+- Bump types-requests from 2.28.11.16 to 2.29.0.0. ([\#15511](https://github.com/matrix-org/synapse/issues/15511))
+- Bump setuptools-rust from 1.5.2 to 1.6.0. ([\#15512](https://github.com/matrix-org/synapse/issues/15512))
+- Update the check_schema_delta script to account for when the schema version has been bumped locally. ([\#15466](https://github.com/matrix-org/synapse/issues/15466))
+
+
 Synapse 1.82.0 (2023-04-25)
 ===========================
 

diff --git a/changelog.d/15025.misc b/changelog.d/15025.misc
@@ -0,0 +1 @@
+Use oEmbed to generate URL previews for YouTube Shorts.
diff --git a/changelog.d/15224.feature b/changelog.d/15224.feature
@@ -0,0 +1 @@
+Add `forget_rooms_on_leave` config option to automatically forget rooms when users leave them or are removed from them.
diff --git a/changelog.d/15284.misc b/changelog.d/15284.misc
diff --git a/changelog.d/15315.feature b/changelog.d/15315.feature
diff --git a/changelog.d/15318.feature b/changelog.d/15318.feature
diff --git a/changelog.d/15344.feature b/changelog.d/15344.feature
diff --git a/changelog.d/15356.misc b/changelog.d/15356.misc
diff --git a/changelog.d/15361.bugfix b/changelog.d/15361.bugfix
diff --git a/changelog.d/15387.feature b/changelog.d/15387.feature
diff --git a/changelog.d/15411.doc b/changelog.d/15411.doc
diff --git a/changelog.d/15417.bugfix b/changelog.d/15417.bugfix
diff --git a/changelog.d/15418.misc b/changelog.d/15418.misc
diff --git a/changelog.d/15437.misc b/changelog.d/15437.misc
@@ -0,0 +1 @@
+Make the `thread_id` column on `event_push_actions`, `event_push_actions_staging`, and `event_push_summary` non-null.
diff --git a/changelog.d/15458.misc b/changelog.d/15458.misc
diff --git a/changelog.d/15462.misc b/changelog.d/15462.misc
diff --git a/changelog.d/15465.misc b/changelog.d/15465.misc
diff --git a/changelog.d/15466.bugfix b/changelog.d/15466.bugfix
diff --git a/changelog.d/15468.misc b/changelog.d/15468.misc
diff --git a/changelog.d/15471.misc b/changelog.d/15471.misc
diff --git a/changelog.d/15473.misc b/changelog.d/15473.misc
diff --git a/changelog.d/15474.misc b/changelog.d/15474.misc
diff --git a/changelog.d/15475.misc b/changelog.d/15475.misc
diff --git a/changelog.d/15476.misc b/changelog.d/15476.misc
diff --git a/changelog.d/15479.misc b/changelog.d/15479.misc
diff --git a/changelog.d/15482.feature b/changelog.d/15482.feature
diff --git a/changelog.d/15494.bugfix b/changelog.d/15494.bugfix
diff --git a/changelog.d/15495.misc b/changelog.d/15495.misc
diff --git a/changelog.d/15496.misc b/changelog.d/15496.misc
diff --git a/changelog.d/15497.misc b/changelog.d/15497.misc
diff --git a/changelog.d/15498.doc b/changelog.d/15498.doc
diff --git a/changelog.d/15507.misc b/changelog.d/15507.misc
diff --git a/changelog.d/15508.misc b/changelog.d/15508.misc
diff --git a/changelog.d/15509.misc b/changelog.d/15509.misc
@@ -0,0 +1 @@
+Bump pyicu from 2.10.2 to 2.11.
diff --git a/changelog.d/15510.misc b/changelog.d/15510.misc
diff --git a/changelog.d/15511.misc b/changelog.d/15511.misc
diff --git a/changelog.d/15512.misc b/changelog.d/15512.misc
diff --git a/changelog.d/15514.misc b/changelog.d/15514.misc
diff --git a/changelog.d/15522.misc b/changelog.d/15522.misc
@@ -0,0 +1 @@
+Remove references to supporting per-user flag for [MSC2654](https://github.com/matrix-org/matrix-spec-proposals/pull/2654) (#15522).
diff --git a/changelog.d/15527.misc b/changelog.d/15527.misc
@@ -0,0 +1 @@
+Don't use a trusted key server when running the demo scripts.
diff --git a/changelog.d/15529.misc b/changelog.d/15529.misc
@@ -0,0 +1 @@
+Speed up rebuilding of the user directory for local users.
diff --git a/changelog.d/15531.misc b/changelog.d/15531.misc
@@ -0,0 +1 @@
+Speed up deleting of old rows in `event_push_actions`.
diff --git a/changelog.d/15532.misc b/changelog.d/15532.misc
@@ -0,0 +1 @@
+Install the `xmlsec` package and switch back to the upstream [cachix/devenv](https://github.com/cachix/devenv) repo in the nix development environment.
diff --git a/changelog.d/15533.misc b/changelog.d/15533.misc
@@ -0,0 +1 @@
+Install the `xmlsec` package and switch back to the upstream [cachix/devenv](https://github.com/cachix/devenv) repo in the nix development environment.
diff --git a/debian/changelog b/debian/changelog
@@ -1,3 +1,9 @@
+matrix-synapse-py3 (1.83.0~rc1) stable; urgency=medium
+
+  * New Synapse release 1.83.0rc1.
+
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 02 May 2023 15:56:38 +0100
+
 matrix-synapse-py3 (1.82.0) stable; urgency=medium
 
   * New Synapse release 1.82.0.

diff --git a/demo/start.sh b/demo/start.sh
@@ -46,7 +46,7 @@ for port in 8080 8081 8082; do
             echo ''
 
 			# Warning, this heredoc depends on the interaction of tabs and spaces.
-			# Please don't accidentaly bork me with your fancy settings.
+			# Please don't accidentally bork me with your fancy settings.
 			listeners=$(cat <<-PORTLISTENERS
 			# Configure server to listen on both $https_port and $port
 			# This overides some of the default settings above
@@ -80,12 +80,8 @@ for port in 8080 8081 8082; do
             echo "tls_certificate_path: \"$DIR/$port/localhost:$port.tls.crt\""
             echo "tls_private_key_path: \"$DIR/$port/localhost:$port.tls.key\""
 
-            # Ignore keys from the trusted keys server
-            echo '# Ignore keys from the trusted keys server'
-            echo 'trusted_key_servers:'
-            echo '  - server_name: "matrix.org"'
-            echo '    accept_keys_insecurely: true'
-            echo ''
+            # Request keys directly from servers contacted over federation
+            echo 'trusted_key_servers: []'
 
 			# Allow the servers to communicate over localhost.
 			allow_list=$(cat <<-ALLOW_LIST

diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md
@@ -57,6 +57,7 @@
       - [Account Validity](admin_api/account_validity.md)
       - [Background Updates](usage/administration/admin_api/background_updates.md)
       - [Event Reports](admin_api/event_reports.md)
+      - [Experimental Features](admin_api/experimental_features.md)
       - [Media](admin_api/media_admin_api.md)
       - [Purge History](admin_api/purge_history_api.md)
       - [Register Users](admin_api/register_api.md)

diff --git a/docs/admin_api/experimental_features.md b/docs/admin_api/experimental_features.md
@@ -1,10 +1,12 @@
 # Experimental Features API
 
 This API allows a server administrator to enable or disable some experimental features on a per-user
-basis. Currently supported features are [msc3026](https://github.com/matrix-org/matrix-spec-proposals/pull/3026): busy 
-presence state enabled, [msc2654](https://github.com/matrix-org/matrix-spec-proposals/pull/2654): enable unread counts,
-[msc3881](https://github.com/matrix-org/matrix-spec-proposals/pull/3881): enable remotely toggling push notifications 
-for another client, and [msc3967](https://github.com/matrix-org/matrix-spec-proposals/pull/3967): do not require
+basis. The currently supported features are: 
+- [MSC3026](https://github.com/matrix-org/matrix-spec-proposals/pull/3026): busy 
+presence state enabled
+- [MSC3881](https://github.com/matrix-org/matrix-spec-proposals/pull/3881): enable remotely toggling push notifications 
+for another client 
+- [MSC3967](https://github.com/matrix-org/matrix-spec-proposals/pull/3967): do not require
 UIA when first uploading cross-signing keys. 
 
 
@@ -19,7 +21,7 @@ provide a body containing the user id and listing the features to enable/disable
 {
    "features": {
       "msc3026":true,
-      "msc2654":true
+      "msc3881":true
    }
 }
 ```
@@ -46,7 +48,6 @@ user like so:
 {
    "features": {
       "msc3026": true,
-      "msc2654": true,
       "msc3881": false,
       "msc3967": false
    }

diff --git a/docs/usage/configuration/config_documentation.md b/docs/usage/configuration/config_documentation.md
@@ -3699,6 +3699,16 @@ default_power_level_content_override:
    trusted_private_chat: null
    public_chat: null
 ```
+---
+### `forget_rooms_on_leave`
+
+Set to true to automatically forget rooms for users when they leave them, either
+normally or via a kick or ban. Defaults to false.
+
+Example configuration:
+```yaml
+forget_rooms_on_leave: false
+```
 
 ---
 ## Opentracing

diff --git a/flake.lock b/flake.lock
diff --git a/flake.nix b/flake.nix
@@ -50,11 +50,7 @@
     # Output a development shell for x86_64/aarch64 Linux/Darwin (MacOS).
     systems.url = "github:nix-systems/default";
     # A development environment manager built on Nix. See https://devenv.sh.
-    # This is temporarily overridden to a fork that fixes a quirk between
-    # devenv's service and python language features. This can be removed
-    # when https://github.com/cachix/devenv/pull/559 is merged upstream.
-    devenv.url = "github:anoadragon453/devenv/anoa/fix_languages_python";
-    #devenv.url = "github:cachix/devenv/main";
+    devenv.url = "github:cachix/devenv/main";
     # Rust toolchains and rust-analyzer nightly.
     fenix = {
       url = "github:nix-community/fenix";
@@ -97,6 +93,7 @@
 
                   # Native dependencies for unit tests (SyTest also requires OpenSSL).
                   openssl
+                  xmlsec
 
                   # Native dependencies for running Complement.
                   olm

diff --git a/poetry.lock b/poetry.lock
diff --git a/pyproject.toml b/pyproject.toml
@@ -89,7 +89,7 @@ manifest-path = "rust/Cargo.toml"
 
 [tool.poetry]
 name = "matrix-synapse"
-version = "1.82.0"
+version = "1.83.0rc1"
 description = "Homeserver for the Matrix decentralised comms protocol"
 authors = ["Matrix.org Team and Contributors <packages@matrix.org>"]
 license = "Apache-2.0"

diff --git a/synapse/config/room.py b/synapse/config/room.py
@@ -75,3 +75,7 @@ def read_config(self, config: JsonDict, **kwargs: Any) -> None:
                         % preset
                     )
                 # We validate the actual overrides when we try to apply them.
+
+        # When enabled, users will forget rooms when they leave them, either via a
+        # leave, kick or ban.
+        self.forget_on_leave = config.get("forget_rooms_on_leave", False)
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Use oEmbed to generate URL previews for YouTube Shorts.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Add `forget_rooms_on_leave` config option to automatically forget rooms when users leave them or are removed from them.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Make the `thread_id` column on `event_push_actions`, `event_push_actions_staging`, and `event_push_summary` non-null.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Remove references to supporting per-user flag for [MSC2654](https://github.com/matrix-org/matrix-spec-proposals/pull/2654) (#15522).
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Don't use a trusted key server when running the demo scripts.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Speed up rebuilding of the user directory for local users.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Speed up deleting of old rows in `event_push_actions`.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Install the `xmlsec` package and switch back to the upstream [cachix/devenv](https://github.com/cachix/devenv) repo in the nix development environment.