MSC3575 (Sliding Sync) add well-known proxy support (#12307)

EdGeraghty · t3chguy · web-flow · commit bb4f57583f9c · 2024-04-30T18:11:11.000Z
* Initial commit Signed-off-by: Ed Geraghty <ed@geraghty.family> * Remove commented code Signed-off-by: Ed Geraghty <ed@geraghty.family> * Change function to reflect it's proxy not native support Signed-off-by: Ed Geraghty <ed@geraghty.family> * Re-add check for servers with native support Signed-off-by: Ed Geraghty <ed@geraghty.family> * Add native support check back in Signed-off-by: Ed Geraghty <ed@geraghty.family> * Re-add endpoint health check function Signed-off-by: Ed Geraghty <ed@geraghty.family> * Use inbuilt `getWellKnown` function Signed-off-by: Ed Geraghty <ed@geraghty.family> * Change the error message to the correct function Signed-off-by: Ed Geraghty <ed@geraghty.family> * Stop storing the proxyurl in the settings for now Signed-off-by: Ed Geraghty <ed@geraghty.family> * Make the logger messages more useful Signed-off-by: Ed Geraghty <ed@geraghty.family> * Start moving the checking logic directly into the controller Signed-off-by: Ed Geraghty <ed@geraghty.family> * Add missing import Signed-off-by: Ed Geraghty <ed@geraghty.family> * Get the client rather than passing it in to the functions Signed-off-by: Ed Geraghty <ed@geraghty.family> * remove invalid `function` keyword Signed-off-by: Ed Geraghty <ed@geraghty.family> * Fix imports Signed-off-by: Ed Geraghty <ed@geraghty.family> * Our new functions are private We shouldn't(?) have to use these check in future elsewhere Signed-off-by: Ed Geraghty <ed@geraghty.family> * Change our proxy check function to return a boolean Signed-off-by: Ed Geraghty <ed@geraghty.family> * Make `nativeSlidingSyncSupport` also return boolean, add in health check Signed-off-by: Ed Geraghty <ed@geraghty.family> * Disable the sliding sync option if the server doesn't support Signed-off-by: Ed Geraghty <ed@geraghty.family> * Only enable the setting if it passes (again) Signed-off-by: Ed Geraghty <ed@geraghty.family> * Update our comments to better match what's going on Signed-off-by: Ed Geraghty <ed@geraghty.family> * Remove unused dialog Signed-off-by: Ed Geraghty <ed@geraghty.family> * Add a well-known check on start-up, if sliding sync has been enabled Signed-off-by: Ed Geraghty <ed@geraghty.family> * Check against the correct endpoint... Signed-off-by: Ed Geraghty <ed@geraghty.family> * Extract baseUrl as we'll reuse it Signed-off-by: Ed Geraghty <ed@geraghty.family> * Make the logs differentiate between the types of proxy Signed-off-by: Ed Geraghty <ed@geraghty.family> * Grab the client well-known directly for use Can't use the client object at this point, it hasn't read in the well-known Signed-off-by: Ed Geraghty <ed@geraghty.family> * Add myself to the copyright assignation I wrote the majority of this file... Signed-off-by: Ed Geraghty <ed@geraghty.family> * Only return `true` if it's actually there Signed-off-by: Ed Geraghty <ed@geraghty.family> * Correct the `proxySlidingSyncSupport` function comment to match the code Signed-off-by: Ed Geraghty <ed@geraghty.family> * Correct the `nativeSlidingSyncSupport`function comment to match the code Signed-off-by: Ed Geraghty <ed@geraghty.family> * Another comment/functionality paring Signed-off-by: Ed Geraghty <ed@geraghty.family> * Remove duplicated types from the doc Signed-off-by: Ed Geraghty <ed@geraghty.family> * Move await to the previous line Removes brackets, and corrects `wellKnown` from being a `Promise` Signed-off-by: Ed Geraghty <ed@geraghty.family> * use `waitForClientWellKnown` to avoid a race condition with the request Signed-off-by: Ed Geraghty <ed@geraghty.family> * Move getting the client out of the `if`, use `waitForClientWellKnown` Signed-off-by: Ed Geraghty <ed@geraghty.family> * Remove `beforeChange` override Signed-off-by: Ed Geraghty <ed@geraghty.family> * Move proxy setup logic into `SlidingSyncManager` Signed-off-by: Ed Geraghty <ed@geraghty.family> * Swap `configure` to private, we call it from `setup` which handles proxy Signed-off-by: Ed Geraghty <ed@geraghty.family> * Promises are always `true` TIL. Signed-off-by: Ed Geraghty <ed@geraghty.family> * use `timeoutSignal` Signed-off-by: Ed Geraghty <ed@geraghty.family> * Change message when there's no server support Signed-off-by: Ed Geraghty <ed@geraghty.family> * Refactor `slidingSyncHealthCheck` Signed-off-by: Ed Geraghty <ed@geraghty.family> * Refactor `nativeSlidingSyncSupport` with try/catch Signed-off-by: Ed Geraghty <ed@geraghty.family> * Change comment to hotlink Signed-off-by: Ed Geraghty <ed@geraghty.family> * Try and make the toggle disabled when there's no endpoint Signed-off-by: Ed Geraghty <ed@geraghty.family> * Move the if statement outside the refactored fn to avoid an await Signed-off-by: Ed Geraghty <ed@geraghty.family> * Revert "Swap `configure` to private, we call it from `setup` which handles proxy" This reverts commit c80a00b. * Remove unused import Signed-off-by: Ed Geraghty <ed@geraghty.family> * Further refactor `slidingSyncHealthCheck` `proxySlidingSyncSupport` already checks the client well-known is there Signed-off-by: Ed Geraghty <ed@geraghty.family> * Make `proxySlidingSyncSupport` log on success Signed-off-by: Ed Geraghty <ed@geraghty.family> * Clarify log message for proxy being up Signed-off-by: Ed Geraghty <ed@geraghty.family> * Move the logic into SlidingSyncManager All so we can set a static variable because the disabled check isn't asynchronous :) Signed-off-by: Ed Geraghty <ed@geraghty.family> * Obviously this isn't a return so don't overwrite with false! Signed-off-by: Ed Geraghty <ed@geraghty.family> * Remove outdated comment Signed-off-by: Ed Geraghty <ed@geraghty.family> * No need to pass in the client Signed-off-by: Ed Geraghty <ed@geraghty.family> * Activating SS should probably be info level logs Signed-off-by: Ed Geraghty <ed@geraghty.family> * If we've not enabled sliding sync, push the logs down a bit Signed-off-by: Ed Geraghty <ed@geraghty.family> * Update i18n error message Signed-off-by: Ed Geraghty <ed@geraghty.family> * Remove unused i18n strings Signed-off-by: Ed Geraghty <ed@geraghty.family> * Correct log message Signed-off-by: Ed Geraghty <ed@geraghty.family> * Prettier Signed-off-by: Ed Geraghty <ed@geraghty.family> * Remove many of the log messages Signed-off-by: Ed Geraghty <ed@geraghty.family> * Short out of `checkSupport` if it's `true` Signed-off-by: Ed Geraghty <ed@geraghty.family> * Add the endpoint back into the log when we're enabling it Signed-off-by: Ed Geraghty <ed@geraghty.family> * Note in the comment that `feature_sliding_sync_proxy_url` is legacy Signed-off-by: Ed Geraghty <ed@geraghty.family> * Expand the well-known liveness check log Signed-off-by: Ed Geraghty <ed@geraghty.family> * No need to stall the client waiting for sliding sync support * `AutoDiscovery.findClientConfig` throws if the baseUrl is blank * Fix `getProxyFromWellKnown` (?) * Add missing semicolon Sorry, linter! Signed-off-by: Ed Geraghty <ed@geraghty.family> * Pass our `MatrixClient` through instead of trying to grab it Signed-off-by: Ed Geraghty <ed@geraghty.family> * Add missing return in function comment Signed-off-by: Ed Geraghty <ed@geraghty.family> * Actually pass through our Client, not the Peg object Signed-off-by: Ed Geraghty <ed@geraghty.family> * Remove SonarCube smell complaint Signed-off-by: Ed Geraghty <ed@geraghty.family> * Neew to make our other two methods public to test Signed-off-by: Ed Geraghty <ed@geraghty.family> * First passing test Hurrah! Signed-off-by: Ed Geraghty <ed@geraghty.family> * Two more tests, this time on `checkSupport` Signed-off-by: Ed Geraghty <ed@geraghty.family> * Reset our `serverSupportsSlidingSync` between tests Signed-off-by: Ed Geraghty <ed@geraghty.family> * Check the static member is being set Signed-off-by: Ed Geraghty <ed@geraghty.family> * Move the static assignation down to the relevant tests Signed-off-by: Ed Geraghty <ed@geraghty.family> * Pull getProxyFromWellKnown mocking up Signed-off-by: Ed Geraghty <ed@geraghty.family> * Check we /haven't/ shorted out Signed-off-by: Ed Geraghty <ed@geraghty.family> * Move our spy up so we can reuse it Signed-off-by: Ed Geraghty <ed@geraghty.family> * Check spidering is being called Signed-off-by: Ed Geraghty <ed@geraghty.family> * Test the proxy is declared Signed-off-by: Ed Geraghty <ed@geraghty.family> * Test entered manually Signed-off-by: Ed Geraghty <ed@geraghty.family> * Sorry, linter * I guess these strings are wrong? * Replace any with string Co-authored-by: Michael Telatynski <7t3chguy@gmail.com> --------- Signed-off-by: Ed Geraghty <ed@geraghty.family> Co-authored-by: Michael Telatynski <7t3chguy@gmail.com>
diff --git a/src/MatrixClientPeg.ts b/src/MatrixClientPeg.ts
@@ -273,17 +273,9 @@ class MatrixClientPegClass implements IMatrixClientPeg {
         opts.threadSupport = true;
 
         if (SettingsStore.getValue("feature_sliding_sync")) {
-            const proxyUrl = SettingsStore.getValue("feature_sliding_sync_proxy_url");
-            if (proxyUrl) {
-                logger.log("Activating sliding sync using proxy at ", proxyUrl);
-            } else {
-                logger.log("Activating sliding sync");
-            }
-            opts.slidingSync = SlidingSyncManager.instance.configure(
-                this.matrixClient,
-                proxyUrl || this.matrixClient.baseUrl,
-            );
-            SlidingSyncManager.instance.startSpidering(100, 50); // 100 rooms at a time, 50ms apart
+            opts.slidingSync = await SlidingSyncManager.instance.setup(this.matrixClient);
+        } else {
+            SlidingSyncManager.instance.checkSupport(this.matrixClient);
         }
 
         // Connect the matrix client to the dispatcher and setting handlers
diff --git a/src/SlidingSyncManager.ts b/src/SlidingSyncManager.ts
@@ -44,7 +44,7 @@ limitations under the License.
  *                      list ops)
  */
 
-import { MatrixClient, EventType } from "matrix-js-sdk/src/matrix";
+import { MatrixClient, EventType, AutoDiscovery, Method, timeoutSignal } from "matrix-js-sdk/src/matrix";
 import {
     MSC3575Filter,
     MSC3575List,
@@ -56,6 +56,9 @@ import {
 import { logger } from "matrix-js-sdk/src/logger";
 import { defer, sleep } from "matrix-js-sdk/src/utils";
 
+import SettingsStore from "./settings/SettingsStore";
+import SlidingSyncController from "./settings/controllers/SlidingSyncController";
+
 // how long to long poll for
 const SLIDING_SYNC_TIMEOUT_MS = 20 * 1000;
 
@@ -323,4 +326,93 @@ export class SlidingSyncManager {
             firstTime = false;
         }
     }
+
+    /**
+     * Set up the Sliding Sync instance; configures the end point and starts spidering.
+     * The sliding sync endpoint is derived the following way:
+     *   1. The user-defined sliding sync proxy URL (legacy, for backwards compatibility)
+     *   2. The client `well-known` sliding sync proxy URL [declared at the unstable prefix](https://github.com/matrix-org/matrix-spec-proposals/blob/kegan/sync-v3/proposals/3575-sync.md#unstable-prefix)
+     *   3. The homeserver base url (for native server support)
+     * @param client The MatrixClient to use
+     * @returns A working Sliding Sync or undefined
+     */
+    public async setup(client: MatrixClient): Promise<SlidingSync | undefined> {
+        const baseUrl = client.baseUrl;
+        const proxyUrl = SettingsStore.getValue("feature_sliding_sync_proxy_url");
+        const wellKnownProxyUrl = await this.getProxyFromWellKnown(client);
+
+        const slidingSyncEndpoint = proxyUrl || wellKnownProxyUrl || baseUrl;
+
+        this.configure(client, slidingSyncEndpoint);
+        logger.info("Sliding sync activated at", slidingSyncEndpoint);
+        this.startSpidering(100, 50); // 100 rooms at a time, 50ms apart
+
+        return this.slidingSync;
+    }
+
+    /**
+     * Get the sliding sync proxy URL from the client well known
+     * @param client The MatrixClient to use
+     * @return The proxy url
+     */
+    public async getProxyFromWellKnown(client: MatrixClient): Promise<string | undefined> {
+        let proxyUrl: string | undefined;
+
+        try {
+            const clientWellKnown = await AutoDiscovery.findClientConfig(client.baseUrl);
+            proxyUrl = clientWellKnown?.["org.matrix.msc3575.proxy"]?.url;
+        } catch (e) {
+            // client.baseUrl is invalid, `AutoDiscovery.findClientConfig` has thrown
+        }
+
+        if (proxyUrl != undefined) {
+            logger.log("getProxyFromWellKnown: client well-known declares sliding sync proxy at", proxyUrl);
+        }
+        return proxyUrl;
+    }
+
+    /**
+     * Check if the server "natively" supports sliding sync (at the unstable endpoint).
+     * @param client The MatrixClient to use
+     * @return Whether the "native" (unstable) endpoint is up
+     */
+    public async nativeSlidingSyncSupport(client: MatrixClient): Promise<boolean> {
+        try {
+            await client.http.authedRequest<void>(Method.Post, "/sync", undefined, undefined, {
+                localTimeoutMs: 10 * 1000, // 10s
+                prefix: "/_matrix/client/unstable/org.matrix.msc3575",
+            });
+        } catch (e) {
+            return false; // 404, M_UNRECOGNIZED
+        }
+
+        logger.log("nativeSlidingSyncSupport: sliding sync endpoint is up");
+        return true; // 200, OK
+    }
+
+    /**
+     * Check whether our homeserver has sliding sync support, that the endpoint is up, and
+     * is a sliding sync endpoint.
+     *
+     * Sets static member `SlidingSyncController.serverSupportsSlidingSync`
+     * @param client The MatrixClient to use
+     */
+    public async checkSupport(client: MatrixClient): Promise<void> {
+        if (await this.nativeSlidingSyncSupport(client)) {
+            SlidingSyncController.serverSupportsSlidingSync = true;
+            return;
+        }
+
+        const proxyUrl = await this.getProxyFromWellKnown(client);
+        if (proxyUrl != undefined) {
+            const response = await fetch(proxyUrl + "/client/server.json", {
+                method: Method.Get,
+                signal: timeoutSignal(10 * 1000), // 10s
+            });
+            if (response.status === 200) {
+                logger.log("checkSupport: well-known sliding sync proxy is up at", proxyUrl);
+                SlidingSyncController.serverSupportsSlidingSync = true;
+            }
+        }
+    }
 }
diff --git a/src/components/views/dialogs/SlidingSyncOptionsDialog.tsx b/src/components/views/dialogs/SlidingSyncOptionsDialog.tsx
diff --git a/src/i18n/strings/en_EN.json b/src/i18n/strings/en_EN.json
@@ -1460,16 +1460,9 @@
         "rust_crypto_optin_warning": "Switching to the Rust cryptography requires a migration process that may take several minutes. To disable you will need to log out and back in; use with caution!",
         "rust_crypto_requires_logout": "Once enabled, Rust cryptography can only be disabled by logging out and in again",
         "sliding_sync": "Sliding Sync mode",
-        "sliding_sync_checking": "Checking…",
-        "sliding_sync_configuration": "Sliding Sync configuration",
         "sliding_sync_description": "Under active development, cannot be disabled.",
-        "sliding_sync_disable_warning": "To disable you will need to log out and back in, use with caution!",
         "sliding_sync_disabled_notice": "Log out and back in to disable",
-        "sliding_sync_proxy_url_label": "Proxy URL",
-        "sliding_sync_proxy_url_optional_label": "Proxy URL (optional)",
-        "sliding_sync_server_no_support": "Your server lacks native support",
-        "sliding_sync_server_specify_proxy": "Your server lacks native support, you must specify a proxy",
-        "sliding_sync_server_support": "Your server has native support",
+        "sliding_sync_server_no_support": "Your server lacks support",
         "under_active_development": "Under active development.",
         "unrealiable_e2e": "Unreliable in encrypted rooms",
         "video_rooms": "Video rooms",
diff --git a/src/settings/Settings.tsx b/src/settings/Settings.tsx
@@ -407,7 +407,7 @@ export const SETTINGS: { [setting: string]: ISetting } = {
         controller: new SlidingSyncController(),
     },
     "feature_sliding_sync_proxy_url": {
-        // This is not a distinct feature, it is a setting for feature_sliding_sync above
+        // This is not a distinct feature, it is a legacy setting for feature_sliding_sync above
         supportedLevels: LEVELS_DEVICE_ONLY_SETTINGS_WITH_CONFIG,
         default: "",
     },
diff --git a/src/settings/controllers/SlidingSyncController.ts b/src/settings/controllers/SlidingSyncController.ts
@@ -1,5 +1,6 @@
 /*
 Copyright 2022 The Matrix.org Foundation C.I.C.
+Copyright 2024 Ed Geraghty <ed@geraghty.family>
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -16,18 +17,11 @@ limitations under the License.
 
 import SettingController from "./SettingController";
 import PlatformPeg from "../../PlatformPeg";
-import { SettingLevel } from "../SettingLevel";
-import { SlidingSyncOptionsDialog } from "../../components/views/dialogs/SlidingSyncOptionsDialog";
-import Modal from "../../Modal";
 import SettingsStore from "../SettingsStore";
 import { _t } from "../../languageHandler";
 
 export default class SlidingSyncController extends SettingController {
-    public async beforeChange(level: SettingLevel, roomId: string, newValue: any): Promise<boolean> {
-        const { finished } = Modal.createDialog(SlidingSyncOptionsDialog);
-        const [value] = await finished;
-        return newValue === value; // abort the operation if we're already in the state the user chose via modal
-    }
+    public static serverSupportsSlidingSync: boolean;
 
     public async onChange(): Promise<void> {
         PlatformPeg.get()?.reload();
@@ -38,6 +32,9 @@ export default class SlidingSyncController extends SettingController {
         if (SettingsStore.getValue("feature_sliding_sync")) {
             return _t("labs|sliding_sync_disabled_notice");
         }
+        if (!SlidingSyncController.serverSupportsSlidingSync) {
+            return _t("labs|sliding_sync_server_no_support");
+        }
 
         return false;
     }
diff --git a/test/SlidingSyncManager-test.ts b/test/SlidingSyncManager-test.ts
