Skip to content

THREADS_FORK.md

Latest commit

 

History

History
317 lines (275 loc) · 22.4 KB

THREADS_FORK.md

File metadata and controls

317 lines (275 loc) · 22.4 KB
Raw

Threads-view fork handoff

Branch, build, and release process for this fork live in ROOMLIST_FORK.md; this document covers only the threads work. Product work lands on develop.

What changed

Upstream's Threads Activity Centre (TAC) was a popup in the space panel listing rooms that happened to contain unread threads. Finding a thread took two navigations — pick a room, then find the thread in that room's panel — and the popup showed nothing about the conversations themselves.

It is replaced by a full-page, scrollable feed of individual threads across every room, modelled on Slack's threads view. Each card shows the room it belongs to, who has spoken in it, the thread root, and the most recent replies. Expanding a card in place reveals the conversation and a composer; one card is expanded at a time.

The space panel keeps a threads button in the same position with the same notification indicator, but it now navigates to the page instead of opening a menu.

Deleted

  • ThreadsActivityCentre.tsx, ThreadsActivityCentreButton.tsx, and the directory's index.ts.
  • ThreadsActivityCentre-test.tsx and its snapshot.

useUnreadThreadRooms.ts survives, moved to views/spaces/threads/, and still drives the nav button's indicator. Its 500 ms sync throttle and the Notifications.tac_only_notifications setting it reads are both unchanged, so badge behaviour matches upstream.

Why the feed is built the way it is

Matrix has no cross-room threads endpoint. /threads is per-room only, so the feed has to aggregate client-side. It does so in two stages:

  1. Seed, no network. Every visible room's room.getThreads() is read for threads already in memory from sync. This is what makes the page appear instantly.
  2. Progressive backfill. Rooms are then fetched one batch at a time, ordered by most recent activity, driven by scrolling. On an account with ~1,000 rooms, eagerly fetching every room is a non-starter: fetchRoomThreads() issues two requests per room.

Backfill is gated on Thread.hasServerSideListSupport. This is the important one. When that flag is unset, Room.createThreadsTimelineSets() falls through to getThreadListFilter(), which creates a persistent server-side filter, and fetchRoomThreads() then requests the room's entire history with limit: Number.MAX_SAFE_INTEGER. Doing that across a large account would litter the account with thousands of filters that every other client also syncs. Against such a server the feed therefore stays limited to threads already in memory rather than backfilling. Do not remove this gate.

"Mentioned in" is read from the events, not from unread state. Slack shows threads you follow, and you follow a thread by replying to it or being @mentioned in it. Matrix has no follow concept, and this fork may not persist one (see compatibility below), so membership is derived: hasParticipated() plus wasMentioned(), the latter reading m.mentions.user_ids off the thread's loaded events. Deriving it from the thread's notification level instead — the obvious shortcut — is wrong twice over: a mention read in any client clears the highlight and would silently drop the thread out of the feed, and NotificationLevel.Unsent outranks Highlight, so a thread with a failed local echo would appear under "Mentions".

Only rooms that changed are rescanned. Entries are cached per room in useThreadsFeed. Room-scoped events (RoomEvent.Timeline, RoomEvent.Receipt, MatrixEventEvent.Decrypted) mark just their own room dirty and rescan it on a trailing 500 ms throttle. ClientEvent.Sync names no room and is the only signal for the room list itself changing, so it drives a whole-account rescan on a much longer 5 s throttle. Rebuilding the entire feed on every sync meant a determineUnreadState call per thread per room twice a second, which is real main-thread cost on a large account.

The whole-account pass keeps the entry objects it already had. Those objects are what ThreadCard's memo compares, so producing fresh ones every 5 s re-rendered every mounted card and every EventTile inside it, whether anything had happened or not — around 2 ms per card, and the render window only grows. scanRooms() therefore hands back the cached array for any room whose entries still say the same thing, decided field by field in entriesEqual(). The per-room dirty pass deliberately does not do this: it is the only thing that tells a card somebody else's read receipt has landed, and a receipt changes none of the fields an entry carries, so entries comparing equal is exactly the case where a card still has to re-render. Extending reuse to that path would freeze displayed receipts, and would need a per-card receipt subscription first.

Backfill cannot be driven by scrolling alone. A feed that does not overflow its container never fires a scroll event, so if the first rooms yield few or no threads there is nothing to scroll and nothing to trigger the next batch. The result is the worst possible outcome on the account this fork exists for: "No threads yet" while 98% of rooms have never been looked at. ThreadsView therefore also backfills from an effect whenever the content is too short to scroll, and only shows an empty state once every room has actually been searched.

Cards render EventTiles directly. TimelinePanel owns a ScrollPanel plus its own SDK listeners and pagination; one per card would be ruinous, and nesting scroll containers breaks the single-scroll feel. The page is one scroll surface, as Slack's is.

Cards must filter the thread timeline themselves. The SDK puts reactions and edits into a thread's timeline alongside real replies, and TimelinePanel filters them out before building tiles. A card that skips that step renders "This event could not be displayed" wherever someone reacted — routinely, since people react to the newest message, which is exactly what a collapsed card previews. getReplies() applies the same haveRendererForEvent + shouldHideEvent pair, and the participant summary reads the filtered list so a reaction-only sender is not credited with taking part.

Cards must also add the pending replies the timeline does not contain. Element runs with detached pending-event ordering, so a local echo lives in the room's pending list, not in the thread timeline, and raises no RoomEvent.Timeline. Reading only the timeline means sending from the feed clears the composer and displays nothing — and a failed send offers no retry or cancel, because the event carrying those affordances was never rendered. getPendingReplies() appends them the way TimelinePanel does, and the feed listens for RoomEvent.LocalEchoUpdated.

Cards read receipts off the thread, never the room. collectThreadReadReceipts() mirrors MessagePanel.getReadReceiptsForEvent, dropping the reader's own receipt and ignored users, but takes them from the thread's receipt store. Falling back to the room's would report main-timeline reads as thread reads, for the same reason receipts are never sent against a root. It is called on every render rather than memoized, because receipts change while the events they hang off stay put, so a cache keyed on the rendered events would serve stale ones; the work is a Map lookup per event.

Cards pass no readReceiptMap, and that is load-bearing. The map exists so a receipt moving between events can animate out of its old position, which costs a positioned container mounted on every event whether it has receipts or not. Nothing animates across a feed of separate conversations, so the cards omit it — and ReadReceiptGroup treats its absence as licence to render the gutter alone rather than that container's four nodes. The gutter itself stays: group layout floats it right at a fixed width, which is what decides where a message's first line wraps, so dropping it would make receipt-less events wrap wider than receipted ones in the same card. The room timeline is untouched, since MessagePanel still passes a real map.

The card body carries mx_ThreadView. Thread-mode EventTile styling — 175 lines of it — is scoped in _EventTile.pcss to that class, not to the rendering type the tiles are given. Without it the tiles silently fall back to room-timeline layout, showing room names and mispositioning hidden events. The thread panel's own layout rules are compounded onto .mx_ThreadPanel, so they are not inherited along with it.

The thread being read is exempt from the filter. Reading a thread is precisely what stops it matching "Unread", so a card expanded under that filter would otherwise delete itself, and the composer being typed into, the moment its read receipt landed. filterEntries() takes the expanded thread ID and always keeps it.

Backfill progress is a set of room IDs, not an index. The room list is rebuilt whenever rooms are joined, left, or upgraded, and an index into the old list points somewhere unrelated in the new one. Tracking searched rooms by ID means the queue can be reordered or replaced freely: newly joined rooms get searched, departed rooms drop out, and no pass has to be disowned when the list changes.

Room visibility is not a per-room property. getVisibleRooms() hides a room that has been upgraded, which it can only determine by examining every room's predecessors. A per-room predicate cannot reproduce that, so dirty rescans check the room against the set found by the last whole-account scan; otherwise an event in an upgraded room puts it back into the feed.

Each card owns its own reply and edit state. This is subtle and easy to regress. EventTile's reply and edit controls identify their target with nothing but TimelineRenderingType.Thread: upstream never has more than one thread timeline on screen, so RoomViewStore skips thread replies outright and ThreadView claims them all. A feed has one timeline per card, so ThreadCard claims only the actions whose payload.event.getThread()?.id matches its own thread, and expands itself to show the composer. Without that check, a reply started on one card is captured by whichever card has a composer open and sent to the wrong thread.

Only the expanded card gets a RoomUploadContextProvider. A provider accepts any Action.ComposerFileInsert whose timelineRenderingType is Thread, and the module API's openFileUploadConfirmation(files, { view: "thread" }) payload names no room or thread. One provider per card therefore meant a module's files being uploaded into every thread in the feed at once — a cross-room disclosure. One provider on the expanded card matches upstream's one-thread-composer assumption exactly. It wraps the whole card body rather than just the composer, because with feature_wysiwyg_composer enabled editing a message renders EditWysiwygComposer, which needs the same context.

The feed is not virtualized. Cards vary widely in height, change height when expanded, and contain focusable controls including a composer — all of which fight both height measurement and the roving-focus model that the room list's FlatVirtualizedList implements for uniform rows. Instead the feed renders a bounded window of cards and grows it as the user scrolls, which keeps DOM size bounded without either problem. react-virtuoso is also not a dependency of apps/web today.

Growing the render window cannot be driven by scrolling alone. ThreadsView's advance() is called both from the scroll handler and from an effect, because two situations produce no scroll event at all: a feed shorter than its container, and a feed already scrolled to its end when a thread arrives. The second is reachable specifically because held order puts arrivals last, which can place them beyond the render window — leaving a card that cannot be reached by scrolling down, since there is nothing left to scroll. Calling it from an effect also keeps the at-top flag honest when content shrinks under the viewport, which moves the scroll offset without the user touching it.

The held order is recorded after painting, and while frozen as well as live. paintedOrder in ThreadsView is written in an effect rather than during render: a render React discards must not be able to hold the feed to an order that was never shown. It is recorded while the order is held too, not just while it is live — otherwise every thread that arrived during a freeze would stay tied for last place, and each new arrival would re-rank the ones before it, which is the reshuffling the freeze exists to prevent. Because the recording happens on every commit, resetting it elsewhere (on a filter change, say) has no lasting effect and is not worth doing.

Compatibility with the upstream client

A profile used with this fork has to stay usable in stock Element and alongside other Matrix clients. Unlike the People section, which does write an m.tag (see ROOMLIST_FORK.md), the threads work writes no shared state at all:

  • No new account data event types. Nothing is written under m.*, im.vector.*, or io.element.*, and nothing is added to the account-level im.vector.web.settings event.
  • No account-level settings. The filter selection is component state and is not persisted at all. If persistence is ever wanted, use SettingLevel.DEVICE only: that writes to the mx_local_settings localStorage blob, which stock Element reads and ignores unknown keys from. An account-level setting would sync to every client.
  • #/threads degrades gracefully. Stock Element has no such screen, and MatrixChat.showScreen() sends unrecognised screens to home. A profile left on the threads page opens on Home in stock Element.
  • mx_last_room_id is never written by the threads page. That key is shared with stock Element, which has no threads screen to restore, so viewThreads() deliberately leaves it pointing at the last real room.
  • Read receipts are ordinary threaded receipts. Reading a card sends a normal m.read/m.read.private carrying the thread's ID, exactly as opening the thread panel would. Nothing fork-specific is involved, and stock Element reads these back as its own. What the receipt is sent against matters: never the thread root, because the SDK counts a root as main-timeline and would advance the room's receipt with it, marking messages the user has never opened. See threadReceiptTarget().
  • "Mark all as read" is the one bulk write. It sends a threaded receipt per unread thread in the feed, and receipts do not come back. That is the feature working, but it is worth knowing it is the only control here that changes a lot of shared state at once, and that stock Element will honour every one of those receipts afterwards.

The one shared-schema compromise is analytics: InteractionName and ScreenName are closed unions from @matrix-org/analytics-events, so the nav button reuses upstream's WebThreadsActivityCentreButton interaction name, and the threads page has no screen name. PosthogTrackers.trackPage() now skips the $pageview when a page type has no mapped screen name, rather than reporting $current_url: undefined — which also fixes that same latent problem for module-provided pages.

Implementation locations

  • apps/web/src/viewmodels/threads/threadsFeed.ts — pure selection, ordering, and filtering. Which threads qualify lives here: participated in, or mentioned in.
  • apps/web/src/viewmodels/threads/useThreadsFeed.ts — listeners, per-room caching and throttling, and the backfill queue. Note it subscribes only to events the client actually re-emits; ThreadEvent.* is emitted on Room and never reaches the client, so new replies are picked up via RoomEvent.Timeline.
  • apps/web/src/components/structures/ThreadsView.tsx — page shell and render window.
  • apps/web/src/components/views/threads/ThreadCard.tsx — a card, collapsed and expanded, plus makeThreadRelation().
  • apps/web/src/components/views/threads/useThreadCardRoomContext.ts — the per-room RoomContext that EventTile needs outside a RoomView.
  • apps/web/src/components/views/threads/threadReadReceipts.ts — the receipts a card's tiles display, thread-scoped.
  • apps/web/src/components/views/spaces/threads/ThreadsNavButton.tsx — space panel entry.
  • Routing: PageTypes.ts, dispatcher/actions.ts, MatrixChat.tsx (viewThreads, showScreen), LoggedInView.tsx.

Known trade-offs

  • The render window only grows. Scrolling far enough eventually mounts every card, and each mounted card's EventTiles carry their own listeners. Scoping the upload provider to the expanded card removed the largest per-card cost, but a session that scrolls the whole feed still accumulates DOM. Virtualizing is the real fix and was rejected for the reasons above; a cheaper alternative is to unmount cards well above the viewport.
  • Receipts on replies a card has not rendered are dropped. A collapsed card shows the last two replies, and an unexpanded thread may not be paginated at all, so a reader who got further than the card shows appears nowhere on it. MessagePanel handles the equivalent case by folding receipts from hidden events onto the last shown event, which is not wanted here: those events are replies that exist and were left out, so folding would claim the reader had only got as far as the card happens to display. Expanding the card shows them in the right place.
  • Mention detection only sees loaded events. wasMentioned() reads m.mentions from a thread's loaded timeline, so a mention in an unpaginated reply, or one from a client predating m.mentions, is missed. Such threads still reach the feed while they carry an unread highlight, which is why that clause is kept in collectRoomEntries().
  • Expanding a card paginates that thread's timeline through client.paginateEventTimeline(). One page is fetched automatically on expand so the "Show N more replies" count is not immediately replaced by a "Load earlier replies" button, but reading a very long thread is still several round trips behind that button rather than the seamless scrollback TimelinePanel gives in the thread panel.
  • A backfill request that fails is retried once and then skipped, so a room that errors twice contributes no threads until the page is reopened.
  • The nav badge and the feed do not agree. The badge comes from useUnreadThreadRooms, which counts any room with an unread thread regardless of whether the user takes part in it, while the feed only shows threads the user participated in or was mentioned in. So the badge can point at a thread the page will not list under any filter. Making them agree means running the feed's per-thread selection from the space panel, which is mounted always and everywhere — the cost the per-room caching exists to avoid. Left inconsistent deliberately.
  • Re-sorting is deferred, and the deferral is only advertised when scrolling caused it. Sorting by latest activity means a reply anywhere in the account can move cards, so the order is held (applyHeldOrder()) whenever a card is expanded or the feed is scrolled off the top, and released when it returns to the top with nothing expanded. Threads arriving while it is held go to the end of the order — which, since the feed only renders a window of that order, usually means below what is rendered rather than visibly "at the bottom"; they are rendered once the window reaches them. A "New activity" control appears when the held order no longer matches the sort, and returns to the top. It is deliberately not shown when the hold is due to an expanded card, because the only way to release that is to collapse the card, which discards the reply being written — so a user composing a reply still gets no signal that the feed is out of date. Unlike Slack, there is no count of what is waiting: the held order alone cannot distinguish a genuinely new thread from an existing one that merely moved.
  • getBoundingClientRect() cannot be spread to position a ContextMenu. A real DOMRect exposes its properties as prototype accessors, so {...rect} is an empty object and the menu renders unpositioned; use the aboveLeftOf/aboveRightOf helpers. jsdom returns a plain object that spreads fine, so this class of bug never surfaces in unit tests — the coordinate assertion in ThreadsViewFilterMenu-test.tsx exists because of that blind spot.

Before changing

  1. Keep the Thread.hasServerSideListSupport gate on backfill. Removing it creates server-side filters per room on older homeservers.
  2. Do not persist threads-view state at account level, and do not introduce a new account data event type, without revisiting the compatibility section above.
  3. Re-run apps/web/test/unit-tests/viewmodels/threads/ and apps/web/playwright/e2e/spaces/threads/ after changing selection or ordering. useThreadsFeed-test.tsx drives the throttled rescans with fake timers; the throttle constants are duplicated there, so change both together.
  4. Fixture threads must be ones the user took part in or was pinged in, or the feed will correctly refuse to show them. populateThreads does this deliberately; a thread built only from other people's messages cannot be used as fixture data. The specs inherited from the Threads Activity Centre suite listed any unread thread, which is why this was wrong at first.
  5. Validate against the ~1,000-room benchmark account before widening the backfill batch sizes or loosening the throttle.
  6. Do not give the per-room dirty rescan the entry reuse the whole-account pass has. Fresh entry identity there is what refreshes displayed read receipts, and useThreadsFeed-test.tsx only covers the whole-account side of that.

Running the Playwright specs on this machine

Port 8080 is held by thunderbird-accounts-stalwart-1, and this fails in a way that wastes a lot of time: Stalwart's admin UI answers /config.json with HTTP 200, which is exactly the readiness probe playwright.config.ts uses, and reuseExistingServer is true — so the whole suite runs against a mail admin page and every test fails for unrelated reasons. Moving to another port does not help on its own, because routeConfigJson in packages/playwright-common hardcodes http://localhost:8080/config.json* as its interception pattern, so the app silently receives no test config. Either free 8080, or serve on another port and write a local apps/web/config.json mirroring the harness's CONFIG_JSON (gitignored) — noting that the per-test config and labsFlags fixtures do not apply under that workaround.