Skip to content

home-mixer: keep VF verdicts separate per safety level in VFCandidateHydrator - #6

Merged
Pitchfork-and-Torch merged 1 commit into
mainfrom
cursor/vf-hydrator-per-level-verdicts-058e
Sep 4, 2026
Merged

Pitchfork-and-Torch merged 1 commit into
mainfrom
cursor/vf-hydrator-per-level-verdicts-058e

Conversation

@Pitchfork-and-Torch

Copy link
Copy Markdown
Owner

What broke

VFCandidateHydrator (home-mixer/candidate_hydrators/vf_candidate_hydrator.rs) sends two visibility-filtering requests per page:

  • TimelineHome for in-network candidates and repost sources
  • TimelineHomeRecommendations for out-of-network candidates, ancestors, and quoted posts

It then merged both answers into one HashMap<u64, ...> keyed by tweet id only:

all_results.extend(in_network_result);
all_results.extend(oon_result);

The same tweet id is routinely in both sets. The most common case is a followed author's own thread: the root post is an in-network candidate, and the reply in the same thread lists the root in ancestors (Thunder populates ancestors for in-network replies). The same collision happens whenever any other selected candidate quotes or replies to an in-network post. DedupConversationFilter runs after VF, so several posts from one conversation are present at this stage.

In every such case the recommendations verdict overwrote the in-network verdict. The followed author's post was then judged under the rules that visibility-filtering/rules/registry.rs applies only to recommendations (SpamHighRecall, NsfwHighRecall, DoNotAmplify, NsfwText, FosnrAbuseInsults, NSFW author/tweet flags, DMCA and geo-restricted media, and the OON-only user labels), and VFFilter removed it from the feed.

Expected: an in-network post is evaluated at TimelineHome. README.md (Filtering): "The same post is allowed to a follower."
Actual: if that post is also an ancestor or quoted post of another candidate, it gets the TimelineHomeRecommendations verdict and is dropped.

The collision also runs the other way: an out-of-network candidate that is also the source of a followed account's repost gets whichever verdict the merge order favours. Neither order is correct for both cases.

Why it matters

High-recall labels (spam, NSFW, abuse-insults) are OON-only on purpose because they trade precision for recall. "Freedom of speech, not reach" labels are likewise meant to limit recommendation, not remove a post from people who chose to follow the author. This bug silently converts those reach limits into removal for followers, and does so most often for authors whose posts get replies and quotes. Nothing in the Under the Hood report or the VF response indicates the wrong safety level was used.

Fix

Keep the two result maps separate (VfVerdicts { in_network, oon }) and route each lookup to the map matching how the id was bucketed:

  • a candidate's own verdict: map for its in_network flag
  • ancestors and quoted posts: recommendations map (unchanged behaviour)
  • repost source: in-network map

No VF rule changes, no additional RPCs, no change to which ids are requested at which level.

Tests

The file had no tests. Added unit tests for both collision directions, ancillary routing, tombstoned ancestors, interstitials, and error propagation, plus an end-to-end hydrate() test using a fake VfClient that answers Allow at TimelineHome and Drop at TimelineHomeRecommendations.

home-mixer ships without a Cargo manifest, so the file was compiled and tested verbatim (via #[path]) in a scratch crate with stub crates for the internal dependencies, on Rust 1.89. All 9 tests pass. Running only the end-to-end test against the unpatched file fails with the root post carrying Some(PossiblyUndesirable) instead of None.

Relation to upstream (xai-org/x-algorithm)

Upstream commit 6384ca7 (2026-09-01) changed the merge to oon_result.into_iter().chain(in_network_result) so the in-network verdict wins on collision. That fixes the follower-suppression direction but keeps a single map keyed by id, so an out-of-network candidate that is also a followed account's repost source now receives the more lenient TimelineHome verdict. This fork is based on the pre-6384ca7 tree and still has the original merge order.

Upstream-ready note: the same VfVerdicts split applies cleanly on top of upstream tip (the surrounding code is identical apart from the TweetVisibility wrapper and the in_network_ids dedup from xai-org#88). It removes the residual collision without changing which safety level each id is requested at.

Open in Web Open in Cursor 

…Hydrator

VFCandidateHydrator asks visibility filtering twice per request: once at
TimelineHome for in-network candidates (plus repost sources) and once at
TimelineHomeRecommendations for out-of-network candidates (plus ancestors
and quoted posts). It then merged both answers into one HashMap keyed by
tweet id, with the recommendations map applied last.

A tweet id can be in both sets. The common case is a followed author's
own thread: the root post is an in-network candidate, and the reply in
the same thread lists the root as an ancestor. The same happens whenever
another selected candidate quotes or replies to an in-network post. In
every such case the recommendations verdict overwrote the in-network
verdict, so the in-network post was judged under the rules that are
meant to apply only to recommendations from accounts the viewer does not
follow (SpamHighRecall, NsfwHighRecall, DoNotAmplify, NsfwText,
FosnrAbuseInsults, the NSFW author/tweet flags, DMCA and geo-restricted
media, and the OON-only user labels in
visibility-filtering/rules/registry.rs). VFFilter then removed the post
from the viewer's For You feed even though the viewer follows the author
and README.md states that "the same post is allowed to a follower".

The same collision runs the other way for an out-of-network candidate
that is also the source of a followed account's repost: the merge order
decides which verdict wins, and neither order is right for both cases.

Keep the two result maps separate and route every lookup to the map
matching how the id was requested: a candidate's own verdict comes from
the map for its in_network flag; ancestors and quoted posts read the
recommendations map; repost sources read the in-network map. No VF rule
changes and no extra RPCs.

Tests cover both collision directions, the ancillary routing, tombstoned
ancestors, interstitials, error propagation, and an end-to-end hydrate()
run with a client that answers Allow at TimelineHome and Drop at
TimelineHomeRecommendations. The end-to-end test fails on the previous
code with the root post carrying the recommendations-only drop reason.

Co-authored-by: Jon Bailey <Pitchfork-and-Torch@users.noreply.github.com>
@Pitchfork-and-Torch
Pitchfork-and-Torch marked this pull request as ready for review September 4, 2026 06:24
@Pitchfork-and-Torch
Pitchfork-and-Torch merged commit 34015e9 into main Sep 4, 2026
@Pitchfork-and-Torch
Pitchfork-and-Torch deleted the cursor/vf-hydrator-per-level-verdicts-058e branch September 4, 2026 06:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants