Repository navigation
home-mixer: keep VF verdicts separate per safety level in VFCandidateHydrator - #6
Merged
Pitchfork-and-Torch merged 1 commit intoSep 4, 2026
Conversation
…Hydrator VFCandidateHydrator asks visibility filtering twice per request: once at TimelineHome for in-network candidates (plus repost sources) and once at TimelineHomeRecommendations for out-of-network candidates (plus ancestors and quoted posts). It then merged both answers into one HashMap keyed by tweet id, with the recommendations map applied last. A tweet id can be in both sets. The common case is a followed author's own thread: the root post is an in-network candidate, and the reply in the same thread lists the root as an ancestor. The same happens whenever another selected candidate quotes or replies to an in-network post. In every such case the recommendations verdict overwrote the in-network verdict, so the in-network post was judged under the rules that are meant to apply only to recommendations from accounts the viewer does not follow (SpamHighRecall, NsfwHighRecall, DoNotAmplify, NsfwText, FosnrAbuseInsults, the NSFW author/tweet flags, DMCA and geo-restricted media, and the OON-only user labels in visibility-filtering/rules/registry.rs). VFFilter then removed the post from the viewer's For You feed even though the viewer follows the author and README.md states that "the same post is allowed to a follower". The same collision runs the other way for an out-of-network candidate that is also the source of a followed account's repost: the merge order decides which verdict wins, and neither order is right for both cases. Keep the two result maps separate and route every lookup to the map matching how the id was requested: a candidate's own verdict comes from the map for its in_network flag; ancestors and quoted posts read the recommendations map; repost sources read the in-network map. No VF rule changes and no extra RPCs. Tests cover both collision directions, the ancillary routing, tombstoned ancestors, interstitials, error propagation, and an end-to-end hydrate() run with a client that answers Allow at TimelineHome and Drop at TimelineHomeRecommendations. The end-to-end test fails on the previous code with the root post carrying the recommendations-only drop reason. Co-authored-by: Jon Bailey <Pitchfork-and-Torch@users.noreply.github.com>
Pitchfork-and-Torch
marked this pull request as ready for review
September 4, 2026 06:24
Pitchfork-and-Torch
deleted the
cursor/vf-hydrator-per-level-verdicts-058e
branch
September 4, 2026 06:32
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What broke
VFCandidateHydrator(home-mixer/candidate_hydrators/vf_candidate_hydrator.rs) sends two visibility-filtering requests per page:TimelineHomefor in-network candidates and repost sourcesTimelineHomeRecommendationsfor out-of-network candidates, ancestors, and quoted postsIt then merged both answers into one
HashMap<u64, ...>keyed by tweet id only:The same tweet id is routinely in both sets. The most common case is a followed author's own thread: the root post is an in-network candidate, and the reply in the same thread lists the root in
ancestors(Thunder populatesancestorsfor in-network replies). The same collision happens whenever any other selected candidate quotes or replies to an in-network post.DedupConversationFilterruns after VF, so several posts from one conversation are present at this stage.In every such case the recommendations verdict overwrote the in-network verdict. The followed author's post was then judged under the rules that
visibility-filtering/rules/registry.rsapplies only to recommendations (SpamHighRecall,NsfwHighRecall,DoNotAmplify,NsfwText,FosnrAbuseInsults, NSFW author/tweet flags, DMCA and geo-restricted media, and the OON-only user labels), andVFFilterremoved it from the feed.Expected: an in-network post is evaluated at
TimelineHome.README.md(Filtering): "The same post is allowed to a follower."Actual: if that post is also an ancestor or quoted post of another candidate, it gets the
TimelineHomeRecommendationsverdict and is dropped.The collision also runs the other way: an out-of-network candidate that is also the source of a followed account's repost gets whichever verdict the merge order favours. Neither order is correct for both cases.
Why it matters
High-recall labels (spam, NSFW, abuse-insults) are OON-only on purpose because they trade precision for recall. "Freedom of speech, not reach" labels are likewise meant to limit recommendation, not remove a post from people who chose to follow the author. This bug silently converts those reach limits into removal for followers, and does so most often for authors whose posts get replies and quotes. Nothing in the Under the Hood report or the VF response indicates the wrong safety level was used.
Fix
Keep the two result maps separate (
VfVerdicts { in_network, oon }) and route each lookup to the map matching how the id was bucketed:in_networkflagNo VF rule changes, no additional RPCs, no change to which ids are requested at which level.
Tests
The file had no tests. Added unit tests for both collision directions, ancillary routing, tombstoned ancestors, interstitials, and error propagation, plus an end-to-end
hydrate()test using a fakeVfClientthat answers Allow atTimelineHomeand Drop atTimelineHomeRecommendations.home-mixerships without a Cargo manifest, so the file was compiled and tested verbatim (via#[path]) in a scratch crate with stub crates for the internal dependencies, on Rust 1.89. All 9 tests pass. Running only the end-to-end test against the unpatched file fails with the root post carryingSome(PossiblyUndesirable)instead ofNone.Relation to upstream (xai-org/x-algorithm)
Upstream commit
6384ca7(2026-09-01) changed the merge tooon_result.into_iter().chain(in_network_result)so the in-network verdict wins on collision. That fixes the follower-suppression direction but keeps a single map keyed by id, so an out-of-network candidate that is also a followed account's repost source now receives the more lenientTimelineHomeverdict. This fork is based on the pre-6384ca7tree and still has the original merge order.Upstream-ready note: the same
VfVerdictssplit applies cleanly on top of upstream tip (the surrounding code is identical apart from theTweetVisibilitywrapper and thein_network_idsdedup from xai-org#88). It removes the residual collision without changing which safety level each id is requested at.