Skip to content

orb(content-lane): frontmatter parser drops a zero-indent YAML sequence #9664

Description

@JSONbored

⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.

Context

parseSimpleFrontmatter in src/review/content-lane/duplicates.ts:73-116 states its contract at
:65-72:

Parse YAML frontmatter into a flat key→string map, capturing each top-level field's value REGARDLESS
of scalar style — inline, quoted, block literal (|), folded (>), or a block/flow sequence. A
regex parser that silently DROPS block/folded/list values would let a contributor hide a
protected-field edit and bypass the protected-edit + duplicate gates.

The sequence branch does not hold that contract. It only consumes lines that begin with whitespace:

} else if (inline === "") {
  const items: string[] = [];
  while (i < lines.length && /^\s/.test(lines[i] ?? "") && (lines[i] ?? "").trim() !== "") {   // :105
    items.push((lines[i] ?? "").replace(/^\s*-\s*/, "").trim());
    i += 1;
  }
  fields[key] = items.join(", ");
}

A zero-indent block sequence is valid YAML and is what js-yaml/gray-matter (the site build's own
parser) accept:

---
author:
- Alice
downloadUrl:
- https://good.example/x.zip
---

Running the parser verbatim over that input yields {"author": "", "downloadUrl": ""}. The same file
with - Mallory / - https://evil.example/x.zip yields the identical {"author": "", "downloadUrl": ""}. protectedFrontmatterChanges (src/review/content-lane/duplicates.ts:168-178)
then compares normalizeProtectedValue("") !== normalizeProtectedValue("") for every protected field,
returns [], and a changed author/downloadUrl produces no protected close — exactly the bypass
the doc comment says is prevented.

The parser was hand-copied into src/review/content-lane/source-evidence.ts:130-186 (the copy the
already-closed #8016 added the sequence branch to); it carries the identical zero-indent gap at
src/review/content-lane/source-evidence.ts:173, where the effect is that the field's URLs are never
fetched or verified at all. findDuplicateFrontmatterKeys
(src/review/content-lane/duplicates.ts:125-155) has the same while at :151 and must stay in
lockstep with the parser (its own doc: "mirroring the parser's block-scalar / sequence skipping").

NOT a duplicate of #8016 (CLOSED): that issue added the sequence branch to source-evidence.ts
because it was missing entirely. This issue is that the branch — in both copies — never handled the
zero-indent form.

Requirements

  • The sequence branch in parseSimpleFrontmatter (src/review/content-lane/duplicates.ts) must
    consume a following line that starts with - at ANY indentation, including zero, and stop at the
    first line that is neither indented nor a - item (i.e. the next top-level key or a blank line).
  • The identical change must be applied to the copy in src/review/content-lane/source-evidence.ts:173.
  • findDuplicateFrontmatterKeys's skip loop (src/review/content-lane/duplicates.ts:151) must skip
    the same lines the parser now consumes, so a zero-indent - item is never mistaken for a top-level
    key and never reported as a duplicate key.
  • A top-level key line must still win over a sequence item: a line matching
    /^([A-Za-z][A-Za-z0-9_]*):/ terminates the sequence even at zero indent.
  • Behaviour for indented sequences, block scalars, flow sequences, and inline scalars must be unchanged.

⚠️ Required pattern: keep the two parser copies byte-equivalent in this branch, and keep
findDuplicateFrontmatterKeys's skip loop structurally identical to the parser's consume loop — the
two are already documented as mirrors. What does NOT satisfy this issue: fixing only
duplicates.ts and leaving source-evidence.ts:173; adding a YAML library dependency (the module
header states this parser exists specifically to avoid one); extracting a shared helper into a new
module without updating both call sites and the duplicate-key skipper; or handling zero-indent by
loosening the loop to accept any non-empty line (that would swallow the next top-level key).

Deliverables

  • parseSimpleFrontmatter in src/review/content-lane/duplicates.ts returns
    {author: "Alice", downloadUrl: "https://good.example/x.zip"} for the zero-indent example above,
    asserted by a new named case in test/unit/content-lane-duplicates.test.ts.
  • parseSimpleFrontmatter in src/review/content-lane/source-evidence.ts returns the same for the
    same input, asserted in test/unit/content-lane-source-evidence.test.ts.
  • A named regression test asserts protectedFrontmatterChanges returns ["author"] when the
    before/after files differ only in a zero-indent author: sequence value.
  • A test asserts findDuplicateFrontmatterKeys returns [] (not a false duplicate) for a file
    containing two different keys each carrying a zero-indent sequence.
  • A test asserts a zero-indent sequence terminates at the next top-level key (the following key's
    value is parsed correctly, not swallowed into the sequence).

All Deliverables above are required in a single PR. A PR that satisfies only some of them — for
example fixing duplicates.ts and its test (Deliverables 1 and 3) while leaving the
source-evidence.ts copy untouched — does not resolve this issue.

Test Coverage Requirements

This repo enforces 99%+ Codecov patch coverage, branch-counted, and vitest.config.ts's
coverage.include covers src/**/*.ts — both files are measured. Both arms of the widened loop
condition need coverage: an indented item, a zero-indent - item, a following top-level key that
terminates the sequence, and a blank line that terminates it. The protected-edit regression test in
Deliverable 3 is required by name.

Expected Outcome

A contributor cannot hide an edit to a protected frontmatter field (author, downloadUrl, …) by
authoring it as a zero-indent YAML sequence, and the source-evidence gate actually fetches and verifies
source URLs written that way. Both parser copies and the duplicate-key scanner agree on where a
sequence ends.

Links & Resources

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions