Skip to content

core: let from end with a step that names each subject - #66

Merged
jbpros merged 3 commits into
mainfrom
core/each-in-from
Oct 2, 2026
Merged

jbpros merged 3 commits into
mainfrom
core/each-in-from

Conversation

@jbpros

@jbpros jbpros commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

First half of #52. Replaces #42.

from can end with {"each_as": "name"}. It isn't called each, because each already means a path into nested lists on all and any:

  • Over an object, every entry is a subject, identified by its key and sorted by key. "keys": [...] says which entries must be there. A missing one is still a subject and fails as absent. Entries that aren't listed aren't checked.
  • Over a list, every item is a subject, as before.
  • A path that starts with $name reads the subject. This matters inside all and any, where a check on an item can now compare it with its subject ("right": ["$pr", "author"]). The row shows the value read through the name beside the list.
"from": ["build", "test_runs", {"each_as": "run", "keys": ["unit-test", "integration-test", "system-test"]}],

A badly written step (not last, more than one, a name that isn't a string or starts with $, keys that isn't a list, or an extra field) fails $well_formed and gives no subjects.

Not in this PR, coming in the second half: as on all/any to name the item, and refs that start with a name. For now {"ref": ["$pr", ...]} fails closed as <invalid ref>. keys can't be a ref yet either.

Changes the report

  • A path whose first key starts with a single $ is now a name. A key like $schema must be written as {"literal": "$schema"}. Without that, the read fails as absent.
  • Inside all/any, a path read through a name or $$input is now in the row's inputs and counted for the cause. A missing $$input value inside an element check used to give cause value, and now gives absent.
  • A missing $$input value in the path of a check inside all/any ("path": ["$$input", ...]) now fails as absent, not value.
  • A literal or selector in from now fails $well_formed. from never read them, so such a requirement found nothing and passed with min_subjects: 0.
  • The $well_formed description, expression and inputs only change for requirements whose from has a step.
  • Custom operators must read the document through value_at and $$input. While ergo checks, input holds ergo's names, not the policy's input.

How names reach the checks

Names travel in input, set once by report and per subject for requirements with a step. A with on data nested inside report's own with data.ergo_document copies the whole document each time, which made every policy about 20 times slower. With input, 1,000 subjects take 0.35s without a step (main: 0.27s) and 0.94s with one, and the time grows in line with the number of subjects (4,000: 1.3s and 3.7s). The evidence can't give names, because report replaces input for the whole evaluation.

Checked

  • opa check --strict, opa fmt, opa test: 458/458, with every line covered. All 416 tests from main pass unchanged.
  • Differential fuzz against main: 4,000 generated policies without naming steps, with $x keys wrapped in literal, give byte-identical reports and violations. Without the wrapping, 34 of 500 differ, so the fuzz does catch the intended change.
  • Property fuzz: 2,000 cases where from: [items, {each_as: "it"}] with every path prefixed by $it gives the same verdicts and causes as plain from: [items].
  • Mutations: dropping each new guard (scope in filter causes, the $ and last-step rules, extra fields, refs to names, missing key as null, name reads in causes, from in $well_formed) makes at least one test fail.
  • Not checked: the Wasm runtime.

🤖 Generated with Claude Code

A from can end with {"each": "name"}. Over an object, every entry becomes a
subject identified by its key, and "keys" says which entries must be there,
so a missing one fails as absent. A path that starts with $name reads the
subject, so a check on a list item can compare it with its subject.

Changes the report: a path whose first key starts with a single $ is now a
name, so a key like $schema has to be written as {"literal": "$schema"}.
Inside all and any, a path read through a name or $$input is now shown in the
row and counted for the cause, so a missing one fails as absent, not value.

Part of #52. Replaces #42.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jbpros jbpros added core Relates to the ergo library or format themselves. impacts-ergo-policies Impacts ergo policy schema. Careful design attention needed. impacts-ergo-reports Impacts ergo report schema. Careful design attention needed. labels Oct 2, 2026
each already means a path into nested lists on all and any, so the step that
names each subject is called each_as instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A with on data nested under report's own with copies the whole document,
which made every policy about 20 times slower. Names now travel in input,
which report sets once, so requirements without a step pay nothing and
ones with a step grow in line with their subjects.

Also count a name or $$input read by an element check's own path towards
its cause, let an id start with a name, name a badly written from as
invalid in $min_subjects, and reword the $well_formed description.

Changes the report: a missing $$input value in the path of a check inside
all or any now fails as absent, not value. A literal or selector in from now
fails $well_formed, where it used to find nothing and could pass with
min_subjects: 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jbpros
jbpros marked this pull request as ready for review October 2, 2026 07:16
@jbpros
jbpros marked this pull request as draft October 2, 2026 07:19
@jbpros
jbpros marked this pull request as ready for review October 2, 2026 07:25
@jbpros
jbpros merged commit e1b4466 into main Oct 2, 2026
3 checks passed
@jbpros
jbpros deleted the core/each-in-from branch October 2, 2026 07:25
jbpros added a commit that referenced this pull request Oct 2, 2026
A misspelt name in an all or any's each path read nothing, so the list had
no items and the check failed with cause value. As a filter, that ruled
every subject out. The read now counts, so it fails as absent.

Changes the report: an all or any whose each path starts with a name that
can't be read now fails as absent, not value. Main has this since names
were added in #66.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jbpros added a commit that referenced this pull request Oct 2, 2026
…e other (#69)

Second half of #52.

An `all` or `any` can now hold another `all` or `any`, an `any_of`
option can hold an `all` or `any`, and `as` names the outer item so the
inner check can still read it. This is what SDLC-CTRL-0007 needs to drop
its glue. Its peer approval needs three things of the same approver, so
they sit in one `any_of` option:

```rego
"from": ["pull_requests", {"each_as": "pr"}],
...
{"op": "any", "path": ["approvers"], "as": "approver", "check": {"op": "any_of", "options": {"peer": [
	{"op": "equals", "path": ["state"], "value": "APPROVED"},
	{"op": "compare", "left": ["username"], "right": ["$pr", "author"], "cmp": "ne"},
	{"op": "all", "path": ["$pr", "commits"], "check":
		{"op": "compare_time", "left": ["$approver", "timestamp"], "right": ["timestamp"], "cmp": "gt"}},
]}}}
```

It renders as `some approvers as $approver: one of: peer(state ==
APPROVED and username ne $pr.author and every $pr.commits:
$approver.timestamp gt timestamp)`.

I rewrote 0007 in a scratch copy of sdlc-policies with all its glue
removed (148 lines to 120): 23 of its 25 tests pass against this branch.
The other two expect cause `value` for a pull request with no author. It
now fails as `absent`, with `$pr.author: null` in the row, which is more
accurate. A commit with no timestamp fails the check, which is the bug
in 0007's glue described in #52.

## Rules

- `as` takes the same names as `each_as`. With `each`, it names the
inner item.
- A name given twice along a chain (by `from`, an outer `as`, or an
inner `as`), or a badly written one, fails the check with cause `absent`
and shows as `<name given twice>` or `<invalid name>`. It isn't `value`,
because in `applies_to` that would rule every subject out and let the
requirement pass with `min_subjects: 0`. Separate checks can reuse a
name.
- One level of nesting. An `any_of` doesn't count as a level, but an
`all`/`any` in its options does. A third `all`/`any` fails as `absent`
and shows `<nested too deep>`. It can't be allowed by accident: letting
the innermost check be another list check makes Rego reject the policy
as recursive.
- Inner lists must be non-empty, like outer ones.

## Rows

Option A from the discussion: the row shows the lists and the values
read through outer names (`approvers[]`, `$pr.commits`), but not which
approver failed. Names given by `as` belong to one item, so reads inside
the list check that gives them aren't shown in the row and don't decide
the cause. A read of the same name anywhere else counts, and fails as
`absent` because nothing gives it there. A path starting with one is
shown as a path inside the item (`approvers[].timestamp`). Showing which
item failed is #68.

## Changes the report

- An `all`/`any` whose `each` path starts with a name that can't be read
now fails as `absent`, not `value`. That hole has been on main since
#66.

- An `all`/`any` in an `any_of` option used to fail with cause `value`
whatever the input, and render as a blank. It's now evaluated and
rendered.

- An `all`/`any` whose inner check reads a path starting with `$$input`
or a name now shows the list beside that value. Before, only the value
was shown. The test from #64 that pins "shown once" still holds, with
the list added.

## Checked

- `opa check --strict`, `opa fmt`, `opa test`: 499/499, with every line
covered.
- Differential fuzz against main: 3,000 generated policies give
byte-identical reports and violations.
- Property fuzz, 2,000 cases with data shaped so a good share of rows
pass (about 1,400 passing rows in total):
- `as: "it"` with `$it` paths gives the same verdicts and causes as no
`as`.
- `all xs: all k: check` gives the same verdicts as `all xs` with `each:
["k"]`.
- Nested `all`/`any` with `as` names in their paths give the same
verdicts as without.
- Property fuzz, 1,600 more cases: a list check wrapped alone in an
`any_of` option gives the same verdicts as the list check, at the top
and inside an outer list, and adding a condition to that option never
makes it pass more.
- Mutations: dropping any of the 25 guards (8 for nesting and names, 7
for the causes and expressions of badly written list checks, 10 for list
checks in `any_of`) makes a test fail, and so does dropping any of the 3
that keep a name to its own list. One mutation hit a guard that turned
out to be redundant, so it and two copies of it were removed.
- Oracle fuzz, 3,600 rows: random nested checks with names given,
misspelt, clashing or out of scope. A Python oracle works out from the
structure which ones are broken or read a name nothing gives. Broken
ones always fail as `absent`, unbound reads never fail as `value`, and
as a filter none lets the requirement pass. Run against the commit
before the scoping fix, it finds that regression, so it can see this
kind of bug.
- Speed: 1,000 PRs × 3 approvers × 5 commits take 3.4s, and time grows
in line with input size (500: 1.8s, 2,000: 6.6s).
- Wasm: nested checks with names give the same rows as `opa eval`, apart
from the check order already in #67.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Relates to the ergo library or format themselves. impacts-ergo-policies Impacts ergo policy schema. Careful design attention needed. impacts-ergo-reports Impacts ergo report schema. Careful design attention needed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant