Repository navigation
Formula guardrail: cel-js arithmetic silently returns null (double × int + bare identifiers) #1928
Description
Activity
Feasibility verified (cel-js 7.6.1) — both compile-time AND runtime fixes work
Spiked both approaches against the pinned cel-js. Yes, these bugs can be caught at metadata-compile time — and there's an even cleaner runtime fix that removes the author burden entirely.
A. Compile-time type check (answers "can the build catch it?") ✅
The build env currently uses
unlistedVariablesAreDyn: true, so every field isdynand nothing type-checks. If the build-time validator instead declaresrecord/input/previouswith the schema's real field types and runs cel-jscheck():const env = new Environment({ unlistedVariablesAreDyn: false }) .registerType('Rec', { fields: { amount: 'double', probability: 'double', status: 'string' } }) .registerVariable('record', 'Rec'); env.parse('record.amount / 100').check(); // → { valid: false, error: 'no such overload: double / int' } env.parse('record.amount / 100.0').check(); // → { valid: true } env.parse('record.amont / 100.0').check(); // → { valid: false, error: 'No such key: amont' }
So a typed check() catches a superset of what we hand-fixed in #1927:
double / intarithmetic overload → flagged- unknown/typo field (
amont) → flagged - bare identifiers (
statusinstead ofrecord.status) → flagged as undeclared variable (withunlistedVariablesAreDyn:false)
This is a build-time-only environment (runtime intentionally stays
dyn+ string-hydration retry), wired intovalidate-expressions.ts(ADR-0032) which already has the object schema. Needs a field-type → CEL-type map (number/currency/percent→double, text/select→string, checkbox→bool, date/datetime→google.protobuf.Timestamp, lookup→nested type ordyn).B. Runtime fix via
registerOperator(makes the common case Just Work) ✅My earlier note said cel-js rejects operator overloads — that was the wrong API (
registerFunction('_/_…')). The correct API isregisterOperator('double / int', impl):for (const op of ['/','*','+','-','%']) for (const sig of [`double ${op} int`, `int ${op} double`]) env.registerOperator(sig, NUMERIC_IMPL[op]); env.evaluate('record.amount / 100', { record:{ amount:120000 } }); // → 1200 env.evaluate('record.amount * record.probability / 100', { record:{amount:120000,probability:70} }); // → 84000 env.evaluate('7 / 2', {}); // → 3 (int/int untouched — overload only on MIXED)
~10 lines in
buildEnv/registerStdLib(cel-engine.ts).amount / 100then works with no float-literal burden, and pure int/int keeps integer-division semantics. Composes with the existing string-hydration retry.Recommendation
Ship both: B removes the footgun for authors (no
.0ritual), A turns the genuinely wrong cases (typos, type mismatches, bare identifiers) into actionable build errors. With B in place, the/ 100.0workaround in #1927'sexpected_revenuecan revert to plain/ 100.Strategy: validate every expression, tiered by what can be checked soundly
Re-evaluated against the mission (all metadata is AI-authored → catch mistakes at build). The goal is yes, validate every expression site. But there's a hard design law, learned the hard way:
The build-time checker may only reject what the runtime evaluator would also fail. Anything stricter is a false positive — and since the authors are AIs generating the whole ecosystem, a false-positive gate breaks the very builds it's meant to protect.
So coverage is staged by soundness, not by ambition. Three findings pin the tiers:
- Per-site scope differs. Flow/automation conditions flatten the record's fields to top-level (
statusis valid);Field.formulaand validation predicates bind only therecordnamespace (statusis a silent-null bug). A single rule is wrong for both. - Real-typed checking false-positives. Declaring fields with true types makes
record.lead_score != nullfault (no such overload: double != null) — null-guards are everywhere and correct at runtime. So fields must bedynin the checker, not their real types. - Flow conditions also carry flow variables (
expiring_deals,new_opportunity) that aren't schema-knowable, so a bare identifier there can't be told apart from a typo.
Tiers
Tier Catches Soundness Status 1. Bare ref in record-scope (formula + validation) bare amount→ nullzero false positives (verified) #1931 (hard error) 2. record.Xtypos at all sitesrecord.amontalready sound (regex existence) shipped 3. Bare-field typos in flow conditions staus == "done"needs flow-var awareness → can't hard-gate; edit-distance warning only proposed, advisory 4. Full type-soundness (string vs number, date misuse) wrong-type compares high false-positive surface; must mirror runtime exactly + bake as warnings deferred Tiers 1+2 are the dominant AI-error classes (silent-null formulas, silent-skip rules) and are now hard build errors with zero false positives. Tiers 3–4 are real but need a warning channel (two-tier guardrail) before they can gate —
validateStackExpressionscurrently hard-fails on every issue, so addingseverity: warningis the prerequisite. Recommend tackling that next if we want flow-condition typo coverage.- Per-site scope differs. Flow/automation conditions flatten the record's fields to top-level (
Tier 3 implemented in #1933 (advisory did-you-mean warnings for likely field typos in flow conditions; non-blocking). Tiers now shipped: 1 (record-scope bare refs, hard error — #1931), 2 (record.X existence, regex — pre-existing), 3 (flow-condition typos, warning — #1933). Tier 4 (full type-soundness) remains deferred — it needs the checker to mirror runtime string-hydration/optional-type semantics exactly and a longer warning bake. Runtime arithmetic fix: #1930.
- added 5 commits that reference this issue
on Jun 16, 2026 Status — platform guardrails shipped
The AI-authoring guardrail series for the formula/flow subsystem is now largely landed:
Tier What PR State Runtime double × intarithmetic overloads (amount / 100works)#1930 ✅ 9.7.0 Build (error) bare field refs in record-scope CEL sites #1931 ✅ 9.7.0 Build (warn) did-you-mean for field typos in flattened flow conditions #1933 ✅ 9.7.0 Catalog register the 14 advertised-but-missing CEL fns (daysBetween/abs/round/min/max/upper/lower/contains/…/date/datetime) + drift-guard #1949 ✅ merged (9.8.0) Build (warn) date-EQUALITY ( $in/$eq) vs time values in query filters#1950 ✅ merged (9.8.0) Build (warn) phantom aggregation ( aggregations/groupBy/… → silent no-op)#1950 ✅ merged (9.8.0) Remaining (deferred)
Field.datestores a full timestamp → exact equality against a re-computed timestamp silently never matches. We now lint this (feat(cli): two flow anti-pattern lints — date-equality filters (#1874) + phantom aggregation (#1870) #1950) and the templates use day-window ranges, but the ROOT semantic gap stands: authors naturally writedate_field == X. A real fix = normalizeField.dateto date-only (00:00:00Z) on write, or compare date-only in the query layer. Needs an ADR — changes storage/query semantics, migration risk.- Tier 4 — full expression type-soundness. Still deferred under the design law (the build checker may only reject what runtime would also fail).
Closing the loop on the catalog/runtime mismatch and the date-equality footgun; leaving this open for the two items above.
Remaining item #1 (Field.date date-only semantics) is now spec'd as a proposed ADR: #1962 (ADR-0053). It frames the root fix as the industry-standard date(tz-naive)/datetime(instant) split, staged: Phase 1 = date-as-YYYY-MM-DD-string at the driver boundary (low risk, fixes equality), Phase 2 = reference-timezone model for today()/daysFromNow()/rendering (needs review). Awaiting decision before implementation.
- added 4 commits that reference this issue
on Jul 18, 2026 Closing — the formula/flow guardrail series is fully landed
Both footguns from the original report, plus the whole systemic-prevention program that grew out of them, are shipped. Summary of everything that closed this out:
The two original footguns
double × intarithmetic silently null → fixed at runtime viaregisterOperatormixed-numeric overloads.record.amount / 100now Just Works, no.0ritual. — fix(formula): register mixed double/int arithmetic overloads (#1928) #1930- Bare identifiers in record-scoped expressions (
statusinstead ofrecord.status) → hard build error with a corrective message. — feat(validate): flag bare field refs in record-scoped CEL sites (#1928) #1931
Build-time guardrail tiers (ADR-0032)
Tier What it catches Severity PR 1 bare field refs in record-scoped CEL sites error #1931 2 record.Xfield-existence / typoserror (pre-existing regex) 3 did-you-mean field typos in flattened flow conditions warning #1933 4 text/boolean field misused in arithmetic/ordering vs a number warning #3178 Tier 4 was the last deferred item — it needed a warning channel (shipped in tier 3) and a checker that mirrors runtime exactly (number/date fields →
dyn, equality excluded), so it flags only what the runtime would also fail. It coversField.formula, validation/action/hook/sharing predicates, and flow conditions.Runtime parity
- Schema-aware validation at
registerFlow— a flow registered dynamically (API/Studio, bypassingobjectstack build) now gets the same tier-2/3/4 checks as advisory warnings, via an opt-in object-schema resolver. Strictly additive; the fatal set is unchanged. — feat(automation): schema-aware flow-condition validation at registration (#1928) #3190
Adjacent hardening that came out of this
- CEL stdlib function catalog (
daysBetween/abs/round/upper/…) registered + drift-guarded. — feat(formula): align CEL stdlib catalog with runtime (#1928) #1949 - Date-equality lint + phantom-aggregation lint in query filters. — feat(cli): two flow anti-pattern lints — date-equality filters (#1874) + phantom aggregation (#1870) #1950
- ADR-0053 (
date= tz-naive calendar day,datetime= instant): Phase 1 driver-boundary normalization (fix(driver-sql): Field.date is a tz-naive YYYY-MM-DD calendar day (ADR-0053 Phase 1) #1968) + Phase 2 reference-timezone model — resolver feat(runtime): resolve a reference timezone onto ExecutionContext #1993, read-time formula context fix(objectql): thread execution context into read-time formula evaluation #1988, tz-awaretoday()/daysFromNow()feat(formula): timezone-aware today() / daysFromNow() / daysAgo() #1998, datetime rendering feat(formula,email): render datetime in reference timezone (ADR-0053 Phase 2) (#1981) #2003, localization settings feat(settings): localization settings — platform timezone/language/formats (ADR-0053 Phase 2) (#1928) #2006.
All merged to
main. Closing as completed.
Generated by Claude Code
- added 5 commits that reference this issue
on Jul 18, 2026 - added a commit that references this issue
on Aug 4, 2026
Summary
Two formula-authoring footguns let an AI write a reasonable-looking
Field.formulathat passesobjectstack buildbut silently evaluates tonullat runtime — the failure class our guardrails are meant to eliminate. Both were found during 9.5.x release-prep testing ofexample-crm(fixed there in #1927; this issue tracks the systemic prevention).1. No
double × intarithmetic overload (primary)cel-js (
@marcbachmann/cel-js) types a record field number asdoubleand a bare integer literal asint, and has no mixed arithmetic overload:So
amount / 100,price * 2,total - fee(int literal) all silently null. Only float literals (100.0,2.0) work. Integer literals in arithmetic are ubiquitous → high-frequency footgun.Constraint: cel-js rejects registering operator overloads —
env.registerFunction('_/_(double, int): double', …)→Invalid signature. So there is no trivial overload-registration fix.Options
validate-expressions.ts(ADR-0032), flag a number-typed field used as an arithmetic operand against anintliteral; message: "use a float literal (e.g./ 100.0)". Cheapest, matches the advisory-warning guardrail strategy.intliterals that are arithmetic operands todouble. Must NOT touchint-typed function args (e.g.daysFromNow(30)). More robust (fixes silently), more invasive, needs tests.2. Bare identifiers in formula expressions (secondary)
Formula scope exposes only
{record, previous, input, os}(seebuildScope,packages/formula/src/stdlib.ts) — no field flattening. So:validate-expressions.tsdoesn't flag bare identifiers that fail to resolve. A lint that requiresrecord./input./os.-qualified references in formula expressions would catch it at build.Acceptance
expected_revenue = amount * probability / 100is either rejected at build with an actionable message, or compiles to a workingdoubleformula.Refs: #1927 (example-crm fixes), ADR-0032. Same "build-green / runtime-silent" family as #1877 / #1870 / #1876.