feat: add JSON Schema validation support with custom chai assertion#7301
feat: add JSON Schema validation support with custom chai assertion#7301sanish-bruno wants to merge 2 commits intousebruno:mainfrom
Conversation
- Introduced a new custom assertion for JSON Schema validation in chai, allowing users to validate response bodies against defined schemas. - Updated the postman translation logic to translate `pm.response.to.have.jsonSchema` to the new assertion format. - Enhanced tests to cover various scenarios for JSON Schema validation, ensuring accurate translations and functionality. - Updated package dependencies to include the latest versions of relevant libraries.
sanish-bruno
requested review from
bijin-bruno,
helloanoop,
lohit-bruno and
naman-bruno
as code owners
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughThis PR adds JSON Schema validation support to Bruno's assertion system. It enables translation of Postman's Changes
Sequence DiagramsequenceDiagram
participant User as User Script
participant Translator as Postman-to-Bruno<br/>Translator
participant Runtime as Runtime<br/>Assertion
participant Ajv as Ajv<br/>Validator
participant Response as Response<br/>Body
User->>Translator: pm.response.to.have.jsonSchema(schema)
Translator->>Translator: Complex Transform
Translator->>Runtime: expect(res.getBody()).to.have.jsonSchema(schema)
Runtime->>Ajv: new Ajv(options)
Runtime->>Ajv: compile(schema)
Runtime->>Response: getBody()
Ajv->>Ajv: validate(compiledSchema, body)
alt Valid
Ajv-->>Runtime: ✓ Success
Runtime-->>User: Assertion passed
else Invalid
Ajv-->>Runtime: ✗ Errors
Runtime-->>User: Assertion failed with errors
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
packages/bruno-converters/tests/postman/postman-translations/transpiler-tests/response.test.js (1)
680-720: Add one guard-case test for over-translation.Consider adding a case where a non-Postman chain (e.g.
customObj.to.have.jsonSchema(schema)) stays unchanged. That protects against false-positive rewrites while this rule expands.As per coding guidelines: “Cover both the 'happy path' and the realistically problematic paths. Validate expected success behaviour, but also validate error handling, edge cases, and degraded-mode behaviour”.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/bruno-converters/tests/postman/postman-translations/transpiler-tests/response.test.js` around lines 680 - 720, Add a guard-case test to ensure the jsonSchema translator doesn't overreach: create a new it block in response.test.js that calls translateCode with a non-Postman chain like "customObj.to.have.jsonSchema(schema);" and assert the output remains unchanged (e.g., expect(translatedCode).toBe('customObj.to.have.jsonSchema(schema);')). Place this alongside the existing pm.response tests so the translator (translateCode) is validated for both correct rewrites and a no-op pass-through when the receiver is not pm.response (or resp alias).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/bruno-js/src/sandbox/bundle-libraries.js`:
- Line 4: The package is requiring '@rollup/plugin-json' (see
require('@rollup/plugin-json') and the json variable used in the Rollup config),
but it isn't declared in packages/bruno-js/package.json; add
'@rollup/plugin-json' to packages/bruno-js package.json devDependencies (use the
same version used in the repo root or a compatible semver) so the module
resolves reliably across package managers, then run install to update the
lockfile.
In `@packages/bruno-js/src/sandbox/quickjs/shims/test.js`:
- Around line 87-100: The jsonSchema assertion (proto.jsonSchema) should handle
negation and schema compile errors: wrap ajv.compile(schema) in try/catch and
convert compile errors into a normalized assertion error, then use
this.assert(condition, positiveMsg, negativeMsg) instead of throwing
DummyChaiAssertionError directly so Chai will respect .not; specifically, catch
errors from ajv.compile and rethrow as a Chai assertion failure, compute isValid
via the compiled validate(data), and call this.assert(isValid, 'expected value
to match JSON schema, validation errors: ' + JSON.stringify(validate.errors),
'expected value to not match JSON schema'); reference proto.jsonSchema,
ajv.compile, validate, validate.errors, and this.assert when making the changes.
---
Nitpick comments:
In
`@packages/bruno-converters/tests/postman/postman-translations/transpiler-tests/response.test.js`:
- Around line 680-720: Add a guard-case test to ensure the jsonSchema translator
doesn't overreach: create a new it block in response.test.js that calls
translateCode with a non-Postman chain like
"customObj.to.have.jsonSchema(schema);" and assert the output remains unchanged
(e.g., expect(translatedCode).toBe('customObj.to.have.jsonSchema(schema);')).
Place this alongside the existing pm.response tests so the translator
(translateCode) is validated for both correct rewrites and a no-op pass-through
when the receiver is not pm.response (or resp alias).
ℹ️ Review info
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (8)
packages/bruno-converters/src/postman/postman-translations.jspackages/bruno-converters/src/utils/postman-to-bruno-translator.jspackages/bruno-converters/tests/postman/postman-translations/transpiler-tests/response.test.jspackages/bruno-js/package.jsonpackages/bruno-js/src/runtime/assert-runtime.jspackages/bruno-js/src/sandbox/bundle-libraries.jspackages/bruno-js/src/sandbox/quickjs/shims/test.jspackages/bruno-tests/collection/scripting/inbuilt modules/tv4/json-schema-assertion.bru
| const rollup = require('rollup'); | ||
| const { nodeResolve } = require('@rollup/plugin-node-resolve'); | ||
| const commonjs = require('@rollup/plugin-commonjs'); | ||
| const json = require('@rollup/plugin-json'); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== Searching package manifests for `@rollup/plugin-json` ==="
fd -a package.json | while read -r manifest; do
dep="$(jq -r '.dependencies["@rollup/plugin-json"] // empty' "$manifest")"
devdep="$(jq -r '.devDependencies["@rollup/plugin-json"] // empty' "$manifest")"
if [ -n "$dep" ] || [ -n "$devdep" ]; then
echo "$manifest => dependencies:$dep devDependencies:$devdep"
fi
done
echo ""
echo "=== Checking packages/bruno-js/package.json devDependencies ==="
if [ -f packages/bruno-js/package.json ]; then
jq -r '.devDependencies | keys[] | select(startswith("@rollup"))' packages/bruno-js/package.json || true
fi
echo ""
echo "=== Checking bundle-libraries.js for `@rollup/plugin-json` usage ==="
if [ -f packages/bruno-js/src/sandbox/bundle-libraries.js ]; then
echo "Lines with `@rollup/plugin-json`:"
rg "@rollup/plugin-json" packages/bruno-js/src/sandbox/bundle-libraries.js -A 2 -B 2 || true
fiRepository: usebruno/bruno
Length of output: 839
Add @rollup/plugin-json as a devDependency in packages/bruno-js/package.json.
The code requires @rollup/plugin-json (line 4) and uses it in the Rollup configuration (line 63), but it's not declared in the package manifest. While the dependency may be available via workspace hoisting from the root package.json, it should be explicitly declared to ensure reliable resolution across different package manager configurations.
Proposed fix
--- a/packages/bruno-js/package.json
+++ b/packages/bruno-js/package.json
@@
"devDependencies": {
"@rollup/plugin-commonjs": "^23.0.2",
+ "@rollup/plugin-json": "^6.1.0",
"@rollup/plugin-node-resolve": "^15.0.1",
"rollup": "3.29.5",
"rollup-plugin-terser": "^7.0.2"
}Also applies to: lines 63
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/bruno-js/src/sandbox/bundle-libraries.js` at line 4, The package is
requiring '@rollup/plugin-json' (see require('@rollup/plugin-json') and the json
variable used in the Rollup config), but it isn't declared in
packages/bruno-js/package.json; add '@rollup/plugin-json' to packages/bruno-js
package.json devDependencies (use the same version used in the repo root or a
compatible semver) so the module resolves reliably across package managers, then
run install to update the lockfile.
| proto.jsonSchema = function(schema, ajvOptions) { | ||
| var Ajv = require('ajv'); | ||
| var ajv = new Ajv(ajvOptions || { allErrors: true }); | ||
| var validate = ajv.compile(schema); | ||
| var data = this._obj; | ||
| var isValid = validate(data); | ||
|
|
||
| if (!isValid) { | ||
| throw new DummyChaiAssertionError( | ||
| 'expected value to match JSON schema, validation errors: ' + JSON.stringify(validate.errors) | ||
| ); | ||
| } | ||
| return this; | ||
| }; |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
find . -name "test.js" -path "*/sandbox/quickjs/shims/*" -type fRepository: usebruno/bruno
Length of output: 111
🏁 Script executed:
cat -n packages/bruno-js/src/sandbox/quickjs/shims/test.js | head -120Repository: usebruno/bruno
Length of output: 4240
🏁 Script executed:
rg "DummyChaiAssertionError" packages/bruno-js/src/sandbox/quickjs/shims/test.js -B 2 -A 2Repository: usebruno/bruno
Length of output: 683
🏁 Script executed:
rg "this.__flags" packages/bruno-js/src/sandbox/quickjs/shims/ -A 2 -B 2Repository: usebruno/bruno
Length of output: 40
🌐 Web query:
Chai custom assertion how to handle .not negation this.assert vs manual __flags
💡 Result:
Use this.assert(...) (or this.assert + proper “positive” and “negated” messages) and let Chai handle .not for you. Chai stores .not as the negate flag, and Assertion#assert automatically flips pass/fail based on that flag. (chaijs.com)
Recommended pattern (don’t manually read __flags)
export default function (chai, utils) {
chai.Assertion.addMethod('startsWith', function (prefix) {
const actual = utils.flag(this, 'object'); // same as this._obj
this.assert(
typeof actual === 'string' && actual.startsWith(prefix), // “truth test”
'expected #{this} to start with #{exp}', // when NOT negated
'expected #{this} to not start with #{exp}', // when negated (.not)
prefix, // expected
actual // actual (optional; defaults to _obj)
);
});
}this.assert uses the negated message automatically when the chain includes .not. (chaijs.com)
When (and how) to check negation explicitly
Only check negation if your assertion’s behavior must change (not just the message). Don’t touch this.__flags (private/internal); use the public flag utility:
const isNegated = utils.flag(this, 'negate');Chai documents negate as the flag set by .not, and utils.flag as the supported API. (chaijs.com)
If you create a new Assertion internally
If you do new chai.Assertion(...) inside your method, copy flags so .not, .deep, etc. propagate:
const a = new chai.Assertion(someValue);
utils.transferFlags(this, a);
a.to.equal(123);transferFlags is the supported way to carry flags between assertion objects. (chaijs.com)
Citations:
- 1: https://www.chaijs.com/guide/plugins/?utm_source=openai
- 2: https://www.chaijs.com/guide/plugins/?utm_source=openai
- 3: https://www.chaijs.com/guide/plugins/?utm_source=openai
- 4: https://www.chaijs.com/api/plugins/?utm_source=openai
🏁 Script executed:
rg "require.*chai" packages/bruno-js/src/sandbox/quickjs/shims/ -A 5Repository: usebruno/bruno
Length of output: 672
jsonSchema method doesn't handle .not negation and misses schema compile errors.
The method only throws when validation fails, breaking expect(value).to.not.have.jsonSchema(schema) semantics—a valid payload should fail the negation assertion but currently passes. Also, schema compilation errors aren't caught and normalized to assertion errors.
The fix exists in your codebase (see the json property at line 74): use this.assert(condition, positivMsg, negatedMsg) which Chai handles negation automatically. Wrap ajv.compile() in try-catch for schema errors.
💡 Suggested approach
proto.jsonSchema = function(schema, ajvOptions) {
var Ajv = require('ajv');
var ajv = new Ajv(ajvOptions || { allErrors: true });
- var validate = ajv.compile(schema);
+ var validate;
+ try {
+ validate = ajv.compile(schema);
+ } catch (error) {
+ throw new DummyChaiAssertionError('Invalid JSON schema: ' + error.message);
+ }
var data = this._obj;
var isValid = validate(data);
- if (!isValid) {
- throw new DummyChaiAssertionError(
- 'expected value to match JSON schema, validation errors: ' + JSON.stringify(validate.errors)
- );
- }
+ this.assert(
+ isValid,
+ 'expected value to match JSON schema, validation errors: ' + JSON.stringify(validate.errors),
+ 'expected value not to match JSON schema'
+ );
return this;
};🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/bruno-js/src/sandbox/quickjs/shims/test.js` around lines 87 - 100,
The jsonSchema assertion (proto.jsonSchema) should handle negation and schema
compile errors: wrap ajv.compile(schema) in try/catch and convert compile errors
into a normalized assertion error, then use this.assert(condition, positiveMsg,
negativeMsg) instead of throwing DummyChaiAssertionError directly so Chai will
respect .not; specifically, catch errors from ajv.compile and rethrow as a Chai
assertion failure, compute isValid via the compiled validate(data), and call
this.assert(isValid, 'expected value to match JSON schema, validation errors: '
+ JSON.stringify(validate.errors), 'expected value to not match JSON schema');
reference proto.jsonSchema, ajv.compile, validate, validate.errors, and
this.assert when making the changes.
1 participant
Summary
Adds JSON Schema validation support to Bruno's test assertions using Ajv, enabling users to validate response bodies against JSON schemas directly in their tests.
What changed:
jsonSchemachai assertion (expect(res.getBody()).to.have.jsonSchema(schema, options?)) powered by Ajv@rollup/plugin-jsonfor Ajv's internal JSON imports)pm.response.to.have.jsonSchema(...)in both the AST transpiler and regex fallbackquickjs-emscriptenfrom 0.29.2 to 0.32.0.brutest file exercising the new assertion (pass, with options, and expected failure)Key files:
packages/bruno-js/src/runtime/assert-runtime.js— Node-side chai pluginpackages/bruno-js/src/sandbox/quickjs/shims/test.js— QuickJS-side assertion shimpackages/bruno-js/src/sandbox/bundle-libraries.js— Ajv bundlingpackages/bruno-converters/src/utils/postman-to-bruno-translator.js— AST transformpackages/bruno-converters/src/postman/postman-translations.js— Regex fallbackpackages/bruno-tests/collection/scripting/inbuilt modules/tv4/json-schema-assertion.bru— E2E testTest plan
npm run test --workspace=packages/bruno-converters)json-schema-assertion.brufilejsonSchemaassertion and confirm it passes/fails as expected🤖 Generated with Claude Code