Add --only-failures flag to bun:test #23312
Conversation
This flag only displays test failures, similar to --dots but without printing dots for each test. This is useful for CI environments or when running large test suites where you only care about failures. Usage: bun test --only-failures The flag works by: - Suppressing output for passing, skipped, and todo tests - Only showing full error details for failing tests - Still showing the final test summary with pass/fail counts 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
|
Updated 5:20 PM PT - Oct 8th, 2025
❌ @pfgithub, your commit b28d83c has 1 failures in
🧪 To try this PR locally: bunx bun-pr 23312That installs a local version of the PR into your bun-23312 --bun |
WalkthroughAdds an --only-failures reporter mode and wires it through CLI args and bunfig parsing, TestCommand reporter state, and bun.js test reporters; updates reporter conditionals to suppress non-failure output and dot printing; and adds fixtures and tests validating the behavior. Changes
Possibly related PRs
Suggested reviewers
Pre-merge checks❌ Failed checks (1 warning)
✅ Passed checks (1 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
test/js/bun/test/only-failures.test.ts (1)
59-79: Consider using inline snapshots for consistency.The multi-file test uses manual assertions instead of snapshots. For consistency and better maintainability, consider using
toMatchInlineSnapshot()ortoMatchSnapshot()to capture the full output, similar to the first test.Apply this diff to use inline snapshots:
- expect(exitCode).toBe(1); - expect(normalizeBunSnapshot(stderr)).toContain("(fail) failing test"); - expect(normalizeBunSnapshot(stderr)).toContain("(fail) another failing test"); - expect(normalizeBunSnapshot(stderr)).not.toContain("(pass)"); + expect({ + exitCode, + stderr: normalizeBunSnapshot(stderr), + }).toMatchInlineSnapshot();Then update the snapshot to capture the expected output. This provides better documentation and catches unexpected output changes.
src/cli/test_command.zig (1)
904-904: Consider optimizing the repeat buffer printing logic.The logic correctly prevents queueing output when
only_failuresis true. However, the repeat buffer printing at lines 1532-1563 doesn't explicitly checkonly_failures, so it will attempt to print from empty buffers when the flag is set. While harmless, you could optimize by addingand !reporter.reporters.only_failuresto the condition at line 1532.Apply this diff to optimize the repeat buffer printing:
- if (reporter.summary().pass > 20 and !Output.isAIAgent() and !reporter.reporters.dots) { + if (reporter.summary().pass > 20 and !Output.isAIAgent() and !reporter.reporters.dots and !reporter.reporters.only_failures) {
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (7)
src/bun.js/test/bun_test.zig(1 hunks)src/bun.js/test/jest.zig(1 hunks)src/cli.zig(1 hunks)src/cli/Arguments.zig(2 hunks)src/cli/test_command.zig(3 hunks)test/js/bun/test/only-failures.fixture.ts(1 hunks)test/js/bun/test/only-failures.test.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (11)
src/**/*.zig
📄 CodeRabbit inference engine (.cursor/rules/building-bun.mdc)
Implement debug logs in Zig using
const log = bun.Output.scoped(.${SCOPE}, false);and invokinglog("...", .{})In Zig code, manage memory carefully and use defer for cleanup of allocations/resources
src/**/*.zig: In Zig source files, place all @import statements at the bottom of the file
Use @import("bun") instead of @import("root").bun when importing Bun in Zig
Files:
src/bun.js/test/bun_test.zigsrc/cli.zigsrc/cli/test_command.zigsrc/cli/Arguments.zigsrc/bun.js/test/jest.zig
**/*.zig
📄 CodeRabbit inference engine (.cursor/rules/javascriptcore-class.mdc)
**/*.zig: Declare the extern C symbol in Zig and export a Zig-friendly alias for use
Wrap the Bun____toJS extern in a Zig method that takes a JSGlobalObject and returns JSC.JSValue
Files:
src/bun.js/test/bun_test.zigsrc/cli.zigsrc/cli/test_command.zigsrc/cli/Arguments.zigsrc/bun.js/test/jest.zig
src/bun.js/**/*.zig
📄 CodeRabbit inference engine (.cursor/rules/zig-javascriptcore-classes.mdc)
src/bun.js/**/*.zig: In Zig binding structs, expose generated bindings via pub const js = JSC.Codegen.JS and re-export toJS/fromJS/fromJSDirect
Constructors and prototype methods should return bun.JSError!JSC.JSValue to integrate Zig error handling with JS exceptions
Use parameter name globalObject (not ctx) and accept (*JSC.JSGlobalObject, *JSC.CallFrame) in binding methods/constructors
Implement getters as get(this, globalObject) returning JSC.JSValue and matching the .classes.ts interface
Provide deinit() for resource cleanup and finalize() that calls deinit(); use bun.destroy(this) or appropriate destroy pattern
Access JS call data via CallFrame (argument(i), argumentCount(), thisValue()) and throw errors with globalObject.throw(...)
For properties marked cache: true, use the generated Zig accessors (NameSetCached/GetCached) to work with GC-owned values
In finalize() for objects holding JS references, release them using .deref() before destroy
Files:
src/bun.js/test/bun_test.zigsrc/bun.js/test/jest.zig
test/**
📄 CodeRabbit inference engine (.cursor/rules/writing-tests.mdc)
Place all tests under the test/ directory
Files:
test/js/bun/test/only-failures.test.tstest/js/bun/test/only-failures.fixture.ts
test/js/**/*.{js,ts}
📄 CodeRabbit inference engine (.cursor/rules/writing-tests.mdc)
Place JavaScript and TypeScript tests under test/js/
Files:
test/js/bun/test/only-failures.test.tstest/js/bun/test/only-failures.fixture.ts
test/js/bun/**/*.{js,ts}
📄 CodeRabbit inference engine (.cursor/rules/writing-tests.mdc)
Place Bun API tests under test/js/bun/, separated by category (e.g., test/js/bun/glob/)
Files:
test/js/bun/test/only-failures.test.tstest/js/bun/test/only-failures.fixture.ts
test/**/*.{js,ts}
📄 CodeRabbit inference engine (.cursor/rules/writing-tests.mdc)
test/**/*.{js,ts}: Write tests in JavaScript or TypeScript using Bun’s Jest-style APIs (test, describe, expect) and run with bun test
Prefer data-driven tests (e.g., test.each) to reduce boilerplate
Use shared utilities from test/harness.ts where applicable
Files:
test/js/bun/test/only-failures.test.tstest/js/bun/test/only-failures.fixture.ts
test/**/*.test.ts
📄 CodeRabbit inference engine (test/CLAUDE.md)
test/**/*.test.ts: Name test files*.test.tsand usebun:test
Do not write flaky tests: never wait for arbitrary time; wait for conditions instead
Never hardcode port numbers in tests; useport: 0to get a random port
When spawning Bun in tests, usebunExe()andbunEnvfromharness
Preferasync/awaitin tests; for a single callback, usePromise.withResolvers()
Do not set explicit test timeouts; rely on Bun’s built-in timeouts
UsetempDir/tempDirWithFilesfromharnessfor temporary files and directories in tests
For large/repetitive strings in tests, preferBuffer.alloc(count, fill).toString()over"A".repeat(count)
Import common test utilities fromharness(e.g.,bunExe,bunEnv,tempDirWithFiles,tmpdirSync, platform checks, GC helpers)
In error tests, assert non-zero exit codes for failing processes and usetoThrowfor synchronous errors
Usedescribeblocks for grouping,describe.eachfor parameterized tests, snapshots withtoMatchSnapshot, and lifecycle hooks (beforeAll,beforeEach,afterEach); track resources for cleanup inafterEach
Useusing/await usingwith Bun resources (e.g., Bun.listen/connect/spawn/serve) to ensure cleanup in tests
Files:
test/js/bun/test/only-failures.test.ts
test/js/**
📄 CodeRabbit inference engine (test/CLAUDE.md)
Organize unit tests for specific features under
test/js/by module
Files:
test/js/bun/test/only-failures.test.tstest/js/bun/test/only-failures.fixture.ts
test/**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
test/**/*.test.{ts,tsx}: Test files must live under test/ and end with .test.ts or .test.tsx
In tests, always use port: 0; do not hardcode ports or roll your own random port
Prefer normalizeBunSnapshot for snapshotting test output instead of asserting raw strings
Do not write tests that assert absence of crashes (e.g., 'no panic' or 'no uncaught exception')
Use Bun’s Jest-compatible runner (import { test, expect } from "bun:test") for tests
Avoid shell commands like find or grep in tests; use Bun’s Glob and built-in tools instead
Prefer running tests via bun bd test and use provided harness utilities (bunEnv, bunExe, tempDir)
Use Bun.spawn with proper stdio handling and await proc.exited in process-spawning tests
Files:
test/js/bun/test/only-failures.test.ts
test/js/bun/**
📄 CodeRabbit inference engine (CLAUDE.md)
Place Bun-specific API tests under test/js/bun/
Files:
test/js/bun/test/only-failures.test.tstest/js/bun/test/only-failures.fixture.ts
🧠 Learnings (14)
📚 Learning: 2025-10-04T09:51:30.294Z
Learnt from: CR
PR: oven-sh/bun#0
File: CLAUDE.md:0-0
Timestamp: 2025-10-04T09:51:30.294Z
Learning: Applies to test/**/*.test.{ts,tsx} : Use Bun.spawn with proper stdio handling and await proc.exited in process-spawning tests
Applied to files:
test/js/bun/test/only-failures.test.tstest/js/bun/test/only-failures.fixture.ts
📚 Learning: 2025-09-03T17:10:13.486Z
Learnt from: CR
PR: oven-sh/bun#0
File: test/CLAUDE.md:0-0
Timestamp: 2025-09-03T17:10:13.486Z
Learning: Applies to test/**/*.test.ts : Name test files `*.test.ts` and use `bun:test`
Applied to files:
test/js/bun/test/only-failures.test.tstest/js/bun/test/only-failures.fixture.ts
📚 Learning: 2025-09-03T17:10:13.486Z
Learnt from: CR
PR: oven-sh/bun#0
File: test/CLAUDE.md:0-0
Timestamp: 2025-09-03T17:10:13.486Z
Learning: Applies to test/**/*.test.ts : When spawning Bun in tests, use `bunExe()` and `bunEnv` from `harness`
Applied to files:
test/js/bun/test/only-failures.test.tstest/js/bun/test/only-failures.fixture.ts
📚 Learning: 2025-10-04T09:51:30.294Z
Learnt from: CR
PR: oven-sh/bun#0
File: CLAUDE.md:0-0
Timestamp: 2025-10-04T09:51:30.294Z
Learning: Applies to test/**/*.test.{ts,tsx} : Use Bun’s Jest-compatible runner (import { test, expect } from "bun:test") for tests
Applied to files:
test/js/bun/test/only-failures.test.tstest/js/bun/test/only-failures.fixture.ts
📚 Learning: 2025-10-04T09:51:30.294Z
Learnt from: CR
PR: oven-sh/bun#0
File: CLAUDE.md:0-0
Timestamp: 2025-10-04T09:51:30.294Z
Learning: Applies to test/**/*.test.{ts,tsx} : Prefer normalizeBunSnapshot for snapshotting test output instead of asserting raw strings
Applied to files:
test/js/bun/test/only-failures.test.ts
📚 Learning: 2025-08-30T00:12:56.803Z
Learnt from: CR
PR: oven-sh/bun#0
File: .cursor/rules/writing-tests.mdc:0-0
Timestamp: 2025-08-30T00:12:56.803Z
Learning: Applies to test/**/*.{js,ts} : Write tests in JavaScript or TypeScript using Bun’s Jest-style APIs (test, describe, expect) and run with bun test
Applied to files:
test/js/bun/test/only-failures.test.tstest/js/bun/test/only-failures.fixture.ts
📚 Learning: 2025-08-30T00:12:56.803Z
Learnt from: CR
PR: oven-sh/bun#0
File: .cursor/rules/writing-tests.mdc:0-0
Timestamp: 2025-08-30T00:12:56.803Z
Learning: Applies to test/cli/**/*.{js,ts} : When testing Bun as a CLI, use spawn with bunExe() and bunEnv from harness, and capture stdout/stderr via pipes
Applied to files:
test/js/bun/test/only-failures.test.tstest/js/bun/test/only-failures.fixture.ts
📚 Learning: 2025-10-04T09:51:30.294Z
Learnt from: CR
PR: oven-sh/bun#0
File: CLAUDE.md:0-0
Timestamp: 2025-10-04T09:51:30.294Z
Learning: Applies to test/**/*.test.{ts,tsx} : Prefer running tests via bun bd test <file> and use provided harness utilities (bunEnv, bunExe, tempDir)
Applied to files:
test/js/bun/test/only-failures.test.tstest/js/bun/test/only-failures.fixture.ts
📚 Learning: 2025-08-30T00:12:56.803Z
Learnt from: CR
PR: oven-sh/bun#0
File: .cursor/rules/writing-tests.mdc:0-0
Timestamp: 2025-08-30T00:12:56.803Z
Learning: Applies to test/js/bun/**/*.{js,ts} : Place Bun API tests under test/js/bun/, separated by category (e.g., test/js/bun/glob/)
Applied to files:
test/js/bun/test/only-failures.test.tstest/js/bun/test/only-failures.fixture.ts
📚 Learning: 2025-09-08T00:41:12.052Z
Learnt from: CR
PR: oven-sh/bun#0
File: src/bun.js/bindings/v8/CLAUDE.md:0-0
Timestamp: 2025-09-08T00:41:12.052Z
Learning: Applies to src/bun.js/bindings/v8/test/v8/v8.test.ts : Add a corresponding test case in test/v8/v8.test.ts that invokes checkSameOutput with the new function
Applied to files:
test/js/bun/test/only-failures.test.ts
📚 Learning: 2025-10-04T09:51:30.294Z
Learnt from: CR
PR: oven-sh/bun#0
File: CLAUDE.md:0-0
Timestamp: 2025-10-04T09:51:30.294Z
Learning: Applies to test/js/bun/** : Place Bun-specific API tests under test/js/bun/
Applied to files:
test/js/bun/test/only-failures.fixture.ts
📚 Learning: 2025-09-03T17:10:13.486Z
Learnt from: CR
PR: oven-sh/bun#0
File: test/CLAUDE.md:0-0
Timestamp: 2025-09-03T17:10:13.486Z
Learning: Applies to test/**/*-fixture.ts : Name test fixture files that are spawned by tests with the suffix `-fixture.ts`
Applied to files:
test/js/bun/test/only-failures.fixture.ts
📚 Learning: 2025-09-03T17:10:13.486Z
Learnt from: CR
PR: oven-sh/bun#0
File: test/CLAUDE.md:0-0
Timestamp: 2025-09-03T17:10:13.486Z
Learning: Applies to test/**/*.test.ts : Do not set explicit test timeouts; rely on Bun’s built-in timeouts
Applied to files:
test/js/bun/test/only-failures.fixture.ts
📚 Learning: 2025-09-20T03:39:41.770Z
Learnt from: pfgithub
PR: oven-sh/bun#22534
File: test/regression/issue/21830.fixture.ts:14-63
Timestamp: 2025-09-20T03:39:41.770Z
Learning: Bun's test runner supports async describe callbacks, unlike Jest/Vitest where describe callbacks must be synchronous. The syntax `describe("name", async () => { ... })` is valid in Bun.
Applied to files:
test/js/bun/test/only-failures.fixture.ts
🧬 Code graph analysis (1)
test/js/bun/test/only-failures.test.ts (1)
test/harness.ts (2)
bunExe(103-106)normalizeBunSnapshot(1809-1840)
🔇 Additional comments (8)
src/bun.js/test/jest.zig (1)
18-18: LGTM! Consistent integration of only_failures flag.The addition of
reporter.reporters.only_failuresto the condition follows the same pattern as the dots reporter, ensuring that file title and prefix information is properly deferred when the only-failures mode is active.src/cli.zig (1)
357-357: LGTM! Proper field addition.The
only_failuresfield is correctly added to the reporters struct with an appropriate default value offalse, following the same pattern as the existingdotsfield.src/bun.js/test/bun_test.zig (1)
176-176: LGTM! Appropriate error output handling.The addition of
reporter.reporters.only_failuresto the condition ensures that error output is formatted correctly when the only-failures mode is active, consistent with the dots reporter behavior.src/cli/Arguments.zig (1)
464-467: LGTM! Proper flag handling.The flag is correctly parsed and sets the appropriate option field.
test/js/bun/test/only-failures.fixture.ts (1)
1-27: LGTM! Well-designed test fixture.The fixture provides comprehensive test coverage with a good mix of passing, failing, skipped, todo, and error cases, which is ideal for validating the --only-failures flag behavior.
test/js/bun/test/only-failures.test.ts (1)
4-57: LGTM! Comprehensive test coverage.The test properly validates the --only-failures flag behavior using bunExe(), bunEnv, and normalizeBunSnapshot as per coding guidelines. The inline snapshot provides clear documentation of expected output.
src/cli/test_command.zig (2)
582-582: LGTM!The field addition is straightforward and correctly defaults to
false.
878-879: LGTM!The logic correctly suppresses non-failure output when either AI agent mode or
--only-failuresis active. The defensive null check onbuntest.reporteris appropriate.
src/cli/Arguments.zig
Outdated
| clap.parseParam("--reporter <STR> Test output reporter format. Available: 'junit' (requires --reporter-outfile), 'dots'. Default: console output.") catch unreachable, | ||
| clap.parseParam("--reporter-outfile <STR> Output file path for the reporter format (required with --reporter).") catch unreachable, | ||
| clap.parseParam("--dots Enable dots reporter. Shorthand for --reporter=dots.") catch unreachable, | ||
| clap.parseParam("--only-failures Only display test failures. Similar to --dots but without printing dots.") catch unreachable, |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
🧩 Analysis chain
Verify flag interaction between --dots and --only-failures.
The description states that --only-failures is "Similar to --dots but without printing dots," suggesting these are alternative modes. However, the code doesn't prevent using both flags simultaneously.
Should these flags be mutually exclusive? If both are provided, what should the expected behavior be? Consider adding validation to ensure only one reporter mode is active at a time, or document the combined behavior if it's intentional.
🏁 Script executed:
#!/bin/bash
# Verify if there's any logic in test_command.zig that handles the interaction between dots and only_failures
rg -n -A 5 -B 5 "reporters\.(dots|only_failures)" src/cli/test_command.zigLength of output: 3976
🏁 Script executed:
#!/bin/bash
rg -n -C3 "\"--dots" src/cli/Arguments.zig
rg -n -C3 "\"--only-failures" src/cli/Arguments.zigLength of output: 2198
Enforce mutual exclusion between --dots and --only-failures
Use clap’s ArgGroup or add a manual check to error when both --dots and --only-failures are passed, or explicitly document that --dots takes precedence if both are provided.
🤖 Prompt for AI Agents
In src/cli/Arguments.zig around line 210, the new `--only-failures` param can be
passed together with `--dots`, but there is no mutual-exclusion enforcement; add
an ArgGroup to the clap definition that contains both `--dots` and
`--only-failures` and mark it as conflicts, or if ArgGroup isn’t
available/desired add a post-parse check that detects both flags and returns a
user-facing error (or explicitly document and implement precedence such that
`--dots` overrides `--only-failures`), ensuring the CLI fails fast with a clear
message when both are supplied.
🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
docs/runtime/bunfig.md (1)
263-286: Consider documenting the relationship between test.onlyFailures and test.reporter.The documentation shows both
test.onlyFailuresandtest.reporter.dotsas separate top-level options under[test]and[test.reporter]respectively. However, from the code changes insrc/bunfig.zig, it appearsonlyFailuresis parsed directly under the[test]object, not nested under[test.reporter]. The documentation structure is correct, but it might be helpful to clarify that:
test.onlyFailuresis a boolean flag that affects output behavior globallytest.reporter.dotsandtest.reporter.junitare reporter-specific configurationsThis distinction helps users understand the configuration hierarchy.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (5)
docs/runtime/bunfig.md(1 hunks)src/bun.js/test/jest.zig(1 hunks)src/bunfig.zig(1 hunks)src/cli/test_command.zig(4 hunks)test/js/bun/test/only-failures.test.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (11)
src/**/*.zig
📄 CodeRabbit inference engine (.cursor/rules/building-bun.mdc)
Implement debug logs in Zig using
const log = bun.Output.scoped(.${SCOPE}, false);and invokinglog("...", .{})In Zig code, manage memory carefully and use defer for cleanup of allocations/resources
src/**/*.zig: In Zig source files, place all @import statements at the bottom of the file
Use @import("bun") instead of @import("root").bun when importing Bun in Zig
Files:
src/bunfig.zigsrc/cli/test_command.zigsrc/bun.js/test/jest.zig
**/*.zig
📄 CodeRabbit inference engine (.cursor/rules/javascriptcore-class.mdc)
**/*.zig: Declare the extern C symbol in Zig and export a Zig-friendly alias for use
Wrap the Bun____toJS extern in a Zig method that takes a JSGlobalObject and returns JSC.JSValue
Files:
src/bunfig.zigsrc/cli/test_command.zigsrc/bun.js/test/jest.zig
test/**
📄 CodeRabbit inference engine (.cursor/rules/writing-tests.mdc)
Place all tests under the test/ directory
Files:
test/js/bun/test/only-failures.test.ts
test/js/**/*.{js,ts}
📄 CodeRabbit inference engine (.cursor/rules/writing-tests.mdc)
Place JavaScript and TypeScript tests under test/js/
Files:
test/js/bun/test/only-failures.test.ts
test/js/bun/**/*.{js,ts}
📄 CodeRabbit inference engine (.cursor/rules/writing-tests.mdc)
Place Bun API tests under test/js/bun/, separated by category (e.g., test/js/bun/glob/)
Files:
test/js/bun/test/only-failures.test.ts
test/**/*.{js,ts}
📄 CodeRabbit inference engine (.cursor/rules/writing-tests.mdc)
test/**/*.{js,ts}: Write tests in JavaScript or TypeScript using Bun’s Jest-style APIs (test, describe, expect) and run with bun test
Prefer data-driven tests (e.g., test.each) to reduce boilerplate
Use shared utilities from test/harness.ts where applicable
Files:
test/js/bun/test/only-failures.test.ts
test/**/*.test.ts
📄 CodeRabbit inference engine (test/CLAUDE.md)
test/**/*.test.ts: Name test files*.test.tsand usebun:test
Do not write flaky tests: never wait for arbitrary time; wait for conditions instead
Never hardcode port numbers in tests; useport: 0to get a random port
When spawning Bun in tests, usebunExe()andbunEnvfromharness
Preferasync/awaitin tests; for a single callback, usePromise.withResolvers()
Do not set explicit test timeouts; rely on Bun’s built-in timeouts
UsetempDir/tempDirWithFilesfromharnessfor temporary files and directories in tests
For large/repetitive strings in tests, preferBuffer.alloc(count, fill).toString()over"A".repeat(count)
Import common test utilities fromharness(e.g.,bunExe,bunEnv,tempDirWithFiles,tmpdirSync, platform checks, GC helpers)
In error tests, assert non-zero exit codes for failing processes and usetoThrowfor synchronous errors
Usedescribeblocks for grouping,describe.eachfor parameterized tests, snapshots withtoMatchSnapshot, and lifecycle hooks (beforeAll,beforeEach,afterEach); track resources for cleanup inafterEach
Useusing/await usingwith Bun resources (e.g., Bun.listen/connect/spawn/serve) to ensure cleanup in tests
Files:
test/js/bun/test/only-failures.test.ts
test/js/**
📄 CodeRabbit inference engine (test/CLAUDE.md)
Organize unit tests for specific features under
test/js/by module
Files:
test/js/bun/test/only-failures.test.ts
test/**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
test/**/*.test.{ts,tsx}: Test files must live under test/ and end with .test.ts or .test.tsx
In tests, always use port: 0; do not hardcode ports or roll your own random port
Prefer normalizeBunSnapshot for snapshotting test output instead of asserting raw strings
Do not write tests that assert absence of crashes (e.g., 'no panic' or 'no uncaught exception')
Use Bun’s Jest-compatible runner (import { test, expect } from "bun:test") for tests
Avoid shell commands like find or grep in tests; use Bun’s Glob and built-in tools instead
Prefer running tests via bun bd test and use provided harness utilities (bunEnv, bunExe, tempDir)
Use Bun.spawn with proper stdio handling and await proc.exited in process-spawning tests
Files:
test/js/bun/test/only-failures.test.ts
test/js/bun/**
📄 CodeRabbit inference engine (CLAUDE.md)
Place Bun-specific API tests under test/js/bun/
Files:
test/js/bun/test/only-failures.test.ts
src/bun.js/**/*.zig
📄 CodeRabbit inference engine (.cursor/rules/zig-javascriptcore-classes.mdc)
src/bun.js/**/*.zig: In Zig binding structs, expose generated bindings via pub const js = JSC.Codegen.JS and re-export toJS/fromJS/fromJSDirect
Constructors and prototype methods should return bun.JSError!JSC.JSValue to integrate Zig error handling with JS exceptions
Use parameter name globalObject (not ctx) and accept (*JSC.JSGlobalObject, *JSC.CallFrame) in binding methods/constructors
Implement getters as get(this, globalObject) returning JSC.JSValue and matching the .classes.ts interface
Provide deinit() for resource cleanup and finalize() that calls deinit(); use bun.destroy(this) or appropriate destroy pattern
Access JS call data via CallFrame (argument(i), argumentCount(), thisValue()) and throw errors with globalObject.throw(...)
For properties marked cache: true, use the generated Zig accessors (NameSetCached/GetCached) to work with GC-owned values
In finalize() for objects holding JS references, release them using .deref() before destroy
Files:
src/bun.js/test/jest.zig
🧠 Learnings (10)
📚 Learning: 2025-10-04T09:51:30.294Z
Learnt from: CR
PR: oven-sh/bun#0
File: CLAUDE.md:0-0
Timestamp: 2025-10-04T09:51:30.294Z
Learning: Applies to test/**/*.test.{ts,tsx} : Use Bun.spawn with proper stdio handling and await proc.exited in process-spawning tests
Applied to files:
test/js/bun/test/only-failures.test.ts
📚 Learning: 2025-09-03T17:10:13.486Z
Learnt from: CR
PR: oven-sh/bun#0
File: test/CLAUDE.md:0-0
Timestamp: 2025-09-03T17:10:13.486Z
Learning: Applies to test/**/*.test.ts : Name test files `*.test.ts` and use `bun:test`
Applied to files:
test/js/bun/test/only-failures.test.ts
📚 Learning: 2025-08-30T00:12:56.803Z
Learnt from: CR
PR: oven-sh/bun#0
File: .cursor/rules/writing-tests.mdc:0-0
Timestamp: 2025-08-30T00:12:56.803Z
Learning: Applies to test/**/*.{js,ts} : Write tests in JavaScript or TypeScript using Bun’s Jest-style APIs (test, describe, expect) and run with bun test
Applied to files:
test/js/bun/test/only-failures.test.ts
📚 Learning: 2025-08-30T00:12:56.803Z
Learnt from: CR
PR: oven-sh/bun#0
File: .cursor/rules/writing-tests.mdc:0-0
Timestamp: 2025-08-30T00:12:56.803Z
Learning: Applies to test/js/bun/**/*.{js,ts} : Place Bun API tests under test/js/bun/, separated by category (e.g., test/js/bun/glob/)
Applied to files:
test/js/bun/test/only-failures.test.ts
📚 Learning: 2025-09-03T17:10:13.486Z
Learnt from: CR
PR: oven-sh/bun#0
File: test/CLAUDE.md:0-0
Timestamp: 2025-09-03T17:10:13.486Z
Learning: Applies to test/**/*.test.ts : When spawning Bun in tests, use `bunExe()` and `bunEnv` from `harness`
Applied to files:
test/js/bun/test/only-failures.test.ts
📚 Learning: 2025-10-04T09:51:30.294Z
Learnt from: CR
PR: oven-sh/bun#0
File: CLAUDE.md:0-0
Timestamp: 2025-10-04T09:51:30.294Z
Learning: Applies to test/**/*.test.{ts,tsx} : Prefer running tests via bun bd test <file> and use provided harness utilities (bunEnv, bunExe, tempDir)
Applied to files:
test/js/bun/test/only-failures.test.ts
📚 Learning: 2025-10-04T09:51:30.294Z
Learnt from: CR
PR: oven-sh/bun#0
File: CLAUDE.md:0-0
Timestamp: 2025-10-04T09:51:30.294Z
Learning: Applies to test/**/*.test.{ts,tsx} : Use Bun’s Jest-compatible runner (import { test, expect } from "bun:test") for tests
Applied to files:
test/js/bun/test/only-failures.test.ts
📚 Learning: 2025-10-04T09:51:30.294Z
Learnt from: CR
PR: oven-sh/bun#0
File: CLAUDE.md:0-0
Timestamp: 2025-10-04T09:51:30.294Z
Learning: Applies to test/**/*.test.{ts,tsx} : Prefer normalizeBunSnapshot for snapshotting test output instead of asserting raw strings
Applied to files:
test/js/bun/test/only-failures.test.ts
📚 Learning: 2025-08-30T00:12:56.803Z
Learnt from: CR
PR: oven-sh/bun#0
File: .cursor/rules/writing-tests.mdc:0-0
Timestamp: 2025-08-30T00:12:56.803Z
Learning: Applies to test/cli/**/*.{js,ts} : When testing Bun as a CLI, use spawn with bunExe() and bunEnv from harness, and capture stdout/stderr via pipes
Applied to files:
test/js/bun/test/only-failures.test.ts
📚 Learning: 2025-10-04T09:51:30.294Z
Learnt from: CR
PR: oven-sh/bun#0
File: CLAUDE.md:0-0
Timestamp: 2025-10-04T09:51:30.294Z
Learning: Applies to test/js/bun/** : Place Bun-specific API tests under test/js/bun/
Applied to files:
test/js/bun/test/only-failures.test.ts
🧬 Code graph analysis (1)
test/js/bun/test/only-failures.test.ts (1)
test/harness.ts (3)
bunExe(101-104)normalizeBunSnapshot(1807-1838)tempDir(276-283)
🔇 Additional comments (10)
docs/runtime/bunfig.md (1)
252-261: LGTM! Clear and concise documentation.The documentation for
test.onlyFailuresis well-structured and provides a clear example. The note about the CLI flag equivalence is helpful.test/js/bun/test/only-failures.test.ts (3)
4-57: LGTM! Comprehensive snapshot test for single file.The test properly:
- Uses
bunExe()andbunEnvfrom harness- Captures stdout and stderr
- Normalizes output with
normalizeBunSnapshot- Validates the complete output structure with an inline snapshot
The snapshot confirms that only failing tests are displayed while the summary includes all test counts.
59-79: LGTM! Good multi-file test with targeted assertions.The test validates that
--only-failuresworks across multiple files by:
- Running tests from two different fixture files
- Checking for presence of "(fail)" markers
- Confirming absence of "(pass)" markers
- Verifying the exit code is 1 (due to failures)
This approach is more maintainable than a full snapshot for multi-file output.
81-120: LGTM! Proper bunfig.toml integration test with resource cleanup.The test correctly:
- Uses
usingsyntax withtempDirfor automatic cleanup (line 82)- Creates a temporary directory with bunfig.toml and test file
- Spawns bun test with the correct working directory (line 109)
- Validates that bunfig configuration is respected
The use of
normalizeBunSnapshot(stderr, dir)on line 117 properly normalizes the temporary directory path.src/bun.js/test/jest.zig (1)
18-18: LGTM! Consistent with the new only_failures flag architecture.The condition now checks
reporter.reporters.only_failuresinstead of directly detecting AI agent mode. This is correct because:
- The
only_failuresflag is set based on AI agent detection intest_command.zig(lines 1367-1369)- This change maintains the same behavior while centralizing the AI agent logic
- It makes the code more explicit about when early clearing should occur
The change aligns with the broader refactoring to use an explicit flag rather than scattered AI agent checks.
src/bunfig.zig (1)
242-245: LGTM! Follows established parsing patterns.The parsing logic for
onlyFailures:
- Correctly checks for boolean type with
try this.expect(expr, .e_boolean)- Sets the appropriate context field
this.ctx.test_options.reporters.only_failures- Follows the exact same pattern as other boolean test options (e.g.,
coverage,smol)The placement within the
TestCommandconfiguration block is appropriate.src/cli/test_command.zig (4)
582-582: LGTM! Field properly declared.The
only_failuresboolean field is correctly:
- Added to the
reportersstruct alongsidedotsandjunit- Initialized with a default value of
false- Placed logically with other reporter configuration
904-904: LGTM! Consistent with the only_failures flag.The condition correctly prevents queuing of skips, todos, and failures for later repetition when either
dotsoronly_failuresmode is active. This is consistent with the printing logic above and ensures that buffered output is only accumulated in normal mode.
1365-1369: LGTM! Appropriate default behavior for AI agents.The initialization logic:
- First checks the explicit CLI flag/bunfig setting (line 1365-1366)
- Falls back to enabling
only_failuresfor AI agents (lines 1367-1369)This is a sensible default that reduces verbosity in AI-assisted development contexts while still allowing users to override it via configuration.
878-880: No changes needed: dots mode takes precedence over only_failuresThe code checks
reporters.dotsfirst, so when both flags are set non-failures print dots and failures print full lines. Logic correctly handles all non-failure statuses.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
src/cli/Arguments.zig (1)
465-468: Clarify or enforce the interaction between --dots and --only-failures.Currently, both
--dotsand--only-failurescan be set simultaneously without validation. When both flags are provided:
- Passing tests print dots (via
reporters.dotscheck on line 864 in test_command.zig)- Failing tests print full output (via logic on line 878-891 in test_command.zig)
However, the PR description states
--only-failuresis "Similar to --dots but without printing dots," which suggests they are alternative modes rather than combinable options.Recommendation: Either:
- Enforce mutual exclusion (as suggested in past review comments) by checking if both flags are provided and returning an error, or
- Document the combined behavior explicitly if it's intentional, explaining what users should expect when both flags are used together.
#!/bin/bash # Verify if there's any documentation or handling for simultaneous use of both flags rg -n -C3 'dots.*only_failures|only_failures.*dots' src/
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (4)
docs/runtime/bunfig.md(1 hunks)src/bun.js/test/jest.zig(1 hunks)src/cli/Arguments.zig(2 hunks)src/cli/test_command.zig(4 hunks)
🧰 Additional context used
📓 Path-based instructions (3)
**/*.zig
📄 CodeRabbit inference engine (.cursor/rules/javascriptcore-class.mdc)
**/*.zig: Declare the extern C symbol in Zig and export a Zig-friendly alias for use
Wrap the Bun____toJS extern in a Zig method that takes a JSGlobalObject and returns JSC.JSValue
Files:
src/cli/Arguments.zigsrc/bun.js/test/jest.zigsrc/cli/test_command.zig
src/**/*.zig
📄 CodeRabbit inference engine (CLAUDE.md)
In Zig code, manage memory carefully and use defer for cleanup of allocations/resources
src/**/*.zig: Use the # prefix to declare private fields in Zig structs (e.g., struct { #foo: u32 })
Prefer decl literals when initializing values in Zig (e.g., const decl: Decl = .{ .binding = 0, .value = 0 })
Place @import directives at the bottom of Zig files
Use @import("bun") instead of @import("root").bunWhen adding debug logs in Zig, create a scoped logger and log via Bun APIs:
const log = bun.Output.scoped(.${SCOPE}, .hidden);thenlog("...", .{})
Files:
src/cli/Arguments.zigsrc/bun.js/test/jest.zigsrc/cli/test_command.zig
src/bun.js/**/*.zig
📄 CodeRabbit inference engine (.cursor/rules/zig-javascriptcore-classes.mdc)
src/bun.js/**/*.zig: In Zig binding structs, expose generated bindings via pub const js = JSC.Codegen.JS and re-export toJS/fromJS/fromJSDirect
Constructors and prototype methods should return bun.JSError!JSC.JSValue to integrate Zig error handling with JS exceptions
Use parameter name globalObject (not ctx) and accept (*JSC.JSGlobalObject, *JSC.CallFrame) in binding methods/constructors
Implement getters as get(this, globalObject) returning JSC.JSValue and matching the .classes.ts interface
Provide deinit() for resource cleanup and finalize() that calls deinit(); use bun.destroy(this) or appropriate destroy pattern
Access JS call data via CallFrame (argument(i), argumentCount(), thisValue()) and throw errors with globalObject.throw(...)
For properties marked cache: true, use the generated Zig accessors (NameSetCached/GetCached) to work with GC-owned values
In finalize() for objects holding JS references, release them using .deref() before destroy
Files:
src/bun.js/test/jest.zig
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Format
🔇 Additional comments (5)
src/bun.js/test/jest.zig (1)
18-18: LGTM! AI agent handling properly consolidated.The removal of
Output.isAIAgent()is correct—this logic has been moved to test_command.zig (lines 1368-1370) whereonly_failuresis now set to true for AI agents. This consolidates the AI agent handling in one place.src/cli/test_command.zig (4)
582-582: LGTM! Field addition is correct.The new
only_failuresboolean field follows the existing pattern fordotsandjunitreporters and is properly initialized tofalse.
878-879: LGTM! Conditional logic correctly implements only-failures mode.The condition properly checks:
- Test result is NOT a failure:
((comptime result.basicResult()) != .fail)- Reporter has
only_failuresenabled:buntest.reporter.?.reporters.only_failuresThis correctly skips printing for non-failure tests when the flag is set, allowing only failures to be displayed.
904-904: LGTM! Buffering behavior is consistent with dots reporter.The updated condition
!this.reporters.dots and !this.reporters.only_failurescorrectly prevents buffering failures for later display when either flag is enabled. This is consistent because:
- The "repeat failures" functionality (lines 1559-1569) only makes sense for the default verbose output
- Both
dotsandonly_failuresmodes print output immediately as tests complete- The repeat is already disabled for dots mode, so this extends the same logic to only_failures
1366-1370: LGTM! Proper initialization with AI agent fallback.The initialization logic correctly:
- Checks the explicit
only_failuresflag from context- Falls back to enabling it for AI agents if not explicitly set
- Maintains the existing AI agent behavior by consolidating the check here (previously scattered across multiple files)
This is a good refactoring that centralizes the AI agent handling.
| ### `test.onlyFailures` | ||
|
|
||
| When enabled, only failed tests are displayed in the output. This helps reduce noise in large test suites by hiding passing tests. Default `false`. | ||
|
|
||
| ```toml | ||
| [test] | ||
| onlyFailures = true | ||
| ``` | ||
|
|
||
| This is equivalent to using the `--only-failures` flag when running `bun test`. |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Document the interaction with dots reporter.
While this documentation is clear, consider adding a note about the behavior when test.onlyFailures is used alongside test.reporter.dots. Based on the code in test_command.zig, when both are enabled, dots are printed for passing tests and full output is shown for failures. This interaction should be documented to avoid user confusion.
Apply this diff to clarify the interaction:
### `test.onlyFailures`
When enabled, only failed tests are displayed in the output. This helps reduce noise in large test suites by hiding passing tests. Default `false`.
```toml
[test]
onlyFailures = trueThis is equivalent to using the --only-failures flag when running bun test.
+
+Note: When combined with test.reporter.dots, passing tests will display as dots while failures show full output.
<details>
<summary>🤖 Prompt for AI Agents</summary>
In docs/runtime/bunfig.md around lines 252 to 261, the docs for
test.onlyFailures lack a note about how it interacts with test.reporter.dots;
add a short sentence after the existing paragraph that states: "Note: When
combined with test.reporter.dots, passing tests will display as dots while
failures show full output." so users understand that dots are still printed for
passes and full failure output is shown.
</details>
<!-- This is an auto-generated comment by CodeRabbit -->
Summary
Adds a new
--only-failuresflag tobun testthat only displays test failures, similar to--dotsbut without printing dots for each test.Motivation
When running large test suites or in CI environments, users often only care about test failures. The existing
--dotsreporter reduces verbosity by showing dots, but still requires visual scanning to find failures. The--only-failuresflag provides a cleaner output by completely suppressing passing tests.Changes
--only-failuresCLI flag inArguments.zigonly_failuresboolean to the test reporters struct incli.zigtest_command.zigto skip non-failures when flag is setjest.zigandbun_test.zigto handle the new flagonly-failures.test.tsUsage
bun test --only-failuresExample output (only shows failures):
Test Plan
--only-failuresflag only shows failing tests--dotsreporter still works correctly🤖 Generated with Claude Code