Skip to content

Conversation

@mlwelles
Copy link
Contributor

@mlwelles mlwelles commented Jan 29, 2026

Summary

This PR eliminates manual setup for running integration tests on macOS and introduces a unified test interface via Make.

Key Features

1. Automatic Cross-Compilation (macOS)

Previously, macOS developers had to manually cross-compile a Linux binary and juggle environment variables. Now, make install handles everything automatically.

2. Unified Test Interface

A single make test entry point that routes to the appropriate test runner based on variables:

# Run all tests (default)
make test

# Use variables to control what runs
make test SUITE=systest              # Run systest suite via t/ runner
make test TAGS=integration2          # Run integration2 tests via go test
make test FUZZ=1 PKG=dql             # Fuzz test the dql package
make test TAGS=upgrade PKG=acl       # Run ACL upgrade tests

3. Convenient Shortcuts

Every SUITE and TAGS value has a corresponding helper target:

# SUITE-based targets
make test-all           # All test suites (i.e. 'make test SUITE=all')
make test-unit          # Unit tests, no Docker (i.e. 'make test SUITE=unit')
make test-core          # Core tests (i.e. 'make test SUITE=core')
make test-systest       # System integration tests (i.e. 'make test SUITE=systest')
make test-vector        # Vector search tests (i.e. 'make test SUITE=vector')
make test-ldbc          # LDBC benchmark tests (i.e. 'make test SUITE=ldbc')
make test-load          # Heavy load tests (i.e. 'make test SUITE=load')

# TAGS-based targets
make test-integration   # Integration tests (i.e. 'make test TAGS=integration')
make test-integration2  # Integration2 tests (i.e. 'make test TAGS=integration2')
make test-upgrade       # Upgrade tests (i.e. 'make test TAGS=upgrade')

# Other
make test-fuzz          # Fuzz tests (i.e. 'make test FUZZ=1')
make test-benchmark PKG=posting  # Benchmark a specific package

4. Dynamic Discovery

make help dynamically discovers available SUITE values from t/t.go and TAGS values from test file build directives.

Changes

  • Build automation: make install auto-detects the OS and builds both native and Linux binaries on non-Linux systems
  • Unified test target: make test routes to t/ runner or go test based on SUITE, TAGS, FUZZ variables
  • Complete helper targets: Every SUITE and TAGS value has a corresponding test-* target
  • Dynamic help: make help shows available SUITE/TAGS values discovered from the codebase
  • Environment variable: Introduced LINUX_GOBIN to specify where Linux binaries are stored for Docker tests
  • Docker compose: All systest docker-compose files updated to use ${LINUX_GOBIN:-$GOPATH/bin} for binary mounting
  • Documentation: Updated TESTING.md, CONTRIBUTING.md with comprehensive examples

Before (macOS)

make install
mv $GOPATH/bin/dgraph $GOPATH/bin/dgraph_osx
GOOS=linux make install
mv $GOPATH/bin/linux_arm64/dgraph $GOPATH/bin/dgraph
export DGRAPH_BINARY=$GOPATH/bin/dgraph_osx
cd t && ./t --pkg=<package>

After (macOS or Linux)

make install
make test SUITE=systest PKG=systest/export

make help Output

Usage: make [target] [VAR=value ...]

Targets:
  all                  Build all targets
  dgraph               Build dgraph binary
  docker-image         Build Docker image (dgraph/dgraph:$VERSION)
  help                 Show available targets and variables
  image-local          Alias for local-image
  install              Install dgraph binary
  local-image          Build local Docker image (dgraph/dgraph:local)
  test-all             All test suites via t/ runner (i.e. 'make test SUITE=all')
  test-benchmark       Go benchmarks (i.e. 'go test -bench')
  test-core            Core tests (i.e. 'make test SUITE=core')
  test-fuzz            Fuzz tests, auto-discovers packages (i.e. 'make test FUZZ=1')
  test-integration     Integration tests (i.e. 'make test TAGS=integration')
  test-integration2    Integration2 tests via dgraphtest (i.e. 'make test TAGS=integration2')
  test-ldbc            LDBC benchmark tests (i.e. 'make test SUITE=ldbc')
  test-load            Heavy load tests (i.e. 'make test SUITE=load')
  test-systest         System integration tests (i.e. 'make test SUITE=systest')
  test-unit            Unit tests, no Docker (i.e. 'make test SUITE=unit')
  test-upgrade         Upgrade tests (i.e. 'make test TAGS=upgrade')
  test-vector          Vector search tests (i.e. 'make test SUITE=vector')
  test                 Run tests (see 'make help' for options)
  uninstall            Uninstall dgraph binary
  version              Show build version info

Variables that can be passed to 'test':
  SUITE     Select t/ runner suite (e.g., make test SUITE=systest)
  TAGS      Go build tags - bypasses t/ runner (e.g., make test TAGS=integration2)
  PKG       Limit to specific package (e.g., make test PKG=systest/export)
  TEST      Run specific test function (e.g., make test TEST=TestGQLSchema)
  FUZZ      Enable fuzz testing (e.g., make test FUZZ=1)
  FUZZTIME  Fuzz duration per package (e.g., make test FUZZ=1 FUZZTIME=60s)

  Available SUITE values: all  ldbc  load  unit  systest  vector  core
  Available TAGS values:  integration integration2 upgrade

Examples:
  make test TAGS=integration2 PKG=systest/vector        # integration2 tests for vector
  make test TAGS=upgrade PKG=acl TEST=TestACL           # specific upgrade test
  make test FUZZ=1 PKG=dql FUZZTIME=30s                 # fuzz dql package for 30s
  make test SUITE=systest PKG=systest/backup/filesystem # systest for backup pkg
  make test-benchmark PKG=posting                       # benchmark posting package

Test plan

  • Run make install on macOS and verify both binaries are created
  • Run make check in t/ directory
  • Verify make test-unit routes to t/ runner with --suite=unit
  • Verify make test TAGS=integration2 PKG=types runs go test directly
  • Verify make test FUZZ=1 PKG=dql FUZZTIME=5s runs fuzz tests
  • Verify make test-benchmark PKG=types runs benchmarks
  • Verify make help shows all targets, variables, and discovered SUITE/TAGS values
  • Verify every SUITE value has a corresponding test-* target
  • Verify every TAGS value has a corresponding test-* target
  • Verify Linux CI still works as expected

On non-Linux systems, `make install` now automatically builds both:
- Native binary at $GOPATH/bin/dgraph
- Linux binary at $GOPATH/linux_<arch>/dgraph

This eliminates the manual cross-compilation steps previously required
for macOS users to run Docker-based integration tests.

Introduces GOPATH_LINUX_BIN env var to locate the Linux binary.
- t/Makefile now uses GOPATH_LINUX_BIN for binary verification
- t/t.go automatically sets GOPATH_LINUX_BIN based on runtime.GOOS
- Add modular dependency check scripts in t/scripts/

The test runner now seamlessly works on macOS without manual
environment variable setup.
Update all systest docker-compose.yml files to mount the dgraph
binary from GOPATH_LINUX_BIN instead of hardcoded GOPATH/bin path.

This enables Docker-based tests to find the correct Linux binary
on both Linux and macOS systems.
- Add TESTING.md as comprehensive testing guide (renamed from
  TESTING_GUIDE.md)
- Update CONTRIBUTING.md to link to TESTING.md with quick start examples
- Simplify t/README.md macOS section to reflect automated build system
- Add cross-references between documentation files

The macOS testing instructions are now much simpler since the build
system handles cross-compilation automatically.
@mlwelles mlwelles requested a review from a team as a code owner January 29, 2026 05:54
@github-actions github-actions bot added area/testing Testing related issues area/documentation Documentation related issues. go Pull requests that update Go code labels Jan 29, 2026
Restore the witty ~~complex~~ sophisticated strikethrough and tighten
prose while preserving voice. Move TESTING.md reference to end of
section as a natural "learn more" exit ramp.
Add note explaining that manual installation isn't required since
AUTO_INSTALL=true make check/test handles dependencies automatically.
- Add language specifier to code block (MD040)
- Rename duplicate "How to Run" headings (MD024)
- Remove trailing colons from anti-pattern headings (MD026)
Shorter, clearer name for the environment variable that specifies
where Linux binaries are stored for Docker tests.
Add design document for unified `make test` entry point that supports:
- Environment variables (TAGS, SUITE, PKG, TEST, FUZZ, FUZZTIME)
- Helper targets (test-unit, test-integration2, etc.)
- Auto-generated help from ## comments
- Auto-discovery of fuzz test packages
Add three-way routing to the test target based on environment variables:
- TAGS: bypass t/ runner, run go test directly with build tags
- FUZZ: auto-discover and run fuzz tests in packages
- SUITE (default): delegate to existing t/ runner

Precedence: TAGS > FUZZ > SUITE

The TAGS path is for integration2/upgrade tests that don't need the
t/ runner's Docker Compose orchestration. The FUZZ path discovers
packages containing Fuzz* functions and runs each with configurable
FUZZTIME (default 300s).
- Change help output to use conventional Make syntax (make test VAR=value)
- Add benchmark example showing PKG variable with test-benchmark
- Update TESTING.md with all 11 test shortcuts and new syntax
- Update CONTRIBUTING.md testing section with new examples
- Align all documentation with current make help output
- Extract SUITE values from t/t.go's allowed list
- Discover TAGS values by scanning test files for //go:build directives
- Display both in make help output for better discoverability
…-all

- Change i.e. examples to put vars after 'make test' (more intuitive)
- Add test-all target for SUITE=all (completes helper target coverage)
- Update TESTING.md to match new help output format
- All SUITE and TAGS values now have corresponding helper targets
@blacksmith-sh
Copy link
Contributor

blacksmith-sh bot commented Jan 30, 2026

Found 1 test failure on Blacksmith runners:

Failure

Test View Logs
github.com/dgraph-io/dgraph/v25/edgraph/TestDropNamespaceErr View Logs

Fix in Cursor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/documentation Documentation related issues. area/testing Testing related issues go Pull requests that update Go code

Development

Successfully merging this pull request may close these issues.

2 participants