Skip to content

Commit 70fb8d8

Browse files
committed
Add gotest adapter for standard Go tests
Similar to the Cypress adapter, this provides utilities for running standard Go tests (*_test.go) via OTE without modifying test code. Key features: - Operators compile test packages into binaries (go test -c) - Binaries are embedded in OTE extension binary (go:embed) - At runtime, binaries are extracted and executed - No hardcoded logic - operators provide all metadata Example usage: //go:embed compiled_tests/*.test var embeddedTestBinaries embed.FS metadata := []gotest.GoTestConfig{ { TestName: "[sig-api-machinery] TestOperatorNamespace [Serial]", BinaryName: "e2e.test", TestPattern: "TestOperatorNamespace", Tags: []string{"Serial"}, Lifecycle: "Informing", }, } specs, err := gotest.BuildExtensionTestSpecsFromGoTestMetadata( metadata, embeddedTestBinaries, "compiled_tests", )
1 parent 356b66a commit 70fb8d8

File tree

6 files changed

+953
-0
lines changed

6 files changed

+953
-0
lines changed

pkg/gotest/README.md

Lines changed: 250 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,250 @@
1+
# Go Test Adapter for OpenShift Tests Extension
2+
3+
This package provides a reusable adapter for running standard Go tests within the OpenShift Tests Extension (OTE) framework.
4+
5+
## Overview
6+
7+
The Go test adapter allows you to:
8+
- Run standard `*_test.go` files as OTE tests
9+
- Embed compiled test binaries in your test extension
10+
- Configure test metadata (tags, timeouts, lifecycle) via source code comments
11+
- Automatically discover tests without manual registration
12+
- Support both serial and parallel test execution
13+
14+
## Quick Start
15+
16+
### 1. Write Standard Go Tests
17+
18+
Write normal Go tests with metadata in comments:
19+
20+
```go
21+
package eventttl_test
22+
23+
import "testing"
24+
25+
// Timeout: 60m
26+
// Tags: Serial
27+
// Lifecycle: Blocking
28+
func TestEventTTLController(t *testing.T) {
29+
// Your test code here
30+
}
31+
32+
// Tags: Parallel
33+
// Lifecycle: Informing
34+
func TestEventTTLValidation(t *testing.T) {
35+
// This test can run in parallel
36+
}
37+
```
38+
39+
### 2. Configure Your Build
40+
41+
Add to your `Makefile`:
42+
43+
```makefile
44+
# Directory structure
45+
TEST_EXTENSION_DIR := cmd/my-operator-tests-ext
46+
GOTEST_DIR := $(TEST_EXTENSION_DIR)/gotest
47+
COMPILED_TESTS_DIR := $(GOTEST_DIR)/compiled_tests
48+
49+
# Build individual test binaries
50+
$(COMPILED_TESTS_DIR)/%.test: test/%
51+
@mkdir -p $(COMPILED_TESTS_DIR)
52+
go test -c -o $@ ./$<
53+
54+
# Generate metadata.json from source comments
55+
$(COMPILED_TESTS_DIR)/metadata.json: test/*
56+
@mkdir -p $(COMPILED_TESTS_DIR)
57+
go run github.com/openshift-eng/openshift-tests-extension/pkg/gotest/cmd/generate-metadata \
58+
$@ test/eventttl test/encryption
59+
60+
# Build all test binaries
61+
.PHONY: build-test-binaries
62+
build-test-binaries: $(COMPILED_TESTS_DIR)/eventttl.test $(COMPILED_TESTS_DIR)/metadata.json
63+
```
64+
65+
### 3. Integrate with Your Test Extension
66+
67+
In your test extension's main.go:
68+
69+
```go
70+
package main
71+
72+
import (
73+
"embed"
74+
"log"
75+
76+
"github.com/openshift-eng/openshift-tests-extension/pkg/extension"
77+
"github.com/openshift-eng/openshift-tests-extension/pkg/gotest"
78+
)
79+
80+
//go:embed gotest/compiled_tests/*.test gotest/compiled_tests/metadata.json
81+
var compiledTests embed.FS
82+
83+
func main() {
84+
// Configure the gotest adapter
85+
config := gotest.GoTestAdapterConfig{
86+
TestPrefix: "[sig-api-machinery] my-operator",
87+
CompiledTests: compiledTests,
88+
CompiledTestsDir: "gotest/compiled_tests",
89+
MetadataFileName: "metadata.json",
90+
}
91+
92+
// Build test specs
93+
specs, err := gotest.BuildExtensionTestSpecs(config)
94+
if err != nil {
95+
log.Fatalf("Failed to build test specs: %v", err)
96+
}
97+
98+
// Register with OTE
99+
ext := extension.New("my-operator-tests")
100+
ext.AddTestSpecs(specs)
101+
ext.Run()
102+
}
103+
```
104+
105+
## Metadata Format
106+
107+
Test metadata is specified in comments above test functions:
108+
109+
### Timeout
110+
111+
Specifies how long the test can run:
112+
113+
```go
114+
// Timeout: 60m
115+
func TestLongRunning(t *testing.T) { ... }
116+
```
117+
118+
### Tags
119+
120+
Controls test execution mode:
121+
122+
```go
123+
// Tags: Serial
124+
func TestSerial(t *testing.T) { ... } // Runs only in serial mode
125+
126+
// Tags: Parallel
127+
func TestParallel(t *testing.T) { ... } // Can run in parallel
128+
129+
// Tags: Serial, Slow
130+
func TestSlowSerial(t *testing.T) { ... } // Multiple tags
131+
```
132+
133+
**Important**: By default, tests are marked `Serial` for safety. Use `Parallel` tag only if your test is thread-safe.
134+
135+
### Lifecycle
136+
137+
Determines whether test failures block CI:
138+
139+
```go
140+
// Lifecycle: Blocking
141+
func TestCritical(t *testing.T) { ... } // Failures block CI
142+
143+
// Lifecycle: Informing
144+
func TestOptional(t *testing.T) { ... } // Failures don't block CI (default)
145+
```
146+
147+
## Architecture
148+
149+
### Build Time
150+
151+
1. **Source Code**`generate-metadata` tool scans `*_test.go` files
152+
2. **metadata.json** generated with all test metadata
153+
3. **Test Binaries** compiled using `go test -c`
154+
4. Both embedded in test extension binary via `//go:embed`
155+
156+
### Runtime
157+
158+
1. **Extraction**: Embedded binaries extracted to temp directory
159+
2. **Discovery**: Each binary queried with `-test.list` to find all tests
160+
3. **Metadata Lookup**: Test metadata loaded from `metadata.json`
161+
4. **Registration**: ExtensionTestSpecs created for each test
162+
5. **Execution**: Tests run via `-test.run` with exact test name
163+
164+
## Advanced Usage
165+
166+
### Custom Test Prefix
167+
168+
The `TestPrefix` appears in all test names:
169+
170+
```go
171+
config := gotest.GoTestAdapterConfig{
172+
TestPrefix: "[sig-auth][Late] openshift-apiserver",
173+
// ...
174+
}
175+
```
176+
177+
Results in test names like:
178+
```
179+
[sig-auth][Late] openshift-apiserver TestAuthentication [Serial] [Timeout:30m]
180+
```
181+
182+
### Multiple Test Directories
183+
184+
Generate metadata from multiple directories:
185+
186+
```bash
187+
go run .../generate-metadata metadata.json \
188+
test/eventttl \
189+
test/encryption \
190+
test/validation
191+
```
192+
193+
### Omitting Test Prefix
194+
195+
Set `TestPrefix` to empty string for simple test names:
196+
197+
```go
198+
config := gotest.GoTestAdapterConfig{
199+
TestPrefix: "", // Test names will be just "TestName [Tags]"
200+
// ...
201+
}
202+
```
203+
204+
## Directory Structure Example
205+
206+
```
207+
my-operator/
208+
├── test/
209+
│ ├── eventttl/
210+
│ │ ├── controller_test.go
211+
│ │ └── validation_test.go
212+
│ └── encryption/
213+
│ └── rotation_test.go
214+
├── cmd/
215+
│ └── my-operator-tests-ext/
216+
│ ├── main.go
217+
│ └── gotest/
218+
│ └── compiled_tests/ # Generated at build time
219+
│ ├── eventttl.test # Embedded binary
220+
│ ├── encryption.test # Embedded binary
221+
│ └── metadata.json # Embedded metadata
222+
└── Makefile
223+
```
224+
225+
## Benefits
226+
227+
1. **Reusable**: Same adapter works for all operators
228+
2. **Type-Safe**: Standard Go testing package
229+
3. **IDE Support**: Full IDE support for Go tests
230+
4. **No Manual Registration**: Tests auto-discovered
231+
5. **Build-Time Metadata**: No runtime parsing needed
232+
6. **OTE Integration**: Full OTE features (parallel execution, lifecycle, etc.)
233+
234+
## Comparison with Ginkgo Adapter
235+
236+
| Feature | Go Test Adapter | Ginkgo Adapter |
237+
|---------|----------------|----------------|
238+
| Test Framework | `testing.T` | Ginkgo/Gomega |
239+
| Discovery | `-test.list` | Ginkgo outline |
240+
| Metadata | Comments | Ginkgo labels |
241+
| Learning Curve | Low (standard Go) | Medium (Ginkgo DSL) |
242+
| Complexity | Simple | More features |
243+
244+
## Contributing
245+
246+
When contributing to other operators:
247+
- Use consistent `TestPrefix` format: `[sig-xxx] operator-name`
248+
- Default to `Serial` and `Informing` for safety
249+
- Document timeout requirements
250+
- Use `Blocking` lifecycle only for critical tests

pkg/gotest/adapter.go

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
package gotest
2+
3+
import (
4+
"embed"
5+
"encoding/json"
6+
"fmt"
7+
"io/fs"
8+
"os"
9+
"path/filepath"
10+
11+
et "github.com/openshift-eng/openshift-tests-extension/pkg/extension/extensiontests"
12+
)
13+
14+
// GoTestAdapterConfig configures the gotest adapter for an operator's test suite
15+
type GoTestAdapterConfig struct {
16+
// TestPrefix is prepended to all test names (e.g., "[sig-api-machinery] kube-apiserver operator")
17+
TestPrefix string
18+
19+
// CompiledTests is the embedded filesystem containing test binaries and metadata.json
20+
CompiledTests embed.FS
21+
22+
// CompiledTestsDir is the directory path within the embed.FS (e.g., "compiled_tests")
23+
CompiledTestsDir string
24+
25+
// MetadataFileName is the name of the metadata file (default: "metadata.json")
26+
MetadataFileName string
27+
}
28+
29+
// testMetadataCache holds parsed metadata for quick lookup
30+
var testMetadataCache map[string]TestMetadataEntry
31+
32+
// MetadataFile holds all test metadata (must match generate_metadata.go)
33+
type MetadataFile struct {
34+
TestDirectory string `json:"testDirectory"`
35+
Tests []TestMetadataEntry `json:"tests"`
36+
}
37+
38+
// TestMetadataEntry holds metadata for a single test (must match generate_metadata.go)
39+
type TestMetadataEntry struct {
40+
Name string `json:"name"`
41+
Timeout string `json:"timeout,omitempty"`
42+
Tags []string `json:"tags,omitempty"`
43+
Lifecycle string `json:"lifecycle,omitempty"`
44+
}
45+
46+
// BuildExtensionTestSpecs discovers all Go tests from embedded test binaries
47+
// This follows the OTE adapter pattern (similar to Cypress/Ginkgo)
48+
func BuildExtensionTestSpecs(config GoTestAdapterConfig) (et.ExtensionTestSpecs, error) {
49+
// Set defaults
50+
if config.MetadataFileName == "" {
51+
config.MetadataFileName = "metadata.json"
52+
}
53+
54+
var specs et.ExtensionTestSpecs
55+
56+
// Load metadata from embedded JSON
57+
if err := loadMetadata(config); err != nil {
58+
return nil, fmt.Errorf("failed to load test metadata: %w", err)
59+
}
60+
61+
// Extract embedded test binaries to temporary directory
62+
tmpDir, err := os.MkdirTemp("", "gotest-adapter-*")
63+
if err != nil {
64+
return nil, fmt.Errorf("failed to create temp dir: %w", err)
65+
}
66+
67+
// Walk through embedded test binaries
68+
err = fs.WalkDir(config.CompiledTests, config.CompiledTestsDir, func(path string, d fs.DirEntry, err error) error {
69+
if err != nil {
70+
return err
71+
}
72+
73+
if d.IsDir() || !d.Type().IsRegular() {
74+
return nil
75+
}
76+
77+
// Skip metadata file - it's not a binary
78+
if filepath.Base(path) == config.MetadataFileName {
79+
return nil
80+
}
81+
82+
// Extract binary to temp directory
83+
data, err := config.CompiledTests.ReadFile(path)
84+
if err != nil {
85+
return fmt.Errorf("failed to read embedded file %s: %w", path, err)
86+
}
87+
88+
binaryName := filepath.Base(path)
89+
tmpBinary := filepath.Join(tmpDir, binaryName)
90+
91+
if err := os.WriteFile(tmpBinary, data, 0755); err != nil {
92+
return fmt.Errorf("failed to write binary %s: %w", tmpBinary, err)
93+
}
94+
95+
// Discover tests from this binary
96+
binarySpecs, err := discoverTestsFromBinary(tmpBinary, binaryName, config.TestPrefix)
97+
if err != nil {
98+
return fmt.Errorf("failed to discover tests from %s: %w", binaryName, err)
99+
}
100+
101+
specs = append(specs, binarySpecs...)
102+
return nil
103+
})
104+
105+
if err != nil {
106+
return nil, fmt.Errorf("failed to walk embedded tests: %w", err)
107+
}
108+
109+
return specs, nil
110+
}
111+
112+
// loadMetadata loads test metadata from embedded JSON file
113+
func loadMetadata(config GoTestAdapterConfig) error {
114+
// Read metadata file from embedded filesystem
115+
metadataPath := filepath.Join(config.CompiledTestsDir, config.MetadataFileName)
116+
data, err := config.CompiledTests.ReadFile(metadataPath)
117+
if err != nil {
118+
return fmt.Errorf("failed to read %s: %w", metadataPath, err)
119+
}
120+
121+
// Parse JSON
122+
var metadataFiles []MetadataFile
123+
if err := json.Unmarshal(data, &metadataFiles); err != nil {
124+
return fmt.Errorf("failed to parse %s: %w", metadataPath, err)
125+
}
126+
127+
// Build lookup map: testName -> metadata
128+
testMetadataCache = make(map[string]TestMetadataEntry)
129+
for _, file := range metadataFiles {
130+
for _, test := range file.Tests {
131+
testMetadataCache[test.Name] = test
132+
}
133+
}
134+
135+
return nil
136+
}

0 commit comments

Comments
 (0)