Skip to content

Commit

Permalink
[chore] Update instructions to run E2E tests (#16084)
Browse files Browse the repository at this point in the history
Update the instructions to run the E2E tests to use the Make target.

I chose to use the e2e-test Make target since it will guarantee the local Collector binary is generated and up to date. I also removed the output screenshots since I think they likely don't provide much value.
  • Loading branch information
evan-bradley authored Nov 7, 2022
1 parent a5a98ef commit 9d2ecd2
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 14 deletions.
Binary file removed testbed/CCRepo_result.png
Binary file not shown.
26 changes: 12 additions & 14 deletions testbed/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ Generally, when designing a test for new exporter and receiver components, devel
* `GenConfigYAMLStr()` - Generate a config string to place in exporter part of collector config so that it can send data to this receiver.
* `ProtocolName()` - Return protocol name to use in collector config pipeline.

* `Testing` - This part may vary from what kind of testing developers would like to do. In existing implementation, we can refer to [End-to-End testing](https://github.com/open-telemetry/opentelemetry-collector/blob/main/testbed/tests/e2e_test.go), [Metrics testing](https://github.com/open-telemetry/opentelemetry-collector/blob/main/testbed/tests/metric_test.go), [Traces testing](https://github.com/open-telemetry/opentelemetry-collector/blob/main/testbed/tests/trace_test.go), [Correctness Traces testing](https://github.com/open-telemetry/opentelemetry-collector/blob/main/testbed/correctness/traces/correctness_test.go), and [Correctness Metrics testing](https://github.com/open-telemetry/opentelemetry-collector/blob/main/testbed/correctness/metrics/metrics_correctness_test.go). For instance, if developers would like to design a trace test for a new exporter and receiver:
* `Testing` - This part may vary from what kind of testing developers would like to do. In existing implementation, we can refer to [End-to-End testing](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/testbed/tests/e2e_test.go), [Metrics testing](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/testbed/tests/metric_test.go), [Traces testing](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/testbed/tests/trace_test.go), [Correctness Traces testing](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/testbed/correctnesstests/traces/correctness_test.go), and [Correctness Metrics testing](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/testbed/correctnesstests/metrics/metrics_correctness_test.go). For instance, if developers would like to design a trace test for a new exporter and receiver:

* ```go
func TestTrace10kSPS(t *testing.T) {
Expand Down Expand Up @@ -107,29 +107,27 @@ Generally, when designing a test for new exporter and receiver components, devel
}
```

## Run Tests and Get Results
## Running the tests

Here providing some examples of how to run and get the results of testing.
To run the tests, use the `e2e-test` Makefile target, which will compile the Collector and run the end-to-end test suites against it.

1. Under the [collector-contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib) repo, the following runs all the tests:
### Run all tests

Run the following at the root of the repo:

```
cd ./testbed
./runtests.sh
make e2e-test
```
​ Then get the result:
![collector-contrib tests result](CCRepo_result.png)
### Run a particular test suite
2. Under the [collector-contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib) repo, the following runs the correctness tests only:
Run the following at the root of the repo:
```
cd ./testbed
TESTS_DIR=correctnesstests/metrics ./runtests.sh
TESTS_DIR=correctnesstests/metrics make e2e-test
```
Then get the result:
### Advanced usage
![collector correctness tests result](correctness_result.png)
A Makefile is also located at [`testbed/Makefile`](./Makefile) that offers targets to directly run certain test suites. Note that these targets will not compile the Collector before running.
Binary file removed testbed/correctness_result.png
Binary file not shown.

0 comments on commit 9d2ecd2

Please sign in to comment.