Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added performance benchmarking doc #4169

Merged
merged 6 commits into from
Oct 4, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ For experimental package changes, see the [experimental CHANGELOG](experimental/

### :books: (Refine Doc)

* docs(contributing): added guidelines for adding benchmark tests [#4169](https://github.com/open-telemetry/opentelemetry-js/pull/4169)

### :house: (Internal)

* test: added a performance benchmark test for span creation [#4105](https://github.com/open-telemetry/opentelemetry-js/pull/4105)
Expand Down
53 changes: 53 additions & 0 deletions doc/contributing/benchmark-tests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@

# Performance Benchmark Testing Guide

Benchmark tests are intended to measure performance of small units of code.

It is recommended that operations that have a high impact on the performance of the SDK (or potential for) are accompanied by a benchmark test. This helps end-users understand the performance trend over time, and it also helps maintainers catch performance regressions.

Benchmark tests are run automatically with every merge to main, and the results are available at <https://open-telemetry.github.io/opentelemetry-js/benchmark>.

## Running benchmark tests

Performance benchmark tests can be run from the root for all modules or from a single module directory only for that module:

``` bash
# benchmark all modules
npm run test:bench

# benchmark a single module
cd packages/opentelemetry-sdk-trace-base
npm run test:bench
```

## Adding a benchmark test

Unlike unit tests, benchmark tests should be written in plain JavaScript (not Typescript).

Add a new test file in folder `test/performance/benchmark` using the following as a template:

``` javascript
const Benchmark = require('benchmark');

const suite = new Benchmark.Suite();

suite.on('cycle', event => {
console.log(String(event.target));
});

suite.add('new benchmark test', function() {
// write code to test ...
});

suite.run();
```

## Automatically running benchmark tests

If you want your test to run automatically with every merge to main (to track trend over time), register the new test file by requiring it in `test/performance/benchmark/index.js`.

Add the `test:bench` script in package.json, if the module does not contain it already.

``` json
"test:bench": "node test/performance/benchmark/index.js | tee .benchmark-results.txt"
```
4 changes: 2 additions & 2 deletions doc/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,7 @@ await myTask()

## Describing a instrument measurement

Using attributes, kind, and the related [semantic conventions](https://github.com/open-telemetry/opentelemetry-specification/tree/main/specification/metrics/semantic_conventions), we can more accurately describe the measurement in a way our metrics backend will more easily understand. The following example uses these mechanisms, which are described below, for recording a measurement
Using attributes, kind, and the related [semantic conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/metrics.md), we can more accurately describe the measurement in a way our metrics backend will more easily understand. The following example uses these mechanisms, which are described below, for recording a measurement
of a HTTP request.

Each metric instruments allows to associate a description, unit of measure, and the value type.
Expand Down Expand Up @@ -343,7 +343,7 @@ One problem with metrics names and attributes is recognizing, categorizing, and

The use of semantic conventions is always recommended where applicable, but they are merely conventions. For example, you may find that some name other than the name suggested by the semantic conventions more accurately describes your metric, you may decide not to include a metric attribute which is suggested by semantic conventions for privacy reasons, or you may wish to add a custom attribute which isn't covered by semantic conventions. All of these cases are fine, but please keep in mind that if you stray from the semantic conventions, the categorization of metrics in your metrics backend may be affected.

_See the current metrics semantic conventions in the OpenTelemetry Specification repository: <https://github.com/open-telemetry/opentelemetry-specification/tree/main/specification/metrics/semantic_conventions>_
_See the current metrics semantic conventions in the OpenTelemetry Specification repository: <https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/metrics.md>_

[spec-overview]: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/overview.md

Expand Down
4 changes: 2 additions & 2 deletions doc/tracing.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ server.on("GET", "/user/:id", onGet);

## Describing a Span

Using span relationships, attributes, kind, and the related [semantic conventions](https://github.com/open-telemetry/opentelemetry-specification/tree/main/specification/trace/semantic_conventions), we can more accurately describe the span in a way our tracing backend will more easily understand. The following example uses these mechanisms, which are described below.
Using span relationships, attributes, kind, and the related [semantic conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/trace.md), we can more accurately describe the span in a way our tracing backend will more easily understand. The following example uses these mechanisms, which are described below.

```typescript
import { NetTransportValues, SemanticAttributes } from '@opentelemetry/semantic-conventions';
Expand Down Expand Up @@ -209,6 +209,6 @@ Consumer spans represent the processing of a job created by a producer and may s

One problem with span names and attributes is recognizing, categorizing, and analyzing them in your tracing backend. Between different applications, libraries, and tracing backends there might be different names and expected values for various attributes. For example, your application may use `http.status` to describe the HTTP status code, but a library you use may use `http.status_code`. In order to solve this problem, OpenTelemetry uses a library of semantic conventions which describe the name and attributes which should be used for specific types of spans. The use of semantic conventions is always recommended where applicable, but they are merely conventions. For example, you may find that some name other than the name suggested by the semantic conventions more accurately describes your span, you may decide not to include a span attribute which is suggested by semantic conventions for privacy reasons, or you may wish to add a custom attribute which isn't covered by semantic conventions. All of these cases are fine, but please keep in mind that if you stray from the semantic conventions, the categorization of spans in your tracing backend may be affected.

_See the current trace semantic conventions in the OpenTelemetry Specification repository: <https://github.com/open-telemetry/opentelemetry-specification/tree/main/specification/trace/semantic_conventions>_
_See the current trace semantic conventions in the OpenTelemetry Specification repository: <https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/trace.md>_

[spec-overview]: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/overview.md