Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch observer #1137

Merged
merged 30 commits into from
Jun 30, 2020
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
5511aa9
chore: adding batch observer, some metrics refactoring
obecny Jun 3, 2020
72e00fb
chore: undo changes after testing
obecny Jun 3, 2020
8e26b73
chore: undo changes after testing
obecny Jun 3, 2020
2e8c892
chore: addressing comments
obecny Jun 4, 2020
8f1d3e6
Merge branch 'master' into batch-observer
obecny Jun 4, 2020
2d55754
chore: renaming observer into value observer, fixing few spotted issues
obecny Jun 4, 2020
54448d4
chore: missing renamed for ValueObserver
obecny Jun 4, 2020
e8cf27b
chore: removing unused class
obecny Jun 4, 2020
36fdc5b
chore: cleanup
obecny Jun 4, 2020
353c476
chore: refactoring, renaming aggregators
obecny Jun 8, 2020
4983115
chore: merge branch 'master' into batch-observer
obecny Jun 8, 2020
f3c2a55
chore: refactoring observer to have base class that can be extended
obecny Jun 8, 2020
02f3259
chore: changing aggregator for ValueObserver, exposing batcher so it …
obecny Jun 10, 2020
7948bb2
chore: merge branch 'master' into batch-observer
obecny Jun 10, 2020
c227d41
chore: addressing comments
obecny Jun 10, 2020
91b32e0
chore: addressing comments
obecny Jun 10, 2020
9111c34
chore: preventing user from updating observer after timeout or update
obecny Jun 10, 2020
aff4fc1
chore: aligning aggregators for value observer and recorder with rega…
obecny Jun 11, 2020
91a517b
Merge branch 'master' into batch-observer
obecny Jun 11, 2020
74be6fc
chore: fixing test
obecny Jun 11, 2020
4bfe858
chore: merge branch 'master' into batch-observer
obecny Jun 12, 2020
e42ca77
chore: fixes after merge
obecny Jun 12, 2020
7385657
chore: changes after review
obecny Jun 12, 2020
e22b982
chore: changes after review with some additional fixes around typing
obecny Jun 12, 2020
1874414
chore: changes after review
obecny Jun 12, 2020
18a8a86
chore: merge branch 'master' into batch-observer
obecny Jun 22, 2020
133625f
chore: lint
obecny Jun 22, 2020
5e52e03
chore: merge branch 'master' into batch-observer
obecny Jun 30, 2020
2214c9f
chore: reviews
obecny Jun 30, 2020
f1ac953
chore: typo
obecny Jun 30, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions examples/metrics/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ npm run start:observer

### Prometheus

1. In prometheus search for "metric_observer"
1. In prometheus search for "cpu_core_usage", "cpu_temp_per_app", "cpu_usage_per_app"

### Links

Expand All @@ -35,7 +35,9 @@ npm run start:observer

### Example

<p align="center"><img src="metrics/observer.png"/></p>
![Screenshot of the running example](metrics/observer.png)
![Screenshot of the running example](metrics/observer_batch.png)
![Screenshot of the running example](metrics/observer_batch2.png)

## Useful links

Expand Down
82 changes: 65 additions & 17 deletions examples/metrics/metrics/observer.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
'use strict';

const { MeterProvider, MetricObservable } = require('@opentelemetry/metrics');
const { MeterProvider } = require('@opentelemetry/metrics');
const { ConsoleLogger, LogLevel } = require('@opentelemetry/core');
const { PrometheusExporter } = require('@opentelemetry/exporter-prometheus');

const exporter = new PrometheusExporter(
Expand All @@ -19,25 +20,72 @@ const meter = new MeterProvider({
interval: 2000,
}).getMeter('example-observer');

const otelCpuUsage = meter.createObserver('metric_observer', {
meter.createValueObserver('cpu_core_usage', {
monotonic: false,
obecny marked this conversation as resolved.
Show resolved Hide resolved
description: 'Example of a observer',
labelKeys: ['core'],
obecny marked this conversation as resolved.
Show resolved Hide resolved
description: 'Example of a sync value observer with callback',
}, (observerResult) => { // this callback is called once per each interval
observerResult.observe(getRandomValue(), { core: '1' });
observerResult.observe(getRandomValue(), { core: '2' });
});

function getCpuUsage() {
return Math.random();
}
// no callback as they will be updated in batch observer
const tempMetric = meter.createValueObserver('cpu_temp_per_app', {
monotonic: false,
description: 'Example of sync value observer used with async batch observer',
});

const observable = new MetricObservable();
// no callback as they will be updated in batch observer
const cpuUsageMetric = meter.createValueObserver('cpu_usage_per_app', {
monotonic: false,
description: 'Example of sync value observer used with async batch observer',
});

setInterval(() => {
observable.next(getCpuUsage());
}, 5000);
meter.createBatchObserver('metric_batch_observer', (observerBatchResult) => {
Promise.all([
someAsyncMetrics(),
// simulate waiting
new Promise((resolve, reject) => {
setTimeout(resolve, 300);
}),
]).then(([apps, waiting]) => {
apps.forEach(app => {
observerBatchResult.observe({ app: app.name, core: '1' }, [
tempMetric.observation(app.core1.temp),
cpuUsageMetric.observation(app.core1.usage),
]);
observerBatchResult.observe({ app: app.name, core: '2' }, [
tempMetric.observation(app.core2.temp),
cpuUsageMetric.observation(app.core2.usage),
]);
});
});
}, {
maxTimeoutUpdateMS: 500,
dyladan marked this conversation as resolved.
Show resolved Hide resolved
logger: new ConsoleLogger(LogLevel.DEBUG)
},
);

otelCpuUsage.setCallback((observerResult) => {
observerResult.observe(getCpuUsage, { pid: process.pid, core: '1' });
observerResult.observe(getCpuUsage, { pid: process.pid, core: '2' });
observerResult.observe(getCpuUsage, { pid: process.pid, core: '3' });
observerResult.observe(getCpuUsage, { pid: process.pid, core: '4' });
observerResult.observe(observable, { pid: process.pid, core: '5' });
});
function someAsyncMetrics() {
return new Promise((resolve) => {
setTimeout(() => {
const stats = [
{
name: 'app1',
core1: { usage: getRandomValue(), temp: getRandomValue() * 100 },
core2: { usage: getRandomValue(), temp: getRandomValue() * 100 },
},
{
name: 'app2',
core1: { usage: getRandomValue(), temp: getRandomValue() * 100 },
core2: { usage: getRandomValue(), temp: getRandomValue() * 100 },
},
];
resolve(stats);
}, 200);
});
}

function getRandomValue() {
return Math.random();
}
Binary file modified examples/metrics/metrics/observer.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/metrics/metrics/observer_batch.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/metrics/metrics/observer_batch2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions examples/metrics/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
"url": "https://github.com/open-telemetry/opentelemetry-js/issues"
},
"dependencies": {
"@opentelemetry/core": "^0.8.3",
"@opentelemetry/exporter-prometheus": "^0.8.3",
"@opentelemetry/metrics": "^0.8.3"
},
Expand Down
10 changes: 8 additions & 2 deletions examples/tracer-web/examples/xml-http-request/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,19 @@ providerWithZone.register({

const webTracerWithZone = providerWithZone.getTracer('example-tracer-web');

const getData = (url) => new Promise((resolve, _reject) => {
const getData = (url) => new Promise((resolve, reject) => {
// eslint-disable-next-line no-undef
const req = new XMLHttpRequest();
req.open('GET', url, true);
req.setRequestHeader('Content-Type', 'application/json');
req.setRequestHeader('Accept', 'application/json');
req.send();
req.onload = () => {
resolve();
};
req.onerror = () => {
reject();
};
req.send();
});

// example of keeping track of context between async operations
Expand All @@ -53,6 +56,9 @@ const prepareClickEvent = () => {
getData(url1).then((_data) => {
webTracerWithZone.getCurrentSpan().addEvent('fetching-span1-completed');
span1.end();
}, ()=> {
webTracerWithZone.getCurrentSpan().addEvent('fetching-error');
span1.end();
});
});
}
Expand Down
3 changes: 2 additions & 1 deletion packages/opentelemetry-api/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,14 @@ export * from './context/propagation/NoopHttpTextPropagator';
export * from './context/propagation/setter';
export * from './correlation_context/CorrelationContext';
export * from './correlation_context/EntryValue';
export * from './metrics/BatchObserverResult';
export * from './metrics/BoundInstrument';
export * from './metrics/Meter';
export * from './metrics/MeterProvider';
export * from './metrics/Metric';
export * from './metrics/MetricObservable';
export * from './metrics/NoopMeter';
export * from './metrics/NoopMeterProvider';
export * from './metrics/Observation';
export * from './metrics/ObserverResult';
export * from './trace/attributes';
export * from './trace/Event';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,23 +14,18 @@
* limitations under the License.
*/

import { Labels } from './Metric';
import { Observation } from './Observation';

/**
* Metric Observable class to handle asynchronous metrics
* Interface that is being used in callback function for Observer Metric
* for batch
*/
export interface MetricObservable {
/**
* Sets the next value for observable metric
* @param value
*/
next: (value: number) => void;
/**
* Subscribes for every value change
* @param callback
*/
subscribe: (callback: (value: number) => void) => void;
export interface BatchObserverResult {
/**
* Removes the subscriber
* @param [callback]
* Used to observe (update) observations for certain labels
* @param labels
* @param observations
*/
unsubscribe: (callback?: (value: number) => void) => void;
observe(labels: Labels, observations: Observation[]): void;
}
33 changes: 30 additions & 3 deletions packages/opentelemetry-api/src/metrics/Meter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,16 @@
* limitations under the License.
*/

import { MetricOptions, Counter, ValueRecorder, Observer } from './Metric';
import { BatchObserverResult } from './BatchObserverResult';
import {
MetricOptions,
Counter,
ValueRecorder,
ValueObserver,
BatchObserver,
BatchMetricOptions,
} from './Metric';
import { ObserverResult } from './ObserverResult';

/**
* An interface to allow the recording metrics.
Expand All @@ -41,9 +50,27 @@ export interface Meter {
createCounter(name: string, options?: MetricOptions): Counter;

/**
* Creates a new `Observer` metric.
* Creates a new `ValueObserver` metric.
* @param name the name of the metric.
* @param [options] the metric options.
* @param [callback] the observer callback
*/
createObserver(name: string, options?: MetricOptions): Observer;
createValueObserver(
name: string,
options?: MetricOptions,
callback?: (observerResult: ObserverResult) => void
): ValueObserver;

/**
* Creates a new `BatchObserver` metric, can be used to update many metrics
* at the same time and when operations needs to be async
* @param name the name of the metric.
* @param callback the batch observer callback
* @param [options] the metric batch options.
*/
createBatchObserver(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are the argument orders different? This is confusing

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because some of them are optional and some not. In batch you have to create name and callback, options in most cases you will not need to define. In observer only name is required, then options and still callback might not be needed. Also the options for ValueObserver are not there same, whereas ValueObserver and UpDownCounter has the same options and the same order of params.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could overload templates be added to this function to handle the different optional combos?

name: string,
callback: (batchObserverResult: BatchObserverResult) => void,
options?: BatchMetricOptions
): BatchObserver;
}
31 changes: 22 additions & 9 deletions packages/opentelemetry-api/src/metrics/Metric.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@

import { CorrelationContext } from '../correlation_context/CorrelationContext';
import { SpanContext } from '../trace/span_context';
import { ObserverResult } from './ObserverResult';
import { BoundCounter, BoundValueRecorder } from './BoundInstrument';
import { Observation } from './Observation';
import { Logger } from '../common/Logger';

/**
* Options needed for metric creation
Expand Down Expand Up @@ -63,6 +64,18 @@ export interface MetricOptions {
* @default {@link ValueType.DOUBLE}
*/
valueType?: ValueType;

/**
* User provided logger.
*/
logger?: Logger;
}

export interface BatchMetricOptions extends MetricOptions {
/**
* Indicates how long the batch metric should wait to update before cancel
*/
maxTimeoutUpdateMS?: number;
}

/** The Type of value. It describes how the data is reported. */
Expand Down Expand Up @@ -146,16 +159,16 @@ export interface ValueRecorder extends UnboundMetric<BoundValueRecorder> {
}

/** Base interface for the Observer metrics. */
export interface Observer extends Metric {
/**
* Sets a callback where user can observe value for certain labels. The
* observers are called periodically to retrieve the value.
* @param callback a function that will be called once to set observers
* for values
*/
setCallback(callback: (observerResult: ObserverResult) => void): void;
export interface BaseObserver extends Metric {
observation: (value: number) => Observation;
}

/** Base interface for the Value Observer metrics. */
export type ValueObserver = BaseObserver;

/** Base interface for the Batch Observer metrics. */
export type BatchObserver = Metric;

/**
* key-value pairs passed by the user.
*/
Expand Down
Loading