Add metrics module #967
Comments
|
Sounds great! Do you have any further ideas regarding this feature request? if so, feel free to share them, I'd love to get familiar with the community expectations. 🔥 |
|
Well the list of possible integrations with NestJS is kinda endless. Beside the given integrations which could come out of the box in NestJS, you could also offer metrics about:
You may want to keep in mind that there are other metrics formats beside Prometheus (such as DataDog, Graphite, etc.) which you may want to add at some point. Spring boot already offers such a metrics module, you can probably get further inspiration there: https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-metrics.html |
|
Maybe Trace from risingstack? |
|
By the way, swagger-stats (http://swaggerstats.io/) is very easy to plug into nest.js app, and you get an API endpoint (+ nice UI), that can be used to scrape metrics. However, you can't add your own metrics that easily. |
|
@DavisJaunzems That looks interesting. Maybe we can use this as inspiration for our own metrics module. Two questions I wasn't able to figure out at a first glance:
|
|
@weeco swagger stats stores the data in memory attached to the main node process (when restarted it starts from scratch), and for some metrics it only keeps last 100 entries. Then these values are exposed through prometheus friendly format that prometheus can pull. I have a simple monitoring solution on top of nest - you create injectable X and register values/functions to it. Then there is a global injectable that scans the nest.js context for these X injectables, and reads their values. Then there is a route that takes the global injectable values and displays them when route is accessed. Pretty simple solution, but not sure if it would make sense to be part of the main nest.js. |
|
I've just write a nest module for prometheus (see: https://github.com/digikare/nestjs-prom). |
|
@weeco but exactly what do you need swaggerstats.io provide, and this what @spike008t create solve a own metrics. So basically thread to close, or provide an PR for documentation HOW TO. We're using swaggerstats.io for few our projects, and this is really great. |
|
No @cojack you are wrong. swagger-stats (as the name already says) relies on swagger definitions. It's written a generic JavaScript library and therefore not written for a specific framework.
|
|
I'm pretty sure that you started to work on such integration already @weeco, correct? :) |
|
@kamilmysliwiec Well not really, but I am working on a proof of concept to show you and/or @BrunnerLivio afterwards :-). |
|
Looking forward to it! If you have something to show already, let me know :) |
|
@spike008t thx for the module. I need such integration for prometheus too, will check your module, whether it is enough for us or not. |
|
@weeco @kamilmysliwiec Do you know if any progress has been made on the integration of prometheus into the core of nestjs ? I need to add this functionality next week :-) |
|
@appsolutegeek To be honest I haven't worked on it for months anymore, as I am barely using NodeJS these days. Back in the days I had my own prometheus module + service for each project and added all desired metrics for each project on my own. That was some boilerplate which I wanted to avoid (which is why I created this issue), but it doesn't block you from adding metrics on your own. I am pretty sure you won't get metrics built into NestJS until next week. I am not sure if Kamil or anyone else is working on this at all. Still seems to be the most requested feature though. |
|
Ok, yep, I will have a go at doing something quick myself. I just didn't want to reinvent the wheel if there was something available |
|
@spike008t Just saw your module over at https://github.com/digikare/nestjs-prom Looks great, I will probably try to adapt it for my use. I presume you are not currently updating it ? @kamilmysliwiec have you thought about pulling this module into nest and offering it out of the box like terminus, typeorm etc. I must admit, I haven't tested it yet :-) - But it looks pretty much complete although the author does state there were some things left to improve. |
|
I thought I would relay my findings here, I am currently using the module in its current state. The author doesn't have time to maintain it anymore but its a great start and currently it does what I need - so I am quite happy. So confirmed, tested and using it now in production! |
|
https://github.com/digikare/nestjs-prom Seems to work well. With a small tweak it should be possible to add a middleware or guard to restrict access to |
|
It would be really nice if we could get some more attention with respect to this. Nest has a lot of potential to provide amazing telemetry by integrating Prometheus. Being able to visualize spikes of 400-500 status codes for example would have tremendous value. |
|
@kamilmysliwiec maybe create metrics nestjs module with appmetrics to export metrics nodejs app and nestjs some metrics as some standart. |
|
Hi, My company relies on Nest, and we forked and improved a Github project that integrates with Prometheus. It exports a Nest module and provides metrics for NodeJS and a few standard metrics for HTTP services(both are optional). I can provide the Nest team access to the project for reviewing it if you think that it might be usable for you. |
That would be great @SqueezeToyAliens! |
|
Asking everyone on this thread (and @kamilmysliwiec @weeco @spike008t @SqueezeToyAliens in particular) Any progress with this? It would be awesome to implement something of this kind on our own product. |
|
@omerxx I am using this https://github.com/digikare/nestjs-prom - it works well, i would love it to be merged into nestjs though. Also would be interesting in knowing how the @SqueezeToyAliens implementation is/ |
|
In case it's helpful, that's the approach I use to instrument request durations in Go. I simply create a middleware which gets the "routePattern" (to avoid highly cardinal labels on dynamic routes), measure the request duration and put that along with the response status code into my Histogram: https://github.com/kafka-owl/kafka-owl/blob/master/backend/pkg/common/middleware/instrument.go PS: I'd also highly recommend to add metrics in the logger, which should be kinda easy if you can create a hook for the log events, see: https://github.com/kafka-owl/kafka-owl/blob/master/backend/pkg/common/logging/logger.go#L29-L54
|
|
I totally love nestjs - and it already comes with a lot of powerful tools and extensions. But this would be an enormous improvement! Is there any progress or has this idea been abandoned? |
|
👋 , Hello I am a metrics nerd and primary maintainer of a few metrics/observability/canary libraries Kayenta, Node Measured, Armory's Spinnaker Observability Plugin I am also a typescript advocate, I recently have discovered NestJs and +1 @Avejack's comment. I would like to volunteer to own integrating a vendor-agnostic library and establish patterns for observability for NestJs similar to Springboot and Micrometer. I think Node-Measured could fit the bill here. I wouldn't want to invest the time into ramping up in the NestJs internals and make the PR for this unless there was buy-in from the project owners and community. Should we start and RFC, what would that look like I wonder? https://discord.com/channels/520622812742811698/527863537708695562/725430518815916052 |
kamilmysliwiec
added
the
priority: very low (5)
|
Hi all. Adding another promising library that handles metrics and tracing. OpenTelemetry which describes itself as "An observability framework for cloud-native software." and it's a part of CNCF and it's vendor agnostic. NodeJS library written in Typescript can be found at https://github.com/open-telemetry/opentelemetry-js. I would expect from Nest Module to support some decorators for easier use, such as Ecosystem behind it has support for a bunch of services/providers. So writing a Nest module for it would bring Nest support for all of those services/providers. |
|
@kamilmysliwiec I know how to do that without rewriting nest logic, add and implement @Proxy (build in JavaScript Proxy feature) decorator, that as the argument takes list of the classes for execution for set/get, this will allow us to unlimited operations before ANY method execution on ANY object that come from DI. With great power comes great responsibility, but this will flip table upside down, there is no such a thing in any framework. |
|
@cojack this sounds really interesting. Is this something that can be made as a third party package, or does the |
|
@jmcdo29 so this have to be done in the place where nest is creating instance of injectable class. |
|
@Legion2 what happen?! |
|
NestJS OpenTelemetry
This library provides deeply integrated protocol-agnostic Nestjs OpenTelemetry instrumentations, metrics and SDK. https://github.com/MetinSeylan/Nestjs-OpenTelemetry DescriptionNestjs is a protocol-agnostic framework. That's why this library can able to work with different protocols like RabbitMQ, GRPC and HTTP. Also you can observe and trace Nestjs specific layers like Pipe, Guard, Controller and Provider. It also includes auto trace and metric instrumentations for some popular Nestjs libraries.
Installationnpm install @metinseylan/nestjs-opentelemetry --saveConfigurationThis is a basic configuration without any trace and metric exporter, but includes default metrics and injectors import { OpenTelemetryModule } from '@metinseylan/nestjs-opentelemetry';
@Module({
imports: [OpenTelemetryModule.forRoot()]
})
export class AppModule {}Default Parameters
Distributed TracingSimple setup with Zipkin exporter, including with default trace instrumentations. import { OpenTelemetryModule } from '@metinseylan/nestjs-opentelemetry';
import { ZipkinExporter } from '@opentelemetry/exporter-zipkin';
import { SimpleSpanProcessor } from '@opentelemetry/sdk-trace-base';
@Module({
imports: [
OpenTelemetryModule.forRoot({
spanProcessor: new SimpleSpanProcessor(
new ZipkinExporter({
url: 'your-zipkin-url',
})
),
}),
],
})
export class AppModule {}After setup, your application will be instrumented, so that you can see almost every layer of application in ZipkinUI, including Guards, Pipes, Controllers even global layers like this
List of supported official exporters here. Trace DecoratorsThis library supports auto instrumentations for Nestjs layers, but sometimes you need to define custom span for specific method blocks like providers methods. In this case import { Injectable } from '@nestjs/common';
import { Span } from '@metinseylan/nestjs-opentelemetry';
@Injectable()
export class AppService {
@Span()
getHello(): string {
return 'Hello World!';
}
}Also @Span('hello')Trace ProvidersIn an advanced usage case, you need to access the native OpenTelemetry Trace provider to access them from Nestjs application context. import { Injectable } from '@nestjs/common';
import { Tracer } from '@opentelemetry/sdk-trace-base';
@Injectable()
export class AppService {
constructor(private readonly tracer: Tracer) {}
getHello(): string {
const span = this.tracer.startSpan('important_section_start');
// do something important
span.setAttributes({ userId: 1150 });
span.end();
return 'Hello World!';
}
}
import { Injectable } from '@nestjs/common';
import { TraceService } from '@metinseylan/nestjs-opentelemetry';
@Injectable()
export class AppService {
constructor(private readonly traceService: TraceService) {}
getHello(): string {
const span = this.traceService.startSpan('hello');
// do something
span.end();
return 'Hello World!';
}
}Auto Trace InstrumentationsThe most helpful part of this library is that you already get all of the instrumentations by default if you set up a module without any extra configuration. If you need to avoid some of them, you can use the import { Module } from '@nestjs/common';
import {
OpenTelemetryModule,
ControllerInjector,
EventEmitterInjector,
GuardInjector,
LoggerInjector,
PipeInjector,
ScheduleInjector,
} from '@metinseylan/nestjs-opentelemetry';
import { ZipkinExporter } from '@opentelemetry/exporter-zipkin';
import { SimpleSpanProcessor } from '@opentelemetry/sdk-trace-base';
@Module({
imports: [
OpenTelemetryModule.forRoot({
traceAutoInjectors: [
ControllerInjector,
GuardInjector,
EventEmitterInjector,
ScheduleInjector,
PipeInjector,
LoggerInjector,
],
spanProcessor: new SimpleSpanProcessor(
new ZipkinExporter({
url: 'your-zipkin-url',
}),
),
}),
]
})
export class AppModule {}List of Trace Injectors
Distributed Logging with Trace IDWhen you set up your environment with the
MetricsSimple setup with Prometheus exporter, you need install @opentelemetry/exporter-prometheus import { OpenTelemetryModule } from '@metinseylan/nestjs-opentelemetry';
import { PrometheusExporter } from '@opentelemetry/exporter-prometheus';
@Module({
imports: [OpenTelemetryModule.forRoot({
metricExporter: new PrometheusExporter({
endpoint: 'metrics',
port: 9464,
})
})]
})
export class AppModule {}Now you can access Prometheus exporter with auto collected metrics http://localhost:9464/metrics. Metric DecoratorsIf you need to observe simple block of function, you can use some basic decorators like Counterimport { Injectable } from '@nestjs/common';
import { Counter } from '@metinseylan/nestjs-opentelemetry';
@Injectable()
export class AppService {
@Counter()
getHello(): string {
return 'Hello World!';
}
}
@Counter('call_me_mr_fahrenheit', {
description: 'important function call counting here.'
})And of course, you can configure your decorator metric, the first parameter is "name" and the second one is MetricOptions Observerimport {Injectable} from '@nestjs/common';
import {Observer} from "./Observer";
@Injectable()
export class AppService {
@Observer('nice_one_observer', {
description: 'some description here.',
boundaries: [10, 20, 30],
})
getHello(): string {
return 'Hello World!';
}
}
Metric ProvidersIn advanced usage cases, you need to access the native OpenTelemetry Metric provider to access them from the Nestjs application context. import { Injectable } from '@nestjs/common';
import { Meter } from '@opentelemetry/sdk-metrics-base';
import { Counter } from '@opentelemetry/api-metrics';
@Injectable()
export class AppService {
private readonly counter: Counter;
constructor(private readonly meter: Meter) {
this.counter = this.meter.createCounter('handsome_counter');
}
getHello(): string {
this.counter.add(1);
return 'Hello World!';
}
}Auto Metric ObserversThis library has extendable resource and protocol-specific Auto Observers. All of them come with default module configuration, which you can extend and configure. import { Module } from '@nestjs/common';
import { PrometheusExporter } from '@opentelemetry/exporter-prometheus';
import {
ActiveHandlesMetric,
HttpRequestDurationSeconds,
OpenTelemetryModule,
} from '@metinseylan/nestjs-opentelemetry';
@Module({
imports: [
OpenTelemetryModule.forRoot({
metricAutoObservers: [
HttpRequestDurationSeconds.build({
boundaries: [20, 30, 100],
}),
ActiveHandlesMetric,
],
metricExporter: new PrometheusExporter({
endpoint: 'metrics',
port: 9464,
}),
metricInterval: 1000,
}),
],
})
export class AppModule {}
List Of Auto Observers
Example Output for
|
| Key | Value |
|---|---|
| exception | Empty string or exception instance name |
| method | GET, POST, PUT, PATCH, DELETE |
| outcome | INFORMATIONAL, SUCCESS, REDIRECTION, CLIENT_ERROR, SERVER_ERROR |
| status | number of HttpStatus |
| uri | url path |
Lets Combine All of them
import { Module } from '@nestjs/common';
import { PrometheusExporter } from '@opentelemetry/exporter-prometheus';
import { OpenTelemetryModule } from '@metinseylan/nestjs-opentelemetry';
import { ZipkinExporter } from '@opentelemetry/exporter-zipkin';
import { SimpleSpanProcessor } from '@opentelemetry/sdk-trace-base';
@Module({
imports: [
OpenTelemetryModule.forRoot({
metricExporter: new PrometheusExporter({
endpoint: 'metrics',
port: 9464,
}),
metricInterval: 1000,
spanProcessor: new SimpleSpanProcessor(
new ZipkinExporter({
url: 'your-zipkin-url',
})
),
}),
],
})
export class AppModule {}
|
I've been making some progress with a middleware here willsoto/nestjs-prometheus#950 It covers most of the cases here. |
|
I'm also very interested in this being a NestJS module - I think OpenTelemetry is going to be the "standard" for this stuff going forward. For example it's now officially supported in .NET: https://devblogs.microsoft.com/dotnet/opentelemetry-net-reaches-v1-0/ |
|
If that helps someone here, I ended up creating a new library due to some urgency for some features and only relying on opentelemetry for both tracing and metrics. Ended up solving some problems I had, but also documented how to integrate OTEL with things like structured logging. https://github.com/pragmaticivan/nestjs-otel It mainly uses open telemetry for everything. It covers:
|
|
Howdy! I've launched a new major version of nestjs-otel with improvements and now including full examples. (https://github.com/pragmaticivan/nestjs-otel) A full working example can be found in A dashboard example is also available: Logs are automatically associated with tracing (Loki + Tempo): |
|
Keep an eye for an upcoming package for NestJS self instrumentation with OpenTelemetry! https://github.com/open-telemetry/opentelemetry-js-contrib/pull/553/files Also Fastify might be coming soon: open-telemetry/opentelemetry-js-contrib#611 |
|
https://github.com/honnamkuan/nestjs-status-monitor/ works well without any 3rd-party-tool except socket.io |
|
For those looking for some sample code of how to integrate OTEL into your Nest app, this was the only code that I needed to add to my project to do so: import { Logger } from '@nestjs/common';
import { OTLPMetricExporter } from '@opentelemetry/exporter-metrics-otlp-http';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { registerInstrumentations } from '@opentelemetry/instrumentation';
import { ExpressInstrumentation } from '@opentelemetry/instrumentation-express';
import { GraphQLInstrumentation } from '@opentelemetry/instrumentation-graphql';
import { HttpInstrumentation } from '@opentelemetry/instrumentation-http';
import { Resource } from '@opentelemetry/resources';
import {
ConsoleMetricExporter,
MeterProvider,
MetricReader,
PeriodicExportingMetricReader,
} from '@opentelemetry/sdk-metrics';
import {
BatchSpanProcessor,
ConsoleSpanExporter,
SimpleSpanProcessor,
} from '@opentelemetry/sdk-trace-base';
import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node';
import { SemanticResourceAttributes } from '@opentelemetry/semantic-conventions';
import { PrismaInstrumentation } from '@prisma/instrumentation';
import { environment } from './environments/environment';
const logger = new Logger('OTLP');
if (environment.openTelemetry) {
const resource = Resource.default().merge(
new Resource({
[SemanticResourceAttributes.SERVICE_NAME]: environment.openTelemetry.serviceName,
})
);
const useTracer =
environment.openTelemetry.exporters?.traceConsole || environment.openTelemetry.exporters?.trace;
const tracerProvider = useTracer ? new NodeTracerProvider({ resource }) : undefined;
if (environment.openTelemetry.exporters?.traceConsole) {
tracerProvider?.addSpanProcessor(new SimpleSpanProcessor(new ConsoleSpanExporter()));
logger.log('exporting traces to console');
}
if (environment.openTelemetry.exporters?.trace) {
tracerProvider?.addSpanProcessor(
new BatchSpanProcessor(new OTLPTraceExporter(environment.openTelemetry.exporters.trace))
);
logger.log(
`exporting traces to ${
environment.openTelemetry.exporters.trace.url ?? 'http://localhost:4318/v1/traces'
}`
);
}
tracerProvider?.register();
/**
* Metrics
*/
const useMeter =
environment.openTelemetry.exporters?.meterConsole || environment.openTelemetry.exporters?.meter;
const readers: MetricReader[] = [];
if (environment.openTelemetry.exporters?.meterConsole) {
readers.push(
new PeriodicExportingMetricReader({
exporter: new ConsoleMetricExporter(),
// exportIntervalMillis: 3000,
})
);
logger.log('exporting metrics to console');
}
if (environment.openTelemetry.exporters?.meter) {
readers.push(
new PeriodicExportingMetricReader({
exporter: new OTLPMetricExporter(environment.openTelemetry.exporters.meter),
// exportIntervalMillis: 3000,
})
);
logger.log(
`exporting metrics to ${
environment.openTelemetry.exporters.meter.url ?? 'http://localhost:4318/v1/metrics'
}`
);
}
const meterProvider = useMeter ? new MeterProvider({ resource, readers }) : undefined;
registerInstrumentations({
meterProvider,
instrumentations: [
new HttpInstrumentation(),
new ExpressInstrumentation(),
new GraphQLInstrumentation(),
new PrismaInstrumentation(),
],
});
} |
and others
For a better monitoring of my Nest microservices I'd like to have a metrics endpoint which can be scraped by monitoring systems like Prometheus.
I believe a module written for NestJS could provide quite a couple metrics by default:
Beside these default metrics which could be offered out of the box, it should be easily possible to inject the metrics module into services, so that users can create custom metrics for their business code (e. g. number of registrations).
Using these metrics we can easily attach grafana to it and monitor our services:
The text was updated successfully, but these errors were encountered: