Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose Prometheus metrics with Micrometer #407

Merged
merged 1 commit into from
Jul 13, 2022

Conversation

nscuro
Copy link
Collaborator

@nscuro nscuro commented Jul 11, 2022

Integrate Metrics collection using Micrometer and expose them for Prometheus.

I did not go with Dropwizard Metrics as that framework lacks support for labels or tags, which are important to be able to add dimensions to metrics.

Also decided against implementing this as an AbstractMetricsResource and went with a simple HttpServlet instead. Main reason being that Prometheus metrics wouldn't benefit from access control features in Alpine, as Prometheus doesn't support custom headers when scraping metrics - it wouldn't be possible to supply an API key. For now, users can keep metrics disabled, or add basic authentication at the proxy layer.

Intended behavior:

  • If ALPINE_METRICS_ENABLED is false or not set at all, MetricsServlet will always respond with HTTP 404 and an empty page. Metrics that require additional work being done in order to collect them (e.g. monitoring ExecutorServices) will not be registered.
  • If ALPINE_METRICS_ENABLED is true, system metrics are collected. All metrics are exposed via MetricsServlet.

Additional metrics can be registered with the global registry, both by applications using Alpine and Alpine itself.

Partly addresses #22.

Per default, basic system metrics are exposed:

Screenshot 2022-07-11 at 21 12 38

Additionally, metrics related to ExecutorServices used by Alpine, and other Alpine-specific metrics like published events or notifications are exposed:

Screenshot 2022-07-11 at 22 00 19

import javax.servlet.ServletContextListener;

/**
* @since 2.1.0
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MissingSummary: A summary line is required on public/protected Javadocs.

Reply with "@sonatype-lift help" for info about LiftBot commands.
Reply with "@sonatype-lift ignore" to tell LiftBot to leave out the above finding from this PR.
Reply with "@sonatype-lift ignoreall" to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.

When talking to LiftBot, you need to refresh the page to see its response. Click here to get to know more about LiftBot commands.


Was this a good recommendation?
[ 🙁 Not relevant ] - [ 😕 Won't fix ] - [ 😑 Not critical, will fix ] - [ 🙂 Critical, will fix ] - [ 😊 Critical, fixing now ]

import java.util.concurrent.ExecutorService;

/**
* @since 2.1.0
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MissingSummary: A summary line is required on public/protected Javadocs.

Reply with "@sonatype-lift help" for info about LiftBot commands.
Reply with "@sonatype-lift ignore" to tell LiftBot to leave out the above finding from this PR.
Reply with "@sonatype-lift ignoreall" to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.

When talking to LiftBot, you need to refresh the page to see its response. Click here to get to know more about LiftBot commands.


Was this a good recommendation?
[ 🙁 Not relevant ] - [ 😕 Won't fix ] - [ 😑 Not critical, will fix ] - [ 🙂 Critical, will fix ] - [ 😊 Critical, fixing now ]

import java.io.IOException;

/**
* @since 2.1.0
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MissingSummary: A summary line is required on public/protected Javadocs.

Reply with "@sonatype-lift help" for info about LiftBot commands.
Reply with "@sonatype-lift ignore" to tell LiftBot to leave out the above finding from this PR.
Reply with "@sonatype-lift ignoreall" to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.

When talking to LiftBot, you need to refresh the page to see its response. Click here to get to know more about LiftBot commands.


Was this a good recommendation?
[ 🙁 Not relevant ] - [ 😕 Won't fix ] - [ 😑 Not critical, will fix ] - [ 🙂 Critical, will fix ] - [ 😊 Critical, fixing now ]

@nscuro
Copy link
Collaborator Author

nscuro commented Jul 11, 2022

This can be tested with Dependency-Track as follows:

  • Add the MetricsServlet to web.xml:
diff --git a/src/main/webapp/WEB-INF/web.xml b/src/main/webapp/WEB-INF/web.xml
index aa5d63bb..db0c70f6 100644
--- a/src/main/webapp/WEB-INF/web.xml
+++ b/src/main/webapp/WEB-INF/web.xml
@@ -23,6 +23,9 @@
          xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd"
          version="3.1">
 
+    <listener>
+        <listener-class>alpine.server.metrics.MetricsInitializer</listener-class>
+    </listener>
     <listener>
         <listener-class>org.dependencytrack.RequirementsVerifier</listener-class>
     </listener>
@@ -50,7 +53,7 @@
         <filter-class>alpine.server.filters.WhitelistUrlFilter</filter-class>
         <init-param>
             <param-name>allowUrls</param-name>
-            <param-value>/index.html,/css,/fonts,/img,/js,/static,/favicon.ico,/api,/mirror,/.well-known</param-value>
+            <param-value>/index.html,/css,/fonts,/img,/js,/static,/favicon.ico,/api,/metrics,/mirror,/.well-known</param-value>
         </init-param>
         <init-param>
             <param-name>forwardTo</param-name>
@@ -58,7 +61,7 @@
         </init-param>
         <init-param>
             <param-name>forwardExcludes</param-name>
-            <param-value>/api,/mirror</param-value>
+            <param-value>/api,/metrics,/mirror</param-value>
         </init-param>
     </filter>
     <filter-mapping>
@@ -122,6 +125,16 @@
         <url-pattern>/api/*</url-pattern>
     </servlet-mapping>
 
+    <servlet>
+        <servlet-name>Metrics</servlet-name>
+        <servlet-class>alpine.server.servlets.MetricsServlet</servlet-class>
+        <load-on-startup>1</load-on-startup>
+    </servlet>
+    <servlet-mapping>
+        <servlet-name>Metrics</servlet-name>
+        <url-pattern>/metrics</url-pattern>
+    </servlet-mapping>
+
     <servlet>
         <servlet-name>NVD Mirror</servlet-name>
         <servlet-class>org.dependencytrack.servlets.NvdMirrorServlet</servlet-class>
  • Add this docker-compose.yml to the root directory of DT's repository:
version: "3"

services:
  dtrack-apiserver:
    build:
      context: .
      dockerfile: src/main/docker/Dockerfile
    environment:
      ALPINE_METRICS_ENABLED: "true"
    ports:
    - "127.0.0.1:8080:8080"
    volumes:
    - "dtrack-apiserver-data:/data"
    restart: unless-stopped

  dtrack-frontend:
    image: dependencytrack/frontend:snapshot
    environment:
      API_BASE_URL: "http://localhost:8080"
    ports:
    - "127.0.0.1:8081:8080"
    restart: unless-stopped

  prometheus:
    image: prom/prometheus:v2.36.2
    ports:
    - "127.0.0.1:9090:9090"
    volumes:
    - "./prometheus.yml:/etc/prometheus/prometheus.yml:ro"
    - "prometheus-data:/prometheus"
    restart: unless-stopped

  grafana:
    image: grafana/grafana-oss:9.0.2
    ports:
    - "127.0.0.1:3000:3000"
    restart: unless-stopped

volumes:
  dtrack-apiserver-data: {}
  prometheus-data: {}
  • Create a file called prometheus.yml in DT's root directory with the following content:
scrape_configs:
- job_name: dtrack-apiserver
  scrape_interval: 15s
  scrape_timeout: 15s
  static_configs:
  - targets:
    - dtrack-apiserver:8080
  • Build the executable WAR for DT's apiserver distribution:
mvn clean package -DskipTests -P enhance -P embedded-jetty -Dlogback.configuration.file=src/main/docker/logback.xml 
  • Build the API server container, launch all services:
docker compose build --pull
docker compose up -d

@stevespringett
Copy link
Owner

Excellent work. Is this ready for merge?

Signed-off-by: nscuro <nscuro@protonmail.com>
@nscuro
Copy link
Collaborator Author

nscuro commented Jul 13, 2022

@stevespringett Yup, ready to go!

I just changed the MetricsServlet response from HTTP 204 to HTTP 404 when metrics are disabled.

@stevespringett stevespringett merged commit 8814673 into stevespringett:master Jul 13, 2022
@nscuro nscuro deleted the metrics branch July 13, 2022 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants