Skip to content

New Node.JS Otel samples #307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 52 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
b9b06f2
Initial skeleton app for otel
chughts Jun 2, 2025
99e14f5
Add in skeleton express code
chughts Jun 2, 2025
a39980b
Skeleton get request
chughts Jun 2, 2025
330d991
Skeleton QueueManager file
chughts Jun 2, 2025
2499067
Add parsing for Queue and Qmgr in request
chughts Jun 2, 2025
c0c70e3
Consolidate request parsing
chughts Jun 2, 2025
e3e1fd7
Updated how data is returned
chughts Jun 3, 2025
fdc2a89
Encapsulate action data
chughts Jun 3, 2025
0c94531
Process env.json file
chughts Jun 3, 2025
a25d83a
Add known check to qmgr request
chughts Jun 3, 2025
402f44a
Working envrionment with ibmmq
chughts Jun 4, 2025
4a024a3
Encapsulate contants into a single reference
chughts Jun 4, 2025
01603ed
Add logic to make CNO
chughts Jun 4, 2025
d37d266
Add logic to make connection
chughts Jun 4, 2025
a2cd64d
Establish container and native connections to queue manager
chughts Jun 4, 2025
4edfbf1
Remove dynamic dispatch as instance not preserved
chughts Jun 4, 2025
71bd196
Establish open skeleton
chughts Jun 4, 2025
0bf00c1
Open queue coded
chughts Jun 5, 2025
6cf7de7
Implement put action
chughts Jun 5, 2025
5cf1313
Rename action method
chughts Jun 5, 2025
16b7b4b
Working put framework
chughts Jun 5, 2025
d322672
Allow environment file to be specified as an environment variable
chughts Jun 5, 2025
8bc403d
Implement get functionality
chughts Jun 6, 2025
daab43e
Start unwrapping returned messages
chughts Jun 6, 2025
bd435d8
Add message processing skeleton
chughts Jun 6, 2025
dde36c9
Add logic to process messages
chughts Jun 6, 2025
a314458
Mark some of the messages as damaged
chughts Jun 6, 2025
d7d67d8
Refactor constants for instrumentation
chughts Jun 6, 2025
1965940
instrumentation framework
chughts Jun 6, 2025
b30b6a4
Add Jaeger container to build
chughts Jun 6, 2025
8bdfd88
Remove RFH2 Headers
chughts Jun 6, 2025
bc74268
Add Jaeger to internal network
chughts Jun 6, 2025
5e7b898
Force 400 error if unknown QMGR
chughts Jun 6, 2025
8620705
Pass qmgr to known check
chughts Jun 6, 2025
8a7bd51
Create an active span for the asynchronous processing
chughts Jun 9, 2025
ddfb8eb
Base implementation of extracting traceparent
chughts Jun 9, 2025
bf1b864
Pull out trace parent from message header if present
chughts Jun 9, 2025
ec30b62
Admit defeat to vscode auto tab formatter
chughts Jun 9, 2025
e4745dc
Signal error span when damaged message found
chughts Jun 9, 2025
4c9cfcd
Add simple metrics capture
chughts Jun 10, 2025
a1aff78
Implement bad message count metric
chughts Jun 10, 2025
eec1199
Add metric to count problematic gets
chughts Jun 10, 2025
86169a2
Add links to original spans
chughts Jun 10, 2025
07df8e6
Add application description to README
chughts Jun 10, 2025
262b0f2
Add way to run without instrumentation
chughts Jun 11, 2025
97125d4
Corrected typo
chughts Jun 11, 2025
ba7affb
Code tidy
chughts Jun 12, 2025
f4a08b1
Code tidy
chughts Jun 12, 2025
a85e60b
code tidy
chughts Jun 12, 2025
85dc253
Code tidy
chughts Jun 13, 2025
52cba46
source jaeger and prometheus images from quay.io
chughts Jun 16, 2025
2ca54df
Base Node.js image on ubi9
chughts Jun 16, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions Node.js-OTel/Dockerfile.mqapp
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Copyright 2025 IBM Corp.
# Licensed under the Apache License, Version 2.0 (the 'License');
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at

# http://www.apache.org/licenses/LICENSE-2.0

# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

FROM --platform=linux/amd64 registry.access.redhat.com/ubi9/nodejs-20 as builder

# Need to become root for npm install to work
USER 0

ENV BUILD_APP_DIR /opt/app-root/src/nodejs/app
WORKDIR ${BUILD_APP_DIR}

# Copy project files into the container
COPY ./ ./

# Copy is crude, so may have native build legacy, which should
# be removed before a container based npm install
RUN rm -fr node_modules \
&& npm install --omit=dev \
&& chmod a+rx ${BUILD_APP_DIR}/*

#Cleaning
WORKDIR ${BUILD_APP_DIR}/node_modules/ibmmq/lib
RUN rm mqidefs_aix.js \
&& rm mqidefs_darwin.js \
&& rm mqidefs_linuxPowerLE.js \
&& rm mqidefs_linuxS390.js \
&& rm mqidefs_windows.js \
&& rm mqidefs_linuxARM.js

WORKDIR ${BUILD_APP_DIR}/node_modules/ibmmq/redist
RUN rm -r msg \
&& rm -r bin \
&& rm -r lap

WORKDIR ${BUILD_APP_DIR}/node_modules/ibmmq/redist/lib64
RUN rm libcurl.so \
&& rm libmqz_r.so \
&& rm libmqzsd_r.so \
&& rm libmqcxa64_r.so \
&& rm libmqcxa_r.so \
&& rm libmqdc_r.so \
&& rm libmqecs_r.so \
&& rm libmqic_r.so \
&& rm libmqiz_r.so \
&& rm libmqjx_r.so \
&& rm libmqmcs_r.so \
&& rm libmqmzse.so \
&& rm libmqxzu_r.so

###########################################################
# This starts the RUNTIME phase
###########################################################
# Now that there is a container with the compiled program we can build a smaller
# runtime image. Start from one of the smaller base container images. This container
# is a Node image with the runtime already embedded
FROM --platform=linux/amd64 registry.access.redhat.com/ubi9/nodejs-20-minimal as appImage

# Need to repeat BUILD Directory location as environment option will
# have been lost
ENV BUILD_APP_DIR /opt/app-root/src/nodejs/app
ENV RUN_APP_DIR /opt/app-root/src/nodejs/app
WORKDIR ${RUN_APP_DIR}

# # Copy over the tree containing the program and all its dependencies
COPY --from=builder ${BUILD_APP_DIR}/ ${RUN_APP_DIR}/

EXPOSE 8080

# For container debugging run
# CMD ["sleep", "infinity"]

# Normal application run
# CMD ["npm","run","start"]

# With tracing and metrics to the console
# CMD ["npm","run","trace"]

# With tracing to Jaeger and metrics to Prometheus
CMD ["npm","run","export-trace"]

124 changes: 124 additions & 0 deletions Node.js-OTel/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# IBM MQ Otel samples for Node.js applications
This sample is based on the Node.js (Node.js, serverless and showcase) patterns in this repo. They have been reworked and cutdown, to allow http requests to initiate put and get requests. Otel Instrumentation is optional.

## Components
There are four components that make up this sample.
- *mq* A MQ Server
- *mqapp* HTTP based Node.js app that listens on `/put` and `get` and invokes MQI API calls.
- *jaeger* That captures otel traces issued by *mqapp*
- *prometheus* That captures otel metrics issued by *mqapp*

## Pre-requsites
We have found that this sample runs best with `podman` and `podman compose`

## *mq*
The docker-compose uses either `icr.io/ibm-messaging/mq:latest` as the base image or a custom built image when running on MacOS ARM64 eg. `ibm-mqadvanced-server-dev:9.4.2.0-arm64`.

You can create an Apple ARM64 compatible container by following this [blog](https://community.ibm.com/community/user/blogs/richard-coppen/2023/06/30/ibm-mq-9330-container-image-now-available-for-appl)

admin and app passwords are set as secrets from the files `admin-password.txt` and `app-password.txt`

## *mqapp*
The application listens to:
- `put`
- `get`

Both are expecting the following query parameters
- *QMGR* Queue Manager
- *QUEUE& Queue
- *num* Quantity of messages to put / get, which if not specified is set to a random number. If the number in the query is > 9 then it is set to a random number < 10

The endpoints can be invoked with curl eg.

```
curl "http://localhost:8080/get?QMGR=QM2&QUEUE=DEV.QUEUE.1
```

### Queue Manager check
The application can interact with a number of queue managers.
The connection details for each are provided in the `env.json` file.

- *QMGR*
- *QM_HOST*
- *QM_PORT*
- *APP_USER*
- *APP_PASSWORD*
- *CHANNEL*

They can be overriden by envrionment variables for example as set in the `docker-compose.yaml` file

- *QMGR_0*
- *QM_HOST_0*=mq*
- *QM_PORT_0*
- *APP_USER_0*
- *APP_PASSWORD_0*

On receipt of a request there is a check to ensure that the queue manager specified in the request is known. If it is not known a 400 HTTP response is returned. Otherwise a 200 HTTP response is returned. A check is made on `num`, and if not a number, or negative or too high, a randomly generated value is used as a replacement.

The subsequent message logic is run asynchronously after the 200 response has been sent.

### messaging logic
The underlying Node.js `ibmmq` library is able to detect if the invoking application is using an Otel SDK.

When it is:

- For outbound messages (MQPUT), the binding looks to see if there is an active OTel trace. If so, it takes the context details and inserts message properties to represent them, if they are not already there.

The context then passes through the MQ network before being consumed.

- For inbound messages (MQGET), the binding looks for those context properties in the message. If it finds them, and it also finds that there is an active span/trace in the application process, it creates a link from the active span to that context.

### put logic
The put messaging logic uses the configurable environment variable `ErrorSensitivity` to add a damaged payload to some of the messages sent to the queue. By default this will be 1 in 5

### get logic
The get messaging logic pulls a set of messages and processes them as a group. For every damaged payload it finds it signals an exception to the trace, linking to the origin trace and span information that has been extracted from the context properties in the message.

### Instrumentation
The Otel instrumentation SDK is intialised and started in the file `instrumentation/control.js`

The trace instrumentation is initialised in `instrumentation/span-trace.js`

The metric instrumentation is initialised in `instrumentation/metrics.js`

### Running the sample application
The application can be started in one of three modes

- `npm run start` which starts up the application without any instrumentation
- `npm run trace` which starts up the application with console only instrumentation
- `npm run export-trace` which starts up the application with Jaeger for tracing and Prometheus for metrics.

By default the Docker file created for the *mqapp* container starts the applicaion as `export-trace`

## *jaeger*
The application sends tracing information to *jaeger* on `http://jaeger:4318/v1/traces`

The Jaeger UI is available on http://localhost:16686/

Application `/get` traces that detect a damaged message have an associated exception span, for each damaged message detected. The exception span links back to the `/put` trace that delivered the damaged message.

## *prometheus*
*prometheus* scrapes metrics from the *mqapp* from `http://localhost:9464/metrics`

The Prometheus UI is available on `http://localhost:9090/`

*mqapp* makes the following metrics available

- **MQI-sample-app-damaged-message-count** which holds the total number of damaged messages found.
- **MQI_sample_app_damaged_get_request_count_total** which holds the total number of http `/get` actions that found damaged messages.

Where MQI-sample-app-damaged-message-count >= MQI_sample_app_damaged_get_request_count_total

## Building the containers
Build the containers by running the command:

```
podman compose -f docker-compose.yaml build
```

## Running the containers
Start the containers by running the command:

```
podman compose -f docker-compose.yaml up
```
1 change: 1 addition & 0 deletions Node.js-OTel/admin-password.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
passw0rd
1 change: 1 addition & 0 deletions Node.js-OTel/app-password.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
passw0rd
120 changes: 120 additions & 0 deletions Node.js-OTel/app.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
/**
* Copyright 2025 IBM Corp.
*
* Licensed under the Apache License, Version 2.0 (the 'License');
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
**/

// Set up debug logging options
const debug_info = require('debug')('mqsample:otel:app:info');
const debug_warn = require('debug')('mqsample:otel::app:warn');

debug_info('Application is starting');

const express = require('express');

const {appLimits} = require('./settings/limits.js');
const {QueueManagerInterface} = require('./queue-manager/qm-requests.js');
const {ActionData} = require('./data/action.js');
const {constants} = require('./settings/constants.js');
const {envSettings} = require('./settings/environment.js');

const NO_QMGR_OR_QUEUE = "QMGR / QUEUE is missing from data input";
const UNKNOWN_QMGR = "Queue Manager not known";

const HTTP_PORT = parseInt(process.env.PORT || '8080');
const app = express();

let qmi = new QueueManagerInterface();

app.get('/put', (req, res) => {
debug_info('Put requested');
processRequest(req, res, constants.PUT);
});

app.get('/get', (req, res) => {
debug_info('Get requested')
processRequest(req, res, constants.GET);
});

function processRequest(req, res, type) {
debug_info('Determining number of messages to process');

let {data, err} = parseRequest(req);

if (null === err) {
debug_info(`Will be processing ${data.num} messages`);
switch (type) {
case constants.PUT:
err = qmi.put(data);
break;
case constants.GET:
err = qmi.get(data);
break;
default:
err = "Command not recognised";
}
}

if (null !== err) {
res.status(400).send(err);
return;
}

res.send(JSON.stringify(`Request to process ${data.num} messages on ${data.qmgr} accepted`));
}

function parseRequest(req) {
let data = {};
let err = null;

data = new ActionData();

data.num = determineNumInRequest(req);

data.qmgr = req.query.QMGR ? req.query.QMGR : null;
data.queue = req.query.QUEUE ? req.query.QUEUE : null;

let ok = true;
for (let value of [data.qmgr, data.queue]) {
if (typeof value === 'undefined' || null === value) {
debug_warn(NO_QMGR_OR_QUEUE);
ok = false;
}
}

if (!ok) {
err = NO_QMGR_OR_QUEUE;
} else if (! envSettings.qmgrIsKnown(data.qmgr)) {
err = UNKNOWN_QMGR;
}

return {data, err};
}


function determineNumInRequest(req) {
let number = req.query.num ? parseInt(req.query.num.toString()) : NaN;
if (isNaN(number)) {
number = appLimits.randomNum();
} else {
number = appLimits.applyLimit(number);
}
return number;
}

app.listen(HTTP_PORT, () => {
debug_info(`Listening on port ${HTTP_PORT}`);
});


debug_info('Application has started');
40 changes: 40 additions & 0 deletions Node.js-OTel/data/action.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
/**
* Copyright 2025 IBM Corp.
*
* Licensed under the Apache License, Version 2.0 (the 'License');
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
**/

// Set up debug logging options
const debug_info = require('debug')('mqsample:otel:actiondata:info');
const debug_warn = require('debug')('mqsample:otel:actiondata:warn');

class ActionData {
// private properties only accessible through getters and setters
#num = 0;
#qmgr = null;
#queue = null;

constructor() {
}

get num() {return this.#num;}
get qmgr() {return this.#qmgr;}
get queue() {return this.#queue;}


set num(n) {return this.#num = n;}
set qmgr(qm) {return this.#qmgr = qm;}
set queue(q) {return this.#queue = q;}
}

module.exports = { ActionData };
Loading