Skip to content

Add documention for instrumenting Go function with OpenTelemetry #396

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/edge/open-telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,4 +76,4 @@ Checkout the docs for more details on [how to configure Grafana Alloy to collect

Functions and services deployed on OpenFaaS Edge can use the service name when configuring the telemetry collection endpoint e,g. `alloy:4317` or `alloy:4318`.

For more info on how to instrument and configure telemetry for functions checkout our language guides for: [Python](/languages/python/#opentelemetry-zero-code-instrumentation) or [Node.js](/languages/node/#opentelemetry-zero-code-instrumentation).
For more info on how to instrument and configure telemetry for functions checkout our language guides for: [Python](/languages/python/#opentelemetry-zero-code-instrumentation), [Node.js](/languages/node/#opentelemetry-zero-code-instrumentation) or [Go](/languages/go/#opentelemetry-instrumentation)
120 changes: 120 additions & 0 deletions docs/languages/go.md
Original file line number Diff line number Diff line change
Expand Up @@ -406,3 +406,123 @@ func Handle(w http.ResponseWriter, r *http.Request) {
}
```

## OpenTelemetry instrumentation

We have a separate golang template `golang-otel` that has built-in support OpenTelemetry traces. The template is based on the `golang-middleware` template and allows you to get traces for Golang functions with minimal code changes.

Some of the key benefits of using the `golang-otel` template are:

- **No boilerplate** - Avoid boilerplate code to configure providers and traces in Go functions. No need to fork and maintain your own version of the golang templates.
- **Configuration using environment variables** - Simplify configuration with environment-based settings, reducing the need for code changes.
- **HTTP instrumentation** - Incoming HTTP requests to the function handler are automatically instrumented.
- **Extensibility with custom traces** - Easily add custom spans using the [OpenTelemetry Go Trace API](https://pkg.go.dev/go.opentelemetry.io/otel) without much boilerplate code.

Create a new function with the `golang-otel` template:

```sh
faas-cli new echo --lang golang-otel
```

Use environment variables to configure the traces exporter for the function.

```diff
functions:
echo:
lang: golang-otel
handler: ./echo
image: echo:latest
+ environment:
+ OTEL_TRACES_EXPORTER: console,otlp
+ OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT:-collector:4317}
```

The `golang-otel` template supports a subset of [OTEL SDK environment variables](https://opentelemetry.io/docs/languages/sdk-configuration/) to configure the exporter.

- `OTEL_TRACES_EXPORTER` specifies which tracer exporter to use. In this example traces are exported to `console` (stdout) and with `otlp` to send traces to an endpoint that accepts OTLP via gRPC.
- `OTEL_EXPORTER_OTLP_ENDPOINT` sets the endpoint where telemetry is exported to.
- `OTEL_SERVICE_NAME` sets the name of the service associated with the telemetry and is used to identify telemetry for a specific function. By default `<fn-name>.<fn-namespace>` is used as the service name on Kubernetes or `<fn-name>` when running the function with OpenFaaS Edge, or locally with `faas-cli local-run`.
- `OTEL_EXPORTER_OTLP_TRACES_INSECURE` can be set to true to disable TLS if that is not supported by the OpenTelemetry collector.

The template can be used as a drop-in replacement for functions that already use the `golang-middleware` template to get HTTP invocation traces for your existing functions without any code changes. All that is required is to change the `lang` field in the `stack.yaml` configuration.

```diff
version: 1.0
provider:
name: openfaas
gateway: http://127.0.0.1:8080
functions:
echo:
- lang: golang-middleware
+ lang: golang-otel
handler: ./echo
image: echo:latest
```

### Creating custom traces in the function handler

here may be cases where an instrumentation library is not available or you may want to add custom tracing data for some of your own functions and methods.

When using the `golang-otel` template the registered global trace provider can be retrieved to add custom spans in the function handler. A span represents a unit of work or operation. Spans are the building blocks of traces.

> Check out the [OpenTelemetry docs](https://opentelemetry.io/docs/languages/go/instrumentation/#creating-spans) for more information on how to work with spans.

On your code you can call the [otel.Tracer](https://pkg.go.dev/go.opentelemetry.io/otel#Tracer) function to get a named tracer and start a new span.

Make sure to add the required packages to the function handler:

```sh
go get "go.opentelemetry.io/otel"
```

Add custom spans in the function handler:

```go
package function

import (
"fmt"
"io"
"net/http"
"sync"

"go.opentelemetry.io/otel"
)

func callOpenAI(ctx context.Context, input []byte) {
// Get a tracer and create a new span
ctx, span := otel.Tracer("function").Start(ctx, "call-openAI")
defer span.End()

// Sleep for 2 seconds to simulate some work.
time.Sleep(time.Second * 2)
}


func Handle(w http.ResponseWriter, r *http.Request) {
var input []byte

if r.Body != nil {
defer r.Body.Close()

body, _ := io.ReadAll(r.Body)

input = body
}

// Call function with the request context to pass on any parent spans.
callOpenAI(r.Context(), input)

w.WriteHeader(http.StatusOK)
fmt.Fprintf(w, "Done processing.")
}
```

To create a new span with a tracer, you’ll need a handle on a `context.Context` instance. These will typically come from the request object and may already contain a parent span from an [instrumentation library](https://opentelemetry.io/docs/languages/go/libraries/).

You can now add [attributes](https://opentelemetry.io/docs/languages/go/instrumentation/#span-attributes) and [events](https://opentelemetry.io/docs/languages/go/instrumentation/#events), [set the status](https://opentelemetry.io/docs/languages/go/instrumentation/#set-span-status) and [record errors](https://opentelemetry.io/docs/languages/go/instrumentation/#record-errors) on the span as required.

## OpenTelemetry zero-code instrumentation

The beta Release for [zero-code instrumentation](https://opentelemetry.io/docs/concepts/instrumentation/zero-code/) of Go applications [has recently been announced](https://opentelemetry.io/blog/2025/go-auto-instrumentation-beta/).

Go auto-instrumentation has not been tested with OpenFaaS functions yet. The easiest way to try it out on Kubernetes is probably by using the [Opentelemetry Operator for Kubernetes](https://opentelemetry.io/docs/platforms/kubernetes/operator/automatic/).
14 changes: 5 additions & 9 deletions docs/languages/python.md
Original file line number Diff line number Diff line change
Expand Up @@ -421,17 +421,13 @@ RUN pip install --no-cache-dir --user -r requirements.txt

The `opentelemetry-bootstrap -a install` command reads through the list of packages installed in your active site-packages folder, and installs the corresponding instrumentation libraries for these packages, if applicable. The OpenTelemetry Python agent uses [monkey patching](https://stackoverflow.com/questions/5626193/what-is-monkey-patching) to modify functions in these libraries at runtime.


Update the fprocess ENV in the Dockerfile to start the OpenTelemetry agent:
Update the function process in `template.yaml`to start the OpenTelemetry agent:

```diff
# configure WSGI server and healthcheck
USER app

- ENV fprocess="python index.py"
+ ENV fprocess="opentelemetry-instrument python index.py"
language: python3-http
- fprocess: python index.py
+ fprocess: opentelemetry-instrument python index.py
```

Use your modified template to create a new function.

The OpenTelemetry agent can be configured using environment variables on the function:
Expand All @@ -450,7 +446,7 @@ functions:
+ OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT:-collector:4317}
```

- `OTEL_SERVICE_NAME` sets the name of the service associated with the telemetry and is used to identify telemetry for a specific function. It can be set to any value you want be we recommend using the clear function identifier `<fn-name>.<fn-namespace>`.
- `OTEL_SERVICE_NAME` sets the name of the service associated with the telemetry and is used to identify telemetry for a specific function. It can be set to any value you want but we recommend using the clear function identifier `<fn-name>.<fn-namespace>`.
- `OTEL_TRACES_EXPORTER` specifies which tracer exporter to use. In this example traces are exported to `console` (stdout) and with `otlp`. The `otlp` option tells `opentelemetry-instrument` to send the traces to an endpoint that accepts OTLP via gRPC.
- setting `OTEL_METRICS_EXPORTER` and `OTEL_LOGS_EXPORTER` to `none` we disable the metrics and logs exporters. You can enable them if desired.
- `OTEL_EXPORTER_OTLP_ENDPOINT` sets the endpoint where telemetry is exported to.
Expand Down