[Tracking Issue] Auto-tracing not working in Google Cloud Functions/Firebase Cloud Functions

[SamyPesse]

When using @google-cloud/trace-agent in Firebase Cloud Functions, the custom tracing works, but auto tracing doesn't (no trace for HTTP/HTTPS/gRPC requests).

Even included as the first module, the cloud functions log output:

ERROR:@google-cloud/trace-agent: TraceAgent#start: Tracing might not work as the following modules were loaded before the trace agent was initialized: [accepts, body-parser, bytes, content-disposition, content-type, cookie, cookie-signature, crc, debug, depd, destroy, ee-first, escape-html, etag, express, finalhandler, forwarded, fresh, iconv-lite, ipaddr.js, media-typer, merge-descriptors, methods, mime, mime-db, mime-types, ms, negotiator, on-finished, parseurl, path-to-regexp, proxy-addr, qs, range-parser, raw-body, send, serve-static, type-is, utils-merge, vary]

It may related to how Firebase Cloud Functions works as an overlay of Google Cloud Functions.

[kjin]

We don't expect tracing to work out of the box for Firebase Cloud Functions (or GCF) at the moment (I'll make sure that gets documented from here on out). The error message points out that express was loaded, presumably before your function code was run.

You could try the workaround of wrapping the function contents in runInRootSpan. From the README:

const traceApi = require('@google-cloud/trace-agent').start();
traceApi.runInRootSpan({ name: 'my-root-span' }, (rootSpan) => {
  // ...
  rootSpan.endSpan();
});

You might consider using rootSpan.addLabel(key, value) to add information that is relevant to the function call, as no labels will be included by default (except for the stack trace, which in this context likely has limited usefulness).

Given how background trace publishing operations may not be supported by Cloud Function's CPU throttling model, it might be useful to publish traces as soon as they're created (by specifying that at most one trace should be buffered at a time):

const traceApi = require('@google-cloud/trace-agent').start({ bufferSize: 1 });

Please let us know if there are any issues that arise!

[kjin]

@SamyPesse If you don't mind, I am going to use this as a tracking issue for the progress of getting the Trace Agent to work with GCF.

[SamyPesse]

@kjin sure 👍 we made it works in our Firebase cloud functions, but without auto-tracing.

The lack of auto-tracing seems to come from the code added by Firebase that requires the modules like express before requiring the application.

[kjin]

@SamyPesse Thanks for the update!

We've been working with the GCF team (which I believe also maintains the Firebase Cloud Functions, or at least the functionality should be similar anyway) but I don't think we can support it yet within the next two months or so. Will give updates as I hear about them.

[steren]

Hi (I am a Product Manager on Google Cloud).
We are working to make it super easy to enable Stackdriver Trace for your functions with no setup required. You can sign up for the Alpha at https://goo.gl/forms/MtCO9Ehu3IfSsqiw2. I should send the first invites within a few weeks.

As others said, in the meantime, you should be able to import the trace module, but not everything will be traced as it is not the first module imported.

[shaneosullivan]

I'm having trouble getting the suggested workaround to work. My code looks as follows

import * as TraceAgent from "@google-cloud/trace-agent";
import * as express from "express";

const app = express();
app.post(
  "/graphql",
  graphqlHTTP({
    schema
  })
);

let tracer;

export = functions
  // @ts-ignore
  .runWith(runtimeOpts)
  .https.onRequest(async (req, resp) => {
    tracer = tracer ? tracer : TraceAgent.start({ bufferSize: 1 });

    const requestName =
      "GraphQL: Root " + (req.body ? req.body.operationName : "unspecified");

    tracer
      .runInRootSpan(
        {
          name: requestName
        },
        async rootSpan => {
          let ret;
          try {
            ret = await app(req, resp);
          } finally {
            console.log(requestName + "[end]");
            rootSpan.endSpan();
          }
          return ret;
        }
      )
      .catch(err => {
        console.error("GraphQL tracing error", err);
      });
  });

As suggested, all code runs in the rootSpan, and the bufferSize is set to 1. I still get the error

error: @google-cloud/trace-agent ERROR StackdriverTracer#start: Tracing might not work as the following modules were loaded before the trace agent was initialized: .....

I've signed up for the Alpha (thanks steren!), but I have many Firebase projects that need proper tracing, so any workaround that would help right now would be useful.

Thanks!

[kjin]

Hi @shaneosullivan -- are you still getting traces reported? The error message being shown here is likely a false alarm, as the condition for its appearance is simply that modules were loaded before the Trace Agent (which is true by nature of the workaround); it's just a blanket "warning" that suggests a possible reason why traces might not get published, if they are not.