Possibility to support Application Server in semantic conventions?

[ivomagi]

I will be involved in contributing quite a few auto-instrumentations for different Java application servers over the next few months for OpenTelemetry Java Agent. Support and tests for Jetty, Tomcat, WildFly, GlassFish, JBoss, etc will be built with my involvement.

What I am currently struggling to understand is the means to send the application-server specific information using the OpenTelemetry. What I expect to send is the information that “This span originated from Apache Tomcat 7.0”, probably encoded in two different key-value pairs.

I looked at the semantic conventions about Resources, especially Process, but it does not seem to offer such possibility. process.runtime seems to be be reserved for JVM vendor/version. So I am looking at two (seemingly) bad alternatives as of now:

• send this information just as span attributes (creating redundancy & bloating the protocol) or
• go off the standard and add custom namespace to the Process which would be specific to one vendor (Splunk to be precise).

At least in the Java world, the application server is an extremely important dimension when trying to make sense of the runtime architecture of yours, so I am trying to figure out whether or not this should be part of the semantic conventions. For other platforms the issue might not be as severe, but for example the difference between php-fpm and mod_php for PHP installations and the PyPy vs CPython for the Python world might reveal similar issues.

[kenfinnigan]

I see the application server as being less meaningful in the world of containers.

Does it matter if a container that used "Tomcat" now uses "WildFly", probably not?

I'd also imagine that the Container semantics could be used to infer or deduce what the application server is.

[iNikem]

We are not "in the world of containers" :) Although they are certainly mainstream and are gaining even more popularity, there are a significant number of clients who does not use containers and still use WebSphere or Weblogic.

And even inside container, Tomcat and WildFly may have very different runtime characteristics and failure modes.

[kenfinnigan]

@iNikem I didn't say we were in a world of containers, though I can see how it might appear implied. Being focused on Kubernetes and containers I sometimes forget there are enterprises not using them.

I'd be curious to understand how defining application server semantics facilitates the monitoring of the application?

[Oberon00]

If you read https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/trace/semantic_conventions/http.md#http-server-definitions, the spec already concedes the existence of application servers with wording such as:

Within a single virtual host, some servers support the concepts of an HTTP application (for example in Java, the Servlet JSR defines an application as "a collection of servlets, HTML pages, classes, and other resources that make up a complete application on a Web server" -- SRV.9 in JSR 53; in a deployment of a Python application to Apache, the application would be the PEP 3333 conformant callable that is configured using the WSGIScriptAlias directive of mod_wsgi).

There is still an issue open to report the application ID #335. It was in the original HTTP PR #263 (comment).

[iNikem]

@Oberon00 thanks for pointing this out. This is relevant, but still a separate issue. "Application" vs "Application Server". Resource semantic convention seems to be the perfect place for the latter, but not I am not so sure about the former.

[jkwatson]

I would agree that having some standard way to report app server vendor and version could be very useful in debugging production issues. I would assume this would apply to the ruby/rails ecosystem as well, yes? Knowing which version of rails was being used, for example, seems like a valuable piece of telemetry.

[ivomagi]

Apparently the way to achieve the desired outcome is to set the information in InstrumentationLibrary.name and InstrumentationLibrary.version attributes, from where the collector will pick it up in case the protocol used is OTLP which currently solves the issue at hand.

[Oberon00]

@ivomagi

Apparently the way to achieve the desired outcome is to set the information in InstrumentationLibrary.name and InstrumentationLibrary.version attributes

I think this is a misunderstanding. These are not attributes, and the InstrumentationLibrary should only describe the artifact that does the instrumentation (i.e. creating the spans), not anything that is instrumented. It is mainly a debugging feature.
This is certainly not suitable for the app server.