Telemetry processors bug

[ivan-zaitsev]

Expected behavior

Include filter with non-string value should be correctly applied.

Actual behavior

Currently only string filter fields are supported.

To Reproduce

Create such applicationinsights.json file:

{
    "preview": {
        "processors": [
            {
                "type": "attribute",
                "include": {
                    "matchType": "regexp",
                    "attributes": [
                        {
                            "key": "http.status_code",
                            "value": 400
                        }
                    ]
                },
                "actions": [
                    {
                        "action": "insert",
                        "key": "key",
                        "value": "value"
                    }
                ]
            }
        ]
    }
}

As "http.status_code" is not a string key (AttributeKey.stringKey) but rather long (AttributeKey.longKey), it will not be resolved, and this code will return null instead of actual value.

ApplicationInsights-Java/agent/agent-tooling/src/main/java/com/microsoft/applicationinsights/agent/internal/processors/AgentProcessor.java

Line 88 in 547ec61

Object existingAttributeValue = attributes.get(AttributeKey.stringKey(attribute.key));

Same for regexp:

ApplicationInsights-Java/agent/agent-tooling/src/main/java/com/microsoft/applicationinsights/agent/internal/processors/AgentProcessor.java

Line 126 in 547ec61

AttributeKey.stringKey(attribute.key), Pattern.compile(attribute.value));

ApplicationInsights-Java/agent/agent-tooling/src/main/java/com/microsoft/applicationinsights/agent/internal/processors/AgentProcessor.java

Line 179 in 547ec61

Object existingAttributeValue = attributes.get(attributeEntry.getKey());

[heyams]

@ivan-zaitsev I tested it using non-String type value, it works for me though.

"preview": {
  "processors": [
    {
      "type": "attribute",
      "actions": [
        {
          "key": "attribute1",
          "value": 1,
          "action": "insert"
        },
        {
          "key": "attribute2",
          "value": "testValue2",
          "action": "insert"
        },
        {
          "key": "sensitiveAttribute1",
          "value": "sensitiveData1",
          "action": "insert"
        }
      ]
    }
  ]
}

this is the payload sent to Application Insights:

{
  "ver": 1,
  "name": "Message",
  "time": "2023-11-15T20:25:29.382Z",
  "iKey": "00000000-0000-0000-0000-0FEEDDADBEEF",
  "tags": {
    "ai.internal.sdkVersion": "java:3.4.19-SNAPSHOT",
    "ai.cloud.roleInstance": "testroleinstance",
    "ai.cloud.role": "testrolename"
  },
  "data": {
    "baseType": "MessageData",
    "baseData": {
      "ver": 2,
      "message": "1::testValue2",
      "severityLevel": "Information",
      "properties": {
        "LoggerName": "org.apache.catalina.startup.VersionLoggerListener",
        "SourceType": "Logger",
        "attribute1": "1",
        "ThreadName": "main",
        "attribute2": "testValue2",
        "sensitiveAttribute1": "redacted"
      }
    }
  }
}

[heyams]

Please try to use the latest version of Java Version. Alternatively, you can submit a repro app at https://github.com/microsoft/ApplicationInsights-Java-Repros.

[ivan-zaitsev]

@heyams

I am talking not about actions, but about "include" filters.

                "include": {
                    "matchType": "regexp",
                    "attributes": [
                        {
                            "key": "http.status_code",
                            "value": 400
                        }
                    ]
                },

When a span contains non-string values the include filtering on that attribute will not work because the AgentProcessor will try to work with that as a string attrubute.

ApplicationInsights-Java/agent/agent-tooling/src/main/java/com/microsoft/applicationinsights/agent/internal/processors/AgentProcessor.java

Line 88 in 547ec61

Object existingAttributeValue = attributes.get(AttributeKey.stringKey(attribute.key));

[ivan-zaitsev]

@heyams

Example:

Suppose that there is a span with such attributes:

{
  "data": {
    "baseType": "MessageData",
    "baseData": {
      "ver": 2,
      "message": "1::testValue2",
      "severityLevel": "Information",
      "properties": {
        "LoggerName": "org.apache.catalina.startup.VersionLoggerListener",
        "SourceType": "Logger",
        "attributeInteger": 1,
        "attributeBool": false,
      }
    }
  }
}

You want to insert a new attribute only if an existing attribute with key "attributeInteger" and value 1 exist.
For this you will use such config:

{
    "preview": {
        "processors": [
            {
                "type": "attribute",
                "include": {
                    "matchType": "strict",
                    "attributes": [
                        {
                            "key": "attributeInteger",
                            "value": 1
                        }
                    ]
                },
                "actions": [
                    {
                        "action": "insert",
                        "key": "newAttribute",
                        "value": "newAttributeValue"
                    }
                ]
            }
        ]
    }
}

But it will not work, because "attributeInteger" will be retrieved as a AttributeKey.stringKey in AgentProcessor instead of AttributeKey.longKey which will result to null value and in the end to false condition.

[heyams]

@ivan-zaitsev i tested include and it seems to be working. Please submit a repro app to help me further investigate.

"preview": {
  "processors": [
    {
      "type": "attribute",
      "actions": [
        {
          "key": "attribute1",
          "value": 1,
          "action": "insert"
        }
      ],
      "id": "attributes/insert"
    },
    {
      "type": "attribute",
      "include": {
        "matchType": "strict",
        "attributes": [
          {
            "key": "attribute1",
            "value": 1
          }
        ]
      },
      "actions": [
        {
          "key": "myNewAttributeKey",
          "value": "myNewAttributeValue",
          "action": "insert"
        }
      ],
      "id": "attributes/myNewAttributeValue"
    }
  ]
}

log sent to application insights:

{
  "ver": 1,
  "name": "Message",
  "time": "2023-11-15T21:17:59.755Z",
  "iKey": "00000000-0000-0000-0000-0FEEDDADBEEF",
  "tags": {
    "ai.internal.sdkVersion": "java:3.4.19-SNAPSHOT",
    "ai.operation.id": "ea8260e295ca98e19912078d290931dc",
    "ai.cloud.roleInstance": "testroleinstance",
    "ai.operation.name": "GET /TelemetryProcessors/test",
    "ai.cloud.role": "testrolename",
  },
  "data": {
    "baseType": "MessageData",
    "baseData": {
      "ver": 2,
      "message": "1::testValue2",
      "severityLevel": "Information",
      "properties": {
        "LoggerName": "smoketestappcontroller",
        "SourceType": "Logger",
        **"attribute1": "1",**
        "ThreadName": "http-nio-8080-exec-2",
        "attribute2": "testValue2",
        **"myNewAttributeKey": "myNewAttributeValue"**
      }
    }
  }
}

[ivan-zaitsev]

@heyams

In your example attribute attribute1 contains value "1" as a String, but if you change it to Integer 1 it will not work.

[heyams]

@ivan-zaitsev yes, i will give you a snapshot to try. I think I understand your issue now.

[ivan-zaitsev]

@heyams

I only mentioned Integer case, but actually the value can be any type (boolean, double, long, string, booleanArray, doubleArray, longArray, stringArray)
https://github.com/open-telemetry/opentelemetry-java/blob/release/v1.32.x/api/all/src/main/java/io/opentelemetry/api/common/AttributeKey.java