http_endpoint: 7.14.0 breaks backwards compatibility by encoding numbers as strings

[adriansr]

For confirmed bugs, please report:

• Version: 7.14.0
• Operating System: -
• Discuss Forum URL: -
• Steps to Reproduce: -

Run filebeat with the following conf:

# filebeat.yml snippet
filebeat.inputs:
- type: http_endpoint
  listen_address: localhost
  listen_port: 9999

$ ./filebeat run -e -N -d '*'

Ingest a document containing a number:

$ curl -XPOST 'http://localhost:9999/' -H 'Content-Type: application/json' --data '{"a":42}'

The following document is produced:

2021-08-14T11:20:13.886+0200	DEBUG	[processors]	processing/processors.go:203	Publish event: {
  "@timestamp": "2021-08-14T09:20:13.886Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "8.0.0"
  },
  "json": {
    "a": "42"
  },
  "input": {
    "type": "http_endpoint"
  },
  [...]
}

This can break existing ingest pipelines that are expecting numbers instead of strings.

This is caused by #26279 changing the input to use Decoder.UseNumber flag.

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[andrewkroh]

I guess it's missing a call to

beats/libbeat/common/jsontransform/transform.go

Line 29 in 23e4403

func TransformNumbers(dict common.MapStr) {

.