[out_es] Id_Key + Write_Operation produces unparseable bulk body ("no requests added") on 4.2.4

[vast0906]

When the ES output is configured with Id_Key + Write_Operation index (or create/update/upsert), the bulk request fails — Elasticsearch returns HTTP 400 action_request_validation_exception: Validation Failed: 1: no requests added;.
Removing Id_Key and Write_Operation (letting ES auto-generate _id) makes the same record stream succeed.

A stdout output configured on the same Match tag with the same upstream record prints clean, valid JSON — no NUL byte, no control characters, all keys/values present. So the upstream record is well-formed; something in the ES output's
bulk-body construction path (only exercised when Id_Key / Write_Operation are set, i.e. when the plugin must emit a per-doc action line such as {"index":{"_id":"..."}} before each source doc) is producing a body that ES cannot parse into any
actionable request — hence no requests added.

The records are produced by a [FILTER] Name lua that builds a flat string-only object (timestamp, level, traceid, message, app/tenant ids, an es_index_prefix, a synthetic logid, etc.). All values are sanitized to drop control characters before
being returned. So the malformed bulk body is being assembled by the ES output, not by the source.

To Reproduce

• Rubular link if applicable: n/a
• Example log message if applicable (one record as printed by the upstream stdout output — clean JSON, no control chars; same record causes the ES output to fail with the bulk error below):

  {"date":1779995840.014,"namespace":"ns-xxxx-aaaaa","level":"INFO","classname":"com.example.aaa.bbb.service.SomeService","podname":"aaa-dev-bbbbb-65dd796558-xxxxx","logicname":"logic-xxx","message":"lll\n\n","env":"dev","type":"logic","logid":"
  ns-xxxx-aaaaa_aaa-dev-bbbbb-65dd796558-xxxxx_2026-05-28T19:17:20.014_logic_635","es_index_prefix":"applog_aaaaa_00000000-0000-0000-0000-000000000000_dev","appId":"00000000-0000-0000-0000-000000000000","tenantname":"aaaaa","thread":"NXXXXXXXX_W
  orker-17","linenumber":"21","time":"2026-05-28T19:17:20.014"}

Bulk response returned by Elasticsearch (verbatim):

[2026/05/28 11:27:33.972634049] [ warn] [engine] failed to flush chunk '1-1779967643.971663269.flb', retry in 14 seconds: task_id=0, input=tail.0 > output=es.0 (out_id=0)
[2026/05/28 11:27:36.972668338] [error] [output:es:es.0] HTTP status=400 URI=/_bulk, response:
{"error":{"root_cause":[{"type":"action_request_validation_exception","reason":"Validation Failed: 1: no requests added;"}],"type":"action_request_validation_exception","reason":"Validation Failed: 1: no requests added;"},"status":400}

• Steps to reproduce the problem:

  a. Run fluent-bit 4.2.4 against an Elasticsearch 6.8.x cluster (also reproduced against ES 7).
  b. Use a [FILTER] lua to emit records with multiple short string fields, one of which is the Id_Key source (e.g. logid) and another is used as Logstash_Prefix_Key (e.g. es_index_prefix).
  c. Configure ES output as below — both outputs match the same tag:

  [OUTPUT]
      Name              es
      Match_Regex       ^my(apps|slog)\..*$
      Host              <es-host>
      Port              9200
      HTTP_User         <user>
      HTTP_Passwd       <pass>
      Suppress_Type_Name On
      Logstash_Format   On
      Logstash_Prefix_Key es_index_prefix
      Logstash_Prefix_Separator _
      Logstash_DateFormat %Y-%m-%d
      Id_Key            logid
      Write_Operation   index
      Retry_Limit       False

  [OUTPUT]
      Name              stdout
      Match             myapps.*
      Format            json_lines

d. Observe: stdout prints clean JSON (see record above); the es output keeps failing with the action_request_validation_exception shown above.
e. Remove Id_Key and Write_Operation (keep everything else identical). Bulk requests now succeed and _id is auto-generated by ES.

Expected behavior

Id_Key + Write_Operation should produce a bulk body whose action lines ({"index":{"_id":"..."}}) and source-doc lines are strictly newline-separated valid JSON — i.e. ES should be able to parse at least one request out of the body. No body
produced by the plugin should make ES return no requests added, regardless of which action header is prepended.

Screenshots

n/a — error is in the ES bulk response body, attached verbatim above.

Your Environment

• Version used: fluent-bit 4.2.4
• Configuration: ES output as shown above; upstream pipeline is tail → kubernetes filter → lua filter (record sanitizer drops chars \x00-\x08, \x0B, \x0C, \x0E-\x1F, \x7F; preserves \r \n \t) → es output.
• Environment name and version (e.g. Kubernetes? What version?): Kubernetes 1.24+, fluent-bit DaemonSet
• Server type and version: Elasticsearch 7.x
• Operating System and version: Linux (container base: upstream fluent/fluent-bit:4.2.4 image)
• Filters and plugins: tail, kubernetes, lua (in-process record rewriter — emits only string/number scalars, no nested tables, no binary), es output

Additional context

• action_request_validation_exception: Validation Failed: 1: no requests added; from ES means the _bulk body the server received contained no parseable action+source pairs. Combined with the fact that the very same record streams successfully
  when Id_Key / Write_Operation are removed, this strongly points at the bulk-body builder in the ES output — the per-doc action line (or its newline framing) is being emitted in a way ES cannot parse, so the whole batch is dropped without a
  single request being registered.
• We have also seen, in earlier 4.2.x runs on a different record shape, an ES error of the form json_parse_exception ... Illegal character ((CTRL-CHAR, code 0)) (a NUL byte mid-body). It is unclear whether that and the current no requests
  added are two faces of the same serialization issue or two separate bugs — happy to capture the raw bulk body off the wire and attach.
• Workarounds we tried that did not help: aggressive control-char sanitization in the lua filter (NUL/CTRL stripped, empty strings nilified to avoid empty-string edge cases), disabling Suppress_Type_Name, switching Write_Operation between
  index / create.
• Workaround that does work: drop Id_Key and Write_Operation entirely and let ES auto-generate _id. The cost is losing fluent-bit's own restart-time dedup guarantee — same offset replayed after a DaemonSet restart will produce duplicate docs
  in ES.
• Would appreciate guidance on:
  a. whether this is a known regression in 4.2.x ES output bulk serialization, and
  b. whether there is a way to keep Id_Key-based dedup without triggering this code path.

Happy to provide a packet capture of the offending bulk body and the corresponding stdout JSON side-by-side if it helps