Skip to content

Fields with certain object poison Ingest Common SetProcessor's copyFrom #65

Closed

Description

The IngestCommon SetProcessor's copyFrom implementation will attempt to perform a deep copy of the value, supporting ONLY following object types:

  • java.util.Map<?, ?> (recursively copies values; leaves keys unmodified)
  • java.util.List<?> (recursively copies values)
  • java.util.Set<?> (recursively copies values)
  • byte[]
  • double[][]
  • double[]
  • null
  • java.lang.String
  • java.lang.Integer
  • java.lang.Long
  • java.lang.Float
  • java.lang.Double
  • java.lang.Boolean
  • java.time.ZonedDateTime
  • java.util.Date

When encountering a value that is not on that list, the pipeline fails with unexpected value type [org.logstash.Timestamp] or similar.

Looking at our own Javafier implementation, the following can escape Javafier#deep and trigger the issue:

  • org.logstash.Timestamp
  • java.math.BigInteger
  • java.math.BigDecimal

Of these, org.logstash.Timestamp can be easily represented by java.time.ZonedDateTime, and can be converted back losslessly when reconstituting the Event.

I believe the others can be solved on the Elasticsearch side.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions