Date string format within aggregation (NEST)

[fredeil]

NEST/Elasticsearch.Net version:
7.5.1
Elasticsearch version:
7.5

Description of the problem including expected versus actual behavior:

Aggregation doing min/max on DateTime property fails when using a format.

Example:

"aggregations" : {
          "first_seen" : {
            "min": {
              "field": "custom.time",
                          "format" : "strict_date_optional_time"
            }
          },
          "last_seen" : {
            "max" : {
              "field" : "custom.time",
              "format" : "strict_date_optional_time"
            }
          }

This works fine from the dev tool inside kibana, but when porting to C#/Nest this error is returned:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "x_content_parse_exception",
        "reason" : "[1:62] [avg] field doesn't support values of type: START_OBJECT",
        "stack_trace" : "[[1:62] [avg] field doesn't support values of type: START_OBJECT]; nested: XContentParseException[[1:62] [avg] field doesn't support values of type: START_OBJECT]"
      }
    ],
    "type" : "x_content_parse_exception",
    "reason" : "[1:62] [avg] field doesn't support values of type: START_OBJECT",
    "stack_trace" : "org.elasticsearch.common.xcontent.XContentParseException: [1:62] [avg] field doesn't support values of type: START_OBJECT "
  },
  "status" : 400
}

I'm not sure why its even complaining that [avg] field doesn't support the formatting since i'm using min and max.

Here is the part of the C# code that fails..

      .Aggregations(a => a
           .Composite("c2_requests", date => date
               .Size(100)
                .Sources(s => s
                  .Terms("ip", t => t
                    .Field(f => f.Custom.Ipaddr)))
                    .Aggregations(a => a
                       .Min("first_seen", m => m
                         .Field(new Field("custom.time", format: "strict_date_optional_time")))
                       .Max("last_seen", m => m
                         .Field(new Field("custom.time", format: "strict_date_optional_time")))))

Works fine if I remove the "format" parameter

[andrx]

Could you also provide rendered request? the value in response.ApiCall.DebugInformation

[russcam]

The issue is that Min and Max aggregation in the client do not currently expose a Format property to be able to pass a property. The Format property of the Field type cannot be used in this manner for aggregations, as it ends up serializing to

{
  "field": { 
    "field": "custom.time", 
    "format": "strict_date_optional_time"
  }
}

when the min and max aggregation expect a format as:

{
  "field": "custom.time", 
  "format": "strict_date_optional_time"
}

This is a consequence of modelling all the different representations of a field with one type, Field.

I'll look at adding a Format property to the aggregations for the next release.

[fredeil]

Thank you @russcam. That would be very useful.

[russcam]

I've opened #4331 to address