Index Lifecycle Policy max_size results in 'unit is missing' error

[octocat-mona]

NEST/Elasticsearch.Net version:
6.7.0

Elasticsearch version:
6.7.1

Description of the problem including expected versus actual behavior:
Creating an ILM policy with a rollover max_size field gives an unit is missing or unrecognized error.

Compared to the RolloverIndexAsync there are a few differences:

1. MaxSize accepts a string parameter (which includes a unit) instead of a long?
2. MaximumDocuments and MaximumSize naming is not consistent with MaxSize and MaxDocs

Of course the first point is the main issue here, a long seems not accepted at all by ES?

Steps to reproduce:

1. Create policy:

await _client.PutLifecycleAsync("my-policy", descriptor => descriptor
    .Policy(policy => policy
        .Phases(phases => phases
            .Hot(hot => hot
                .Actions(actions => actions
                    .Rollover(rollover => rollover
                        .MaximumDocuments(1_000_000)
                        .MaximumSize(40_000_000_000)// Note the size type is 'long?'
                    )
                )
            )
        )
    )
);

// Compared to Rollover:
/*await _client.RolloverIndexAsync("my-alias", d => d
    .Conditions(c => c
        .MaxDocs(1_000_000)
        .MaxSize("40gb")// Note the size type is 'string'
    )
);*/

Which results in http call:

PUT _ilm/policy/my-policy
{
	"policy": {
		"phases": {
			"hot": {
				"actions": {
					"rollover": {
						"max_docs": 1000000,
						"max_size": 40000000000
					}
				}
			}
		}
	}
}

This will return a 400 error:

{
	"error": {
		"root_cause": [{
				"type": "parse_exception",
				"reason": "failed to parse setting [max_size] with value [40000000000] as a size in bytes: unit is missing or unrecognized"
			}
		],
		"type": "x_content_parse_exception",
		"reason": "[8:19] [put_lifecycle_request] failed to parse field [policy]",
		"caused_by": {
			"type": "x_content_parse_exception",
			"reason": "[8:19] [lifecycle_policy] failed to parse field [phases]",
			"caused_by": {
				"type": "x_content_parse_exception",
				"reason": "[8:19] [phases] failed to parse field [hot]",
				"caused_by": {
					"type": "x_content_parse_exception",
					"reason": "[8:19] [phase] failed to parse field [actions]",
					"caused_by": {
						"type": "x_content_parse_exception",
						"reason": "[8:19] [actions] failed to parse field [rollover]",
						"caused_by": {
							"type": "x_content_parse_exception",
							"reason": "[8:19] [rollover] failed to parse field [max_size]",
							"caused_by": {
								"type": "parse_exception",
								"reason": "failed to parse setting [max_size] with value [40000000000] as a size in bytes: unit is missing or unrecognized"
							}
						}
					}
				}
			}
		}
	},
	"status": 400
}

[codebrain]

Thanks for raising this issue @KPStolk.

You are indeed correct on both counts - this is a bug in 6.7 - I have opened a PR to address this.

For 7.x we will ensure both properties use the full name prefix of Maximum instead of Max. RolloverIndex* is an older method in the client and needs to be renamed.

[octocat-mona]

@codebrain Thanks for confirming the bug, will use the LowLevel client for now until the fix is released!

[codebrain]

@KPStolk - 6.8.0 was just published to nuget, it should be available soon.

You will have the opportunity to use either the MaximumSize and set the value in bytes, or MaximumSizeAsString and then use the notation shortcut (e.g. "40gb").

Please let me know if I can close this ticket :)

[octocat-mona]

@codebrain Thanks, it works fine now!

Any reason you created a new method MaximumSizeAsString instead of an overload for MaximumSize?

[codebrain]

It was to maintain binary compatibility - we had a debate about what to do in this instance, as technically we are fixing a bug, but to add AsString as the suffix is a common pattern already in the codebase.

Happy that this has fixed the issue for you, I will close this ticket now.

Thanks!