Rename "after" field

[dakrone]

This commit renames the "after" field to "index_age". This makes it more
explicit that the phase is dependent on the age of the index, not on the
completion time of the previous phase.

Relates to #29823

[elasticmachine]

Pinging @elastic/es-core-infra

[talevy]

thanks for this! I think index_age is an improvement over after. I think index_age's downfall of not being able to describe our adjustment for rolled-over data timing was shared with after.

+1 to this change and making sure we overcome the ambiguity in documentation

[colings86]

I'm not sure about this change because of the fact that once we've rolled over he age is based on the data age (more specifically the timestamp we rolled over on) rather than the time since index creation so calling it index_age could leads to an equal amount of confusion as we had before. I would like to get input form @eskibars at least here. This is a low priority change anyway so I don't think its going to hurt at all for it to wait a few days before we merge to get this input.

[dakrone]

Yeah this is no hurry other than the conflicts that pile up as we add stuff, we can wait and see what @eskibars thinks

[eskibars]

I think after is really difficult for a novice user to reason about. I think most people would assume after means "how much after the last stage ends is this one?" index_age is a lot closer to what the age actually is and a lot less dangerous if you consider data retention the safer outcome (which it often is). That is, I could imagine someone setting warm after to 7 days, cold after to 14 days, and delete after to 30 days and being very upset with the results.

rollover_age seems even more explicit, though probably even more confusing if you're not using rollover. It strikes me that the naming difficulty is really a function of the convention being non-deterministic at the time of index creation. That is, the time before transition is not just a function of age of data, but when you happened to have some other attribute of the index triggered (e.g. size of the last block or number of documents or time since the last rollover). I think if we ever want a completely reasonable (and consistent) name, we'll end up having to enforce some more consistency on the policy like "must have rollover."

For now, I'd recommend something like index_or_rollover_age. It's longer to type out but helps reflects the ambiguity and would force a user to look up the meaning if they didn't understand how policies were executed. What do you think?

[dakrone]

@eskibars my concern with index_or_rollover_age is that we're hijacking the "rollover" term for a feature we already have. Also, I think it's really long :)

[eskibars]

I concede it's long, but it seems there's likely a tradeoff between ambiguity and length in this case given how it works. I'm not sure what you mean by it hijacking the "rollover" term though.

[dakrone]

I'm not sure what you mean by it hijacking the "rollover" term though.

Since we use "rollover" as an action, I'm concerned we're hijacking that age (rollover every 7 days for instance) to be different, consider:

hot: {
  index_or_rollover_age: 24h,
  actions: {
   rollover: {
     max_age: 12h
   }        
  }
}

We have "rollover" twice there, once associated with 12 hours and once with 24 hours, I'm concerned that it's confusing to conflate the index age for a phase with an action that occurs in a phase

[dakrone]

Alternatively, we could call it just "age" if we wanted

[eskibars]

age has the same name collision as index_or_rollover_age, just a different part of the rollover API.

How about minimum_age?

[dakrone]

How about minimum_age?

I like it more than after and less than index_age :)

[eskibars]

I like it more than index_age, since it sets no expectations of index age vs time of rollover. And I like it better than "after," since the "age" part implies it relates to absolute value rather than an incremental value from the previous step. It's less explicit than "index_age" about what the age of the thing is, but that's intentional since we actually don't know whether we're telling the user index age vs rollover date at that point.

Is there more info on why you like it less than index_age?

[dakrone]

Is there more info on why you like it less than index_age?

I like the description-subject relationship more and it's probably more of a personal preference thing. I'm not -1 on minimum_age, it's better than "after".

[dakrone]

After discussion we've decided to go with minimum_age, I'll open a new PR for this as this one is a bit stale now.