[FEATURE] Add a new ActionFilter to support notification when long running operations complete #703
Description
Is your feature request related to a problem?
support sending notification when long running index operation completed.
opensearch-project/index-management-dashboards-plugin#615
What solution would you like?
Add a new ActionFilter to filter all requests at transport layer, and for reindex, resize, open, force_merge, wrap ActionListener to add notification capability. Once the ActionListener callback is called, that means operation is completed. However, for resize, open we need to double check for all shards are started, if not, register a ActiveShardsObserver to continue monitoring shard status until all primary shards are started.
What alternatives have you considered?
Add a IndexOperationListener to watching on .tasks index. If there has task result write into this index, that means a task just run complete, we can parse the document and send out notification accordingly. This will only works on long running operation run in async way.
Do you have any additional context?
Add any other context or screenshots about the feature request here.