Adding task cancellation timestamp in task API

[sgup432]

Description

Adding task cancellation start time and running time since cancellation details as discussed here - #6953

Related Issues

Partially resolves - #6953

Manual testing

Spun up an opensearch cluster and added deliberate delay in search request via sleep to mock and test this functionality.
Without cancellation

curl -XGET localhost:9200/_tasks?pretty
{
  "nodes" : {
    "2t7D3gtQQoydmYbreSltTw" : {
      "name" : "runTask-0",
      "transport_address" : "127.0.0.1:9300",
      "host" : "127.0.0.1",
      "ip" : "127.0.0.1:9300",
      "roles" : [
        "cluster_manager",
        "data",
        "ingest",
        "remote_cluster_client"
      ],
      "attributes" : {
        "testattr" : "test",
        "shard_indexing_pressure_enabled" : "true"
      },
      "tasks" : {
        "2t7D3gtQQoydmYbreSltTw:90" : {
          "node" : "2t7D3gtQQoydmYbreSltTw",
          "id" : 90,
          "type" : "transport",
          "action" : "cluster:monitor/tasks/lists",
          "start_time_in_millis" : 1683265864323,
          "running_time_in_nanos" : 248904,
          "cancellable" : false,
          "cancelled" : false,
          "headers" : { }
        },
        "2t7D3gtQQoydmYbreSltTw:91" : {
          "node" : "2t7D3gtQQoydmYbreSltTw",
          "id" : 91,
          "type" : "direct",
          "action" : "cluster:monitor/tasks/lists[n]",
          "start_time_in_millis" : 1683265864323,
          "running_time_in_nanos" : 116795,
          "cancellable" : false,
          "cancelled" : false,
          "parent_task_id" : "2t7D3gtQQoydmYbreSltTw:90",
          "headers" : { }
        },
        "2t7D3gtQQoydmYbreSltTw:67" : {
          "node" : "2t7D3gtQQoydmYbreSltTw",
          "id" : 67,
          "type" : "direct",
          "action" : "indices:data/read/search[phase/fetch/id]",
          "start_time_in_millis" : 1683265829337,
          "running_time_in_nanos" : 34986856773,
          "cancellable" : true,
          "cancelled" : false,
          "parent_task_id" : "2t7D3gtQQoydmYbreSltTw:61",
          "headers" : { }
        },
        "2t7D3gtQQoydmYbreSltTw:61" : {
          "node" : "2t7D3gtQQoydmYbreSltTw",
          "id" : 61,
          "type" : "transport",
          "action" : "indices:data/read/search",
          "start_time_in_millis" : 1683265829284,
          "running_time_in_nanos" : 35039394343,
          "cancellable" : true,
          "cancelled" : false,
          "headers" : { }
        }
      }
    }
  }
}

After cancellation parent Search Task

curl -XGET localhost:9200/_tasks?pretty
{
  "nodes" : {
    "2t7D3gtQQoydmYbreSltTw" : {
      "name" : "runTask-0",
      "transport_address" : "127.0.0.1:9300",
      "host" : "127.0.0.1",
      "ip" : "127.0.0.1:9300",
      "roles" : [
        "cluster_manager",
        "data",
        "ingest",
        "remote_cluster_client"
      ],
      "attributes" : {
        "testattr" : "test",
        "shard_indexing_pressure_enabled" : "true"
      },
      "tasks" : {
        "2t7D3gtQQoydmYbreSltTw:164" : {
          "node" : "2t7D3gtQQoydmYbreSltTw",
          "id" : 164,
          "type" : "direct",
          "action" : "indices:data/read/search[phase/fetch/id]",
          "start_time_in_millis" : 1683265979351,
          "running_time_in_nanos" : 24063541772,
          "cancellable" : true,
          "cancelled" : true,
          "parent_task_id" : "2t7D3gtQQoydmYbreSltTw:61",
          "headers" : { },
          "cancelled_at_millis" : 1683265995821
        },
        "2t7D3gtQQoydmYbreSltTw:181" : {
          "node" : "2t7D3gtQQoydmYbreSltTw",
          "id" : 181,
          "type" : "transport",
          "action" : "cluster:monitor/tasks/lists",
          "start_time_in_millis" : 1683266003414,
          "running_time_in_nanos" : 243712,
          "cancellable" : false,
          "cancelled" : false,
          "headers" : { }
        },
        "2t7D3gtQQoydmYbreSltTw:61" : {
          "node" : "2t7D3gtQQoydmYbreSltTw",
          "id" : 61,
          "type" : "transport",
          "action" : "indices:data/read/search",
          "start_time_in_millis" : 1683265829284,
          "running_time_in_nanos" : 174130192988,
          "cancellable" : true,
          "cancelled" : true,
          "headers" : { },
          "cancelled_at_millis" : 1683265995820
        },
        "2t7D3gtQQoydmYbreSltTw:182" : {
          "node" : "2t7D3gtQQoydmYbreSltTw",
          "id" : 182,
          "type" : "direct",
          "action" : "cluster:monitor/tasks/lists[n]",
          "start_time_in_millis" : 1683266003415,
          "running_time_in_nanos" : 120782,
          "cancellable" : false,
          "cancelled" : false,
          "parent_task_id" : "2t7D3gtQQoydmYbreSltTw:181",
          "headers" : { }
        }
      }
    }
  }
}

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/14938/
• CommitID: 706fbb8
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      2 org.opensearch.remotestore.SegmentReplicationRemoteStoreIT.testReplicaHasDiffFilesThanPrimary

• URL: https://build.ci.opensearch.org/job/gradle-check/15072/
• CommitID: fc49144
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov-commenter]

Codecov Report

Merging #7445 (2d43f9e) into main (b4b3724) will increase coverage by 70.72%.
The diff coverage is 80.00%.

@@             Coverage Diff             @@
##             main    #7445       +/-   ##
===========================================
+ Coverage        0   70.72%   +70.72%     
- Complexity      0    56132    +56132     
===========================================
  Files           0     4680     +4680     
  Lines           0   266104   +266104     
  Branches        0    39070    +39070     
===========================================
+ Hits            0   188192   +188192     
- Misses          0    61938    +61938     
- Partials        0    15974    +15974     

Impacted Files	Coverage Δ
...r/src/main/java/org/opensearch/tasks/TaskInfo.java	89.37% <73.33%> (ø)
...ain/java/org/opensearch/tasks/CancellableTask.java	82.60% <83.33%> (ø)
...erver/src/main/java/org/opensearch/tasks/Task.java	80.90% <100.00%> (ø)

... and 4677 files with indirect coverage changes

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.search.backpressure.SearchBackpressureIT.testSearchTaskCancellationWithHighCpu

• URL: https://build.ci.opensearch.org/job/gradle-check/15154/
• CommitID: 3994b25
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[Bukhtawar]

Just one minor comment. Otherwise LGTM

[Bukhtawar]

Just one minor comment. Otherwise LGTM

[Bukhtawar]

Just one minor comment. Otherwise LGTM

[andrross]

@dblock Can you take a look? I think the thing you requested changes on has been addressed.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/15960/
• CommitID: 4dc0994
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/15959/
• CommitID: 1d1e919
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/15968/
• CommitID: 4dc0994
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/15971/
• CommitID: 4dc0994
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[dblock]

Let's walk back from suggestions and help this PR get in. What are your must haves @Bukhtawar @andrross?

[andrross]

@dblock Personally I'm fine with it as-is. Just waiting on resolving your request changes :) I do think the @Nullable annotation is a good idea now that you called it out. I just pushed a merge commit from main that should fix the gradle check errors.

[Bukhtawar]

No strong opinion from my side as well. There is only a small overhead of the non-primitive Long wrt to the primitive long. That should be fine as long as it aids readability

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.search.backpressure.SearchBackpressureIT.testSearchShardTaskCancellationWithHighCpu

• URL: https://build.ci.opensearch.org/job/gradle-check/16057/
• CommitID: 2d43f9e
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.