Create ScheduledJobIdentityManager interface in IdentityPlugin

[cwperks]

Description

Companion PR in security repo: opensearch-project/security#2773
Companion PR in Job Scheduler: opensearch-project/job-scheduler#394

This PR introduces a new interface inside identity plugin for scheduled job identity management. The new interface is called ScheduledJobIdentityManager and contains 3 methods that Job Scheduler will use for extensions use-cases where scheduled jobs run out of process. The current architecture of plugins that extend job scheduler plugin and manage job details in an index owned by the respective plugin (not owned by job scheduler, but made aware to job scheduler) can be changed using this new model.

While the interface is being created here and can be implemented differently, the implementation inside the security plugin being proposed will keep a list of all registered jobs in a single index owned by the security plugin and owner information will be stored in this index. There would be a 1-to-1 mapping of job details stored across all job details indices of plugins that extend job scheduler and this full list owned by the security plugin with owner information. For referential integrity it is stored with the jobId and jobIndex which is the compound key that links back to the document containing jobs details.

With the user information persisted on a secure index, the implementer of this interface can implement another method issueAccessTokenOnBehalfOfUser to issue an access token to be sent to a job runner to be able to execute a job under a user context outside of the user being present.

Related Issues

Resolves opensearch-project/security#2626

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/15451/
• CommitID: 4dec6f0
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/15454/
• CommitID: f41e07e
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringQueryPhase

• URL: https://build.ci.opensearch.org/job/gradle-check/15457/
• CommitID: 437a9aa
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[stephen-crawford]

What do you mean by User here? Are you talking about the user who's credentials are being used for authz or are you talking about the plugin/extension subject performing the job? Are you parsing the user from the threadContext or is it supposed to be associated with the job in some way?

[cwperks]

This would be left up to an IdentityPlugin to implement, but the idea with the security plugin is that this user is the creator of the scheduled job. For the security plugin that does mean getting the authenticated user from the threadcontext and persisting it in an index that would contain all scheduled jobs and the user (owner) associated with them.

See opensearch-project/security#2773 for an implementation in the Security plugin.

[cwperks]

@scrawfor99 This PR has been updated to include a ScheduledJobIdentity which is generic and contains a username and a map of attributes.

[stephen-crawford]

Do you think this is required or do you think we can just wipe the details after job resolution?

[cwperks]

This would be if a user deletes a scheduled job altogether. If the original job details is purged from the index that stores the job details then this hook provides a way for an IdentityPlugin to hook into that and react accordingly - for the Security plugin that would mean deleting the entry from the index containing all scheduled jobs and associated user.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/18471/
• CommitID: 19ed9ba
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/18940/
• CommitID: 50b0bc0
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: null ❌
• URL: null
• CommitID: a31425c
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.snapshots.DedicatedClusterSnapshotRestoreIT.testIndexDeletionDuringSnapshotCreationInQueue
      1 org.opensearch.remotestore.RemoteStoreIT.testStaleCommitDeletionWithoutInvokeFlush
      1 org.opensearch.cluster.allocation.AwarenessAllocationIT.testThreeZoneOneReplicaWithForceZoneValueAndLoadAwareness

• URL: https://build.ci.opensearch.org/job/gradle-check/19276/
• CommitID: a4bf43e
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.remotestore.RemoteStoreIT.testStaleCommitDeletionWithInvokeFlush

• URL: https://build.ci.opensearch.org/job/gradle-check/19277/
• CommitID: 78d22eb
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/19639/
• CommitID: b42bcf7

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.search.SearchWeightedRoutingIT.testSearchAggregationWithNetworkDisruption_FailOpenEnabled

• URL: https://build.ci.opensearch.org/job/gradle-check/19640/
• CommitID: 03f638f
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/20061/
• CommitID: 3b8c09a

[cwperks]

Closing for now. Will re-open at a later date with updates.