[Jobs] Add job submission ID to `get_runtime_context()`

[architkulkarni]

Description

Add an API like the existing API ray.runtime_context.get_runtime_context().get_job_id(), but for the job submission ID.

Use case

The use case is for job driver code to read its job submission ID at runtime. e.g. to save job related artifacts to a remote registry with a separate folder for each job submission id and allow easy artifact retrieval just by the id.

[GuillaumeDesforges]

From Huaiwei Sun on Slack

To clarify, there are two job-related ids:

• job id. Example: 02000000. This is available for all the ray jobs (whether you run it via ray client or job api).
• job submission id. Example: raysubmit_7VLKtzTp2ZUVsmAc. This is available only for jobs submitted via job API for now

[scottsun94]

@architkulkarni To avoid potential confusion,
We probably should haveray.runtime_context.get_runtime_context().get_job_submission_id() for submission id and ray.runtime_context.get_runtime_context().get_job_id() for job id?

[architkulkarni]

Sure, that API makes sense.

[DhavalRepo18]

@architkulkarni any timeline?

[sfriedowitz]

Is it possible to get the submission_id in any way from the running job?

[scottsun94]

cc: @akshay-anyscale for awareness

[allenwang28]

One simple way to approach this:

import ray
from ray.job_submission import JobSubmissionClient


def get_job_submission_id() -> str:
    c = JobSubmissionClient()
    current_job_id = ray.get_runtime_context().get_job_id()
    jobs = c.list_jobs()
    return [job.submission_id for job in jobs if job.job_id == current_job_id][0]

[anyscalesam]

@allenwang28 do you/can you take point in cutting a PR here?

[nima-rezaeian-ai]

This is something we're interested in as well -- any updates on timeline?

[jcotant1]

@allenwang28 any update on this one?