Key error when using task_run_id in a target template

[gryBox]

Description

Similar to one of the issues in #2640 where filename throws a key error when used in a template, so does task_run_id.

(py37moc) :~/prefect_guide$ /miniconda3/envs/py37moc/bin/python /home/ilivni/prefect_guide/tst_map.py
0.11.3
[2020-05-27 18:56:17] INFO - prefect.FlowRunner | Beginning Flow run for 'blah'
[2020-05-27 18:56:17] INFO - prefect.FlowRunner | Starting flow run.
[2020-05-27 18:56:17] INFO - prefect.TaskRunner | Task 'return_list': Starting task run...
[2020-05-27 18:56:17] ERROR - prefect.TaskRunner | Unexpected error: KeyError('task_run_id')
Traceback (most recent call last):
  File "/miniconda3/envs/py37moc/lib/python3.7/site-packages/prefect/engine/runner.py", line 48, in inner
    new_state = method(self, state, *args, **kwargs)
  File "/miniconda3/envs/py37moc/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 660, in check_target
    if result.exists(target, **prefect.context):
  File "//miniconda3/envs/py37moc/lib/python3.7/site-packages/prefect/engine/results/local_result.py", line 123, in exists
    return os.path.exists(os.path.join(self.dir, location.format(**kwargs)))
KeyError: 'task_run_id'

Reproduction

import prefect 

print(prefect.__version__)

from prefect import task, Flow
from prefect.engine.results import LocalResult

lcl_res = LocalResult(dir="~/prefect_guide/results/{flow_name}")

@task(target="{task_name}-{task_run_id}",
)
def return_list():
    return [1, 2, 3]
@task(target="{task_name}/{map_index}.prefect")
def mapped_task(x):
    return x + 1
with Flow("blah", result=lcl_res) as flow:
    mapped_task.map(return_list)


st = flow.run()
flow.visualize(flow_state=st)

Environment

Any additional information about your environment

Optionally run prefect diagnostics from the command line and paste the information here

[joshmeek]

Hey @gryBox having a task run ID requires a use of a backend—either core's server or Cloud. Since each task run ID corresponds to a unique entry in the database. So I would expect some behavior like this to happen. I do think we could raise a more informative error though.

[cicdw]

@joshmeek I think there's an opportunity here for us to polish the templating interface; things like task_run_id don't exist in Core-alone, but I can imagine people doing local development with Cloud-specific context vars like this and it'd be nice if we can find a way to support that. Same issue with the {filename} template with mapping.

[gryBox]

Hi - Currently not exposing the {filename} in core results in an error loop if trying to deploy with docker.

1. Test locally (no {filename}) -> Good
2. Deploy with docker storage -> Error need {filename}
3. Add {filename} -> KeyError
4. Rinse lather repeat

[lauralorenz]

@cicdw on the perspective of 'finding a way to support [Cloud-specific context vars in Core only]', do we think making some context that is ephmeral and meaningless (like a random UUID that won't be stored anywhere in the example of task_run_id) is anywhere towards the right direction?

Alternatively we were also talking about catching KeyErrors and raising our own exception that redirects people to the context docs as a way to deal with all cloud-only context.

[cicdw]

making some context that is ephmeral and meaningless (like a random UUID...)

Yea I think that would work! To ensure the ID is somewhat meaningful and constant across the full lifecycle of a flow run we could randomly generate them all for each task somewhere in this region: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/core/flow.py#L904-L919

The only edge case here is for mapped tasks - when running against a backend, we generate a unique task run id for each child (happens here: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/core/flow.py#L904-L919) so we'd want to consider what to do when running locally in this situation.

Alternatively we were also talking about catching KeyErrors and raising our own exception that redirects people to the context docs as a way to deal with all cloud-only context.

Yea, I def think catching KeyErrors and pointing to docs is a great idea; however, I do think it's important to ensure some amount of consistency between local runs -> cloud runs, so as a goal we should try to have context key parity where it makes sense.

[jcrist]

Was this fixed by #2717, or is there still more to do here?

[cicdw]

@jcrist I think there's still an opportunity here to ensure Cloud <-> Core populate the exact same context keys. Here you can find a list of the keys that Cloud populates that Core alone does not: https://docs.prefect.io/api/latest/utilities/context.html. I'm 90% that most of these can be handled with auto-generated UUIDs somewhere around here: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/core/flow.py#L903