ML Pipelines DatabricksStep doesn't support Run.get_context()

[RothNRK]

While using a DatabricksStep I want to get the appropriate Run context so that I can log information to Azure ML.

I thought this might work:

1. Authenticate with AzureMLTokenAuthentication
2. Get an authenticated Workspace -> Experiment -> Run
3. Use the Run.get_context()

Although I can't use the AzureMLTokenAuthentication to authenticate the Workspace.

Can you please provide more information on Azure's intention on how AzureMLTokenAuthentication should even be used?

The only way I've been able to get a run_context from within Databricks during a DatabricksStep is:

os.environ['AZUREML_RUN_TOKEN'] = AZUREML_RUN_TOKEN
os.environ['AZUREML_RUN_TOKEN_EXPIRY'] = AZUREML_RUN_TOKEN_EXPIRY
os.environ['AZUREML_RUN_ID'] = AZUREML_RUN_ID
os.environ['AZUREML_ARM_SUBSCRIPTION'] = AZUREML_ARM_SUBSCRIPTION
os.environ['AZUREML_ARM_RESOURCEGROUP'] = AZUREML_ARM_RESOURCEGROUP
os.environ['AZUREML_ARM_WORKSPACE_NAME'] = AZUREML_ARM_WORKSPACE_NAME
os.environ['AZUREML_ARM_PROJECT_NAME'] = AZUREML_ARM_PROJECT_NAME
os.environ['AZUREML_SERVICE_ENDPOINT'] = AZUREML_SERVICE_ENDPOINT

run = Run.get_context(allow_offline=False)

Which feels like a hack. How does Azure suggest that someone does this.

Any help is appreciated.

Noel

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 4f99a518-b801-fb1c-4f6a-4a520be5aecc
• Version Independent ID: abc91e79-e6ef-801d-1883-4e0fba418317
• Content: azureml.core.authentication.AzureMLTokenAuthentication class - Azure Machine Learning Python
• Content Source: AzureML-Docset/stable/docs-ref-autogen/azureml-core/azureml.core.authentication.AzureMLTokenAuthentication.yml
• Service: machine-learning
• Sub-service: core
• GitHub Login: @DebFro
• Microsoft Alias: jmartens

[RohitMungi-MSFT]

@RothNRK The sample notebook for authentication steps that can be used are available here. CLI or service principal authentication can be used to get your workspace details

@aashishb Could you please advise?

[RothNRK]

@RohitMungi-MSFT Thank you for your response.

CLI Auth doesn't look like it meets my needs.

Using a Service Principal would work but it requires:

1. A Service Principal.
2. A key-vault mounted as a Secret Scope in Databricks.
   This would be good if it were my only option but Azure ML is kind enough to send info to make a secure connection and log to Azure ML using the method I outlined above (which is a bit of a hack but certainly less effort than the Service Principal method).

How do we meaningfully use AzureMLTokenAuthentication (The notebook you linked doesn't include an example with AzureMLTokenAuthentication).

Thank you

[rastala]

@RothNRK The AzureMLTokenAuthentication is intended as Azure ML internal API.

Run.get_context() should simply work when you're within a context of Azure ML run, without any extra authentication required.

[RothNRK]

@rastala Thank you. That is what I expected but while in a DatabricksStep it doesn't seem to simply work.

Run.get_context(allow_offline=False) produces:

KeyError                                  Traceback (most recent call last)
/databricks/python/lib/python3.7/site-packages/azureml/core/run.py in _load_scope(cls)
    215             # Load authentication scope environment variables
--> 216             subscription_id = os.environ['AZUREML_ARM_SUBSCRIPTION']
    217             run_id = os.environ["AZUREML_RUN_ID"]

/local_disk0/pythonVirtualEnvDirs/virtualEnv-6c0c6c97-5068-4f34-91f1-ded12db18057/lib/python3.7/os.py in __getitem__(self, key)
    677             # raise KeyError with the original key value
--> 678             raise KeyError(key) from None
    679         return self.decodevalue(value)

KeyError: 'AZUREML_ARM_SUBSCRIPTION'

The above exception was the direct cause of the following exception:

RunEnvironmentException                   Traceback (most recent call last)
/databricks/python/lib/python3.7/site-packages/azureml/core/run.py in get_context(cls, allow_offline, used_for_context_manager, **kwargs)
    291         try:
--> 292             experiment, run_id = cls._load_scope()
    293 

/databricks/python/lib/python3.7/site-packages/azureml/core/run.py in _load_scope(cls)
    232         except KeyError as key_error:
--> 233             raise_from(RunEnvironmentException(), key_error)
    234         else:

/databricks/python/lib/python3.7/site-packages/six.py in raise_from(value, from_value)

RunEnvironmentException: RunEnvironmentException:
	Message: Could not load a submitted run, if outside of an execution context, use experiment.start_logging to initialize an azureml.core.Run.
	InnerException None
	ErrorResponse 
{
    "error": {
        "message": "Could not load a submitted run, if outside of an execution context, use experiment.start_logging to initialize an azureml.core.Run."
    }
}

During handling of the above exception, another exception occurred:

RunEnvironmentException                   Traceback (most recent call last)
<command--1> in <module>
     13 
     14 with open(filename, "rb") as f:
---> 15   exec(f.read())
     16 

<string> in <module>

<string> in main()

<string> in _main(args)

<string> in _get_run_context(args)

<string> in _get_run()

/databricks/python/lib/python3.7/site-packages/azureml/core/run.py in �[0;36mget_context(cls, allow_offline, used_for_context_manager, **kwargs)
    303             else:
    304                 module_logger.debug("Could not load the run context and allow_offline set to False")
--> 305                 raise RunEnvironmentException(inner_exception=ex)
    306 
    307     @classmethod

RunEnvironmentException: RunEnvironmentException:
	Message: Could not load a submitted run, if outside of an execution context, use experiment.start_logging to initialize an azureml.core.Run.
	InnerException RunEnvironmentException:
	Message: Could not load a submitted run, if outside of an execution context, use experiment.start_logging to initialize an azureml.core.Run.
	InnerException None
	ErrorResponse 
{
    "error": {
        "message": "Could not load a submitted run, if outside of an execution context, use experiment.start_logging to initialize an azureml.core.Run."
    }
}
	ErrorResponse 
{
    "error": {
        "message": "Could not load a submitted run, if outside of an execution context, use experiment.start_logging to initialize an azureml.core.Run."
    }
}

So it looks like it's looking for environment variables that aren't set. If I run it with Run.get_context() I get an error related to the _OfflineRun not working (which makes sense since I need a Run not an _OfflineRun).

[swathi-intelligent]

I am facing the same issue as above, mentioned by @RothNRK . Further I need parent pipeline run ID from the Run.
If I run it with Run.get_context() I am getting an error with the _OfflineRun. Could you please point to an example of the same?

Thanks!

[RothNRK]

Just a friendly reminder that this ticket still exists.

[aashishb]

@rastala , please check and create ICM for pipelines team if needed. Thanks!

[brunocous]

I'm facing the exact same problem as the original poster of this issue. Still couldn't get it to work.