Skip to content

Commit

Permalink
Merge pull request MicrosoftDocs#76108 from sdgilley/sdg-master
Browse files Browse the repository at this point in the history
workspace concept article
  • Loading branch information
Ja-Dunn authored May 17, 2019
2 parents 3e75cef + ad605fd commit 3275ef8
Show file tree
Hide file tree
Showing 14 changed files with 219 additions and 30 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -31,40 +31,24 @@ The machine learning workflow generally follows this sequence:
1. After a satisfactory run is found, register the persisted model in the **model registry**.
1. Develop a scoring script that uses the model and **Deploy the model** as a **web service** in Azure, or to an **IoT Edge device**.

You perform these steps with any of the following:
+ [Azure Machine Learning SDK for Python](https://docs.microsoft.com/python/api/overview/azure/ml/intro?view=azure-ml-py)
+ [Azure Machine Learning CLI](https://docs.microsoft.com/azure/machine-learning/service/reference-azure-machine-learning-cli)
+ The [visual interface (preview) for Azure Machine Learning service](ui-concept-visual-interface.md)

> [!NOTE]
> Although this article defines terms and concepts used by Azure Machine Learning service, it does not define terms and concepts for the Azure platform. For more information about Azure platform terminology, see the [Microsoft Azure glossary](https://docs.microsoft.com/azure/azure-glossary-cloud-terminology).
## Workspace

The workspace is the top-level resource for Azure Machine Learning service. It provides a centralized place to work with all the artifacts you create when you use Azure Machine Learning service.

The workspace keeps a list of compute targets that you can use to train your model. It also keeps a history of the training runs, including logs, metrics, output, and a snapshot of your scripts. You use this information to determine which training run produces the best model.

You register models with the workspace. You use a registered model and scoring scripts to deploy a model to Azure Container Instances, Azure Kubernetes Service, or to a field-programmable gate array (FPGA) as a REST-based HTTP endpoint. You can also deploy the image to an Azure IoT Edge device as a module. Internally, a docker image is created to host the deployed image. If needed, you can specify your own image.

You can create multiple workspaces, and each workspace can be shared by multiple people. When you share a workspace, you can control access to it by assigning users to the following roles:

* Owner
* Contributor
* Reader

For more information on these roles, see the [Manage access to an Azure Machine Learning workspace](how-to-assign-roles.md) article.

When you create a new workspace, it automatically creates several Azure resources that are used by the workspace:

* [Azure Container Registry](https://azure.microsoft.com/services/container-registry/): Registers docker containers that you use during training and when you deploy a model.
* [Azure storage account](https://azure.microsoft.com/services/storage/): Is used as the default datastore for the workspace.
* [Azure Application Insights](https://azure.microsoft.com/services/application-insights/): Stores monitoring information about your models.
* [Azure Key Vault](https://azure.microsoft.com/services/key-vault/): Stores secrets that are used by compute targets and other sensitive information that's needed by the workspace.

> [!NOTE]
> In addition to creating new versions, you can also use existing Azure services.
[The workspace](concept-workspace.md) is the top-level resource for Azure Machine Learning service. It provides a centralized place to work with all the artifacts you create when you use Azure Machine Learning service.

A taxonomy of the workspace is illustrated in the following diagram:

[![Workspace taxonomy](./media/concept-azure-machine-learning-architecture/azure-machine-learning-taxonomy.png)](./media/concept-azure-machine-learning-architecture/azure-machine-learning-taxonomy.png#lightbox)

For more information about workspaces, see [What is an Azure Machine Learning workspace?](concept-workspace.md).

## Experiment

An experiment is a grouping of many runs from a specified script. It always belongs to a workspace. When you submit a run, you provide an experiment name. Information for the run is stored under that experiment. If you submit a run and specify an experiment name that doesn't exist, a new experiment with that newly specified name is automatically created.
Expand Down
99 changes: 99 additions & 0 deletions articles/machine-learning/service/concept-workspace.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
---
title: 'What is a workspace'
titleSuffix: Azure Machine Learning service
description: Learn what a workspace is and why you need one for Azure Machine Learning service.
services: machine-learning
ms.service: machine-learning
ms.subservice: core
ms.topic: conceptual
ms.author: sgilley
author: sdgilley
ms.date: 05/15/2019
# As a data scientist, I want to understand the purpose of a workspace for Azure Machine Learning service.
---


# What is an Azure Machine Learning workspace?

The workspace is the top-level resource for Azure Machine Learning service. It provides a centralized place to work with all the artifacts you create when you use Azure Machine Learning service.

The workspace keeps a history of the training runs, including logs, metrics, output, and a snapshot of your scripts. You use this information to determine which training run produces the best model.

Once you have a model you like, you register it with the workspace. You use the registered model and scoring scripts to deploy to Azure Container Instances, Azure Kubernetes Service, or to a field-programmable gate array (FPGA) as a REST-based HTTP endpoint. You can also deploy the model to an Azure IoT Edge device as a module.

## Taxonomy

A taxonomy of the workspace is illustrated in the following diagram:

[![Workspace taxonomy](./media/concept-azure-machine-learning-architecture/azure-machine-learning-taxonomy.png)](./media/concept-azure-machine-learning-architecture/azure-machine-learning-taxonomy.png#lightbox)

The diagram shows the following components of a workspace:

+ A workspace can contain [Notebook VMs](quickstart-run-cloud-notebook.md), cloud resources configured with the Python environment necessary to run Azure Machine Learning.
+ [User roles](how-to-assign-roles.md) enable you to share your workspace with other users, teams or projects.
+ [Compute targets](concept-azure-machine-learning-architecture.md#compute-target) are used to run your experiments.
+ When you create the workspace, [associated resources](#resources) are also created for you.
+ [Experiments](concept-azure-machine-learning-architecture.md#experiment) are training runs you use to build your models. You can create and run experiments with
+ The [Azure Machine Learning SDK for Python](https://docs.microsoft.com/python/api/overview/azure/ml/intro?view=azure-ml-py).
+ The [automated machine learning experiments (preview)](how-to-create-portal-experiments.md) section in the Azure portal.
+ The [visual interface (preview)](ui-concept-visual-interface.md).
+ [Pipelines](concept-azure-machine-learning-architecture.md#pipeline) are reusable workflows for training and retraining your model.
+ [Datasets](concept-azure-machine-learning-architecture.md#dataset) aid in management of the data you use for model training and pipeline creation.
+ Once you have a model you want to deploy, you create a [registered model](concept-azure-machine-learning-architecture.md#model-registry).
+ Use the registered model and a scoring script to create a [deployment](concept-azure-machine-learning-architecture.md#image-registry).

## Tools for workspace interaction

You can interact with your workspace in the following ways:

+ On the web:
+ The [Azure portal](https://azure.portal.com)
+ The [visual interface (preview)](ui-concept-visual-interface.md)
+ In Python using Azure Machine Learning [SDK](https://docs.microsoft.com/python/api/overview/azure/ml/intro?view=azure-ml-py)
+ On the command line using the Azure Machine Learning [CLI extension](https://docs.microsoft.com/azure/machine-learning/service/reference-azure-machine-learning-cli)

## Machine learning with a workspace

Machine learning tasks read and/or write artifacts to your workspace.

+ Run an experiment to train a model - writes experiment run results to the workspace.
+ Use automated ML to train a model - writes training results to the workspace.
+ Register a model in the workspace.
+ Deploy a model - uses the registered model to create a deployment.
+ Create and run reusable workflows.
+ View machine learning artifacts such as experiments, pipelines, models, deployments.
+ Track and monitor models.

## Workspace management

You can also perform the following workspace management tasks:

| Workspace management task | Portal | SDK | CLI |
|---------------------------|------------------|------------|------------|
| Create a workspace | **✓** | **✓** | **✓** |
| Create and manage compute resources | **✓** | **✓** | **✓** |
| Manage workspace access | **✓** | | **✓** |
| Create a notebook VM | **✓** | | |

Get started with the service by [creating a workspace](setup-create-workspace.md).

## <a name="resources"></a> Associated resources

When you create a new workspace, it automatically creates several Azure resources that are used by the workspace:

+ [Azure Container Registry](https://azure.microsoft.com/services/container-registry/): Registers docker containers that you use during training and when you deploy a model.
+ [Azure Storage account](https://azure.microsoft.com/services/storage/): Is used as the default datastore for the workspace.
+ [Azure Application Insights](https://azure.microsoft.com/services/application-insights/): Stores monitoring information about your models.
+ [Azure Key Vault](https://azure.microsoft.com/services/key-vault/): Stores secrets that are used by compute targets and other sensitive information that's needed by the workspace.

> [!NOTE]
> In addition to creating new versions, you can also use existing Azure services.
## Next steps

To get started with Azure Machine Learning service, see:

+ [Azure Machine Learning service overview](overview-what-is-azure-ml.md)
+ [Create a workspace](setup-create-workspace.md)
+ [Manage a workspace](how-to-manage-workspace.md)
+ [Tutorial: Train a model](tutorial-train-models-with-aml.md)
2 changes: 1 addition & 1 deletion articles/machine-learning/service/how-to-access-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ This how-to shows examples of the following tasks:

## Prerequisites

To use datastores, you first need a [workspace](concept-azure-machine-learning-architecture.md#workspace).
To use datastores, you first need a [workspace](concept-workspace.md).

Start by either [creating a new workspace](setup-create-workspace.md#sdk) or retrieving an existing one:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ ms.custom: seodec18

# Create and manage Azure Machine Learning service workspaces

In this article, you'll create, view, and delete [**Azure Machine Learning service workspaces**](concept-azure-machine-learning-architecture.md#workspace) in the Azure portal for [Azure Machine Learning service](overview-what-is-azure-ml.md). You can also create and delete workspaces [using the CLI](reference-azure-machine-learning-cli.md) or [with Python code](https://aka.ms/aml-sdk).
In this article, you'll create, view, and delete [**Azure Machine Learning service workspaces**](concept-workspace.md) in the Azure portal for [Azure Machine Learning service](overview-what-is-azure-ml.md). You can also create and delete workspaces [using the CLI](reference-azure-machine-learning-cli.md) or [with Python code](https://aka.ms/aml-sdk).

## Create a workspace

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ To facilitate deep learning model training, the Azure Machine Learning Python SD

## Train with an estimator

Once you've created your [workspace](concept-azure-machine-learning-architecture.md#workspace) and set up your [development environment](how-to-configure-environment.md), training a model in Azure Machine Learning involves the following steps:
Once you've created your [workspace](concept-workspace.md) and set up your [development environment](how-to-configure-environment.md), training a model in Azure Machine Learning involves the following steps:
1. Create a [remote compute target](how-to-set-up-training-targets.md) (note you can also use local computer as compute target)
2. Upload your [training data](how-to-access-data.md) to datastore (Optional)
3. Create your [training script](tutorial-train-models-with-aml.md#create-a-training-script)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ The `TensorFlow` estimator also provides a layer of abstraction over execution,

Since the `TensorFlow` estimator class is similar to the base [`Estimator`](https://docs.microsoft.com/python/api/azureml-train-core/azureml.train.estimator.estimator?view=azure-ml-py), we recommend you first read the [base Estimator how-to article](how-to-train-ml-models.md) to understand the overarching concepts.

To get started with Azure Machine Learning service, [complete the quickstart](quickstart-run-cloud-notebook.md). Once finished, you'll have an [Azure Machine Learning workspace](concept-azure-machine-learning-architecture.md#workspace) and all of our [sample notebooks](https://github.com/Azure/MachineLearningNotebooks/tree/master/how-to-use-azureml) including those for training DNNs with TensorFlow and Keras.
To get started with Azure Machine Learning service, [complete the quickstart](quickstart-run-cloud-notebook.md). Once finished, you'll have an [Azure Machine Learning workspace](concept-workspace.md) and all of our [sample notebooks](https://github.com/Azure/MachineLearningNotebooks/tree/master/how-to-use-azureml) including those for training DNNs with TensorFlow and Keras.

## Single-node training

Expand Down
2 changes: 1 addition & 1 deletion articles/machine-learning/service/how-to-vscode-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ To install the Azure Machine Learning extension:

## Get started with Azure Machine Learning

Before you start training and deploying machine learning models in Visual Studio Code, you need to create an [Azure Machine Learning service workspace](concept-azure-machine-learning-architecture.md#workspace) in the cloud. This workspace will contain your models and resources.
Before you start training and deploying machine learning models in Visual Studio Code, you need to create an [Azure Machine Learning service workspace](concept-workspace.md) in the cloud. This workspace will contain your models and resources.

To create a workspace and add your first experiment:

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ The latest release of Azure Machine Learning service includes the following feat
+ A new, more comprehensive Python <a href="https://aka.ms/aml-sdk" target="_blank">SDK</a>.
+ The new expanded [Azure CLI extension](reference-azure-machine-learning-cli.md) for machine learning.

The [architecture](concept-azure-machine-learning-architecture.md) was redesigned for ease of use. Instead of multiple Azure resources and accounts, you only need an [Azure Machine Learning service Workspace](concept-azure-machine-learning-architecture.md#workspace). You can create workspaces quickly in the [Azure portal]((setup-create-workspace.md#portal). By using a workspace, multiple users can store training and deployment compute targets, model experiments, Docker images, deployed models, and so on.
The [architecture](concept-azure-machine-learning-architecture.md) was redesigned for ease of use. Instead of multiple Azure resources and accounts, you only need an [Azure Machine Learning service Workspace](concept-workspace.md). You can create workspaces quickly in the [Azure portal]((setup-create-workspace.md#portal). By using a workspace, multiple users can store training and deployment compute targets, model experiments, Docker images, deployed models, and so on.

Although there are new improved CLI and SDK clients in the current release, the desktop workbench application itself has been retired. Experiments can be managed in the [workspace dashboard in Azure portal](how-to-track-experiments.md#view-the-experiment-in-the-azure-portal). Use the dashboard to get your experiment history, manage the compute targets attached to your workspace, manage your models and Docker images, and even deploy web services.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ If you have an Azure Machine Learning service workspace, skip to the [next secti

1. Wait approximately 4-5 minutes, until the status changes to **Running**.


## Launch Jupyter web interface

After your VM is running, use the **Notebook VMs** section to launch the Jupyter web interface.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ ms.date: 04/19/2019

# Create an Azure Machine Learning service workspace

To use Azure Machine Learning service, you need an [**Azure Machine Learning service workspace**](concept-azure-machine-learning-architecture.md#workspace). This workspace is the top-level resource for the service and provides you with a centralized place to work with all the artifacts you create.
To use Azure Machine Learning service, you need an [**Azure Machine Learning service workspace**](concept-workspace.md). This workspace is the top-level resource for the service and provides you with a centralized place to work with all the artifacts you create.

In this article, you learn how to create a workspace using any of these methods:
* The [Azure portal](#portal) interface
Expand Down
6 changes: 6 additions & 0 deletions articles/machine-learning/service/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,12 @@
- name: How the service works
displayName: architecture concepts definitions
href: concept-azure-machine-learning-architecture.md
- name: Workspace
displayName: concept definitions architecture
href: concept-workspace.md
- name: Visual interface
displayName: concept
href: ui-concept-visual-interface.md
- name: Automated machine learning
displayName: automl, auto ml
href: concept-automated-ml.md
Expand Down
Loading

0 comments on commit 3275ef8

Please sign in to comment.