Skip to content

Commit 6af4d22

Browse files
author
naman-msft
committed
added new doc using AI agent!
1 parent 5c0eebf commit 6af4d22

File tree

4 files changed

+313
-19
lines changed

4 files changed

+313
-19
lines changed

‎scenarios/azure-docs/articles/batch/quick-create-cli.md

Lines changed: 247 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,247 @@
1+
---
2+
title: 'Quickstart: Use the Azure CLI to create a Batch account and run a job'
3+
description: Follow this quickstart to use the Azure CLI to create a Batch account, a pool of compute nodes, and a job that runs basic tasks on the pool.
4+
ms.topic: quickstart
5+
ms.date: 04/12/2023
6+
ms.custom: mvc, devx-track-azurecli, mode-api, linux-related-content, innovation-engine
7+
author: (preserved)
8+
ms.author: (preserved)
9+
---
10+
11+
# Quickstart: Use the Azure CLI to create a Batch account and run a job
12+
13+
This quickstart shows you how to get started with Azure Batch by using Azure CLI commands and scripts to create and manage Batch resources. You create a Batch account that has a pool of virtual machines, or compute nodes. You then create and run a job with tasks that run on the pool nodes.
14+
15+
After you complete this quickstart, you understand the [key concepts of the Batch service](batch-service-workflow-features.md) and are ready to use Batch with more realistic, larger scale workloads.
16+
17+
## Prerequisites
18+
19+
- [!INCLUDE [quickstarts-free-trial-note](~/reusable-content/ce-skilling/azure/includes/quickstarts-free-trial-note.md)]
20+
21+
- Azure Cloud Shell or Azure CLI.
22+
23+
You can run the Azure CLI commands in this quickstart interactively in Azure Cloud Shell. To run the commands in the Cloud Shell, select **Open Cloudshell** at the upper-right corner of a code block. Select **Copy** to copy the code, and paste it into Cloud Shell to run it. You can also [run Cloud Shell from within the Azure portal](https://shell.azure.com). Cloud Shell always uses the latest version of the Azure CLI.
24+
25+
Alternatively, you can [install Azure CLI locally](/cli/azure/install-azure-cli) to run the commands. The steps in this article require Azure CLI version 2.0.20 or later. Run [az version](/cli/azure/reference-index?#az-version) to see your installed version and dependent libraries, and run [az upgrade](/cli/azure/reference-index?#az-upgrade) to upgrade. If you use a local installation, sign in to Azure by using the appropriate command.
26+
27+
>[!NOTE]
28+
>For some regions and subscription types, quota restrictions might cause Batch account or node creation to fail or not complete. In this situation, you can request a quota increase at no charge. For more information, see [Batch service quotas and limits](batch-quota-limit.md).
29+
30+
## Create a resource group
31+
32+
Run the following [az group create](/cli/azure/group#az-group-create) command to create an Azure resource group. The resource group is a logical container that holds the Azure resources for this quickstart.
33+
34+
```azurecli-interactive
35+
export RANDOM_SUFFIX=$(openssl rand -hex 3)
36+
export REGION="canadacentral"
37+
export RESOURCE_GROUP="qsBatch$RANDOM_SUFFIX"
38+
39+
az group create \
40+
--name $RESOURCE_GROUP \
41+
--location $REGION
42+
```
43+
44+
Results:
45+
46+
<!-- expected_similarity=0.3 -->
47+
48+
```JSON
49+
{
50+
"id": "/subscriptions/xxxxx/resourceGroups/qsBatchxxx",
51+
"location": "eastus2",
52+
"managedBy": null,
53+
"name": "qsBatchxxx",
54+
"properties": {
55+
"provisioningState": "Succeeded"
56+
},
57+
"tags": null,
58+
"type": "Microsoft.Resources/resourceGroups"
59+
}
60+
```
61+
62+
## Create a storage account
63+
64+
Use the [az storage account create](/cli/azure/storage/account#az-storage-account-create) command to create an Azure Storage account to link to your Batch account. Although this quickstart doesn't use the storage account, most real-world Batch workloads use a linked storage account to deploy applications and store input and output data.
65+
66+
Run the following command to create a Standard_LRS SKU storage account in your resource group:
67+
68+
```azurecli-interactive
69+
export STORAGE_ACCOUNT="mybatchstorage$RANDOM_SUFFIX"
70+
71+
az storage account create \
72+
--resource-group $RESOURCE_GROUP \
73+
--name $STORAGE_ACCOUNT \
74+
--location $REGION \
75+
--sku Standard_LRS
76+
```
77+
78+
## Create a Batch account
79+
80+
Run the following [az batch account create](/cli/azure/batch/account#az-batch-account-create) command to create a Batch account in your resource group and link it with the storage account.
81+
82+
```azurecli-interactive
83+
export BATCH_ACCOUNT="mybatchaccount$RANDOM_SUFFIX"
84+
85+
az batch account create \
86+
--name $BATCH_ACCOUNT \
87+
--storage-account $STORAGE_ACCOUNT \
88+
--resource-group $RESOURCE_GROUP \
89+
--location $REGION
90+
```
91+
92+
Sign in to the new Batch account by running the [az batch account login](/cli/azure/batch/account#az-batch-account-login) command. Once you authenticate your account with Batch, subsequent `az batch` commands in this session use this account context.
93+
94+
```azurecli-interactive
95+
az batch account login \
96+
--name $BATCH_ACCOUNT \
97+
--resource-group $RESOURCE_GROUP \
98+
--shared-key-auth
99+
```
100+
101+
## Create a pool of compute nodes
102+
103+
Run the [az batch pool create](/cli/azure/batch/pool#az-batch-pool-create) command to create a pool of Linux compute nodes in your Batch account. The following example creates a pool that consists of two Standard_A1_v2 size VMs running Ubuntu 20.04 LTS OS. This node size offers a good balance of performance versus cost for this quickstart example.
104+
105+
```azurecli-interactive
106+
export POOL_ID="myPool$RANDOM_SUFFIX"
107+
108+
az batch pool create \
109+
--id $POOL_ID \
110+
--image canonical:0001-com-ubuntu-server-focal:20_04-lts \
111+
--node-agent-sku-id "batch.node.ubuntu 20.04" \
112+
--target-dedicated-nodes 2 \
113+
--vm-size Standard_A1_v2
114+
```
115+
116+
Batch creates the pool immediately, but takes a few minutes to allocate and start the compute nodes. To see the pool status, use the [az batch pool show](/cli/azure/batch/pool#az-batch-pool-show) command. This command shows all the properties of the pool, and you can query for specific properties. The following command queries for the pool allocation state:
117+
118+
```azurecli-interactive
119+
az batch pool show --pool-id $POOL_ID \
120+
--query "{allocationState: allocationState}"
121+
```
122+
123+
Results:
124+
125+
<!-- expected_similarity=0.3 -->
126+
127+
```JSON
128+
{
129+
"allocationState": "resizing"
130+
}
131+
```
132+
133+
While Batch allocates and starts the nodes, the pool is in the `resizing` state. You can create a job and tasks while the pool state is still `resizing`. The pool is ready to run tasks when the allocation state is `steady` and all the nodes are running.
134+
135+
## Create a job
136+
137+
Use the [az batch job create](/cli/azure/batch/job#az-batch-job-create) command to create a Batch job to run on your pool. A Batch job is a logical group of one or more tasks. The job includes settings common to the tasks, such as the pool to run on. The following example creates a job that initially has no tasks.
138+
139+
```azurecli-interactive
140+
export JOB_ID="myJob$RANDOM_SUFFIX"
141+
142+
az batch job create \
143+
--id $JOB_ID \
144+
--pool-id $POOL_ID
145+
```
146+
147+
## Create job tasks
148+
149+
Batch provides several ways to deploy apps and scripts to compute nodes. Use the [az batch task create](/cli/azure/batch/task#az-batch-task-create) command to create tasks to run in the job. Each task has a command line that specifies an app or script.
150+
151+
The following Bash script creates four identical, parallel tasks called `myTask1` through `myTask4`. The task command line displays the Batch environment variables on the compute node, and then waits 90 seconds.
152+
153+
```azurecli-interactive
154+
for i in {1..4}
155+
do
156+
az batch task create \
157+
--task-id myTask$i \
158+
--job-id $JOB_ID \
159+
--command-line "/bin/bash -c 'printenv | grep AZ_BATCH; sleep 90s'"
160+
done
161+
```
162+
163+
Batch distributes the tasks to the compute nodes.
164+
165+
## View task status
166+
167+
After you create the tasks, Batch queues them to run on the pool. Once a node is available, a task runs on the node.
168+
169+
Use the [az batch task show](/cli/azure/batch/task#az-batch-task-show) command to view the status of Batch tasks. The following example shows details about the status of `myTask1`:
170+
171+
```azurecli-interactive
172+
az batch task show \
173+
--job-id $JOB_ID \
174+
--task-id myTask1
175+
```
176+
177+
The command output includes many details. For example, an `exitCode` of `0` indicates that the task command completed successfully. The `nodeId` shows the name of the pool node that ran the task.
178+
179+
## View task output
180+
181+
Use the [az batch task file list](/cli/azure/batch/task#az-batch-task-file-show) command to list the files a task created on a node. The following command lists the files that `myTask1` created:
182+
183+
```azurecli-interactive
184+
# Wait for task to complete before downloading output
185+
echo "Waiting for task to complete..."
186+
while true; do
187+
STATUS=$(az batch task show --job-id $JOB_ID --task-id myTask1 --query "state" -o tsv)
188+
if [ "$STATUS" == "running" ]; then
189+
break
190+
fi
191+
sleep 10
192+
done
193+
194+
az batch task file list --job-id $JOB_ID --task-id myTask1 --output table
195+
```
196+
197+
Results are similar to the following output:
198+
199+
Results:
200+
201+
<!-- expected_similarity=0.3 -->
202+
203+
```output
204+
Name URL Is Directory Content Length
205+
---------- ---------------------------------------------------------------------------------------- -------------- ----------------
206+
stdout.txt https://mybatchaccount.eastus2.batch.azure.com/jobs/myJob/tasks/myTask1/files/stdout.txt False 695
207+
certs https://mybatchaccount.eastus2.batch.azure.com/jobs/myJob/tasks/myTask1/files/certs True
208+
wd https://mybatchaccount.eastus2.batch.azure.com/jobs/myJob/tasks/myTask1/files/wd True
209+
stderr.txt https://mybatchaccount.eastus2.batch.azure.com/jobs/myJob/tasks/myTask1/files/stderr.txt False 0
210+
```
211+
212+
The [az batch task file download](/cli/azure/batch/task#az-batch-task-file-download) command downloads output files to a local directory. Run the following example to download the *stdout.txt* file:
213+
214+
```azurecli-interactive
215+
az batch task file download \
216+
--job-id $JOB_ID \
217+
--task-id myTask1 \
218+
--file-path stdout.txt \
219+
--destination ./stdout.txt
220+
```
221+
222+
You can view the contents of the standard output file in a text editor. The following example shows a typical *stdout.txt* file. The standard output from this task shows the Azure Batch environment variables that are set on the node. You can refer to these environment variables in your Batch job task command lines, and in the apps and scripts the command lines run.
223+
224+
```text
225+
AZ_BATCH_TASK_DIR=/mnt/batch/tasks/workitems/myJob/job-1/myTask1
226+
AZ_BATCH_NODE_STARTUP_DIR=/mnt/batch/tasks/startup
227+
AZ_BATCH_CERTIFICATES_DIR=/mnt/batch/tasks/workitems/myJob/job-1/myTask1/certs
228+
AZ_BATCH_ACCOUNT_URL=https://mybatchaccount.eastus2.batch.azure.com/
229+
AZ_BATCH_TASK_WORKING_DIR=/mnt/batch/tasks/workitems/myJob/job-1/myTask1/wd
230+
AZ_BATCH_NODE_SHARED_DIR=/mnt/batch/tasks/shared
231+
AZ_BATCH_TASK_USER=_azbatch
232+
AZ_BATCH_NODE_ROOT_DIR=/mnt/batch/tasks
233+
AZ_BATCH_JOB_ID=myJob
234+
AZ_BATCH_NODE_IS_DEDICATED=true
235+
AZ_BATCH_NODE_ID=tvm-257509324_2-20180703t215033z
236+
AZ_BATCH_POOL_ID=myPool
237+
AZ_BATCH_TASK_ID=myTask1
238+
AZ_BATCH_ACCOUNT_NAME=mybatchaccount
239+
AZ_BATCH_TASK_USER_IDENTITY=PoolNonAdmin
240+
```
241+
242+
## Next steps
243+
244+
In this quickstart, you created a Batch account and pool, created and ran a Batch job and tasks, and viewed task output from the nodes. Now that you understand the key concepts of the Batch service, you're ready to use Batch with more realistic, larger scale workloads. To learn more about Azure Batch, continue to the Azure Batch tutorials.
245+
246+
> [!div class="nextstepaction"]
247+
> [Tutorial: Run a parallel workload with Azure Batch](./tutorial-parallel-python.md)

‎scenarios/metadata.json

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -997,5 +997,23 @@
997997
}
998998
]
999999
}
1000+
},
1001+
{
1002+
"status": "active",
1003+
"key": "azure-docs/articles/batch/quick-create-cli.md",
1004+
"title": "Quickstart: Use the Azure CLI to create a Batch account and run a job",
1005+
"description": "Follow this quickstart to use the Azure CLI to create a Batch account, a pool of compute nodes, and a job that runs basic tasks on the pool.",
1006+
"stackDetails": [
1007+
],
1008+
"sourceUrl": "https://raw.githubusercontent.com/MicrosoftDocs/executable-docs/main/scenarios/azure-docs/articles/batch/quick-create-cli.md",
1009+
"documentationUrl": "https://learn.microsoft.com/en-us/azure/batch/quick-create-cli",
1010+
"nextSteps": [
1011+
{
1012+
"title": "Tutorial: Run a parallel workload with Azure Batch",
1013+
"url": "https://learn.microsoft.com/en-us/azure/batch/tutorial-parallel-python"
1014+
}
1015+
],
1016+
"configurations": {
1017+
}
10001018
}
10011019
]

‎tools/converted_doc.md

Lines changed: 28 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -27,25 +27,15 @@ After you complete this quickstart, you understand the [key concepts of the Batc
2727
>[!NOTE]
2828
>For some regions and subscription types, quota restrictions might cause Batch account or node creation to fail or not complete. In this situation, you can request a quota increase at no charge. For more information, see [Batch service quotas and limits](batch-quota-limit.md).
2929
30-
## Setup Environment Variables
31-
32-
Below, we declare environment variables that will be used throughout this Exec Doc. We include a random suffix to uniquely name resources and avoid collisions on repeated executions.
33-
34-
```bash
35-
export RANDOM_SUFFIX=$(openssl rand -hex 3)
36-
export REGION="eastus2"
37-
export RESOURCE_GROUP="qsBatch$RANDOM_SUFFIX"
38-
export STORAGE_ACCOUNT="mybatchstorage$RANDOM_SUFFIX"
39-
export BATCH_ACCOUNT="mybatchaccount$RANDOM_SUFFIX"
40-
export POOL_ID="myPool$RANDOM_SUFFIX"
41-
export JOB_ID="myJob$RANDOM_SUFFIX"
42-
```
43-
4430
## Create a resource group
4531

4632
Run the following [az group create](/cli/azure/group#az-group-create) command to create an Azure resource group. The resource group is a logical container that holds the Azure resources for this quickstart.
4733

4834
```azurecli-interactive
35+
export RANDOM_SUFFIX=$(openssl rand -hex 3)
36+
export REGION="canadacentral"
37+
export RESOURCE_GROUP="qsBatch$RANDOM_SUFFIX"
38+
4939
az group create \
5040
--name $RESOURCE_GROUP \
5141
--location $REGION
@@ -76,6 +66,8 @@ Use the [az storage account create](/cli/azure/storage/account#az-storage-accoun
7666
Run the following command to create a Standard_LRS SKU storage account in your resource group:
7767

7868
```azurecli-interactive
69+
export STORAGE_ACCOUNT="mybatchstorage$RANDOM_SUFFIX"
70+
7971
az storage account create \
8072
--resource-group $RESOURCE_GROUP \
8173
--name $STORAGE_ACCOUNT \
@@ -88,6 +80,8 @@ az storage account create \
8880
Run the following [az batch account create](/cli/azure/batch/account#az-batch-account-create) command to create a Batch account in your resource group and link it with the storage account.
8981

9082
```azurecli-interactive
83+
export BATCH_ACCOUNT="mybatchaccount$RANDOM_SUFFIX"
84+
9185
az batch account create \
9286
--name $BATCH_ACCOUNT \
9387
--storage-account $STORAGE_ACCOUNT \
@@ -109,6 +103,8 @@ az batch account login \
109103
Run the [az batch pool create](/cli/azure/batch/pool#az-batch-pool-create) command to create a pool of Linux compute nodes in your Batch account. The following example creates a pool that consists of two Standard_A1_v2 size VMs running Ubuntu 20.04 LTS OS. This node size offers a good balance of performance versus cost for this quickstart example.
110104

111105
```azurecli-interactive
106+
export POOL_ID="myPool$RANDOM_SUFFIX"
107+
112108
az batch pool create \
113109
--id $POOL_ID \
114110
--image canonical:0001-com-ubuntu-server-focal:20_04-lts \
@@ -121,7 +117,7 @@ Batch creates the pool immediately, but takes a few minutes to allocate and star
121117

122118
```azurecli-interactive
123119
az batch pool show --pool-id $POOL_ID \
124-
--query "allocationState"
120+
--query "{allocationState: allocationState}"
125121
```
126122

127123
Results:
@@ -141,6 +137,8 @@ While Batch allocates and starts the nodes, the pool is in the `resizing` state.
141137
Use the [az batch job create](/cli/azure/batch/job#az-batch-job-create) command to create a Batch job to run on your pool. A Batch job is a logical group of one or more tasks. The job includes settings common to the tasks, such as the pool to run on. The following example creates a job that initially has no tasks.
142138

143139
```azurecli-interactive
140+
export JOB_ID="myJob$RANDOM_SUFFIX"
141+
144142
az batch job create \
145143
--id $JOB_ID \
146144
--pool-id $POOL_ID
@@ -183,14 +181,25 @@ The command output includes many details. For example, an `exitCode` of `0` indi
183181
Use the [az batch task file list](/cli/azure/batch/task#az-batch-task-file-show) command to list the files a task created on a node. The following command lists the files that `myTask1` created:
184182

185183
```azurecli-interactive
186-
az batch task file list \
187-
--job-id $JOB_ID \
188-
--task-id myTask1 \
189-
--output table
184+
# Wait for task to complete before downloading output
185+
echo "Waiting for task to complete..."
186+
while true; do
187+
STATUS=$(az batch task show --job-id $JOB_ID --task-id myTask1 --query "state" -o tsv)
188+
if [ "$STATUS" == "running" ]; then
189+
break
190+
fi
191+
sleep 10
192+
done
193+
194+
az batch task file list --job-id $JOB_ID --task-id myTask1 --output table
190195
```
191196

192197
Results are similar to the following output:
193198

199+
Results:
200+
201+
<!-- expected_similarity=0.3 -->
202+
194203
```output
195204
Name URL Is Directory Content Length
196205
---------- ---------------------------------------------------------------------------------------- -------------- ----------------

‎tools/stdout.txt

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
AZ_BATCH_NODE_MOUNTS_DIR=/mnt/batch/tasks/fsmounts
2+
AZ_BATCH_TASK_WORKING_DIR=/mnt/batch/tasks/workitems/myJobadb33d/job-1/myTask1/wd
3+
AZ_BATCH_TASK_DIR=/mnt/batch/tasks/workitems/myJobadb33d/job-1/myTask1
4+
AZ_BATCH_NODE_SHARED_DIR=/mnt/batch/tasks/shared
5+
AZ_BATCH_TASK_USER=_azbatch
6+
AZ_BATCH_NODE_IS_DEDICATED=true
7+
AZ_BATCH_NODE_STARTUP_DIR=/mnt/batch/tasks/startup
8+
AZ_BATCH_JOB_ID=myJobadb33d
9+
AZ_BATCH_NODE_STARTUP_WORKING_DIR=/mnt/batch/tasks/startup/wd
10+
AZ_BATCH_TASK_ID=myTask1
11+
AZ_BATCH_ACCOUNT_NAME=mybatchaccountadb33d
12+
AZ_BATCH_RESERVED_EPHEMERAL_DISK_SPACE_BYTES=1000000000
13+
AZ_BATCH_NODE_ROOT_DIR=/mnt/batch/tasks
14+
AZ_BATCH_POOL_ID=myPooladb33d
15+
AZ_BATCH_RESERVED_DISK_SPACE_BYTES=1000000000
16+
AZ_BATCH_ACCOUNT_URL=https://mybatchaccountadb33d.canadacentral.batch.azure.com/
17+
AZ_BATCH_NODE_ID=tvmps_1b25c614520a9192d5e81007e1880adf7012f74bc13ba2733718a8d77878cc5b_d
18+
AZ_BATCH_TASK_USER_IDENTITY=PoolNonAdmin
19+
AZ_BATCH_OS_RESERVED_EPHEMERAL_DISK_SPACE_BYTES=1000000000
20+
AZ_BATCH_CERTIFICATES_DIR=/mnt/batch/tasks/workitems/myJobadb33d/job-1/myTask1/certs

0 commit comments

Comments
 (0)