We sometimes have the need to use a batch (console) application to perform some computational tasks. These tasks may run on a schedule, however may not run often enough to justify setting up a dedicated deployment/run pipeline. I would like to run these processors on serverless cloud infrastructure so that you are only billed when service is utilized, and use low-code options for setup.
Below I have described my experiment on how I tried to achieve this objective using Docker, Azure Container Instances service, and Azure Logic Apps.
High-level steps
- Create console app
- Add docker support
- Setup Azure infrastructure
- Build and load image to registry
- Trigger logic app
- Docker Desktop
- .NET SDK
- Azure CLI
- VS.Code
- (Optional) VS Code extensions
- C# v1.25.2
- Docker v1.23.2
- vscode-icons v12.0.1
- YAML v1.10.1
- Create .net application using vs code
Get supported project templates
dotnet new
Create new console project (and configure git repo)
# dotnet new <short name> -n <project name> -o <project folder>
dotnet new console -n helloConsole -o App
cd App
dotnet new gitignore
git init
Open project in vs code
code .
- Update code (Program.cs) to crate sample application that takes value from environment variable
Console.WriteLine("Start long running app");
var counter = 0;
var maxCount = Environment.GetEnvironmentVariable("max_count");
var max = string.IsNullOrEmpty(maxCount) ?-1: Convert.ToInt32(maxCount);
while (max == -1 || counter < max)
{
Console.WriteLine($"Counter: {++counter}");
await Task.Delay(TimeSpan.FromMilliseconds(1_000));
}
Console.WriteLine("End long running app");
- Add debugging files
- Add docker support using docker extension
Example of the docker file thats generated
FROM mcr.microsoft.com/dotnet/runtime:7.0 AS base
WORKDIR /app
# Creates a non-root user with an explicit UID and adds permission to access the /app folder
# For more info, please refer to https://aka.ms/vscode-docker-dotnet-configure-containers
RUN adduser -u 5678 --disabled-password --gecos "" appuser && chown -R appuser /app
USER appuser
FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
WORKDIR /src
COPY ["helloConsole.csproj", "./"]
RUN dotnet restore "helloConsole.csproj"
COPY . .
WORKDIR "/src/."
RUN dotnet build "helloConsole.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "helloConsole.csproj" -c Release -o /app/publish /p:UseAppHost=false
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "helloConsole.dll"]
- Debug project in docker
I used Azure portal to setup resources, however when I had a working version I created bicep and arm template to make setup simpler. Use the "Deploy to Azure" button to deploy an ARM Template to seutp the following resources:
- Azure Container Registry (if required)
- Azure logic app
Note: I have not been able to get logic app connection to the container registry working with biceps. Currently you would need to add a new connection using your credentials.
Logic app has the following workflow
- Triggered by HTTP request
- Set environment variables
- Create Azure Container instance (pull image from container registry and run container) (options )
- Wait until complete
- Get container log
- Delete container instance
Crate container instance
- Build a docker image with docker file in the current directory (.)
# Docker build –t [image name]:[version] -f [docker file location] [path]
image=helloconsole
ver=latest
docker build -t $image:$ver -f Dockerfile .
- Setup script variables
registry=demotkcr
image=helloconsole
ver=latest
sub=[subscription id]
rg=demotk-rg
- Tag local docker image so it can be pushed to docker registry
#docker tag source_image[:tag] target_image[:tag]
docker tag $image:latest $registry.azurecr.io/$image:$ver
- log in to Azure
az login
- Set context to correct subscription
az account set -s $sub
az account list -o table
- Get admin password for ACR and log in to container registry
un=$(az acr credential show -n $registry -g $rg --query username -o tsv)
pw=$(az acr credential show -n $registry -g $rg --query passwords[0].value -o tsv)
docker login $registry.azurecr.io -u $un -p $pw
- Push image to registry
# docker push [docker registry name]/[Image name]:[version]
docker push $registry.azurecr.io/$image:$ver
Log in to Azure portal, select logic app, and trigger a test run.
The process worked well for jobs that are running less than 10 minutes.
For jobs that are running longer the step to delete the container instance did not run, however the container was stopped and we were not billed by Azure once it was stopped
- Investigate how to set up connections to container registry through biceps
- Investigate how to cleanup container instances for longer running processors