Add library support for local deployments #14237

anthony-c-martin · 2024-06-04T16:04:21Z

Add Bicep.Local.Deploy library containing "local deployment" logic.

The full spec is available here - unfortunately only available to Microsoft employees currently.

Microsoft Reviewers: Open in CodeFlow

github-actions · 2024-06-04T16:07:57Z

Test this change out locally with the following install scripts (Action run 9409014946)

VSCode

Mac/Linux

bash <(curl -Ls https://aka.ms/bicep/nightly-vsix.sh) --run-id 9409014946

Windows

iex "& { $(irm https://aka.ms/bicep/nightly-vsix.ps1) } -RunId 9409014946"

Azure CLI

Mac/Linux

bash <(curl -Ls https://aka.ms/bicep/nightly-cli.sh) --run-id 9409014946

Windows

iex "& { $(irm https://aka.ms/bicep/nightly-cli.ps1) } -RunId 9409014946"

github-actions · 2024-06-04T16:20:21Z

Dotnet Test Results

35 files - 62 35 suites - 62 11m 58s ⏱️ - 21m 10s
10 631 tests - 265 10 630 ✅ - 265 1 💤 ±0 0 ❌ ±0
12 833 runs - 25 635 12 832 ✅ - 25 633 1 💤 - 2 0 ❌ ±0

Results for commit bc33931. ± Comparison against base commit 46a412b.

♻️ This comment has been updated with latest results.

src/Bicep.Local.Extension/Rpc/GrpcChannelHelper.cs

Mistake

davidcho23 · 2024-06-05T17:04:42Z

src/Bicep.Local.Extension/Rpc/GrpcChannelHelper.cs

@@ -48,7 +48,7 @@ public static GrpcChannel CreateChannel(string socketPath)
        };

        // The URL is not used, but it must be a valid URI.


What is this URL needed for then?

It's parsed initially by the GrpcChannel API (which throws if it's invalid), but the actual socket connection logic is overridden by setting the HttpHandler

davidcho23 · 2024-06-05T17:07:12Z

src/Bicep.Local.Deploy/LocalDeploymentEngine.cs

+        }
+    }
+
+    private async Task SaveDeployment(DeploymentPlan deploymentPlan)


Just to clarify, SaveDeployment means that the deployment is kept track of locally, correct?

I'm using in-memory data providers, so when we "save" an entity, we are replacing it in memory. This is only available while the process is running - it is not persisted afterwards.

davidcho23 · 2024-06-05T17:10:08Z

src/Bicep.Local.Deploy/LocalDeploymentSettings.cs

+
+    public string[] PreviewFunctionalityExternalPermittedTenants { get; set; } = new string[] { };
+
+    public string[] PreviewFunctionalityPublicFeatures { get; set; } = new string[] {


I'm probably a little confused but where does this get checked compared to the ServiceDirectory.json? Or is it not at all?

It's not so relevant for this particular feature, but I'm forced to define a value for it.

jorgecotillo · 2024-06-05T17:40:12Z

src/Bicep.Local.Deploy/LocalDeployment.cs

+
+    public static async Task<Result> Deploy(LocalExtensibilityHandler extensibilityHandler, string templateString, string parametersString, CancellationToken cancellationToken)
+    {
+        var services = new ServiceCollection()


I assume this is a short-cut, or de we want to register and build the service provider every time .Deploy is invoked?

It's a short-cut for now - https://msazure.visualstudio.com/One/_workitems/edit/28299061 would be the proper implementation for it.

jorgecotillo · 2024-06-05T17:42:23Z

src/Bicep.Local.Deploy/LocalDeployment.cs

+        }
+        finally
+        {
+            await dispatcher.StopAsync();


Where does distacher.Start() happens?

Great question, I think it must be happening inside the deployment engine. It's important to stop regardless, so that we don't leak jobs when running in the language server.

jorgecotillo · 2024-06-05T17:50:30Z

src/Bicep.Local.Deploy/LocalDeploymentEngine.cs

+        return new(
+            deploymentEngine.CreateDeploymentDefinition(entity, requestContext.ApiVersion),
+            operationEntities
+                .Where(operation => operation.TargetResource is not null)


When is TargetResource null?

The deployment last job creates a single operation for "processing outputs" - this filters it down to just resource operations.

jorgecotillo · 2024-06-05T17:55:13Z

src/Bicep.Local.Deploy/LocalDeploymentEngine.cs

+        {
+            await Task.Delay(500, cancellationToken);
+
+            entity = await dataProvider.FindDeployment(context.SubscriptionId, context.ResourceGroupName, context.DeploymentName);


This part is interesting, in ARM, we go off the async header and poll for the status, here we don't have such option, something we could explore at some point is standing up a local deployments server, so we can reuse the same deploymentsWorkflow rather than implementing a local deployment engine.

Also, a TODO to add here is to gracefully handle an exception thrown by volatile datastore also to not loop indefinitely or until the cancellation is requested

It would be nice to avoid polling here - it's a consequence of low visibility into ResourceStack's job processing internals. I've cut the polling down to every 20ms to make it more responsive as it should be very low-cost.

jorgecotillo · 2024-06-05T17:58:30Z

src/Bicep.Local.Deploy/Extensibility/AzExtensibilityProvider.cs

+                var parameters = new JObject {
+                    ["$schema"] = "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
+                    ["contentVersion"] = "1.0.0.0",
+                    ["parameters"] = request.Resource.Properties["parameters"],


TODO, account for parametersLink and templateLink?

Good callout - something to think through.

jorgecotillo · 2024-06-05T18:04:37Z

src/Bicep.Local.Deploy/Extensibility/AzExtensibilityProvider.cs

+    {
+        switch (request.Resource.Type)
+        {
+            case "Microsoft.Resources/deployments": {


If this is Az extensibility provider, can this type be any azure resource type?

Not currently - it's out of scope for the first phase.

jorgecotillo · 2024-06-05T18:06:31Z

src/Bicep.Local.Deploy/Extensibility/LocalExtensibilityHandler.cs

+
+    private async Task<ExtensibilityOperationResponse> CallProvider(string method, IExtensibilityProvider provider, ExtensibilityOperationRequest request, CancellationToken cancellationToken)
+    {
+        return method switch


TODO: move to enum

Good callout!

anthony-c-martin force-pushed the ant/local_libs branch from ce644d7 to 5f01f51 Compare June 4, 2024 16:12

anthony-c-martin force-pushed the ant/local_libs branch 2 times, most recently from e52e251 to 9023c38 Compare June 4, 2024 16:29

Add library support for local deployments

2fa4456

anthony-c-martin force-pushed the ant/local_libs branch from 9023c38 to 2fa4456 Compare June 4, 2024 16:34

anthony-c-martin commented Jun 4, 2024

View reviewed changes

src/Bicep.Local.Extension/Rpc/GrpcChannelHelper.cs Show resolved Hide resolved

davidcho23 previously approved these changes Jun 4, 2024

View reviewed changes

Add explicit reference to Microsoft.AspNet.WebApi.Client

bc33931

anthony-c-martin force-pushed the ant/local_libs branch from 443f0ea to bc33931 Compare June 5, 2024 13:51

anthony-c-martin enabled auto-merge (squash) June 5, 2024 13:53

davidcho23 self-requested a review June 5, 2024 14:33

anthony-c-martin disabled auto-merge June 5, 2024 15:52

davidcho23 reviewed Jun 5, 2024

View reviewed changes

jorgecotillo reviewed Jun 5, 2024

View reviewed changes

jorgecotillo approved these changes Jun 5, 2024

View reviewed changes

anthony-c-martin merged commit f22ddf9 into main Jun 5, 2024
49 of 50 checks passed

anthony-c-martin deleted the ant/local_libs branch June 5, 2024 19:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add library support for local deployments #14237

Add library support for local deployments #14237

anthony-c-martin commented Jun 4, 2024 •

edited

Loading

github-actions bot commented Jun 4, 2024 •

edited

Loading

github-actions bot commented Jun 4, 2024 •

edited

Loading

davidcho23 Jun 5, 2024

anthony-c-martin Jun 5, 2024

davidcho23 Jun 5, 2024

anthony-c-martin Jun 5, 2024

davidcho23 Jun 5, 2024

anthony-c-martin Jun 5, 2024

jorgecotillo Jun 5, 2024

anthony-c-martin Jun 5, 2024

jorgecotillo Jun 5, 2024

anthony-c-martin Jun 5, 2024

jorgecotillo Jun 5, 2024

anthony-c-martin Jun 5, 2024

jorgecotillo Jun 5, 2024

anthony-c-martin Jun 5, 2024

jorgecotillo Jun 5, 2024

anthony-c-martin Jun 5, 2024

jorgecotillo Jun 5, 2024

anthony-c-martin Jun 5, 2024

jorgecotillo Jun 5, 2024

anthony-c-martin Jun 5, 2024

		@@ -48,7 +48,7 @@ public static GrpcChannel CreateChannel(string socketPath)
		};

		// The URL is not used, but it must be a valid URI.


		public string[] PreviewFunctionalityExternalPermittedTenants { get; set; } = new string[] { };

		public string[] PreviewFunctionalityPublicFeatures { get; set; } = new string[] {

Add library support for local deployments #14237

Add library support for local deployments #14237

Conversation

anthony-c-martin commented Jun 4, 2024 • edited Loading

Microsoft Reviewers: Open in CodeFlow

github-actions bot commented Jun 4, 2024 • edited Loading

github-actions bot commented Jun 4, 2024 • edited Loading

Dotnet Test Results

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anthony-c-martin commented Jun 4, 2024 •

edited

Loading

github-actions bot commented Jun 4, 2024 •

edited

Loading

github-actions bot commented Jun 4, 2024 •

edited

Loading