Clarification on DSC v3 for Custom Azure Machine Configuration Packages and Azure Arc IaC Best Practices

[boaz-raz]

Hi everyone,

I'm exploring the best ways to leverage DSC v3 for managing machine configurations for servers connected via Azure Arc, following an Infrastructure as Code (IaC) approach.

My understanding from the documentation (like How to create custom machine configuration package artifacts) is that creating custom machine configuration package artifacts is the recommended method for deploying configurations, including those using DSC.

I have a few specific questions regarding DSC v3 in this context:

1. DSC v3 for Custom Packages: Can we confirm that the process outlined in the linked Microsoft Learn article for creating custom machine configuration packages (developing the .mof, packaging with New-GuestConfigurationPackage, etc.) is fully applicable and the recommended approach when authoring configurations using DSC v3 resources and paradigms? Are there any DSC v3-specific considerations or best practices we should be aware of during this package creation and deployment process, particularly for both Windows and Linux nodes managed by Azure Arc?

2. DSC v3 Features in Machine Configuration: With the advancements in DSC v3 (e.g., new resource manifest, JSON/YAML configurations, potential Bicep integration for CaC), how are these features best utilized when creating and deploying custom machine configuration packages? For instance:

   • Are there updated tools or cmdlets beyond New-GuestConfigurationPackage that are tailored for DSC v3 constructs?
   • What is the current status or roadmap for using Bicep to define DSC v3 configurations that are then packaged for Azure Machine Configuration?

3. Alternative IaC Approaches with DSC v3 and Azure Arc: Besides creating custom .zip packages and deploying them via Azure Policy, are there other recommended or emerging IaC patterns for managing machine state with DSC v3 on Azure Arc-enabled servers? For example, direct application of DSC v3 configurations through other Azure services or CLI commands that align with IaC principles?

I'm keen to ensure we're aligning with the latest capabilities and recommended practices for using DSC v3 with Azure Machine Configuration and Azure Arc. Any insights, examples, or roadmap information would be greatly appreciated!

Thanks!

[Gijsreyn]

H @boaz-raz, the quick answer on all 3 questions: Machine Configuration hasn't implemented DSC v3 (yet). As far as I know, there are some talks about it, but no clear roadmap. Not to say that yes, I would also like to see this :)

[mimachniak]

I think before this implementation, we need to add auto-resource installation of DSC, as now we are working on cache there this is next step.

[boaz-raz]

@Gijsreyn @mimachniak

Thanks for the quick response and clarifying that Azure Machine Configuration doesn't yet support DSC v3. That definitely shapes the current approach for managing Arc-enabled servers.

Given this, and since I'm new to DSC, what's the recommended way to apply DSC v3 as Infrastructure as Code (IaC) for on-premise servers right now?

I was thinking of writing a small PowerShell script to act as a local "agent." This agent would regularly check a UNC path for DSC v3 YAML configuration files, then perform dsc test to see if the server is in the desired state. If not, it would execute dsc set to apply the configuration. (I have a small PoC working for this.)

Would this "agent" approach be a viable and recommended path for immediate on-premise DSC v3 IaC, or are there other patterns or tools you'd suggest considering in the interim? Any feedback on this approach would be greatly appreciated!

[michaeltlombardi]

If I was starting from scratch for on-premises IaC without using any higher order tools, here's the approach I would probably look at taking for experimenting with DSC for managing my servers:

Phase 1: Initializing

1. Drafting - I would write a configuration document to manage a small number of related settings, like setting registry keys.

2. Testing - I would write a small set of acceptance tests for the desired state. Having been around the configuration management space for a long time, I would very strongly recommend having a secondary tool to validate your state outside of your configuration management tool. Any CM tool, including DSC, Puppet, Chef, Ansible, Azure Machine Configuration, etc will faithfully report the state as the resource reports it. If there's a bug in the resource implementation, you might apply a configuration repeatedly and never understand that anything is wrong.

3. Documenting: I would write (minimal) documentation for the configuration document - the document itself tells us "what" and the resources handle the "how", but only you know the "why". It doesn't need to be exhaustive, but adding information about the purpose of the document now will help enormously in the future.

4. Committing: I would commit both the configuration document and tests to a repository, so I can get all the benefits of treating my infrastructure definitions as code. I would use whatever code repository my organization is already using - preferably one where I can track issues and cross-reference other projects. This will also enable me to get a historical state of the configuration document, which can be very useful for onboarding team members, talking with auditors, and so on.

5. Scripting: I would consider creating a small PowerShell module for managing this configuration document with an eye towards maintainability and future expansion. I would probably want to be able to use the functions and any scripts in a continuous integration (CI) context as well as for interactive operations. I might write functions like:

   • Get-ConfigurationDocument - retrieve a configuration document locally or from the repository (or artifact feed)
   • Install-DscCli - Retrieve DSC and install on a machine, depending on how my organization wants software to get to our machines. For example, I could have the command retrieve the published artifact from GitHub, extract it, unblock the files, and ensure the folder is added to the PATH. Alternatively, I could pull DSC from a software package manager repository we're already using. I might even implement this function to operate against remote machines as well as local.
   • Initialize-Dsc - Setup a machine for running DSC as a scheduled task. I'd probably implement it to reuse the Install-DscCli function to get DSC on the machine if it's not already there. Personally I would probably use a .psd1 (or JSON) file to control the schedule and serve as parameters for the DscAgent.ps1 script.

   These are just a few quick examples, what I would actually do depends heavily on the context I'm working within. The point for me would be to make sure I'm developing a maintainable set of tools I can use and build on as we bring more nodes and settings under configuration management.

Phase 2: The first server

I would look to get a single machine under control first and see how things go for a week or so. I'd ask myself and my teammates questions like:

• What's the next setting we should add to this configuration document?
• Are we getting any value from the output? Is there anything we're missing?
• How can we enhance this? Should we tie the scheduled task into any dashboards, log collection, or other tools?
• Is this having any impact on operations? How much friction do we have in our process? Can we reduce that friction?
• When we update the configuration document, how ergonomic is that process?
• Which other servers should we prioritize bringing under control?

I would be careful to remind myself and the team that whatever we've built or done to this point should be considered a prototype and be willing to pivot any component or implementation to suit our needs. We're in the process of building custom tooling to manage our infrastructure and we should always keep in mind who the tools are for and what purposes are most important to us.

Phase 3: Adding Complexity

After a week or two of managing that first server with a small configuration document, I would work with the team on either bringing other servers under control, defining an additional document, or expanding the scope of that document. I wouldn't try to do all of these things at once. Maybe we can start thinking about how we want to deploy the script agent/scheduled task in more detail. Maybe we start to consider a model for representing our environment and how to incorporate that - should we have multiple baselines? Should we use parameters? Should we pre-process configuration documents to go from generic to specific?

Thoughts

Configuration management tooling is a hugely complex space. My go-to advice for everyone for the last decade has been: if you don't have to try building your own tooling from the ground up, don't. If anyone in your organization is already using a higher order configuration management tool, start working with them to see whether and how you can collaborate. If those tools don't yet support DSC, starting the conversation and showing a prototype can help you move forward.

I usually describe the complexity of configuration management as "exploding exponential complexity" - the more servers you need to manage, the more settings you want to control, the more you're going to find yourself wishing you had abstractions and helper tools to ease that problem as it grows by orders of magnitude.

But I've also worked in shops where there wasn't any CM in place and getting a higher order tool was a long uphill battle, so we built some internal tools to learn and bridge the gaps. I can definitely see a way forward for doing that, but it's always going to be contextual and the most important thing to me in this process has never been the specific tools. It's the change in approach and thinking, moving towards declarative idempotent modelling of our infrastructure and seeing how we make it serve the needs of ourselves, our partner teams, our users, and our business. It's a great opportunity to reexamine how we've been managing our infrastructure and choose new models and modes for that work.