Digesting Microsoft's DSC v3.2 release: A practical look at what changed

Microsoft's DSC v3.2 announcement and release were published on the 29th of April. Looking at the table of contents (TOC), a lot has changed since v3.1.

But behind TOC, there are a number of small (to big) changes that aren't obvious directly from the surface. Take the hypothetical TOML example, for instance. How does that actually work in practice? And what about the PowerShell discovery extension?

In this post, we'll digest the Microsoft DSC 3.2 release a bit further. Not by repeating the announcement. Instead, by looking at some of the parts that require a second look, and looking at what they actually do when you work with them.

Wasn't WhatIf not "already" there?

Besides the Bicep support and new built-in resources, there was a section named "Extended WhatIf support." But wasn't that already there?

Well, partially true, as also stated in the article. However, there are two interesting aspects when looking in more depth. The first one is obvious and stated in the article, DSC's subcommand config set already had the --what-if mode, but now it can also be applied on dsc resource set.

But the resource manifest can now declare whatIfReturns field in the resource manifest. This is helpful for resource authors, enabling a richer preview output for resources they develop.

So, how does this work in-depth? Whenever you run dsc config set -w or dsc resource set -w, DSC invokes the resource's set command with an extra flag specified by this whatIfArg manifest field. After the command finishes, DSC interprets the output according to the whatIfReturns manifest field instead of the normal return field.

To take it one step further and make it a complete example, let's illustrate it with a PowerShell DSC resource managing a Windows environment variable. This would be the MyEnvVar.dsc.resource.json file:

{
  "$schema": "https://aka.ms/dsc/schemas/v3/bundled/resource/manifest.json",
  "type": "MyCompany/EnvVar",
  "version": "0.1.0",
  "description": "Manages a Windows user-scope environment variable.",
  "get": {
    "executable": "pwsh",
    "args": [
      "-NonInteractive", "-NoProfile", "-ExecutionPolicy", "Bypass",
      "-File", "MyEnvVar.ps1",
      "get",
      { "jsonInputArg": "--input", "mandatory": true }
    ]
  },
  "set": {
    "executable": "pwsh",
    "args": [
      "-NonInteractive", "-NoProfile", "-ExecutionPolicy", "Bypass",
      "-File", "MyEnvVar.ps1",
      "set",
      { "jsonInputArg": "--input", "mandatory": true },
      { "whatIfArg": "-w" }
    ],
    "return": "state",
    "whatIfReturns": "state"
  },
  "schema": {
    "embedded": {
      "$schema": "https://json-schema.org/draft/2020-12/schema",
      "type": "object",
      "required": ["name"],
      "properties": {
        "name":  { "type": "string", "description": "The environment variable name." },
        "value": { "type": ["string", "null"], "description": "The value to set." },
        "_exist": {
          "type": "boolean",
          "default": true,
          "description": "Whether the variable should exist."
        },
        "_metadata": {
          "type": "object",
          "properties": {
            "whatIf": { "type": "array", "items": { "type": "string" } }
          }
        }
      }
    }
  }
}

In the resource script (MyEnvVar.ps), it looks something like:

[CmdletBinding()]
param(
    [Parameter(Mandatory, Position = 0)]
    [ValidateSet('get', 'set')]
    [string] $Operation,

    [Parameter(Mandatory)]
    [string] $Input,

    # DSC injects -w automatically when running in what-if mode (whatIfArg: "-w")
    [Alias('WhatIf')]
    [switch] $w
)

Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'

class EnvVarState {
    [string] $name
    [string] $value
    [bool]   $_exist

    EnvVarState([string]$name, [string]$value, [bool]$exist) {
        $this.name   = $name
        $this.value  = $value
        $this._exist = $exist
    }
}

function Get-CurrentState {
    param([string] $Name)

    $current = [System.Environment]::GetEnvironmentVariable($Name, 'User')
    if ($null -ne $current) {
        return [EnvVarState]::new($Name, $current, $true)
    }
    return [EnvVarState]::new($Name, $null, $false)
}

function Invoke-Get {
    param([pscustomobject] $Desired)

    $state = Get-CurrentState -Name $Desired.name
    [ordered]@{
        name   = $state.name
        value  = $state.value
        _exist = $state._exist
    } | ConvertTo-Json -Compress -Depth 5
}

# ---------------------------------------------------------------------------
# Set operation (also handles what-if when $WhatIf is $true)
# ---------------------------------------------------------------------------

function Invoke-Set {
    param(
        [pscustomobject] $Desired,
        [bool]           $WhatIf
    )

    $name         = $Desired.name
    $desiredValue = $Desired.value
    $shouldExist  = if ($null -ne $Desired._exist) { [bool]$Desired._exist } else { $true }
    $current      = Get-CurrentState -Name $name

    $messages = [System.Collections.Generic.List[string]]::new()
    if (-not $shouldExist) {
        if ($current._exist) {
            $messages.Add("Would delete environment variable '$name'")
        }

        if ($WhatIf) {
            $projected = [ordered]@{ name = $name; _exist = $false }
            if ($messages.Count -gt 0) {
                $projected['_metadata'] = @{ whatIf = $messages.ToArray() }
            }
            return ($projected | ConvertTo-Json -Compress -Depth 5)
        }

        # Actual deletion: pass $null to remove the variable
        [System.Environment]::SetEnvironmentVariable($name, $null, 'User')
        return ([ordered]@{ name = $name; _exist = $false } | ConvertTo-Json -Compress -Depth 5)
    }

    if (-not $current._exist) {
        $messages.Add("Would create environment variable '$name' with value '$desiredValue'")
    } elseif ($null -ne $desiredValue -and $current.value -ne $desiredValue) {
        $messages.Add("Would change value from '$($current.value)' to '$desiredValue'")
    }

    if ($WhatIf) {
        $projected = [ordered]@{
            name   = $name
            value  = if ($null -ne $desiredValue) { $desiredValue } else { $current.value }
            _exist = $true
        }
        if ($messages.Count -gt 0) {
            $projected['_metadata'] = @{ whatIf = $messages.ToArray() }
        }
        return ($projected | ConvertTo-Json -Compress -Depth 5)
    }

    # Actual set
    if ($null -ne $desiredValue) {
        [System.Environment]::SetEnvironmentVariable($name, $desiredValue, 'User')
    }

    # Return final state after applying changes
    $final = Get-CurrentState -Name $name
    [ordered]@{
        name   = $final.name
        value  = $final.value
        _exist = $final._exist
    } | ConvertTo-Json -Compress -Depth 5
}

# ---------------------------------------------------------------------------
# Entry point
# ---------------------------------------------------------------------------

$desired = $Input | ConvertFrom-Json

switch ($Operation) {
    'get' { Invoke-Get -Desired $desired }
    'set' { Invoke-Set -Desired $desired -WhatIf $w.IsPresent }
}

If the user now runs dsc resource set --what-if -r MyCompany/EnvVar --input '{"name":"MY_VAR","value":"hello"}', DSC is going to read the manifest. It will find the { "whatIfArg": "-w" } in set.args. The -w is inserted because the execution type is WhatIf ending in something like:

pwsh -NonInteractive -NoProfile -ExecutionPolicy Bypass -File MyEnvVar.ps1 set --input '...' -w

The resource script itself receives the -w, so it passes by $w.IsPresent. The script is going to grab what would change, and return the projected JSON without actually calling SetEnvironmentVariable. The returned output would look something like this:

{
  "name": "MY_VAR",
  "value": "hello",
  "_exist": true,
  "_metadata": {
    "whatIf": [
      "Would create environment variable 'MY_VAR' with value 'hello'"
    ]
  }
}

There are more rules when you define stateAndDiff in the resource manifest, but this is roughly the idea about whatIfReturns field.

Code gremlins with arrows

DSC already supports defining ARM functions in configuration documents. They borrowed them and implemented them in the engine. What's new is that there have been two functions introduced that have these weird arrows in them. These are known as lambda expressions.

The first one will be map(), which takes two arguments (inputArray, lambda function). This function can be helpful where you have, for example, a list of short usernames and need to produce the matching corporate email address for each one of them:

$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json
parameters:
  usernames:
    type: array
    description: Short usernames to convert into corporate email addresses.
    defaultValue:
      - Gael
      - Thomas
      - Gijs
resources:
  - name: 
    type: Microsoft.DSC.Debug/Echo
    properties:
      output: >-
        [map(
          parameters('usernames'),
          lambda('name',
            concat(lambdaVariables('name'), '@themembersofdscworkinggroup.com')
          )
        )]

The expected output is an array outputting: ["Gael@themembersofdscworkinggroup.com", "Thomas@themembersofdscworkinggroup.com", "Gijs@themembersofdscworkinggroup.com"].

The second one is an interesting one: filter(). It's already in the wording, but filters an array with a custom filtering function. It accepts the same number of arguments as map() does.

Imagine that you've got a bunch of well-known system ports, say under 1024. Those are all managed by the operating system, but you also have application ports (greater than 1024) that DSC should configure. The filter() function trims the list down to only the ports that DSC owns in the following example:

$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json
parameters:
  allPorts:
    type: array
    description: >-
      All TCP ports referenced in this deployment. Includes both well-known
      system ports managed by the OS and application ports managed by DSC.
    defaultValue:
      - 22     # SSH  — system-managed
      - 80     # HTTP — system-managed
      - 443    # HTTPS — system-managed
      - 3000   # Node.js dev server — DSC-managed
      - 8080   # Application HTTP — DSC-managed
      - 8443   # Application HTTPS — DSC-managed
resources:
  - name: Show only application tier ports
    type: Microsoft.DSC.Debug/Echo
    properties:
      output:
        appPorts: >-
          [filter(
            parameters('allPorts'),
            lambda('port',
              greaterOrEquals(lambdaVariables('port'), 1024)
            )
          )]

Use those functions wisely.

DSC speaks more languages than most developers

And I wasn't talking about the DSC resources themselves. No, we know that DSC configuration documents can be created in Bicep, YAML, and JSON. So why was that hypothetical TOML configuration added in the announcement?

DSC's engine is extensible. And with extensible, I mean that you can extend DSC beyond its original borders as far as it originally speaks the language it needs to (which, spoiler alert, is JSON). To bring that hypothetical example alive, it's possible to create an import extension manifest that performs this transformation. The only problem is: how can it properly read TOML to JSON?

Luckily, there's already a PowerShell module (PSToml) available on the PowerShell Gallery that does such a trick. This module contains a command called ConvertFrom-Toml. Combine this with the ConvertTo-Json and it's possible to create two files:

# convert-toml.ps1
[CmdletBinding()]
param(
    [Parameter(Mandatory)]
    [string]$Path
)

$ErrorActionPreference = 'Stop'

Import-Module PSToml -ErrorAction Stop

ConvertFrom-Toml -InputObject (Get-Content -Path $Path -Raw) | ConvertTo-Json -Depth 10 -Compress

With the relevant extension manifest:

// tomltodsc.dsc.extension.json
{
  "$schema": "https://aka.ms/dsc/schemas/v3/bundled/extension/manifest.json",
  "type": "DSC.Community/TomlDocument",
  "version": "0.1.0",
  "description": "Converts TOML configuration files to valid DSC configuration documents.",
  "import": {
    "fileExtensions": ["toml"],
    "executable": "pwsh",
    "args": [
      "-NoLogo",
      "-NonInteractive",
      "-NoProfile",
      "-Command",
      "./convert-toml.ps1",
      {
        "fileArg": "-Path"
      }
    ]
  }
}

You can now run the following winget.dsc.config.toml perfectly fine against DSC's engine:

"$schema" = "https://aka.ms/dsc/schemas/v3/bundled/config/document.json"
[[resources]]
name = "PowerShell 7 Preview"
type = "Microsoft.WinGet/Package"
[resources.properties]
id = "Microsoft.PowerShell.Preview"

From hypothetical to real. You can insert any format into DSC if it speaks the language.

Conclusion

The list was big. Big enough to cover another article on it. But I wanted to take a couple out, as I thought they were worth more time and a closer look. The hypothetical example was taken and brought to life with the help of PSToml and the extensibility model of DSC's engine.

For other features that haven't been covered, you're in luck! Some of them I already covered here on my blog:

• The PowerShell discovery extension and adapted resource manifest
• Bicep integration via gRPC