How to execute Pester tests in parallel with Pester 6.0

Pester v6.0.0-rc.1 includes one of the most requested test-runner features: an experimental file-level parallel execution.

The work itself was landed by the maintainer Jakub Jareš himself, and the shape of the feature is simple from the user's point of view. If you just have a folder full of independent *.Tests.ps1 files and you run on PowerShell 7 or newer, you can enable Run.Parallel and let Pester run those files in separate runspaces.

That last part of the sentence contains the most important detail. Parallelism happens at the file level. Pester doesn't really split individual It blocks across workers. Instead, each test file becomes the unit of work. That's why this feature is a strong fit for test suites that already have many focused files, such as in my case, command-line integration tests, where each file exercises a separate command area.

For this blog post walkthrough, I used a small slice of the copied Microsoft DSC tests. They are a useful real-world example because many of them spend time outside Pester itself, invoking dsc.exe executable and parsing JSON output. But they add up when running one file after another. That is exactly where file-level parallel execution can help.

Let's take a look at this new experimental feature.

What has changed

Prior to v6.0.0, a Pester run effectively discovered a batch of files and then ran them sequentially in one flow.

The pull request that has been merged in v6.0.0-rc.1 changes how Pester works through test files. Instead of treating discovery and execution as two separate phases for the whole run, Pester now works file by file: it discovers one file, runs it, and then moves to the next. Sequential and parallel runs both use this same flow, so the console output and IDE integrations can behave the same way in both modes.

When you build up a configuration object, the Run.Parallel = $true can be set and Pester uses PowerShell 7's Foreach-Object -Parallel behind the scenes. Each file runs in its own runspace. The worker runspace stays quiet and records the per-container and per-test events. The parent runspace then replays those events in discovery order so the final result object, console summary, and adapter events still look like one normal Pester run.

The first configuration flag is the one you will use most often when you want parallelization:

$configuration = [PesterConfiguration]::Default
$configuration.Run.Path = './dsc/tests'
$configuration.Run.Parallel = $true
$configuration.Run.PassThru = $true

$result = Invoke-Pester -Configuration $configuration

But there are two companion settings worth knowing early:

$configuration.Run.ParallelThrottleLimit = 4
$configuration.Run.BeforeContainer = { . ./test-setup.ps1 }

If you already know Foreach-Object -Parallel, you'll know that ParallelThrottleLimit caps how many files to run at once. The default value just means how many processor counts you have available (0). It's a good starting point for CPU-bound tests, but command-line integration tests sometimes need a lower number because of other hardware-related bottlenecks (think about disk, process startup, network, etc).

The other setting, BeforeContainer is another setting you can use for running a setup before each file is discovered and run. This matters more in parallel mode because each worker starts from a clean runspace. If your current suite relies on functions imported into the parent session before calling Invoke-Pester, move that setup into Run.BeforeContainer or a Pester.BeforeContainer.ps1 file in the repository root.

Benchmarking parallel execution

To make the feature easy to evaluate, I used a small benchmark test that runs the same set of files twice. Once sequentailly and once with the Parallel = $true set. For me, the goal wasn't to make a universal speed number when this was turned on. Instead, I wanted to answer two practical questions:

1. Do the results count the same?
2. Does the shape of the test suite benefit from parallel workers?

The benchmark I took was a small slice of Microsoft's DSC test suite:

Most of these tests are self-contained and used inline YAML configuration, so I added them in an array to test out:

$DscParallelBenchmarkFiles = @(
    'dsc_version.tests.ps1',
    'dsc_output.tests.ps1',
    'dsc_lambda.tests.ps1',
    'dsc_resource_condition.tests.ps1'
)

The benchmark script (which you'll see in a second) keeps it simple. It just compares sequential versus parallel and counts the results. That was also a safety check: a faster run isn't useful if it changes the meaning of the suite.

I called the script dsc-parallel-speed.tests.ps1 and run it as followed:

Invoke-Pester C:\source\Pester\dsc\dsc-parallel-speed.tests.ps1 -Output Detailed

The script itself contained the following code, with the important difference between the two runs is the Run.Parallel value:

Describe 'DSC tests under Pester file-level parallel execution' -Skip:($PSVersionTable.PSVersion.Major -lt 7 -or $null -eq (Get-Command dsc -ErrorAction SilentlyContinue)) {
    BeforeAll {
        $dscParallelBenchmarkFiles = @(
            'dsc_version.tests.ps1',
            'dsc_output.tests.ps1',
            'dsc_lambda.tests.ps1',
            'dsc_resource_condition.tests.ps1'
        )

        $testRoot = Join-Path $PSScriptRoot 'tests'
        $script:DscBenchmarkPaths = foreach ($fileName in $dscParallelBenchmarkFiles) {
            $path = Join-Path $testRoot $fileName
            if (-not (Test-Path -LiteralPath $path -PathType Leaf)) {
                throw "Benchmark file was not found: $path"
            }

            $path
        }

        function Invoke-DscPesterBenchmark {
            [CmdletBinding()]
            param(
                [switch] $Parallel
            )

            $configuration = [PesterConfiguration]::Default
            $configuration.Run.Path = $script:DscBenchmarkPaths
            $configuration.Run.PassThru = $true
            $configuration.Run.Parallel = [bool] $Parallel
            $configuration.Run.ParallelThrottleLimit = [Math]::Min([Environment]::ProcessorCount, 8)
            $configuration.Output.Verbosity = 'None'

            $watch = [System.Diagnostics.Stopwatch]::StartNew()
            $result = Invoke-Pester -Configuration $configuration
            $watch.Stop()

            [pscustomobject]@{
                Mode    = if ($Parallel) { 'Parallel' } else { 'Sequential' }
                Seconds = [Math]::Round($watch.Elapsed.TotalSeconds, 2)
                Total   = $result.TotalCount
                Passed  = $result.PassedCount
                Failed  = $result.FailedCount
                Skipped = $result.SkippedCount
                Result  = $result
            }
        }
    }

    It 'keeps DSC result counts identical while reporting benchmark timings' {
        $sequential = Invoke-DscPesterBenchmark
        $parallel = Invoke-DscPesterBenchmark -Parallel

        $parallel.Total | Should -Be $sequential.Total
        $parallel.Passed | Should -Be $sequential.Passed
        $parallel.Failed | Should -Be $sequential.Failed
        $parallel.Skipped | Should -Be $sequential.Skipped

        $speedup = if ($parallel.Seconds -gt 0) {
            '{0:n2}x' -f ($sequential.Seconds / $parallel.Seconds)
        }
        else {
            'n/a'
        }

        $summary = @(
            [pscustomobject]@{
                Mode    = $sequential.Mode
                Seconds = $sequential.Seconds
                Speedup = '1.00x'
                Total   = $sequential.Total
                Passed  = $sequential.Passed
                Failed  = $sequential.Failed
                Skipped = $sequential.Skipped
            }
            [pscustomobject]@{
                Mode    = $parallel.Mode
                Seconds = $parallel.Seconds
                Speedup = $speedup
                Total   = $parallel.Total
                Passed  = $parallel.Passed
                Failed  = $parallel.Failed
                Skipped = $parallel.Skipped
            }
        )

        Write-Host ''
        Write-Host 'DSC Pester parallel benchmark'
        Write-Host ($summary | Format-Table -AutoSize | Out-String)
    }
}

After the run, the timing table is displayed in the console:

I think the best way to treat these numbers is as a local signal, not as a promise that it will always be faster. A suite with ten very short files usually benefits less than a suite with fifty medium-cost files.

Making a suite parallel-friendly

Each new feature has its caveats, and that was revealed while I was running those DSC files. First of all, the TestDrive is a perfect fit for parallel mode because Pester creates isolated test-drive state for the file that is running. Local variables, BeforeAll, and AfterAll inside a file also stay with that file's worker.

But having things process-wide is different. Environment variables, the current process, external services, shared (temp) paths, and global settings can easily collide. Before enabling parallel execution on those, look for tests that modify this kind of thing. A rough shape of a test file that should make you pause before putting it in the parallel batch:

Describe 'My command-line tool' {
    BeforeAll {
        $script:OriginalPath = $env:PATH
        $env:PATH = "$PSScriptRoot/tools;$env:PATH"
    }

    AfterAll {
        $env:PATH = $script:OriginalPath
    }

    It 'finds the tool on PATH' {
        my-tool --version | Should -Match '1.2.3'
    }
}

That test may be reasonable on its own, but in a parallel run, another file can observe the modified PATH while this file is still running. Prefer just passing explicit paths or use isolated temp folders and output files. Or keep tests like this out of the parallel set until the shared state is removed:

#pester:no-parallel

The above is a directive that can appear as a comment anywhere in a test file. Pester will detect it with the PowerShell tokenizer, so a string containing the same text does not opt the file out. Other fallbacks around parallel execution are:

• It requires PowerShell 7 or newer
• Only applies to file-based runs through Run.Path
• Scriptblock containers fall back to sequential
• Code coverage falls back to sequential (for now)

And lastly, the Run.SkipRemainingOnFailure = 'Run' is also sequential because stopping the entire run after the first failure cannot be coordinated across isolated workers yet.

Takeaway

Testing out this feature myself already gave me a glimpse of the nice part of this experimental change; you don't have to rewrite your tests to use this. You can start with an independent folder containing *.Tests.ps1 files, run it sequentially, and then turn on Run.Parallel to compare the results. If the results are different in the timing table, you know you can switch.

However, if a file depends on shared process state, you can either fix the isolation or just opt the file out. For suites that already have clean (file) boundaries, the payoff can be immediate: the same tests, the same result object, and less time waiting for the final summary.