Pester 5 code coverage file is not compatible with Azure Devops

[petertheautomator]

General summary of the issue

The code coverage XML file which is being generated by Pester-5 New-PesterConfiguration isn't compatible with Azure Devops.

Pester Configuration:

$pesterConfiguration = @{
    Run = @{
        Path = @("$rootFolder\Interfaces")
    }
    Should = @{
        ErrorAction = 'Continue'
    }
    CodeCoverage = @{
        OutputFormat = 'JaCoCo'
        OutputEncoding = 'UTF8'
        OutputPath = "$rootFolder\Pester-Coverage.xml"
        Enabled = $true
    }
    TestResult = @{
        OutputPath = "$rootFolder\Pester-Test.xml"
        OutputFormat = 'NUnitXml'
        OutputEncoding = 'UTF8'
        Enabled = $true
    }
}

#Invoke pester with the configuration hashtable
$config = New-PesterConfiguration -Hashtable $pesterConfiguration
Invoke-Pester -Configuration $config

The code coverage XML file that is being generated by the deprecated Pester 4 switches is functioning properly with Azure Devops.

Invoke-Pester -CodeCoverage $NotTestFiles -CodeCoverageOutputFile $OutCoverageFile

Describe your environment

Pester version : 5.3.1 C:\Users<name>\Documents\PowerShell\Modules\Pester\5.3.1\Pester.psm1
PowerShell version : 7.2.2
OS version : Microsoft Windows NT 10.0.17763.0

Steps to reproduce

Use the pester configuration as described above and upload the code coverage into Azure Devops

- task: PublishCodeCoverageResults@1
  inputs:
    codeCoverageTool: 'JaCoCo'
    summaryFileLocation: '**/Pester-Coverage.xml'
    pathToSources: $(System.DefaultWorkingDirectory)
    failIfCoverageEmpty: true

Expected Behavior

Current Behavior

Possible Solution? (optional)

We managed to fix this issue by altering the XML after it is being created.

[xml]$pesterCoverageOut = get-content -path ".\Pester-Coverage.xml"
foreach ($classNode in $pesterCoverageOut.SelectNodes("//class")) {
    $classNode.sourcefilename = "Interfaces/$($classNode.sourcefilename)"
}
foreach ($sourceFileNode in $pesterCoverageOut.SelectNodes("//sourcefile")) {
    $sourceFileNode.name = "Interfaces/$($sourceFileNode.name)"
}
$pesterCoverageOut.Save(".\Pester-Coverage.xml")

[nohwnd]

I tried this, and you are right. For files that are deeper in the directory structure it does not work. There is another format based on JaCoCo available that is called CoverageGutters which works for me just fine in azdo:
I am using your configuration, I just changed the output format to this:

OutputFormat = 'CoverageGutters'

You can also use this for make coverage work in your VSCode, if you use that:

https://youtu.be/qeiy8fRMHf8?t=4590

[johlju]

Hope this helps. We have the paths 5 folders down, and we point from where should start looking for the relative path using pathToSources.

We get the code coverage looking alright:

https://dev.azure.com/dsccommunity/SqlServerDsc/_build/results?buildId=5546&view=codecoverage-tab

See here:

https://github.com/dsccommunity/SqlServerDsc/blob/2f5513ba98bc6fbc2277b0590f13d110b00424cc/azure-pipelines.yml#L245-L250

- task: PublishCodeCoverageResults@1
  displayName: 'Publish Code Coverage to Azure DevOps'
  inputs:
    codeCoverageTool: 'JaCoCo'
    summaryFileLocation: '$(Build.SourcesDirectory)/$(buildFolderName)/$(testResultFolderName)/JaCoCo_coverage.xml'
    pathToSources: '$(Build.SourcesDirectory)/$(sourceFolderName)/'