Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Cobertura coverage format #2298

Open
wants to merge 32 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
932d9b4
wip: Implement Cobertura coverage format
joeskeen Feb 10, 2023
36af738
fix unit test
joeskeen Feb 10, 2023
14e36af
revert editor settings change
joeskeen Feb 10, 2023
658b358
remove ?? operator
joeskeen Feb 10, 2023
1f6525b
fix attribute ordering
joeskeen Feb 10, 2023
a2e5eb0
make coverage report test work on all platforms
joeskeen Feb 10, 2023
61eefc1
Fix windows paths
joeskeen Feb 10, 2023
6c3db56
fix unit test for Windows paths
joeskeen Feb 10, 2023
3b6e06b
kick the build
joeskeen Feb 10, 2023
14e8ef7
re-implement Cobertura coverage report generation
joeskeen Feb 14, 2023
ab0fa57
fix compatibility issues
joeskeen Feb 14, 2023
70ed47a
fix tests
joeskeen Feb 14, 2023
386cd3c
removing Cobertura from v4 parameter options
joeskeen Feb 14, 2023
f0ac943
fix compatibility with ReportGenerator
joeskeen Feb 14, 2023
a272120
Update src/functions/Coverage.ps1
joeskeen Feb 14, 2023
f9ed4a0
fix whitespace
joeskeen Feb 14, 2023
c20f260
Merge branch 'Cobertura' of github.com:joeskeen/Pester into Cobertura
joeskeen Feb 14, 2023
6a1ddc3
fix output
joeskeen Feb 14, 2023
9d78186
fix windows paths
joeskeen Feb 14, 2023
0bd5244
order packages,classes,methods by name
joeskeen Feb 14, 2023
719047a
change Cobertura DTD to loose
joeskeen Feb 14, 2023
9116f06
Tune coverage report for performance
joeskeen Feb 15, 2023
45bdc68
Merge remote-tracking branch 'upstream/main' into pr/joeskeen/2298
fflaten Jul 10, 2024
13c13a7
Remove outdated condition
fflaten Jul 10, 2024
505afef
Merge remote-tracking branch 'upstream/main' into pr/joeskeen/2298
fflaten Jul 12, 2024
a12b1c7
Add Cobertura DTD file
fflaten Jul 12, 2024
13f0b75
Apply suggestions from code review
fflaten Jul 12, 2024
b1f7f2e
Fix typo and update JaCoCo starttime
fflaten Jul 12, 2024
c643f5e
Fix tests
fflaten Jul 12, 2024
80f2120
Use epoch time for Cobertura and JaCoCo
fflaten Jul 12, 2024
58590c9
Merge remote-tracking branch 'upstream/main' into pr/joeskeen/2298
fflaten Oct 15, 2024
98b7881
Update test
fflaten Oct 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions src/Main.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -382,6 +382,7 @@ function Invoke-Pester {
Default value is: JaCoCo.
Currently supported formats are:
- JaCoCo - this XML file format is compatible with Azure Devops, VSTS/TFS
- Cobertura - this XML file format is compatible with Azure Devops, VSTS/TFS
joeskeen marked this conversation as resolved.
Show resolved Hide resolved

The ReportGenerator tool can be used to consolidate multiple reports and provide code coverage reporting.
https://github.com/danielpalme/ReportGenerator
Expand Down Expand Up @@ -1154,10 +1155,13 @@ function Invoke-Pester {
$configuration = $run.PluginConfiguration.Coverage

if ("JaCoCo" -eq $configuration.OutputFormat -or "CoverageGutters" -eq $configuration.OutputFormat) {
[xml] $jaCoCoReport = [xml] (Get-JaCoCoReportXml -CommandCoverage $breakpoints -TotalMilliseconds $totalMilliseconds -CoverageReport $coverageReport -Format $configuration.OutputFormat)
[xml] $coverageXmlReport = [xml] (Get-JaCoCoReportXml -CommandCoverage $breakpoints -TotalMilliseconds $totalMilliseconds -CoverageReport $coverageReport -Format $configuration.OutputFormat)
}
elseif ("Cobertura" -eq $configuration.OutputFormat) {
[xml] $coverageXmlReport = [xml] (Get-CoberturaReportXml -CommandCoverage $breakpoints -TotalMilliseconds $totalMilliseconds -CoverageReport $coverageReport)
}
else {
throw "CodeCoverage.CoverageFormat must be 'JaCoCo' or 'CoverageGutters', but it was $($configuration.OutputFormat), please review your configuration."
throw "CodeCoverage.CoverageFormat must be 'JaCoCo', 'CoverageGutters', or 'Cobertura' but it was $($configuration.OutputFormat), please review your configuration."
}

$settings = [Xml.XmlWriterSettings] @{
Expand All @@ -1172,7 +1176,7 @@ function Invoke-Pester {
$stringWriter = [Pester.Factory]::CreateStringWriter()
$xmlWriter = [Xml.XmlWriter]::Create($stringWriter, $settings)

$jaCocoReport.WriteContentTo($xmlWriter)
$coverageXmlReport.WriteContentTo($xmlWriter)

$xmlWriter.Flush()
$stringWriter.Flush()
Expand Down
2 changes: 1 addition & 1 deletion src/Pester.RSpec.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -335,7 +335,7 @@ function New-PesterConfiguration {
Enabled: Enable CodeCoverage.
Default value: $false

OutputFormat: Format to use for code coverage report. Possible values: JaCoCo, CoverageGutters
OutputFormat: Format to use for code coverage report. Possible values: JaCoCo, CoverageGutters, Cobertura
Default value: 'JaCoCo'

OutputPath: Path relative to the current directory where code coverage report is saved.
Expand Down
2 changes: 1 addition & 1 deletion src/csharp/Pester/CodeCoverageConfiguration.cs
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ public static CodeCoverageConfiguration ShallowClone(CodeCoverageConfiguration c
public CodeCoverageConfiguration() : base("CodeCoverage configuration.")
{
Enabled = new BoolOption("Enable CodeCoverage.", false);
OutputFormat = new StringOption("Format to use for code coverage report. Possible values: JaCoCo, CoverageGutters", "JaCoCo");
OutputFormat = new StringOption("Format to use for code coverage report. Possible values: JaCoCo, CoverageGutters, Cobertura", "JaCoCo");
OutputPath = new StringOption("Path relative to the current directory where code coverage report is saved.", "coverage.xml");
OutputEncoding = new StringOption("Encoding of the output file.", "UTF8");
Path = new StringArrayOption("Directories or files to be used for code coverage, by default the Path(s) from general settings are used, unless overridden here.", new string[0]);
Expand Down
232 changes: 228 additions & 4 deletions src/functions/Coverage.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -1043,6 +1043,221 @@ function Get-JaCoCoReportXml {
return $xml
}

function Get-CoberturaReportXml {
param (
[parameter(Mandatory = $true)]
$CommandCoverage,
[parameter(Mandatory = $true)]
[object] $CoverageReport,
[parameter(Mandatory = $true)]
[long] $TotalMilliseconds
)

if ($null -eq $CoverageReport -or ($pester.Show -eq [Pester.OutputTypes]::None) -or $CoverageReport.NumberOfCommandsAnalyzed -eq 0) {
return [string]::Empty
}

$now = & $SafeCommands['Get-Date']
$nineteenSeventy = & $SafeCommands['Get-Date'] -Date "01/01/1970"
[long] $endTime = [math]::Floor((New-TimeSpan -start $nineteenSeventy -end $now).TotalMilliseconds)
[long] $startTime = [math]::Floor($endTime - $TotalMilliseconds)
fflaten marked this conversation as resolved.
Show resolved Hide resolved

$folderGroups = $CommandCoverage | & $SafeCommands["Group-Object"] -Property {
& $SafeCommands["Split-Path"] $_.File -Parent
}

$packageList = [System.Collections.Generic.List[psobject]]@()

$report = @{
Instruction = @{ Missed = 0; Covered = 0; Total = 0 }
Line = @{ Missed = 0; Covered = 0; Total = 0 }
Method = @{ Missed = 0; Covered = 0; Total = 0 }
Class = @{ Missed = 0; Covered = 0; Total = 0 }
}

foreach ($folderGroup in $folderGroups) {

$package = @{
Name = $folderGroup.Name
Classes = [ordered] @{ }
Instruction = @{ Missed = 0; Covered = 0; Total = 0 }
Line = @{ Missed = 0; Covered = 0; Total = 0 }
Method = @{ Missed = 0; Covered = 0; Total = 0 }
Class = @{ Missed = 0; Covered = 0; Total = 0 }
}

foreach ($command in $folderGroup.Group) {
$file = $command.File
$function = $command.Function
if (!$function) { $function = '<script>' }
joeskeen marked this conversation as resolved.
Show resolved Hide resolved
$line = $command.StartLine.ToString()

$missed = if ($command.Breakpoint.HitCount) { 0 } else { 1 }
$covered = if ($command.Breakpoint.HitCount) { 1 } else { 0 }
$total = 1

if (!$package.Classes.Contains($file)) {
$package.Class.Missed += $missed
$package.Class.Covered += $covered
$package.Class.Total += $total
$package.Classes.$file = @{
Methods = [ordered] @{ }
Lines = [ordered] @{ }
Instruction = @{ Missed = 0; Covered = 0; Total = 0 }
Line = @{ Missed = 0; Covered = 0; Total = 0 }
Method = @{ Missed = 0; Covered = 0; Total = 0 }
Class = @{ Missed = $missed; Covered = $covered; Total = $total }
}
}

if (!$package.Classes.$file.Methods.Contains($function)) {
$package.Method.Missed += $missed
$package.Method.Covered += $covered
$package.Method.Total += $total
$package.Classes.$file.Method.Missed += $missed
$package.Classes.$file.Method.Covered += $covered
$package.Classes.$file.Method.Total += $total
$package.Classes.$file.Methods.$function = @{
FirstLine = $line
Instruction = @{ Missed = 0; Covered = 0; Total = 0 }
Line = @{ Missed = 0; Covered = 0; Total = 0 }
Method = @{ Missed = $missed; Covered = $covered; Total = $total }
}
}

if (!$package.Classes.$file.Lines.Contains($line)) {
$package.Line.Missed += $missed
$package.Line.Covered += $covered
$package.Line.Total += $total
$package.Classes.$file.Line.Missed += $missed
$package.Classes.$file.Line.Covered += $covered
$package.Classes.$file.Line.Total += $total
$package.Classes.$file.Methods.$function.Line.Missed += $missed
$package.Classes.$file.Methods.$function.Line.Covered += $covered
$package.Classes.$file.Methods.$function.Line.Total += $total
$package.Classes.$file.Lines.$line = @{
Instruction = @{ Missed = 0; Covered = 0; Total = 0 }
}
}

$package.Instruction.Missed += $missed
$package.Instruction.Covered += $covered
$package.Instruction.Total += $total
$package.Classes.$file.Instruction.Missed += $missed
$package.Classes.$file.Instruction.Covered += $covered
$package.Classes.$file.Instruction.Total += $total
$package.Classes.$file.Methods.$function.Instruction.Missed += $missed
$package.Classes.$file.Methods.$function.Instruction.Covered += $covered
$package.Classes.$file.Methods.$function.Instruction.Total += $total
$package.Classes.$file.Lines.$line.Instruction.Missed += $missed
$package.Classes.$file.Lines.$line.Instruction.Covered += $covered
$package.Classes.$file.Lines.$line.Instruction.Total += $total
}

$report.Class.Missed += $package.Class.Missed
$report.Class.Covered += $package.Class.Covered
$report.Class.Total += $package.Class.Total
$report.Method.Missed += $package.Method.Missed
$report.Method.Covered += $package.Method.Covered
$report.Method.Total += $package.Method.Total
$report.Line.Missed += $package.Line.Missed
$report.Line.Covered += $package.Line.Covered
$report.Line.Total += $package.Line.Total
$report.Instruction.Missed += $package.Instruction.Missed
$report.Instruction.Covered += $package.Instruction.Covered
$report.Instruction.Total += $package.Instruction.Total

$packageList.Add($package)
}

$commonParent = Get-CommonParentPath -Path $CoverageReport.AnalyzedFiles
$commonParentLeaf = & $SafeCommands["Split-Path"] $commonParent -Leaf
Fixed Show fixed Hide fixed

# the Cobertura xml format without the doctype, as the XML stuff does not like DTD's.
$xmlDeclaration = '<?xml version="1.0" encoding="utf-8"?>'
$coberturaReport = $xmlDeclaration
$coberturaReport += '<coverage>'
$coberturaReport += "<sources><source>$commonParent</source></sources>"
$coberturaReport += '</coverage>'

[xml] $coberturaReportXml = $coberturaReport

$coverageElement = $coberturaReportXml.coverage
Add-XmlAttribute -Element $coverageElement -Attributes @{
'lines-valid' = $report.Line.Total
'lines-covered' = $report.Line.Covered
'line-rate' = if ($report.Line.Total) { $report.Line.Missed / $report.Line.Total} else { 0 }
# TODO: branch coverage
'branches-valid' = 0
'branches-covered' = 0
'branch-rate' = 1
timestamp = $startTime
complexity = 0
version = 0.1
}
joeskeen marked this conversation as resolved.
Show resolved Hide resolved

$packagesElement = Add-XmlElement -Parent $coverageElement -Name 'packages'

foreach ($package in $packageList) {
$packageRelativePath = Get-RelativePath -Path $package.Name -RelativeTo $commonParent

# "" in root and "sub-dir" elsewhere
$packageName = if ($null -eq $packageRelativePath -or "" -eq $packageRelativePath) {
""
}
else {
$packageRelativePathFormatted = $packageRelativePath.Replace("\", "/")
$packageRelativePathFormatted
}

$packageElement = Add-XmlElement -Parent $packagesElement -Name 'package' -Attributes @{
name = ($packageName -replace "/$", "")
'line-rate' = if ($package.Line.Total) { $package.Line.Missed / $package.Line.Total} else { 0 }
'branch-rate' = 1
}
$classesElement = Add-XmlElement -Parent $packageElement -Name 'classes'

foreach ($file in $package.Classes.Keys) {
$class = $package.Classes.$file
$classElementRelativePath = (Get-RelativePath -Path $file -RelativeTo $commonParent).Replace("\", "/")
$classElementName = "$classElementRelativePath"
$classElementName = $classElementName.Substring(0, $($classElementName.LastIndexOf(".")))
$classElement = Add-XmlElement -Parent $classesElement -Name 'class' -Attributes ([ordered] @{
name = $classElementName
filename = $classElementRelativePath
'line-rate' = if ($class.Line.Total) { $class.Line.Missed / $class.Line.Total} else { 0 }
'branch-rate' = 1
})
$methodsElement = Add-XmlElement -Parent $classElement -Name 'methods'

joeskeen marked this conversation as resolved.
Show resolved Hide resolved
foreach ($function in $class.Methods.Keys) {
$method = $class.Methods.$function
$methodElement = Add-XmlElement -Parent $methodsElement -Name 'method' -Attributes ([ordered] @{
name = $function
hits = $method.Method.Covered
signature = '()'
})

$linesElement = Add-XmlElement -Parent $methodElement -Name 'lines'

foreach ($line in $class.Lines.Keys) {
$null = Add-XmlElement -Parent $linesElement -Name 'line' -Attributes ([ordered] @{
number = $line
hits = $class.Lines.$line.Instruction.Covered
})
}
}

}
}

# There is no pretty way to insert the Doctype, as microsoft has deprecated the DTD stuff.
$coberturaReportDocType = '<!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd"[]>'
joeskeen marked this conversation as resolved.
Show resolved Hide resolved
$xml = $coberturaReportXml.OuterXml.Insert($xmlDeclaration.Length, $coberturaReportDocType)

return $xml
}

function Add-XmlElement {
param (
[parameter(Mandatory = $true)] [System.Xml.XmlNode] $Parent,
Expand All @@ -1051,14 +1266,23 @@ function Add-XmlElement {
)
$element = $Parent.AppendChild($Parent.OwnerDocument.CreateElement($Name))
if ($Attributes) {
foreach ($key in $Attributes.Keys) {
$attribute = $element.Attributes.Append($Parent.OwnerDocument.CreateAttribute($key))
$attribute.Value = $Attributes.$key
}
Add-XmlAttribute -Element $element -Attributes $Attributes
}
return $element
}

function Add-XmlAttribute {
param(
[parameter(Mandatory = $true)] [System.Xml.XmlNode] $Element,
[parameter(Mandatory = $true)] [System.Collections.IDictionary] $Attributes
)

foreach ($key in $Attributes.Keys) {
$attribute = $Element.Attributes.Append($Element.OwnerDocument.CreateAttribute($key))
$attribute.Value = $Attributes.$key
}
}

function Add-JaCoCoCounter {
param (
[parameter(Mandatory = $true)] [ValidateSet('Instruction', 'Line', 'Method', 'Class')] [string] $Type,
Expand Down
Loading