-
-
Notifications
You must be signed in to change notification settings - Fork 473
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Code coverage not listing all missed commands #1465
Comments
https://pester.dev/docs/usage/code-coverage/ : A quick note on the "analyzed commands" numbers. You may have noticed that even though CoverageTest.ps1 is 17 lines long, Pester reports that only 5 commands are being analyzed for coverage. This is a limitation of the current implementation of the coverage analysis, which uses PSBreakpoints to track which commands are executed. Breakpoints can only be triggered by commands in PowerShell, which includes both calls to functions, Cmdlets and programs, as well as expressions and variable assignments. Breakpoints are not triggered by keywords such as else, try, or finally, or on opening or closing braces |
@dlwyatt So, in my test code it is possible to set PSBreakpionts on lines 7 and 12. I've done so in testing. Are the |
Correct, though you can often set a breakpoint on a line with |
@dlwyatt Thanks for the explanation, very helpful! |
@dlwyatt was there maybe some change? Frankly I don't remember how exactly the BPs are setup for code coverage, so I might be trying it incorrectly here: try {
throw
}
catch { }
$bp.HitCount This stops on the BP and reports 1. |
@dlwyatt, @nohwnd I've had a quick look at the source and wonder if the following change would be valid. Function: Coverage.ps1\Get-CommandsInFileLine: 195 before $predicate = {
$args[0] -is [System.Management.Automation.Language.DynamicKeywordStatementAst] -or
$args[0] -is [System.Management.Automation.Language.CommandBaseAst]
} Line: 195 after $predicate = {
$args[0] -is [System.Management.Automation.Language.DynamicKeywordStatementAst] -or
$args[0] -is [System.Management.Automation.Language.CommandBaseAst] -or
$args[0] -is [System.Management.Automation.Language.BreakStatementAst] -or
$args[0] -is [System.Management.Automation.Language.ContinueStatementAst] -or
$args[0] -is [System.Management.Automation.Language.ExitStatementAst] -or
$args[0] -is [System.Management.Automation.Language.ReturnStatementAst] -or
$args[0] -is [System.Management.Automation.Language.ThrowStatementAst]
} I've done some simple tests and it works. I've included statements that can exist by themselves. I may have missed some. I don't know if there are implications to making this change. Updated Test-Function.ps1 filefunction Test-Function {
try {
Out-Null
if ($false) {
break
}
if ($false) {
exit
}
if ($false) {
return
}
}
catch {
Write-Host 'Dummy'
throw
}
foreach ($i in 0..3) {
if ($false) {
continue
}
}
} Results of code coverage:
|
Looks good. Please wait for a bit for confirmation from Dave if there will be any :) Then please PR it with the example that you have as a test. You might need to fix the other tests, and maybe there are going to be some inconsistencies across PowerShell versions. I am prepared to help out with that in the PR. If you are not interested in PRing it, let me know I will mark it as up for grabs. 🙂 |
@nohwnd I'm in the process of testing the code. I've found an issue with the There is also Pester.psm1\Invoke-Pester\Line 1111:
This just executes the tests. It is inside a scriptblock that is called at line: 1137:
After this line is executed the |
This is how it works I think: $sb = {
try {
throw
}
catch { }
return
}
# in Enter-CoverageAnalysis -> New-CoverageBreakpoint
$params = @{
Script = $sb.File
Line = 3
Column = 9
Action = { }
}
$breakpoint = @(
(Set-PSBreakPoint -Script $sb.File -Line 3 -Column 9 -Action { }),
# not hit when we specify column, but hit when we don't specify column
(Set-PSBreakPoint -Script $sb.File -Line 7 -Column 5 -Action { })
)
# this will happen when we execute Describe / Context / It / any covered code
& $sb
$breakpoint[0].HitCount
$breakpoint[1].HitCount
# in Exit-CoverageAnalysis
$breakpoint | Remove-PSBreakpoint |
@nohwnd As you pointed out, when the column is specified the breakpoint is not hit. When it's not specified the breakpoint is hit. So, this looks like a PowerShell issue. Also, I now understand that |
@SteveL-MSFT could you help us understand why the breakpoint on return is not hit, when column is specified, please? Is that by design? |
@nohwnd After a bit of further investigation, based on your sample code:
I added the following to your code and updated the # Not hit
if ($true) {
return
}
# Hit
if ($true) {
return 99
} |
Are you sure the column specified points to a valid location in the file? If the line/column is invalid, you don't get an error and no breakpoint is hit. |
@SteveL-MSFT pretty sure, it hits if the column is not specified, or is before the return. The first breakpoint below is based on the AST extent, the rest are hardcoded, it hits up to 4 but not 5 where return starts.
$sb = {
return
}
$e = $sb.Ast.FindAll({param($t) $t -is [Management.Automation.Language.ReturnStatementAst] }, $true).Extent
# in Enter-CoverageAnalysis -> New-CoverageBreakpoint
$breakpoint = @(
(Set-PSBreakpoint -Script $e.File -Line $e.StartLineNumber -Column $e.StartColumnNumber -Action {}),
(Set-PSBreakPoint -Script $sb.File -Line 2 -Column 1 -Action { }),
(Set-PSBreakPoint -Script $sb.File -Line 2 -Column 2 -Action { }),
(Set-PSBreakPoint -Script $sb.File -Line 2 -Column 3 -Action { }),
(Set-PSBreakPoint -Script $sb.File -Line 2 -Column 4 -Action { }),
(Set-PSBreakPoint -Script $sb.File -Line 2 -Column 5 -Action { }),
(Set-PSBreakPoint -Script $sb.File -Line 2 -Column 6 -Action { }),
(Set-PSBreakPoint -Script $sb.File -Line 2 -Column 7 -Action { }),
(Set-PSBreakPoint -Script $sb.File -Line 2 -Column 8 -Action { }),
(Set-PSBreakPoint -Script $sb.File -Line 2 -Column 9 -Action { }),
(Set-PSBreakPoint -Script $sb.File -Line 2 -Column 11 -Action { }),
(Set-PSBreakPoint -Script $sb.File -Line 2 -Column 12 -Action { }),
(Set-PSBreakPoint -Script $sb.File -Line 2 -Column 13 -Action { }),
(Set-PSBreakPoint -Script $sb.File -Line 2 -Column 14 -Action { }),
(Set-PSBreakPoint -Script $sb.File -Line 2 -Column 15 -Action { }),
(Set-PSBreakPoint -Script $sb.File -Line 2 -Column 16 -Action { })
)
# this will happen when we execute Describe / Context / It / any covered code
& $sb
$breakpoint | select Script, Line, Column, HitCount
# in Exit-CoverageAnalysis
$breakpoint | Remove-PSBreakpoint |
So I was following the example videos for MIcrosoft Virtual Academy, trying to update some of what's written for the latest version of pester, when I did the code-coverage examples, I don't get the missed commands section at all. So I wonder if something changed with how this works for code coverage? Example here: https://docs.microsoft.com/en-us/shows/testing-powershell-with-pester/code-coverage |
Only the summary with percentage etc. is shown by |
1. General summary of the issue
When running code coverage in a file not all
Missed commands
are listed. Test code:Test-Function.ps1
Test-Function.Tests.ps1
Test.ps1
Output
2. Describe Your Environment
3. Expected Behavior
I expected
Missd commands
to include:4. Current Behavior
Missed commands
only includes:The text was updated successfully, but these errors were encountered: