There is a difference on the code coverage result format

[fedegiust]

Summary

There is an inconsistency for running tests between deployment and running them directly.
When running a deployment there is an option for code coverage formatters which give a number of options, which is great! I like the html format.

But when running apex run test the options are limited

Steps To Reproduce

sf apex run test --test-level RunLocalTests --output-dir temp --synchronous --result-format html --code-coverage

[github-actions[bot]]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[github-actions[bot]]

Hello @fedegiust 👋 It looks like you didn't include the full Salesforce CLI version information in your issue.
Please provide the output of version --verbose --json for the CLI you're using (sf or sfdx).

A few more things to check:

• Make sure you've provided detailed steps to reproduce your issue.
  • A repository that clearly demonstrates the bug is ideal.
• Make sure you've installed the latest version of Salesforce CLI. (docs)
  • Better yet, try the rc or nightly versions. (docs)
• Try running the doctor command to diagnose common issues.
• Search GitHub for existing related issues.

Thank you!

[fedegiust]

In this case it happens across all versions

{
  "architecture": "win32-x64",
  "cliVersion": "@salesforce/cli/2.59.6",
  "nodeVersion": "node-v20.17.0",
  "osVersion": "Windows_NT 10.0.19045",
  "rootPath": "C:\\Users\\Usuario\\AppData\\Local\\sf\\client\\2.59.6-767cea4",
  "shell": "C:\\Program Files\\Git\\usr\\bin\\bash.exe",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 3.2.3 (core)",
    "@oclif/plugin-commands 4.0.14 (core)",
    "@oclif/plugin-help 6.2.11 (core)",
    "@oclif/plugin-not-found 3.2.20 (core)",
    "@oclif/plugin-plugins 5.4.7 (core)",
    "@oclif/plugin-search 1.2.8 (core)",
    "@oclif/plugin-update 4.5.6 (core)",
    "@oclif/plugin-version 2.2.12 (core)",
    "@oclif/plugin-warn-if-update-available 3.1.15 (core)",
    "@oclif/plugin-which 3.2.13 (core)",
    "@salesforce/cli 2.59.6 (core)",
    "apex 3.4.9 (core)",
    "api 1.2.1 (core)",
    "auth 3.6.56 (core)",
    "data 3.6.5 (core)",
    "deploy-retrieve 3.12.7 (core)",
    "info 3.4.6 (core)",
    "limits 3.3.30 (core)",
    "marketplace 1.2.25 (core)",
    "org 4.5.8 (core)",
    "packaging 2.8.4 (core)",
    "schema 3.3.29 (core)",
    "settings 2.3.19 (core)",
    "signups 2.5.14 (user) published 45 days ago (Sat Sep 07 2024) (latest is 2.5.23)",
    "sobject 1.4.35 (core)",
    "source 3.5.19 (core)",
    "telemetry 3.6.12 (core)",
    "templates 56.3.18 (core)",
    "trust 3.7.27 (core)",
    "user 3.5.29 (core)",
    "@salesforce/sfdx-scanner 4.6.0 (user) published 28 days ago (Tue Sep 24 2024)",
    "sfdmu 4.36.3 (user) published 32 days ago (Thu Sep 19 2024)",
    "sfdx-git-delta 5.45.0 (user) published 32 days ago (Fri Sep 20 2024) (latest is 5.49.0)",
    "sfdx-plugin-source-read 1.2.0 (user) published 364 days ago (Tue Oct 24 2023)"
  ]
}

[iowillhoit]

Hey @fedegiust, thanks for sharing this! I am going to convert this into a Github Discussion since it is more of a feature request than a bug report. We have a few other things outline for a plugin-apex revamp that we will hopefully get prioritized for next year.

In the meantime as a work-around, you could use the --dry-run flag on sf project deploy start to get your html code coverage output without actually deploying your source

[github-actions[bot]]

Hello @fedegiust 👋 None of the versions of sf you shared match the latest release.

Shared: 2.59.6
Latest: 2.63.9

Update to the latest version of Salesforce CLI (docs) and confirm that you're still seeing your issue.
You can also try the rc and nightly releases! (docs)

After updating, share the full output of sf version --verbose --json

[mcarvin8]

Something to add to this as well.

The coverage reports for deploy and test commands follow different result formats. If you provide the same result format for both commands (JSON for example), you'll see different coverage report formatting for both commands.

Something I'm accounting for with my sonar plugin - https://github.com/mcarvin8/apex-code-coverage-transformer

If the coverage reports could be standardized between both commands, that would be nice.