This is a community project that is not supported by Microsoft.
❗ This version of PBI Inspector only supports the new enhanced metadata file format (PBIR), see https://learn.microsoft.com/en-gb/power-bi/developer/projects/projects-report. For the older PBIR-legacy file format, please use the previous version of PBI Inspector available at https://github.com/NatVanG/PBI-Inspector ❗
To support the new enhanced report format (PBIR), a new "part" custom command has been introduced which helps to navigate to or iterate over the new metadata file format's parts such as "Pages", "Visuals", "Bookmarks" etc. Rules defined against the new format are not backward compatible with the older PBIR-legacy format and vice versa.
Thanks to Michael Kovalsky of Semantic Link Labs fame and Rui Romano for their feedback on this project. Thanks also to Luke Young for creating the PBI Inspector logo.
Special thanks also to David Mitchell for his unwavering support and advocacy of PBI Inspector. Check out David's Microsoft blog post and tooling for automating the migration of workspaces from Power BI Premium to Microsoft Fabric capacities.
Please report issues here.
- Intro
- Releases
- Base rules
- Run from the Graphical user interface
- Run from the Command line
- Interpreting results
- Azure DevOps integration
- Running reports on reports
- Custom rules guide
- Patching
- Examples
- Wiki
- Known issues
- Report an issue
So we've DevOps, MLOps and DataOps... but why not VisOps? How can we ensure that business intelligence charts and other visuals within report pages are published in a consistent, performance optimised and accessible state? For example, are local report settings set in a consistent manner for a consistent user experience? Are visuals deviating from the specified theme by, say, using custom colours? Are visuals kept lean so they render quickly? Are charts axes titles displayed? etc.
With Microsoft Power BI, visuals are placed on a canvas and formatted as desired, images may be included and theme files referenced. Testing the consistency of the visuals output has thus far typically been a manual process. The Power BI Project file format (.pbip) was introduced then more recently enhanced to enable pro developer application lifecycle management and source control also known as CI/CD. PBI Inspector V2 contributes to CI/CD for Power BI reports by providing the ability to define fully configurable testing rules written in json. PBI Inspector V2 is powered by Greg Dennis's Json Logic .NET implementation, see https://json-everything.net/json-logic.
See releases for the Windows application and Command Line interface (CLI) at: https://github.com/NatVanG/PBI-InspectorV2/releases
While PBI Inspector V2 supports custom rules, it also includes the following base rules defined at https://github.com/NatVanG/PBI-InspectorV2/blob/part-concept/Rules/Base-rules.json, some rules allow for user parameters:
- Remove custom visuals which are not used in the report (no user parameters)
- Reduce the number of visible visuals on the page (set parameter
paramMaxVisualsPerPageto the maximum number of allowed visible visuals on the page) - Reduce the number of objects within visuals (override hardcoded
6parameter value the maximum number of allowed objects per visuals) - Reduce usage of TopN filtering visuals by page (set
paramMaxTopNFilteringPerPage) - Reduce usage of Advanced filtering visuals by page (set
paramMaxAdvancedFilteringVisualsPerPage) - Reduce number of pages per report (override hardcoded
10parameter value the maximum number of allowed pages per report) - Avoid setting ‘Show items with no data’ on columns (no user parameters)
- Tooltip and Drillthrough pages should be hidden (no user parameters)
- Ensure charts use theme colours (no user parameters)
- Ensure pages do not scroll vertically (no user parameters)
- Ensure alternativeText has been defined for all visuals (disabled by default, no user parameters)
To modify parameters, save a local copy of the Base-rules.json file at https://github.com/NatVanG/PBI-InspectorV2/blob/part-concept/Rules/Base-rules.json and point PBI Inspector V2 to the new file.
To disable a rule, edit the rule's json to specify "disabled": true. At runtime PBI Inspector V2 will ignore any disabled rule.
Running PBIRInspectorWinForm.exe presents the user with the following interface:
- Browse to your local PBI Desktop File, either the *.pbip file or its parent folder. 📝 Support for .PBIX file using the enhanced report metadata will be added in a subsequent release.
- Either use the base rules file included in the application or select your own.
- Use the "Browse" button to select an output directory to which the results will be written. Alternatively, select the "Use temp files" check box to write the resuls to a temporary folder that will be deleted upon exiting the application.
- Select output formats, either JSON or HTML or both. To simply view the test results in a formatted page select the HTML output.
- Select "Verbose" to output both test passes and fails, if left unselected then only failed test results will be reported.
- Select "Run". The test run log messages are displayed at the bottom of the window. If "Use temp files" is selected (or the Output directory field is left blank) along with the HTML output check box, then the browser will open to display the HTML results.
- Any test run information, warnings or errors are displayed in the console output textbox.
All command line parameters are as follows:
-pbip filepath: Required. The path to the *.pbip file.
-pbix filepath: Not currently supported.
-rules filepath: Required. The filepath to the rules file. Save a local copy of the Base Rules file and modify as required.
-verbose true|false: Optional, false by default. If false then only rule violations will be shown otherwise all results will be listed.
-output directorypath: Optional. If -formats is set to either JSON, HTML or PNG, writes results to the specified directory, any existing files will be overwritten. If not supplied then a temporary directory will be created in the user's temporary files folder.
-formats CONSOLE,JSON,HTML,PNG,ADO: Optional. Comma-separated list of output formats.
- CONSOLE (default) writes results to the console output. If "-formats" is not specified then "CONSOLE" will be used by default.
- JSON writes results to a Json file.
- HTML writes results to a formatted Html page. If no output directory is specified and the HTML format is specified, then a browser page will be opened to display the HTML results. When specifying "HTML" format, report page wireframe images will be created so there is no need to also include the "PNG" format.
- PNG draws report pages wireframes clearly showing any failing visuals.
- ADO outputs Azure DevOps compatible task commands for use in a deployment pipeline. Task commands issued are "task.logissue" and "task.complete", see https://learn.microsoft.com/en-us/azure/devops/pipelines/scripts/logging-commands?view=azure-devops&tabs=bash#task-commands. PBI Inspector V2 rules definition can be given a "logType" attribute of either "warning" or "error" which will be passed to the Azure DevOps task command as follows:
##vso[task.logissue type=warning|error]. When specifying "ADO" all other output format types will be ignored.
Commmand line examples:
- Run "Base-rules.json" rule definitions against PBI report file at "Sales.Report and return results in Json and HTML formats:
PBIRInspectorCLI.exe -pbip "C:\Files\Sales.pbip" -rules ".\Files\Base-rules.json" -output "C:\Files\TestRun" -formats "JSON,HTML"
- Run "Base-rules.json" rule definitions against PBI report file at "Sales.Report and return results to the console only:
PBIRInspectorCLI.exe -pbip "C:\Files\Sales.pbip" -rules ".\Files\Base-rules.json" -output "C:\Files\TestRun" -formats "Console"
- Run "Base-rules.json" rule definitions against PBI report file at "Sales.Report and return results as Azure DevOps compatible log and tasks commands (see https://learn.microsoft.com/en-us/azure/devops/pipelines/scripts/logging-commands?view=azure-devops&tabs=bash#task-commands):
PBIRInspectorCLI.exe -pbip "C:\Files\Sales.pbip" -rules ".\Files\Base-rules.json" -formats "ADO"
If a verbose output was requested, then results for both test passes and failures will be reported. The JSON output is intended to be consumed by a subsequent process, for example a Power BI report may be created that uses the JSON file as a data source and visualises the PBI Inspector V2 test results. The HTML page is a more readable format for humans which also includes report page wireframe images when tests are at the report page level. These images are intended to help the user identify visuals that have failed the test such as in the example image below. The PBI Inspector V2 logo is also displayed at the centre of each failing visuals as an additional identification aid when the wireframe is busy.
Visuals with a dotted border are visuals hidden by default as the following example:
For a tutorial on how to run PBI Inspector V2 as part of an Azure DevOps pipeline job (alongside Tabular Editor's BPA rules), see https://learn.microsoft.com/en-us/power-bi/developer/projects/projects-build-pipelines.
❗ Please note that to work with PBI Inspector V2 the YAML file referenced in the tutorial needs to be updated as follows ContinuousIntegration-Rules-PBIR.yml.
📝 This is a high-level guide to custom rules for a deeper explanation of rules and operators see the PBI Inspector V2 wiki.
Custom rules are defined in a JSON file as an array of rule objects as follows:
{
"rules": [
...
]
}Each rule object has the following properties:
{
"id": "A unique identifier of your choice for the rule",
"name": "A name that is shown in HTML results with wireframe images.",
"description": "Details to help you and others understand what this rule does",
"disabled": true|false(default),
"part": "Optional. One of Report|Pages|PagesHeader|AllPages|Visuals|AllVisuals|Bookmarks|BookmarksHeader|AllBookmarks. If the part specified is an array with multiple items (such as "Pages"), the rule will apply to each array item in sequence."
"test": [
//test logic
,
//optional data variables
,
//expected result
],
"patch":
[
//optional patch logic to fix the issue
]
}For example (without the optional patch logic), the following rule checks that, for each Page, certain charts have both axes titles displayed. The rule returns the names of failing visuals in an array.
{
"id": "SHOW_AXES_TITLES",
"name": "Show visual axes titles",
"description": "Check that certain charts have both axes title showing, returns an array of visual names that fail the test.",
"part": "Pages",
"disabled": false,
"test": [
{
"map": [
{
"filter": [
{
"part": "Visuals"
},
{
"and": [
{
"in": [
{
"var": "visual.visualType"
},
[
"lineChart",
"barChart",
"columnChart",
"clusteredBarChart",
"stackedBarChart"
]
]
},
{
"or": [
{
"==": [
{
"var": "visual.objects.categoryAxis.0.properties.showAxisTitle.expr.Literal.Value"
},
"false"
]
},
{
"==": [
{
"var": "visual.objects.valueAxis.0.properties.showAxisTitle.expr.Literal.Value"
},
"false"
]
}
]
}
]
}
]
},
{
"var": "name"
}
]
},
{
},
[]
]
}Optionally a rule can now also define a patch to fix items (e.g. visuals) failing the test. For example a patch for the test above is as follows. The patch iterates through the failing visual names returned and fixes the "Visuals" part of the report definition by setting the "showAxisTitle" property to "true" for both the category and value axes:
"patch": [
"Visuals",
[
{
"op": "replace",
"path": "/visual/objects/categoryAxis/0/properties/showAxisTitle/expr/Literal/Value",
"value": "true"
},
{
"op": "replace",
"path": "/visual/objects/valueAxis/0/properties/showAxisTitle/expr/Literal/Value",
"value": "true"
}
]
]A patch definition has the following structure:
"patch": [
"One of Report|Pages|PagesHeader|AllPages|Visuals|AllVisuals|Bookmarks|BookmarksHeader|AllBookmarks",
[patch logic operator array]
]The patch logic operator array is defined as per the JSON Patch specification at https://tools.ietf.org/html/rfc6902. PBI Inspector V2 uses the .NET implementation of JSON Patch, for see https://docs.json-everything.net/patch/basics/.
Therefore the full rule example including the patch is as follows:
{
"id": "SHOW_AXES_TITLES",
"name": "Show visual axes titles",
"description": "Check that certain charts have both axes title showing.",
"part": "Pages",
"disabled": true,
"applyPatch": true,
"test": [
{
"map": [
{
"filter": [
{
"part": "Visuals"
},
{
"and": [
{
"in": [
{
"var": "visual.visualType"
},
[
"lineChart",
"barChart",
"columnChart",
"clusteredBarChart",
"stackedBarChart"
]
]
},
{
"or": [
{
"==": [
{
"var": "visual.objects.categoryAxis.0.properties.showAxisTitle.expr.Literal.Value"
},
"false"
]
},
{
"==": [
{
"var": "visual.objects.valueAxis.0.properties.showAxisTitle.expr.Literal.Value"
},
"false"
]
}
]
}
]
}
]
},
{
"var": "name"
}
]
},
{
},
[]
],
"patch": [
"Visuals",
[
{
"op": "replace",
"path": "/visual/objects/categoryAxis/0/properties/showAxisTitle/expr/Literal/Value",
"value": "true"
},
{
"op": "replace",
"path": "/visual/objects/valueAxis/0/properties/showAxisTitle/expr/Literal/Value",
"value": "true"
}
]
]
}Here's another custom rule with patch example that sets the report's default page. First we test the report's activePageName, if it's incorrect we replace it:
{
"id": "ACTIVE_PAGE",
"name": "Ensure report's active page index is set to the correct page",
"description": "",
"part": "PagesHeader",
"applyPatch": true,
"test": [
{
"var": "activePageName"
},
"ReportSection89a9619c7025093ade1c"
],
"patch": [
"PagesHeader",
[
{
"op": "replace",
"path": "/activePageName",
"value": "ReportSection89a9619c7025093ade1c"
}
]
]
}Although somewhat of an anti-pattern, it is possible to vary a rule's test based on the report name which is retrieved from the .platform file as shown below:
{
"id": "VARY_BY_REPORT_NAME",
"name": "Vary by report name",
"description": "Run rule only if report display name is 'Inventory sample'",
"test": [
{
"?:": [
{
"==": [
{
"query": [
{
"part": ".platform"
},
{
"var": "metadata.displayName"
}
]
},
"Inventory sample"
]
},
"Rule output",
"This is another report."
]
},
"Rule output"
]
}For full rule file examples see:
- Base Rules - The set of rules that ships with PBI Inspector V2
- Example Rules - An ever growing library of example rules
- Example Rules with Patches - Examples of patches to fix issues
- Rules Template - A simple rules file template to get you started with your own rules
For an in-depth understanding of PBI Inspector V2 rules and operators see the PBI Inspector V2 wiki.
- Currently page wireframes are only created in a 16:9 aspect ratio so custom report page sizes including tooltip pages may not render as expected as shown in the following tooltip page example. See tooltip page example below:
- Currently page wireframes do not faithfully represents the report page layout when visual groups are present.
Please report issues at https://github.com/NatVanG/PBI-InspectorV2/issues.