quickstart: Model Validation #750

validbeck · 2025-06-11T18:21:53Z

Internal Notes for Reviewers

sc-10572

Quickstart — Model Validation

LIVE PREVIEW

Brand new introductory guide for model validation.

How to test

You can pull this PR down and preview locally, or just click the live preview link above. ;)

Recommended review:

Test the steps to make sure they actually work as outlined. 🚨 NOTE THAT THE JUPYTERHUB LINK IS NOT LIVE YET AND IS CURRENTLY A BROKEN LINK 🚨 — aiming to resolve that before we push the quickstart changes to production. Fix pt1 here: https://github.com/validmind/demo-environment/pull/21
Confirm that the "learning outcome" (summary at the top of the page) is actually covered in the material.

External Release Notes

We've released a brand new quickstart experience for model validators, a user-friendly introduction to using ValidMind for validating models. This quickstart covers the basics of using ValidMind to validate models, walking you through how to use the ValidMind Library to run tests and evaluate a champion model, then add your test results as evidence to your validation report within the ValidMind Platform.

This quickstart is the second in a series of new user journeys — look forward to our upcoming introductory guide for platform administration!

github-actions · 2025-06-11T18:56:21Z

PR Summary

This pull request introduces several enhancements to the documentation and validation templates used within the ValidMind platform. Key changes include:

Documentation Template Enhancements:
- The glossary entry for 'template' has been updated to provide a clearer description of its role in structuring model documentation and validation activities.
- A new entry for 'validation report template' has been added, detailing its purpose in standardizing model validation processes.
Quickstart Guides Updates:
- The 'Quickstart — Setup & Administration' and 'Quickstart — Model Development' guides now include subtitles indicating the estimated time to complete.
- The 'Quickstart — Model Validation' guide has been expanded with detailed instructions on registering models, assigning validator credentials, and using the ValidMind developer tools to run tests and log results.
- Additional guidance is provided on working with validation reports, including linking test results as evidence and managing model findings.
Content and Navigation Improvements:
- Updated links and footnotes in the quickstart guides to improve navigation and provide direct access to related resources.
- Adjustments to the layout and content blocks to enhance readability and user experience.

These changes aim to improve the clarity, usability, and comprehensiveness of the documentation and validation processes within the platform.

Test Suggestions

Verify that the updated glossary entries accurately reflect the intended definitions and are displayed correctly.
Test the navigation links in the quickstart guides to ensure they direct to the correct resources.
Check the functionality of the 'Quickstart — Model Validation' guide, ensuring all steps can be followed without errors.
Ensure that the validation report template correctly allows linking of test results as evidence.
Review the display of subtitles in the quickstart guides to confirm they are correctly formatted and visible.

nrichers

Looks great already, @validbeck! This is a substantial, helpful quickstart that fills in an important content gap. 🚀

I have requested a few stylistic tweaks, see my comments. Will approve after.

site/get-started/validator/quickstart-validator.qmd

Co-authored-by: Nik Richers <nik@validmind.ai>

github-actions · 2025-06-16T16:53:07Z

PR Summary

This pull request focuses on standardizing the terminology used across the documentation by changing instances of 'QuickStart' to 'quickstart'. Additionally, it updates the documentation templates to provide a more comprehensive framework for model validation activities. The changes include:

Terminology Standardization: All occurrences of 'QuickStart' have been updated to 'quickstart' to maintain consistency across the documentation.
Documentation Template Enhancements: The documentation templates have been expanded to include a more detailed structure for model validation, ensuring consistency and completeness in validation reports. This includes outlining required sections, recommended analyses, and expected validation tests.
Content Updates: Various sections of the documentation have been updated to reflect these changes, including the style guide, glossary, and quickstart guides for developers and validators.
Removal of Redundant Content: Some outdated or redundant content has been removed to streamline the documentation.

These changes aim to improve the clarity and usability of the documentation for users engaging with the ValidMind platform.

Test Suggestions

Verify that all instances of 'QuickStart' have been correctly updated to 'quickstart' across the documentation.
Check that the updated documentation templates provide clear guidance for model validation activities.
Ensure that the quickstart guides for developers and validators are consistent with the new terminology and template structure.
Test the links and references in the documentation to ensure they point to the correct updated sections.
Review the documentation for any remaining outdated content that may need updating.

nrichers

@validbeck I made some additional comments — at least the one to fix the broken step numbering should be addressed. LGTM after you resolve these new comments. 🚀

site/get-started/validator/quickstart-validator.qmd

site/get-started/common-steps/_access-quickstart-notebook.qmd

site/get-started/validator/quickstart-validator.qmd

github-actions · 2025-06-16T22:59:06Z

Vale source linter

⚠️ WARN: Found 13165 issues across 605 files — Check the workflow run or download the full report

Show first 30 files with issues

site/404.qmd

Line 2: Use parentheses judiciously. (suggestion)
Line 3: Did you really mean 'html'? (error)
Line 53: Did you really mean 'const'? (error)
Line 53: Use parentheses judiciously. (suggestion)
Line 54: Did you really mean 'const'? (error)
Line 54: Did you really mean 'ctx'? (error)
Line 54: Use parentheses judiciously. (suggestion)
Line 54: Put a nonbreaking space between the number and the unit in '2d'. (error)
Line 58: Did you really mean 'const'? (error)
Line 58: Did you really mean 'fontSize'? (error)
... and 23 more issues

site/about/contributing/style-guide/conventions.qmd

Line 6: Use 'it's' instead of 'it is'. (suggestion)
Line 6: In general, use active voice instead of passive voice ('is received'). (suggestion)
Line 6: Use parentheses judiciously. (suggestion)
Line 9: Use 'capability' or 'feature' instead of 'functionality'. (warning)
Line 9: Use 'shouldn't' instead of 'should not'. (suggestion)
Line 9: In general, use active voice instead of passive voice ('be lost'). (suggestion)
Line 15: Did you really mean 'Palo'? (error)
Line 19: Avoid using 'will'. (warning)
Line 19: In general, use American spelling instead of 'centre'. (warning)
Line 19: Avoid using 'will'. (warning)
... and 182 more issues

site/about/contributing/style-guide/style-guide.qmd

Line 2: Spell out 'USING', if it's unfamiliar to the audience. (suggestion)
Line 2: Spell out 'THE', if it's unfamiliar to the audience. (suggestion)
Line 2: Spell out 'THE', if it's unfamiliar to the audience. (suggestion)
Line 2: Spell out 'TITLE', if it's unfamiliar to the audience. (suggestion)
Line 2: Spell out 'THE', if it's unfamiliar to the audience. (suggestion)
Line 22: Try to avoid using first-person plural like 'we'. (warning)
Line 22: Try to avoid using first-person plural like 'we'. (warning)
Line 22: Try to avoid using first-person plural like 'We'. (warning)
Line 24: Try to avoid using first-person plural like 'Our'. (warning)
Line 24: Try to avoid using first-person plural like 'our'. (warning)
... and 13 more issues

site/about/contributing/style-guide/voice-and-tone.qmd

Line 6: Try to avoid using first-person plural like 'we'. (warning)
Line 8: Try to avoid using first-person plural like 'we'. (warning)
Line 8: Try to avoid using first-person plural like 'our'. (warning)
Line 8: Try to avoid using first-person plural like 'we'. (warning)
Line 8: Try to avoid using first-person plural like 'we'. (warning)
Line 18: Avoid using 'will'. (warning)
Line 18: In general, use active voice instead of passive voice ('be reviewed'). (suggestion)
Line 22: Use parentheses judiciously. (suggestion)
Line 22: Try to avoid using first-person plural like 'we'. (warning)
Line 22: Don't put a space before or after a dash. (error)
... and 39 more issues

site/about/contributing/validmind-community.qmd

Line 8: Use parentheses judiciously. (suggestion)
Line 8: Spell out 'MRM', if it's unfamiliar to the audience. (suggestion)
Line 8: Try to avoid using first-person plural like 'our'. (warning)
Line 8: Don't use exclamation points in text. (error)
Line 8: Try to avoid using first-person plural like 'us'. (warning)
Line 11: Try to avoid using first-person plural like 'our'. (warning)
Line 11: Use parentheses judiciously. (suggestion)
Line 11: Don't use exclamation points in text. (error)
Line 12: Try to avoid using first-person plural like 'our'. (warning)
Line 12: Use parentheses judiciously. (suggestion)
... and 36 more issues

site/about/deployment/_deployment-available-options.qmd

Line 1: Try to avoid using first-person plural like 'Our'. (warning)
Line 1: Use parentheses judiciously. (suggestion)
Line 1: Use 'Google Cloud Platform' or 'GCP' instead of 'Cloud'. (warning)
Line 3: Try to avoid using first-person plural like 'We'. (warning)
Line 5: Don't put a space before or after a dash. (error)
Line 5: Use parentheses judiciously. (suggestion)
Line 5: In general, use active voice instead of passive voice ('be established'). (suggestion)
Line 7: Use parentheses judiciously. (suggestion)
Line 7: Spell out 'VPV', if it's unfamiliar to the audience. (suggestion)
Line 7: Don't put a space before or after a dash. (error)
... and 1 more issues

site/about/deployment/deployment-options.qmd

Line 24: Use parentheses judiciously. (suggestion)
Line 26: Use 'Google Cloud Platform' or 'GCP' instead of 'Cloud'. (warning)
Line 28: Try to avoid using first-person plural like 'our'. (warning)
Line 32: Don't put a space before or after a dash. (error)
Line 32: Did you really mean 'validators'? (error)
Line 34: Don't put a space before or after a dash. (error)
Line 36: Don't put a space before or after a dash. (error)
Line 38: Don't put a space before or after a dash. (error)
Line 42: Try to avoid using first-person plural like 'We'. (warning)
Line 42: Use 'data is' instead of 'data are'. (warning)
... and 12 more issues

site/about/deployment/multi-tenant-cloud.qmd

Line 13: Use 'tap' instead of 'touch'. (warning)
Line 17: Did you really mean 'deployment_'? (error)
Line 19: Use 'it's' instead of 'it is'. (suggestion)
Line 21: Use parentheses judiciously. (suggestion)
Line 29: Don't put a space before or after a dash. (error)
Line 30: Don't put a space before or after a dash. (error)
Line 31: Don't put a space before or after a dash. (error)
Line 32: Don't put a space before or after a dash. (error)
Line 39: In general, use active voice instead of passive voice ('is encrypted'). (suggestion)
Line 40: Use parentheses judiciously. (suggestion)
... and 2 more issues

site/about/deployment/virtual-private-validmind.qmd

Line 2: Spell out 'USING', if it's unfamiliar to the audience. (suggestion)
Line 2: Spell out 'THE', if it's unfamiliar to the audience. (suggestion)
Line 2: Spell out 'THE', if it's unfamiliar to the audience. (suggestion)
Line 2: Spell out 'TITLE', if it's unfamiliar to the audience. (suggestion)
Line 2: Spell out 'THE', if it's unfamiliar to the audience. (suggestion)
Line 3: Use parentheses judiciously. (suggestion)
Line 3: Spell out 'VPV', if it's unfamiliar to the audience. (suggestion)
Line 14: Use 'tap' instead of 'touch'. (warning)
Line 18: Use parentheses judiciously. (suggestion)
Line 18: Spell out 'VPV', if it's unfamiliar to the audience. (suggestion)
... and 8 more issues

site/about/fine-print/data-privacy-policy.qmd

Line 12: Try to avoid using first-person plural like 'our'. (warning)
Line 19: Try to avoid using first-person plural like 'we'. (warning)
Line 21: Try to avoid using first-person plural like 'Our'. (warning)
Line 21: Try to avoid using first-person plural like 'our'. (warning)
Line 21: Try to avoid using first-person plural like 'our'. (warning)
Line 26: Try to avoid using first-person plural like 'our'. (warning)
Line 26: Try to avoid using first-person plural like 'we'. (warning)
Line 26: Try to avoid using first-person plural like 'our'. (warning)
Line 34: Use 'what's' instead of 'What is'. (suggestion)
Line 36: Try to avoid using first-person plural like 'our'. (warning)
... and 49 more issues

site/about/fine-print/license-agreement.qmd

Line 9: Spell out 'READ', if it's unfamiliar to the audience. (suggestion)
Line 9: Spell out 'THIS', if it's unfamiliar to the audience. (suggestion)
Line 9: Use parentheses judiciously. (suggestion)
Line 9: Spell out 'THIS', if it's unfamiliar to the audience. (suggestion)
Line 9: Spell out 'USING', if it's unfamiliar to the audience. (suggestion)
Line 9: Spell out 'THE', if it's unfamiliar to the audience. (suggestion)
Line 9: Spell out 'USING', if it's unfamiliar to the audience. (suggestion)
Line 9: Spell out 'THE', if it's unfamiliar to the audience. (suggestion)
Line 9: Spell out 'YOU', if it's unfamiliar to the audience. (suggestion)
Line 9: Spell out 'ARE', if it's unfamiliar to the audience. (suggestion)
... and 115 more issues

site/about/glossary/_ai.qmd

Line 1: Spell out 'IBM', if it's unfamiliar to the audience. (suggestion)
Line 1: Use parentheses judiciously. (suggestion)
Line 3: Use parentheses judiciously. (suggestion)
Line 7: Use parentheses judiciously. (suggestion)
Line 9: Use parentheses judiciously. (suggestion)
Line 10: In general, use active voice instead of passive voice ('were trained'). (suggestion)
Line 12: Use parentheses judiciously. (suggestion)
Line 12: Spell out 'LLM', if it's unfamiliar to the audience. (suggestion)
Line 13: Use parentheses judiciously. (suggestion)

site/about/glossary/_attestation.qmd

Line 2: Did you really mean 'validator'? (error)
Line 2: In general, use active voice instead of passive voice ('are met'). (suggestion)
Line 5: In general, use active voice instead of passive voice ('is triggered'). (suggestion)
Line 14: Did you really mean 'validators'? (error)

site/about/glossary/_developer-tools.qmd

Line 2: Use parentheses judiciously. (suggestion)
Line 2: Use 'capability' or 'feature' instead of 'functionality'. (warning)
Line 15: Use 'that's' instead of 'that is'. (suggestion)
Line 20: Try to avoid using first-person plural like 'our'. (warning)
Line 30: Use parentheses judiciously. (suggestion)
Line 32: Use parentheses judiciously. (suggestion)

site/about/glossary/_model-documentation.qmd

Line 1: Spell out 'MODEL', if it's unfamiliar to the audience. (suggestion)
Line 1: Spell out 'NEEDS', if it's unfamiliar to the audience. (suggestion)
Line 1: In general, use active voice instead of passive voice ('BE ADDED'). (suggestion)
Line 1: Spell out 'ADDED', if it's unfamiliar to the audience. (suggestion)
Line 3: Use parentheses judiciously. (suggestion)
Line 4: Spell out 'ABC', if it's unfamiliar to the audience. (suggestion)

site/about/glossary/_models.qmd

Line 4: Use parentheses judiciously. (suggestion)
Line 7: In general, use active voice instead of passive voice ('are derived'). (suggestion)
Line 7: In general, use active voice instead of passive voice ('is achieved'). (suggestion)
Line 11: Use parentheses judiciously. (suggestion)
Line 15: Use 'it's' instead of 'it is'. (suggestion)
Line 15: In general, use active voice instead of passive voice ('is decommissioned'). (suggestion)
Line 15: Use parentheses judiciously. (suggestion)
Line 22: Use parentheses judiciously. (suggestion)
Line 22: Spell out 'MRM', if it's unfamiliar to the audience. (suggestion)
Line 23: In general, use active voice instead of passive voice ('are developed'). (suggestion)

site/about/glossary/_monitoring.qmd

Line 1: Spell out 'NEEDS', if it's unfamiliar to the audience. (suggestion)
Line 1: In general, use active voice instead of passive voice ('BE ADDED'). (suggestion)
Line 1: Spell out 'ADDED', if it's unfamiliar to the audience. (suggestion)
Line 3: Use parentheses judiciously. (suggestion)
Line 4: Spell out 'ABC', if it's unfamiliar to the audience. (suggestion)

site/about/glossary/_mrm.qmd

Line 3: Spell out all ordinal numbers ('1st') in text. (error)
Line 3: Put a nonbreaking space between the number and the unit in '1s'. (error)
Line 4: Don't use plurals in parentheses such as in 'unit(s)'. (error)
Line 4: Use parentheses judiciously. (suggestion)
Line 4: Spell out all ordinal numbers ('1st') in text. (error)
Line 4: Put a nonbreaking space between the number and the unit in '1s'. (error)
Line 4: Use 'they're' instead of 'they are'. (suggestion)
Line 6: Spell out all ordinal numbers ('2nd') in text. (error)
Line 7: Spell out all ordinal numbers ('2nd') in text. (error)
Line 7: Did you really mean 'validators'? (error)
... and 11 more issues

site/about/glossary/_validation-reports.qmd

Line 1: Spell out 'NEEDS', if it's unfamiliar to the audience. (suggestion)
Line 1: In general, use active voice instead of passive voice ('BE ADDED'). (suggestion)
Line 1: Spell out 'ADDED', if it's unfamiliar to the audience. (suggestion)
Line 3: Use parentheses judiciously. (suggestion)
Line 4: Spell out 'ABC', if it's unfamiliar to the audience. (suggestion)

site/about/glossary/_validmind-features.qmd

Line 7: In general, use active voice instead of passive voice ('are used'). (suggestion)
Line 7: Use parentheses judiciously. (suggestion)

site/about/glossary/_validmind.qmd

Line 3: In general, use active voice instead of passive voice ('are intertwined'). (suggestion)
Line 5: Use parentheses judiciously. (suggestion)
Line 6: Use 'open source' instead of 'open-source'. (warning)
Line 6: Use parentheses judiciously. (suggestion)
Line 6: Did you really mean 'overfit'? (error)
Line 8: Use parentheses judiciously. (suggestion)
Line 9: Use parentheses judiciously. (suggestion)
Line 9: Did you really mean 'APIs'? (error)

site/about/glossary/glossary.qmd

Line 7: Spell out 'NEW', if it's unfamiliar to the audience. (suggestion)
Line 7: Spell out 'TERM', if it's unfamiliar to the audience. (suggestion)
Line 7: Spell out 'NEEDS', if it's unfamiliar to the audience. (suggestion)
Line 7: In general, use active voice instead of passive voice ('BE ADDED'). (suggestion)
Line 7: Spell out 'ADDED', if it's unfamiliar to the audience. (suggestion)
Line 9: Spell out 'ABC', if it's unfamiliar to the audience. (suggestion)
Line 9: Use parentheses judiciously. (suggestion)
Line 9: Use 'for example' instead of 'e.g.'. (error)
Line 10: Spell out 'KEY', if it's unfamiliar to the audience. (suggestion)
Line 10: Spell out 'TERMS', if it's unfamiliar to the audience. (suggestion)
... and 9 more issues

site/about/glossary/key_concepts/_docs.qmd

Line 4: Use 'app' instead of 'application'. (warning)

site/about/glossary/key_concepts/_inputs.qmd

Line 2: In general, use active voice instead of passive voice ('be evaluated'). (suggestion)
Line 4: In general, use active voice instead of passive voice ('been initialized'). (suggestion)
Line 4: Use parentheses judiciously. (suggestion)
Line 5: In general, use active voice instead of passive voice ('been initialized'). (suggestion)
Line 5: Use parentheses judiciously. (suggestion)
Line 6: In general, use active voice instead of passive voice ('is used'). (suggestion)
Line 7: In general, use active voice instead of passive voice ('is used'). (suggestion)
Line 7: Use parentheses judiciously. (suggestion)

site/about/glossary/key_concepts/_key-concepts.qmd

Line 1: Spell out 'KEY', if it's unfamiliar to the audience. (suggestion)
Line 1: Spell out 'NEEDS', if it's unfamiliar to the audience. (suggestion)
Line 1: In general, use active voice instead of passive voice ('BE ADDED'). (suggestion)
Line 1: Spell out 'ADDED', if it's unfamiliar to the audience. (suggestion)
Line 3: Use parentheses judiciously. (suggestion)
Line 5: Use parentheses judiciously. (suggestion)
Line 5: Use 'for example' instead of 'e.g.'. (error)
Line 5: Spell out 'ABC', if it's unfamiliar to the audience. (suggestion)

site/about/glossary/key_concepts/_metrics.qmd

Line 2: Use 'don't' instead of 'do not'. (suggestion)
Line 2: In general, use active voice instead of passive voice ('be registered'). (suggestion)
Line 2: In general, use active voice instead of passive voice ('be used'). (suggestion)
Line 4: In general, use active voice instead of passive voice ('be thought'). (suggestion)

site/about/glossary/key_concepts/_monitoring-template.qmd

Line 2: In general, use active voice instead of passive voice ('be tracked'). (suggestion)
Line 2: In general, use active voice instead of passive voice ('be conducted'). (suggestion)
Line 2: In general, use active voice instead of passive voice ('be displayed'). (suggestion)
Line 4: Did you really mean 'boilerplates'? (error)

site/about/glossary/key_concepts/_outputs.qmd

Line 2: Use parentheses judiciously. (suggestion)
Line 2: 's.D' should have one space. (error)

site/about/glossary/key_concepts/_parameters.qmd

Line 2: In general, use active voice instead of passive voice ('be passed'). (suggestion)

site/about/glossary/key_concepts/_template.qmd

Line 2: In general, use active voice instead of passive voice ('be run'). (suggestion)
Line 2: In general, use active voice instead of passive voice ('be displayed'). (suggestion)
Line 5: Did you really mean 'validators'? (error)
Line 7: Did you really mean 'boilerplates'? (error)

... and 575 more files with issues

github-actions · 2025-06-16T23:01:00Z

Lighthouse check results

⚠️ WARN: Average accessibility score is 0.78 (required: >0.9) — Check the workflow run

Show Lighthouse scores

Folder depth level checked: 0

Commit SHA: 2ab7744

Modify the workflow to check a different depth:

0: Top-level navigation only — /index.html, /guide/guides.html, ...
1: All first-level subdirectories — /guide/.html, /developer/.html, ...
2: All second-level subdirectories — /guide/attestation/*.html, ...

Page	Accessibility	Performance	Best Practices	SEO
/developer/validmind-library.html	0.75	0.68	1.00	0.89
/get-started/get-started.html	0.76	0.85	1.00	0.80
/guide/guides.html	0.76	0.87	1.00	0.90
/index.html	0.79	0.85	1.00	0.80
/releases/all-releases.html	0.85	0.91	1.00	0.80
/support/support.html	0.82	0.86	1.00	0.90
/training/training.html	0.73	0.76	0.96	0.80

github-actions · 2025-06-16T23:14:05Z

Validate docs site

✓ INFO: A live preview of the docs site is available — Open the preview

validbeck added 4 commits June 11, 2025 09:59

WIP validation quickstart

729007a

Validation quickstart draft

50f98c3

Proofreading

789c118

Quick edit to key concepts

472be81

validbeck self-assigned this Jun 11, 2025

validbeck added documentation Improvements or additions to documentation highlight Feature to be curated in the release notes labels Jun 11, 2025

Edit

1a72bac

validbeck requested review from nrichers and LoiAnsah June 11, 2025 18:55

More proofreading

1558526

validbeck mentioned this pull request Jun 11, 2025

quickstart: Model Development #748

Merged

validbeck added 4 commits June 11, 2025 12:49

QuickStart casing fix

8c32b07

removing future tenses

328b986

common-assets > common-steps

72c5e0b

tabsets

3a2e1e7

nrichers requested changes Jun 12, 2025

View reviewed changes

validbeck and others added 3 commits June 16, 2025 09:44

Update site/get-started/validator/quickstart-validator.qmd

a59d2e0

Co-authored-by: Nik Richers <nik@validmind.ai>

Update site/get-started/validator/quickstart-validator.qmd

81b79ca

Co-authored-by: Nik Richers <nik@validmind.ai>

Tweak

9e63617

validbeck requested a review from nrichers June 16, 2025 16:52

nrichers approved these changes Jun 16, 2025

View reviewed changes

Fixing numbering

2534cfc

validbeck merged commit 7a14c9f into main Jun 17, 2025
5 checks passed

validbeck deleted the beck/sc-10572/new-model-validation-get-started branch June 17, 2025 15:28

quickstart: Model Validation #750

quickstart: Model Validation #750

Uh oh!

Conversation

validbeck commented Jun 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Internal Notes for Reviewers