Skip to content

quickstart: Model Validation #750

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Jun 17, 2025
2 changes: 1 addition & 1 deletion site/about/contributing/style-guide/conventions.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,7 @@ aliases:
##### Hotlinked

```md
{{{< video https://www.youtube.com/embed/rIR8Mql7eGs?si=vnZA_zP4tAjFjI4r title='ValidMind QuickStart' >}}}
{{{< video https://www.youtube.com/embed/rIR8Mql7eGs?si=vnZA_zP4tAjFjI4r title='ValidMind Quickstart' >}}}
```

##### Embedded
Expand Down
2 changes: 1 addition & 1 deletion site/about/contributing/style-guide/voice-and-tone.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ Address the reader directly by using the second person.

| Correct | Incorrect |
|------|-----|
| After completing this QuickStart, you will be able to view your test results as part of your model documentation right in the {{< var validmind.platform >}}. | After completing this QuickStart, the model developer will be able to view the test results as part of the model documentation right in the {{< var validmind.platform >}}. |
| After completing this quickstart, you will be able to view your test results as part of your model documentation right in the {{< var validmind.platform >}}. | After completing this quickstart, the model developer will be able to view the test results as part of the model documentation right in the {{< var validmind.platform >}}. |
: **2nd person** examples {.hover}

### Avoid stiff formality
Expand Down
7 changes: 5 additions & 2 deletions site/about/glossary/key_concepts/_template.qmd
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
template, documentation template
documentation template
: Functions as a test suite and lays out the structure of model documentation, segmented into various sections and sub-sections. Documentation templates define the structure of your model documentation, specifying the tests that should be run, and how the results should be displayed.

{{< var vm.product >}} templates come with pre-defined sections, similar to test placeholders, including boilerplates and spaces designated for documentation and test results. When rendered, produces a document that model developers can use for model validation.
validation report template
: Serves as a standardized framework for conducting and documenting model validation activities. It outlines the required sections, recommended analyses, and expected validation tests, ensuring consistency and completeness across validation reports. The template helps guide validators through a systematic review process while promoting comparability and traceability of validation outcomes.

{{< var vm.product >}} templates come with pre-defined sections, similar to test placeholders, including boilerplates and spaces designated for test results, evidence, or findings.
14 changes: 4 additions & 10 deletions site/developer/samples-jupyter-notebooks.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -40,31 +40,25 @@ Our Jupyter Notebook code samples showcase the capabilities and features of the

::::

## QuickStart for Customer Churn Model Documentation — Full Suite
## Quickstarts

Learn how to use dataset and model documentation function on a simple customer churn model. The easiest way to try the QuickStart is on JupyterHub or Google Colaboratory:
Learn how to develop or validate a simple customer churn model with {{< var vm.product >}}. The easiest way to try the quickstarts are on JupyterHub:

:::: {.flex .flex-wrap .justify-around}

::: {.w-50-ns}

JupyterHub
: A a web-based platform when you can interact with Jupyter Notebook instances on a shared server. It is commonly used as a collaborative and interactive computing environment for data analysis, scientific research, and programming.

::: {.tc}
[{{< fa code >}} Quickstart on JupyterHub]({{< var url.jupyterhub >}}/hub/user-redirect/lab/tree/quickstart/quickstart_model_documentation.ipynb){.button target="_blank"}
[{{< fa code >}} Quickstart for model documentation]({{< var url.jupyterhub >}}/hub/user-redirect/lab/tree/quickstart/quickstart_model_documentation.ipynb){.button target="_blank"}

:::

:::

::: {.w-40-ns}

Google Colaboratory (Colab)
: A free Jupyter Notebook environment that runs in the cloud. There, you can work with our Jupyter Notebooks by saving your own copy, write and execute code, share your work to collaborate with others in real-time.

::: {.tc}
[{{< fa brands google >}} Quickstart on Colab](https://colab.research.google.com/drive/1NLtjCFUZV2I_ttGUkiAL9uwGZK_SGUlw?usp=share_link){.button target="_blank"}
[{{< fa code >}} Quickstart for model validation]({{< var url.jupyterhub >}}/hub/user-redirect/lab/tree/quickstart/quickstart_model_validation.ipynb){.button target="_blank"}

:::

Expand Down
3 changes: 0 additions & 3 deletions site/developer/validmind-library.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,6 @@ listing:
contents:
- ../notebooks/quickstart/quickstart_model_documentation.ipynb
- ../notebooks/quickstart/quickstart_model_validation.ipynb
# - path: https://youtu.be/rIR8Mql7eGs
# title: "{{< fa brands youtube >}} {{< var vm.product >}} QuickStart"
# description: "Watch the walkthrough on YouTube: `https://youtu.be/rIR8Mql7eGs`"
- id: model-development
type: grid
grid-columns: 2
Expand Down
2 changes: 1 addition & 1 deletion site/get-started/_sidebar.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ website:
- id: get-started
title: "Get Started"
contents:
- text: "Quickstart For"
- text: "Quickstarts"
- text: "Model Development"
file: get-started/developer/quickstart-developer.qmd
- text: "Model Validation"
Expand Down
4 changes: 2 additions & 2 deletions site/get-started/administrator/quickstart-administrator.qmd
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "Quickstart — Setup & Administration [coming soon]{.smallercaps .pink}"
subtitle: "10 minutes"
date: last-modified
page-layout: full
---

Set up the foundational units for your organization within the {{< var validmind.platform >}}, prepare your organization for users, then grant access to your first user.

<!-- BECK'S OUTLINE / NOTES -->

<!-- {{< include /get-started/common-assets/_before-you-begin.qmd >}}
<!-- {{< include /get-started/common-steps/_before-you-begin.qmd >}}

## Setting up your organization

Expand Down
8 changes: 0 additions & 8 deletions site/get-started/common-assets/_before-you-begin.qmd

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@

You can either load the hosted version of our notebook, or download the notebook to run in your own local environment:

#### Load the hosted notebook (recommended)
#### Open the hosted notebook (recommended)

1. Sign in to [JupyterHub]({{< var url.jupyterhub >}}) with your {{< var vm.product >}} login.
8 changes: 8 additions & 0 deletions site/get-started/common-steps/_before-you-begin.qmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
::: {.attn}

## {{< fa star-of-life >}} Before you begin

- [x] To try out {{< var vm.product >}}, you need to be a registered user on the {{< var validmind.platform >}}.^[[Register with {{< var vm.product >}}](/guide/configuration/register-with-validmind.qmd)]
- [x] You'll also want to make sure that you're logged into the {{< var validmind.platform >}}.^[ [Log in to {{< var vm.product >}}](/guide/configuration/log-in-to-validmind.qmd)]

:::
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
#### Download the notebooks

##### Option 1: Clone the GitHub repository

1. Open a terminal or command prompt.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#### Get your code snippet

Before we move on, let's also retrieve the unique *code snippet* for your model that will enable you to connect your testing environment with the correct model in the {{< var validmind.platform >}}:
Before we move on, let's also retrieve the unique *code snippet* for your model that enables you to connect your testing environment with the correct model in the {{< var validmind.platform >}}:

1. In the left sidebar that appears for your model, click **{{< fa rocket >}} Getting Started**.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@
To initialize the {{< var validmind.developer >}} which connects your environment to the correct model in the {{< var validmind.platform >}}, you'll need your unique code snippet!^[[Get your code snippet](#get-your-code-snippet)]
:::

1. Follow the instructions to run the first two code cells in the notebook,[^longnote] which will set the stage by installing and initializing {{< var vm.product >}}.^[[Install and initialize the {{< var validmind.developer >}}](/developer/model-documentation/install-and-initialize-validmind-library.qmd)]
1. Follow the instructions to run the first two code cells in the notebook,[^longnote] which sets the stage by installing and initializing {{< var vm.product >}}.^[[Install and initialize the {{< var validmind.developer >}}](/developer/model-documentation/install-and-initialize-validmind-library.qmd)]

1. Under the cell at bottom of the **Get your code snippet** section, confirm that you see a success message verifying that you've connected to the right model.

This message verifies that the {{< var vm.developer >}} works as expected as is connected to the correct model within the {{< var vm.platform >}}.
This message verifies that the {{< var vm.developer >}} works as expected as is connected to the correct model and document type within the {{< var vm.platform >}}.

[^longnote]:

Expand Down
69 changes: 39 additions & 30 deletions site/get-started/developer/quickstart-developer.qmd
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "Quickstart — Model Development"
subtitle: "15 minutes"
date: last-modified
page-layout: full
listing:
- id: whats-next
type: grid
Expand All @@ -23,51 +23,56 @@ aliases:

Register your first model, use the {{< var validmind.developer >}} to run tests and document a model, then review your generated model documentation within the {{< var validmind.platform >}}.

{{< include /get-started/common-assets/_before-you-begin.qmd >}}
{{< include /get-started/common-steps/_before-you-begin.qmd >}}

## Register your first model

The first step to using {{< var vm.product >}} to develop models is to register them in your model inventory:

{{< include /get-started/common-assets/_register-your-first-model.qmd >}}
{{< include /get-started/common-steps/_register-your-first-model.qmd >}}

::: {.column-margin}
![Registration modal with the options filled out for the model development quickstart](developer-new-model.png){width=90% fig-alt="A screenshot showing the registration modal with the options filled out for the model development quickstart" .screenshot}
:::

{{< include /get-started/common-assets/_get-your-code-snippet.qmd >}}
{{< include /get-started/common-steps/_get-your-code-snippet.qmd >}}

::: {.column-margin}
![{{< fa rocket >}} Getting Started page with a sample code snippet](developer-code-snippet.png){width=90% fig-alt="A screenshot showing the Getting Started page with a sample code snippet" .screenshot}
:::

## Using the {{< var validmind.developer >}}

Next, let's set up the {{< var validmind.developer >}} in your developer environment of choice so we can use it to run and log tests, which will help automate the documentation of your models.
Next, let's set up the {{< var validmind.developer >}} in your development environment of choice so we can use it to run and log tests, which helps automate the documentation of your models.

Our companion notebook, **Quickstart for model documentation,**[^3] will walk you through the steps of installing the {{< var vm.developer >}} in your environment, initializing the {{< var vm.developer >}} for use, and running a full suite of documentation tests that will log the results to the {{< var validmind.platform >}}.
Our companion notebook, **Quickstart for model documentation,**[^1] walks you through the steps of installing the {{< var vm.developer >}} in your environment, initializing the {{< var vm.developer >}} for use, and running a full suite of documentation tests that logs results to the {{< var validmind.platform >}}.

{{< include /get-started/common-assets/_access-quickstart-notebook.qmd >}}
{{< include /get-started/common-steps/_access-quickstart-notebook.qmd >}}

<!-- NOT LIVE ON JUPYTERHUB YET, HASN'T BEEN UPDATED -->
1. Open [Quickstart for model documentation]({{< var url.jupyterhub >}}/hub/user-redirect/lab/tree/quickstart/quickstart_model_documentation.ipynb) and [run the quickstart notebook](#run-the-quickstart-notebook).

{{< include /get-started/common-assets/_download-notebooks.qmd >}}
#### Download the notebooks

1. After the cloning process is complete, open `notebooks/quickstart/quickstart_model_documentation.ipynb` in your developer environment and [run the quickstart notebook](#run-the-quickstart-notebook).
::: {.panel-tabset}

{{< include /get-started/common-assets/_download-code-samples.qmd >}}
{{< include /get-started/common-steps/_download-notebooks.qmd >}}

1. Open `notebooks/quickstart/quickstart_model_documentation.ipynb` in your developer environment and [run the quickstart notebook](#run-the-quickstart-notebook).
1. After the cloning process is complete, open `notebooks/quickstart/quickstart_model_documentation.ipynb` in your development environment and [run the quickstart notebook](#run-the-quickstart-notebook).

{{< include /get-started/common-assets/_run-quickstart-notebook.qmd >}}
{{< include /get-started/common-steps/_download-code-samples.qmd >}}

1. Open `notebooks/quickstart/quickstart_model_documentation.ipynb` in your development environment and [run the quickstart notebook](#run-the-quickstart-notebook).

:::

{{< include /get-started/common-steps/_run-quickstart-notebook.qmd >}}

::: {.column-margin}
**Example success message:**

```bash
2024-10-08 22:06:42,610 - INFO(validmind.api_client): 🎉 Connected to {{< var vm.product >}}!
📊 Model: [ValidMind] Model development quickstart (ID: cmalgf3qi02ce199qm3rdkl46)
2025-06-10 22:06:42,610 - INFO(validmind.api_client): 🎉 Connected to {{< var vm.product >}}!
📊 Model: [ValidMind] Model development quickstart (ID: cmbpnxrv6012o0g9w5049nsct)
📁 Document Type: model_documentation
```
:::
Expand All @@ -80,34 +85,38 @@ After you successfully run the notebook, return to the {{< var validmind.platfor

1. In the left sidebar, click **{{< fa cubes >}} Inventory**.

2. Locate or search for the model you registered for this quickstart and click on it to select it.[^4]
2. Locate or search for the model you registered for this quickstart and click on it to select it.[^2]

3. In the left sidebar that appears for the model, click **{{< fa book-open >}} Documentation**.
3. In the left sidebar that appears for the model, click **{{< fa book-open >}} Documentation**:

- Your model's documentation is broken down into sections, defined by your documentation templates.[^5]
- Each section can be assigned a completion status,[^6] allowing you to track the progress of your documentation.
- Your model's documentation is broken down into sections, defined by your documentation templates.[^3]
- Each section can be assigned a completion status,[^4] allowing you to track the progress of your documentation.

4. Click on any section of the documentation to review the test results and drafts of test result summaries generated by the {{< var validmind.developer >}}.

In any subsection, expand the **{{< var vm.product >}} Insights™** panel to view the Documentation Guidelines[^7] that tell you more about what these sections mean and help you with the task of documenting the model as a developer.
In any subsection, expand the **{{< var vm.product >}} Insights™** panel to view the Documentation Guidelines[^5] that tell you more about what these sections mean and help you with the task of documenting the model as a developer.

5. Refine your model documentation by:

- Adding or editing content blocks[^6]
- Adding additional test results after you've logged them[^7]

:::: {.flex .flex-wrap .justify-around}

::: {.w-40-ns .tc}

![Example model details page with documentation outline](developer-model-details.png){width=90% fig-alt="A screenshot showing an example model details page with documentation outline" .screenshot group="documentation"}
![Example model details page with documentation outline](developer-model-details.png){fig-alt="A screenshot showing an example model details page with documentation outline" .screenshot group="documentation"}

:::

::: {.w-40-ns .tc}

![Populated model documentation section with the **{{< var vm.product >}} Insights™** panel open](validmind-insights.png){width=90% fig-alt="A screenshot showing populated model documentation section with the ValidMind Insights™ panel open" .screenshot group="documentation"}
![Populated model documentation section with the **{{< var vm.product >}} Insights™** panel open](validmind-insights.png){fig-alt="A screenshot showing populated model documentation section with the ValidMind Insights™ panel open" .screenshot group="documentation"}

:::

::::


## What's next

Now that you've grasped the basics of using {{< var vm.product >}} for model development, continue on your journey with our end-to-end training course for developers and browse through our resource hub for the {{< var validmind.developer >}}:
Expand All @@ -118,16 +127,16 @@ Now that you've grasped the basics of using {{< var vm.product >}} for model dev

<!-- FOOTNOTES -->

[^1]: [Register with {{< var vm.product >}}](/guide/configuration/register-with-validmind.qmd)
[^1]: [Quickstart for model documentation](/notebooks/quickstart/quickstart_model_documentation.ipynb)

[^2]: [Log in to {{< var vm.product >}}](/guide/configuration/log-in-to-validmind.qmd)
[^2]: [Working with the model inventory](/guide/model-inventory/working-with-model-inventory.qmd#search-filter-and-sort-models)

[^3]: [Quickstart for model documentation](/notebooks/quickstart/quickstart_model_documentation.ipynb)
[^3]: [Working with documentation templates](/guide/templates/working-with-documentation-templates.qmd)

[^4]: [Working with the model inventory](/guide/model-inventory/working-with-model-inventory.qmd#search-filter-and-sort-models)
[^4]: [Assign documentation section statuses](/guide/model-documentation/assign-documentation-section-statuses.qmd)

[^5]: [Working with documentation templates](/guide/templates/working-with-documentation-templates.qmd)
[^5]: [View documentation guidelines](/guide/model-documentation/view-documentation-guidelines.qmd)

[^6]: [Assign documentation section statuses](/guide/model-documentation/assign-documentation-section-statuses.qmd)
[^6]: [Work with content blocks](/guide/model-documentation/work-with-content-blocks.qmd)

[^7]: [View documentation guidelines](/guide/model-documentation/view-documentation-guidelines.qmd)
[^7]: [Work with test results](/guide/model-documentation/work-with-test-results.qmd)
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading