Skip to content

October 22 2024 release #487

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 40 commits into from
Oct 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
09a6957
Edits up until Releases
validbeck Oct 10, 2024
c47ceb2
Edits for 2024-aug-13 releases
validbeck Oct 10, 2024
2f0473a
Adjusted figcaption to center text
validbeck Oct 10, 2024
6e45c75
Edits for 2024-jul-22 releases
validbeck Oct 10, 2024
8d17df9
Edits for 2024-jun-10 releases
validbeck Oct 10, 2024
f3d0dec
Edits to overview guides
validbeck Oct 14, 2024
7cb9494
Releases tweaks up to June 10 2024
validbeck Oct 14, 2024
e3f4012
Edits to march 27 2024 wip
validbeck Oct 14, 2024
745cb50
Final tweaks to march 27 2024
validbeck Oct 14, 2024
6002563
Edited feb 14 2024
validbeck Oct 14, 2024
3a2ffb1
Edited jan 26 2024
validbeck Oct 14, 2024
f4d3035
Edited jan 18 2024
validbeck Oct 14, 2024
60ec26e
May 22 2024 releases WIP
validbeck Oct 14, 2024
a1660e6
Edited may 22 2024
validbeck Oct 14, 2024
5a6141b
Added _how-to-upgrade.qmd single-source
validbeck Oct 14, 2024
1e09edb
Added _how-to-upgrade.qmd to newer releases missing it
validbeck Oct 14, 2024
11a223a
Edited nov 09 2023
validbeck Oct 14, 2024
6a0ddfd
Edited oct 25 2023
validbeck Oct 14, 2024
deef0be
WIP Sep 27 2023
validbeck Oct 15, 2024
2babaf0
Edited Sep 27 2023
validbeck Oct 15, 2024
5622182
Tweaks to variable in buttons & links
validbeck Oct 15, 2024
e7763db
Edited aug 15 2023
validbeck Oct 15, 2024
e6e8a41
Edited jul 24 2023
validbeck Oct 15, 2024
43c5d12
Edited jun 22 2023
validbeck Oct 15, 2024
1aee690
Edited may 30 2023
validbeck Oct 15, 2024
86cf3db
Visual tweaks
validbeck Oct 16, 2024
d91e05b
More cleanup
validbeck Oct 16, 2024
cf27de1
Some more tweaks
validbeck Oct 16, 2024
733a21d
Final? edits
validbeck Oct 16, 2024
97aca81
Fixed merge conflicts
validbeck Oct 16, 2024
504b4de
Feedback from Nik
validbeck Oct 17, 2024
60053cd
Maybe the python docs need to be updated on remote?
validbeck Oct 17, 2024
32ee29d
Nope, just a docs-demo preview issue
validbeck Oct 17, 2024
75de804
Merge pull request #478 from validmind/beck/sc-6527/docs-audit-p4-about
validbeck Oct 17, 2024
fa597f0
Merge remote-tracking branch 'origin/main' into staging
github-actions[bot] Oct 17, 2024
b7aafa8
Merge pull request #482 from validmind/update-staging-11394261078
github-actions[bot] Oct 17, 2024
3798d37
Release notes to sprint 60 (#483)
nrichers Oct 22, 2024
5dbd8db
Merge remote-tracking branch 'origin/main' into staging
github-actions[bot] Oct 22, 2024
f4efc31
Merge pull request #485 from validmind/update-staging-11469924270
github-actions[bot] Oct 22, 2024
66dcbb3
Merge branch 'staging' into prod-october-22-2024-release
nrichers Oct 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 0 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,14 +101,6 @@ When constructing links, refer to the `.qmd` file as Quarto will properly render
|---|---|
| `[Get started with JupyterHub](get-started/developer/try-with-jupyterhub.qmd)` | `[Get started with JupyterHub](get-started/developer/try-with-jupyterhub.html)` |

The only exceptions are within HTML, such as CSS-styled buttons which necessitate usage of `.html` href links:

```html
<a class="btn btn-primary" role="button" style="padding: 12px;color: var(--bs-white);background: #DE257E;border-radius: 4px;margin-top: 40px;font-family: 'Inter';font-style: normal;font-weight: 700;font-size: 14px;line-height: 100%;border: 1px solid var(--bs-pink);" href="get-started/developer/try-with-jupyterhub.html">QuickStart</a>
```

> You'll need to pay special attention to these links and manually test them to ensure that they are not broken whenever any changes to file destinations are made.

When constructing filepaths, including while using [Quarto's Includes](https://quarto.org/docs/authoring/includes.html) (single-sourcing feature), you'll also want start with the root directory whenever possible as this minimizes usage of unclear relative paths:

| Correct | Incorrect |
Expand Down
1 change: 1 addition & 0 deletions site/_quarto.yml
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ website:
- text: "---"
- text: "RELEASES"
# MAKE-RELEASE-NOTES-EMBED-MARKER
- releases/2024-oct-22/release-notes.qmd
- releases/2024-sep-25/release-notes.qmd
- releases/2024-sep-09/release-notes.qmd
- releases/2024-aug-13/release-notes.qmd
Expand Down
46 changes: 46 additions & 0 deletions site/about/contributing/style-guide/conventions.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,52 @@ Column 2, 50% wide
- Keep hyperlinks in instructional text minimal — too many links can misdirect the user from the task at hand.
- When possible — such as any time instructions are not single-sourced — instead of multiple inline links, make use of margin footnotes.[^5]
- Whenever possible, make the link the title of the destination article. This also solves the issue of links with nebulous descriptions that do not match article titles, or links whose destinations are not clear to the user such as "Read more."
- When constructing links, refer to the `.qmd` file as Quarto will properly render these into `.html` links on your behalf and check to see if the destinations are able to be resolved.
- When constructing filepaths, you'll also want start with the root directory whenever possible as this minimizes usage of unclear relative paths.

| Correct | Incorrect |
|------|-----|
| `[Set up model workflows](/guide/model-workflows/set-up-model-workflows.qmd)` | `[Learn more ...](/../../set-up-model-workflows.html)` |
: **Inline links** examples {.hover}

#### Buttons

Sometimes, it's helpful to highlight a call to action with a button that takes you to a topic or to a notebook on JupyterHub.

Change any Markdown link into a our theme-styled button by appending `{.button}`:

:::: {.flex .flex-wrap .justify-around}

::: {.w-30-ns}
**Correct:**

[Set up model workflows](/guide/model-workflows/set-up-model-workflows.qmd){.button}

```markdown
[Set up model workflows](/guide/model-workflows/set-up-model-workflows.qmd){.button}
```

::: {.callout}
Using a markdown button also enables you to link to to the `.qmd` path instead of the rendered `.html` file.
:::

:::

::: {.w-60-ns}
**Incorrect:**

<form method="get" action="../../../guide/model-workflows/set-up-model-workflows.html">
<button type="submit" style="color: #de257e; background-color: white; border-radius: 8px; border: none; font-size: 16px; padding: 6.25px 12.5px; margin-left: 16px; margin-bottom: 20px;border: 1px solid #de257e">Set up model workflows</button>
</form>

```html
<form method="get" action="../../../guide/model-workflows/set-up-model-workflows.html">
<button type="submit" style="color: white; background-color: #de257e; border-radius: 8px; border: none; font-size: 16px; padding: 6.25px 12.5px; margin-left: 16px; margin-bottom: 20px;">Set up model workflows</button>
</form>
```
:::

::::

#### Margin footnotes

Expand Down
24 changes: 19 additions & 5 deletions site/about/overview-model-documentation.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,16 @@ filters:
- tachyons
aliases:
- ../guide/overview-model-documentation.html
listing:
id: quickstart
type: grid
grid-columns: 2
# image-height: 100%
contents:
- path: ../get-started/quickstart.qmd
title: "QuickStart"
description: "Our QuickStart is the quickest and easiest way to try out our product features."
fields: [title, description, reading-time]
---

<!---
Expand All @@ -15,17 +25,19 @@ Overview & updated graphic
### Closed beta: LLM testing & documentation
--->

The ValidMind Developer Framework streamlines the process of documenting various types of models. The {{< var vm.developer >}} automates the documentation process, ensuring that your model documentation and testing aligns with regulatory and compliance standards.
The {{< var validmind.developer >}} streamlines the process of documenting various types of models. The {{< var vm.developer >}} automates the documentation process, ensuring that your model documentation and testing aligns with regulatory and compliance standards.

::: {.prereq}

## {{< fa code >}} The {{< var validmind.developer >}}

The {{< var validmind.developer >}} is a {{< var vm.developer >}} and documentation engine designed to streamline the process of documenting various types of models, including traditional statistical models, legacy systems, artificial intelligence/machine learning models, and large language models (LLMs). It offers model developers a systematic approach to documenting and testing risk models with repeatability and consistency, ensuring alignment with regulatory and compliance standards.
The {{< var validmind.developer >}} is a {{< var vm.developer >}} and documentation engine designed to streamline the process of documenting various types of models, including traditional statistical models, legacy systems, artificial intelligence/machine learning models, and large language models (LLMs).

It offers model developers a systematic approach to documenting and testing risk models with repeatability and consistency, ensuring alignment with regulatory and compliance standards.

<!-- Using the variable in alt text messes up the image display -->

![](overview-developer-framework.png){width=80% fig-align="center" fig-alt="An image showing the two main components of ValidMind. The developer framework that integrates with your existing developer environment, and the ValidMind AI risk platform."}
![The two main components of ValidMind. The developer framework that integrates with your existing developer environment, and the ValidMind AI risk platform.](overview-developer-framework.png){width=80% fig-align="center" fig-alt="An image showing the two main components of ValidMind. The developer framework that integrates with your existing developer environment, and the ValidMind AI risk platform."}

The {{< var validmind.developer >}} consists of a client-side library, API integration for models and testing, and validation tests that streamline the model development process. Implemented as a series of independent libraries in Python and R, our {{< var vm.developer >}} ensures compatibility and flexibility with diverse sets of developer environments and requirements.

Expand Down Expand Up @@ -117,7 +129,7 @@ ValidMind imports the following artifacts into the documentation via our API end
- Output from test and test suites that have been run.
- Images, plots, visuals that were generated as part of extracting metrics and running tests.

![](fine-print/overview-api-integration.png){fig-alt="A representation of artifacts imported into the documentation via our API"}
![Artifacts imported into the documentation via our SaaS API](fine-print/overview-api-integration.png){width=90% fig-alt="A representation of artifacts imported into the documentation via our SaaS API"}

::: {.callout-important title="ValidMind does NOT:"}
- Send any personal identifiable information (PII) when generating documentation reports.
Expand All @@ -126,7 +138,9 @@ ValidMind imports the following artifacts into the documentation via our API end

## Ready to try out ValidMind?

Our [QuickStart](/get-started/quickstart.qmd) is the quickest and easiest way to try out our product features.
:::{#quickstart}
:::



<!-- FOOTNOTES -->
Expand Down
66 changes: 46 additions & 20 deletions site/about/overview-model-risk-management.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,16 @@ filters:
- tachyons
aliases:
- ../guide/overview-model-risk-management.html
listing:
id: quickstart
type: grid
grid-columns: 2
# image-height: 100%
contents:
- path: ../get-started/quickstart.qmd
title: "QuickStart"
description: "Our QuickStart is the quickest and easiest way to try out our product features."
fields: [title, description, reading-time]
---

<!---
Expand All @@ -26,17 +36,17 @@ The {{< var validmind.platform >}} provides a comprehensive suite of tools, guid

<!-- Using the variable in alt text messes up the image display -->

![](overview-platform-ui.png){width=80% fig-align="center" fig-alt="An image showing the two main components of ValidMind. The developer framework that integrates with your existing developer environment, and the ValidMind AI risk platform."}
![The two main components of ValidMind. The developer framework that integrates with your existing developer environment, and the ValidMind AI risk platform.](overview-platform-ui.png){width=80% fig-align="center" fig-alt="An image showing the two main components of ValidMind. The developer framework that integrates with your existing developer environment, and the ValidMind AI risk platform."}

The {{< var vm.platform >}} employs a multi-tenant architecture, hosting the cloud-based user interface, APIs, databases, and internal services. The design ensures efficient resource utilization and offers a highly scalable solution for organizations of varying sizes.

With the {{< var vm.platform >}}, you can:

- **Track your model inventory** — Manage the model lifecycle, track the workflow status for models, plan for upcoming validation dates, and more.
- **Work on validation initiatives** — Collaborate with developers and validators to review documentation, add findings, keep track of review statuses, and generate validation reports.
- **Configure workflows** — Set up ValidMind to follow your existing model risk management processes, manage statuses for different parts of the workflow, and get an end-to-end view of workflows and who is involved.
- **Use, create, or edit tests, test suites, and templates** — Create and/or configure required validation tests, test suites, and documentation templates for specific model use cases, tailoring it to your own specific needs.
- **Integrate with your stack** — Import and export model documentation and validation reports.
- **Track your model inventory**[^1] — Manage the model lifecycle, track the workflow status for models, plan for upcoming validation dates, and more.
- **Work on validation initiatives**[^2] — Collaborate with developers and validators to review documentation, add findings, keep track of review statuses, and generate validation reports.
- **Configure workflows**[^3] — Set up ValidMind to follow your existing model risk management processes, manage statuses for different parts of the workflow, and get an end-to-end view of workflows and who is involved.
- **Use, create, or edit tests, test suites, and templates**[^4] — Create and/or configure required validation tests, test suites, and documentation templates for specific model use cases, tailoring it to your own specific needs.
- **Integrate with your stack**[^5] — Import and export model documentation and validation reports.

:::

Expand All @@ -51,18 +61,21 @@ Examples of regulations or policies include:
::: {.w-75-ns}

### SR 11-7: Guidance on Model Risk Management
The Supervisory Guidance on model risk management issued by the Board of Governors of the Federal Reserve System and the Office of the Comptroller of the Currency in the United States in 2011. It provides comprehensive guidance to financial institutions on developing and maintaining a robust model risk management framework, covering aspects like model development, implementation, use, and validation. SR 11-7 is widely recognized and has become a benchmark in the industry for model risk management practices.

The Supervisory Guidance on model risk management issued by the Board of Governors of the Federal Reserve System and the Office of the Comptroller of the Currency in the United States in 2011. It provides comprehensive guidance to financial institutions on developing and maintaining a robust model risk management framework, covering aspects like model development, implementation, use, and validation.

:::

::: {.w-20-ns .content-center}

![](federal-reserve.svg){width=70% fig-alt="Board of Governers Federal Reserve logo"}
![Board of Governors Federal Reserve logo](federal-reserve.svg){width=70% fig-alt="Board of Governors Federal Reserve logo"}

:::

::::

SR 11-7 is widely recognized and has become a benchmark in the industry for model risk management practices.

SR 11-7 outlines these core requirements:

Model Risk Management
Expand All @@ -89,18 +102,22 @@ The regulation also mandates ongoing monitoring and periodic reviews to ensure m

::: {.w-20-ns .content-center}

![](bank-of-england.svg){fig-alt="Bank of England logo"}
![Bank of England logo](bank-of-england.svg){fig-alt="Bank of England logo"}

:::

::: {.w-75-ns}

### SS1/23 – Model Risk Management Principles for Banks
A policy issued by the Prudential Regulation Authority (PRA) in the UK. It encapsulates the final model risk management principles following feedback on the earlier consultation paper CP6/22. The statement provides guidelines for banks in the UK on managing model risk effectively, with particular emphasis on strategic planning and technical capabilities. It outlines principles and amendments, like clarifications on model complexity factors, senior management function responsibilities, and inclusion of dynamic adjustments in model change management, aiming to standardize MRM practices across UK banks and foster the safe adoption of emerging technologies, such as machine learning, artificial intelligence, and large language models (LLMs).

A policy issued by the Prudential Regulation Authority (PRA) in the UK. It encapsulates the final model risk management principles following feedback on the earlier consultation paper CP6/22. The statement provides guidelines for banks in the UK on managing model risk effectively, with particular emphasis on strategic planning and technical capabilities.

:::

::::

It outlines principles and amendments, like clarifications on model complexity factors, senior management function responsibilities, and inclusion of dynamic adjustments in model change management, aiming to standardize MRM practices across UK banks and foster the safe adoption of emerging technologies, such as machine learning, artificial intelligence, and large language models (LLMs).

SS1/23 outlines these core principles:

Model Identification and Model Risk Classification
Expand Down Expand Up @@ -163,7 +180,7 @@ ValidMind, as a robust tool for implementing Model Risk Management (MRM) best pr

:::: {.flex .flex-wrap .justify-around}

::: {.w-50-ns}
::: {.w-40-ns}

First line of defense — model developers
: ValidMind offers a suite of tools for model developers, facilitating thorough documentation and rigorous testing of models, aligning with the regulatory expectations of both SR 11-7 and SS1/23, particularly for models under regulatory purview.
Expand All @@ -177,14 +194,11 @@ Third line of defense — auditors
Model inventory
: The Model Inventory feature encapsulates a centralized repository for all models, aiding in streamlined tracking, management, and monitoring, simplifying compliance with the inventory mandates specified in SR 11-7 and SS1/23.

Lifecycle management and custom workflows
: ValidMind’s capabilities extend to effective model lifecycle management through configurable workflows. This structured approach to managing model risks across various lifecycle stages significantly aids in meeting the rigorous management and oversight expectations set by SR 11-7 and SS1/23.

:::

::: {.w-40-ns}

![](/assets/img/validmind-demo-compact.png){width=90% fig-alt="An image showing the main ValidMind UI main dashboard"}
::: {.w-50-ns}
Lifecycle management and custom workflows
: ValidMind’s capabilities extend to effective model lifecycle management through configurable workflows. This structured approach to managing model risks across various lifecycle stages significantly aids in meeting the rigorous management and oversight expectations set by SR 11-7 and SS1/23.

Model documentation automation
: By automating model documentation through configurable templates and test plans, ValidMind ensures consistent and accurate documentation capture, directly aligning with the documentation standards stipulated in these regulatory guidelines.
Expand All @@ -198,10 +212,22 @@ Communication and tracking

::::



By integrating these features, ValidMind provides a comprehensive platform that not only simplifies the path to compliance with SR 11-7 and SS1/23 but also embeds a culture of rigorous and transparent model risk management within the organization.

## Ready to try out ValidMind?

Our [QuickStart](/get-started/quickstart.qmd) is the quickest and easiest way to try out our product features.
:::{#quickstart}
:::


<!-- FOOTNOTES -->

[^1]: [Working with the model inventory](/guide/model-inventory/working-with-model-inventory.qmd)

[^2]: [Preparing validation reports](/guide/model-validation/preparing-validation-reports.qmd)

[^3]: [Working with model workflows](/guide/model-workflows/working-with-model-workflows.qmd)

[^4]: [Working with documentation templates](/guide/model-documentation/working-with-documentation-templates.qmd)

[^5]: [Export documentation](/guide/model-documentation/export-documentation.qmd)
Loading
Loading