Skip to content

Fix merge conflicts for prod push #645 #646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Feb 3, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions site/developer/model-testing/testing-overview.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,9 @@ listing:
contents:
- ../../notebooks/code_samples/custom_tests/implement_custom_tests.ipynb
- ../../notebooks/code_samples/custom_tests/integrate_external_test_providers.ipynb
- path: /notebooks/how_to/add_context_to_llm_descriptions.ipynb
title: "Add context to LLM-generated test descriptions"
description: "Learn how to add custom use case and test-specific context to your LLM-generated test descriptions."
---

{{< var vm.product >}} provides many built-in tests and test suites, which help you produce documentation during stages of the model development lifecycle where you need to validate that your work satisfies MRM (model risk management) requirements.
Expand Down
9 changes: 8 additions & 1 deletion site/faq/faq-testing.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,11 @@ Yes, {{< var vm.product >}} allows tests to be manipulated at several levels:
- You can change the thresholds and parameters for default tests already available in the {{< var vm.developer >}} — for instance, changing the threshold parameter for the class imbalance flag.[^5]
- You can also connect your own custom tests with the {{< var validmind.developer >}}. These custom tests are configurable and are able to run programmatically, just like the rest of the {{< var vm.developer >}}.[^6]

::: {.callout}
In addition to custom tests, you can also add use case and test-specific context for any test to enhance the LLM-generated test descriptions using the {{< var validmind.developer >}}.[^7]

:::

{{< include _faq-explainability.qmd >}}

{{< include _faq-synthetic-datasets.qmd >}}
Expand All @@ -62,4 +67,6 @@ Yes, {{< var vm.product >}} allows tests to be manipulated at several levels:

[^5]: [`ClassImbalance()`](/validmind/validmind/tests/data_validation/ClassImbalance.html#ClassImbalance)

[^6]: [Can I use my own tests?](/developer/model-testing/testing-overview.qmd#can-i-use-my-own-tests)
[^6]: [Can I use my own tests?](/developer/model-testing/testing-overview.qmd#can-i-use-my-own-tests)

[^7]: [Add context to LLM-generated test descriptions](/notebooks/how_to/add_context_to_llm_descriptions.ipynb)
24 changes: 15 additions & 9 deletions site/guide/model-documentation/work-with-content-blocks.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,11 @@ While editing a simple text block, you can insert math equations using the formu

### Generate Text with AI <sup>[beta]{.smallcaps}</sup>

::: {.callout title="Have you logged your tests?"}
Generating content drafts for your model documentation works best after you've logged tests and test descriptions for your model's tests with the {{< var validmind.developer >}},[^8] as existing test descriptions provide more context for the large language model (LLM) to draw upon.

:::

While editing a simple text block, you can have {{< var vm.product >}} assist you with generating content drafts:

1. Click **{{< fa diamond >}} <sup>[beta]{.smallcaps}</sup> (Generate Text with AI)** in the toolbar while editing a content block.
Expand All @@ -92,19 +97,18 @@ While editing a simple text block, you can have {{< var vm.product >}} assist yo
3. Review the draft composed by the {{< var vm.product >}} AI Content Builder for accuracy and relevance, then:

- Click **{{< fa download >}} Accept Text** to insert the draft into your content block.
- Click **{{< fa rotate >}} Try Again** to regenerate a different draft.
- Click **{{< fa rotate >}} Try Again** to regenerate a different draft.g
- Click **Cancel** to discard the draft and return to your documentation section.

4. After you insert the AI-generated draft, click on the text box to make the necessary edits and adjustments to your copy:

- Ensure that content is in compliance with the quality guidelines outlined by your organization.
- Use the content editing toolbar[^8] just as you would with any other text block.
- Use the content editing toolbar[^9] just as you would with any other text block.

![Generating content with AI within a simple text block](generate-with-ai.gif){width=90% fig-alt="An animation that showcases the Generate with AI feature within a simple text block" .screenshot}

::: {.callout}
When generating content drafts with AI, accepted versions and edits are retained in your {{< fa wifi >}} Model Activity[^9] just like other updates to your documentation, reports, or plans.
:::
When generating content drafts with AI, accepted versions and edits are retained in your {{< fa wifi >}} Model Activity[^10] just like other updates to your documentation, reports, or plans.


## Remove content blocks

Expand All @@ -114,7 +118,7 @@ Test-driven or metric over time blocks can be re-added later on but **text block

1. In the left sidebar, click **{{< fa cubes >}} Inventory**.

2. Select a model or find your model by applying a filter or searching for it.[^10]
2. Select a model or find your model by applying a filter or searching for it.[^11]

3. In the left sidebar that appears for your model, click **{{< fa book-open >}} Documentation**, **{{< fa shield >}} Validation Report**, or **{{< fa desktop >}} Ongoing Monitoring**.

Expand Down Expand Up @@ -159,8 +163,10 @@ Test-driven or metric over time blocks can be re-added later on but **text block

[^7]: [Collaborate with others](/guide/model-documentation/collaborate-with-others.qmd)

[^8]: [Content editing toolbar](#content-editing-toolbar)
[^8]: [Run tests and test suites](/developer/model-testing/testing-overview.qmd)

[^9]: [Content editing toolbar](#content-editing-toolbar)

[^9]: [View model activity](/guide/model-inventory/view-model-activity.qmd)
[^10]: [View model activity](/guide/model-inventory/view-model-activity.qmd)

[^10]: [Working with the model inventory](/guide/model-inventory/working-with-model-inventory.qmd#search-filter-and-sort-models)
[^11]: [Working with the model inventory](/guide/model-inventory/working-with-model-inventory.qmd#search-filter-and-sort-models)
Binary file modified site/notebooks.zip
Binary file not shown.
Loading
Loading