Release notes to sprint 60 #483

nrichers · 2024-10-18T17:32:42Z

Internal Notes for Reviewers

This PR includes release notes up to Sprint 60: LIVE PREVIEW — October 22, 2024

Frontend

Developer framework

Documentation

https://github.com/validmind/documentation/releases/tag/v2.5.18

Other improvements

New button class for Markdown links

I got tired of our hard-coded HTML buttons and added a new class for buttons to simplify the process. You can now add {.button} to any Markdown link to turn it into a button:

Before	After
`<form method="get" action="../../guide/monitoring/ongoing-monitoring.html"><button type="submit" style="color: white; background-color: #de257e; border-radius: 8px; border: none; font-size: 16px; padding: 6.25px 12.5px; margin-left: 16px; margin-bottom: 20px;">Learn about monitoring</button></form>`	`[Learn about monitoring](../../guide/monitoring/ongoing-monitoring.qmd){.button}`

Rounded edges for screenshots

Quite subtle, but our class="screenshot" was very rectilinear. I added the same rounding for corners as for the button:

Before	After

External Release Notes

github-actions · 2024-10-19T01:08:28Z

A PR preview is available: Preview URL

github-actions · 2024-10-19T01:28:46Z

PR Summary

This pull request introduces several enhancements and bug fixes to the ValidMind platform, focusing on improving user experience, functionality, and documentation.

Key Enhancements

Support for Ongoing Monitoring: Introduces ongoing monitoring for model risk management, allowing users to enable monitoring for both existing and new models. This feature includes a new notebook for logging monitoring results.
Archive and Delete Models: Adds functionality to archive and delete models, helping maintain an accurate model inventory. Archived models are read-only and can be restored by an administrator.
Bias and Fairness Tests: Introduces new metrics for evaluating model bias and fairness, along with a new Jupyter Notebook for hands-on analysis.
Metric Over Time Content Block: Adds a new content block type for displaying metrics over time, enhancing model documentation and monitoring.
Column Filtering in Tests: Allows filtering of specific columns when running tests, simplifying data analysis.
Email Verification Improvements: Enhances the email verification process with a new error page and resend functionality.
Custom Field Management: Enhances custom field functionality with Python support, a new code editor, and testing features.
Attachment Management: Introduces attachment fields for model inventory and findings, with enhanced permissions for managing attachments.
New Currency Field Type: Adds a currency type for number fields, allowing for formatted currency data.
Delete Finding Button: Allows validators to delete findings with confirmation dialogs and error handling.
Role Management Enhancements: Enables renaming and editing of organization roles, except for the Customer Admin role.
Model Activity Filtering: Improves filtering options for model activity logs, making it easier to find relevant information.
Rich Text Editor for Validation Guidelines: Introduces rich text editing for validation guideline descriptions, improving content management.

Bug Fixes

Scrollbar Display on Windows: Fixes an issue with unnecessary scrollbars in the layout sidebar on Windows, improving the user interface.

Documentation Updates

Training Content: Updates training modules with a new color scheme and improved navigation.
Model Validation Videos: Adds a series of videos to guide users through the model validation process.
Deployment Options Overview: Expands documentation on deployment options for ValidMind.
GitHub Link for Code Samples: Adds a link to the code samples page for easier access to open-source software.

Test Suggestions

Test the ongoing monitoring feature by enabling it for a model and verifying data population in the monitoring template.
Verify the archive and delete functionality by archiving a model and attempting to delete it.
Run bias and fairness tests using the new metrics and confirm the results are as expected.
Add a 'Metric Over Time' block to model documentation and check the display of metrics over a specified time range.
Test column filtering in dataset tests to ensure only selected columns are analyzed.
Simulate an email verification error and verify the new error page and resend functionality.
Create and test a custom field with Python code to ensure correct execution and output.
Add and manage attachments in model inventory and findings, checking for correct permissions.
Create a currency field and verify the formatting and precision settings.
Delete a finding and confirm the confirmation dialog and error handling work correctly.
Rename an organization role and verify the changes are reflected in the settings.
Filter model activity logs and ensure the filters work as intended.
Use the rich text editor for validation guidelines and check the formatting capabilities.

github-actions · 2024-10-19T01:28:51Z

PR Summary

This pull request introduces several enhancements and bug fixes to the ValidMind platform, focusing on improving user experience, functionality, and documentation.

Key Enhancements

Support for Ongoing Monitoring: Introduces ongoing monitoring for model risk management, allowing users to enable monitoring for both existing and new models. This feature includes a new notebook for logging monitoring results.
Archive and Delete Models: Adds functionality to archive and delete models, helping maintain an accurate model inventory. Archived models are read-only and can be restored by an administrator.
Bias and Fairness Tests: Introduces new metrics for evaluating model bias and fairness, along with a new Jupyter Notebook for hands-on analysis.
Metric Over Time Content Block: Adds a new content block type for displaying metrics over time, enhancing model documentation and monitoring.
Column Filtering in Tests: Allows filtering of specific columns when running tests, simplifying data analysis.
Email Verification Improvements: Enhances the email verification process with a new error page and resend functionality.
Custom Field Management: Enhances custom field functionality with Python support, a new code editor, and testing features.
Attachment Management: Introduces attachment fields for model inventory and findings, with enhanced permissions for managing attachments.
New Currency Field Type: Adds a currency type for number fields, allowing for formatted currency data.
Delete Finding Button: Allows validators to delete findings with confirmation dialogs and error handling.
Role Management Enhancements: Enables renaming and editing of organization roles, except for the Customer Admin role.
Model Activity Filtering: Improves filtering options for model activity logs, making it easier to find relevant information.
Rich Text Editor for Validation Guidelines: Introduces rich text editing for validation guideline descriptions, enhancing content management.

Bug Fixes

Scrollbar Display on Windows: Fixes an issue with unnecessary scrollbars in the layout sidebar on Windows, improving the user interface.

Documentation Updates

Training Content: Updates training modules with a new color scheme and improved navigation.
Model Validation Videos: Adds a series of videos to guide users through the model validation process.
Deployment Options Overview: Expands documentation on deployment options for ValidMind.
GitHub Link for Code Samples: Adds a link to the code samples page for easier access to open-source software.

Test Suggestions

Test the ongoing monitoring feature by enabling it for both new and existing models.
Verify the archive and delete functionality for models, ensuring models can be archived, deleted, and restored.
Evaluate the bias and fairness tests using the new metrics and Jupyter Notebook.
Test the 'Metric Over Time' content block by adding it to model documentation and monitoring.
Run tests with column filtering to ensure only specified columns are analyzed.
Check the email verification process, including the new error page and resend functionality.
Test the custom field management with Python support, including the code editor and testing features.
Verify attachment management in model inventory and findings, ensuring permissions are respected.
Test the new currency field type for number fields, checking formatting and precision options.
Ensure the delete finding button works as expected, with proper confirmation and error handling.
Test role management enhancements by renaming and editing roles, except for the 'Customer Admin' role.
Verify model activity filtering options for accuracy and ease of use.
Test the rich text editor for validation guidelines, ensuring content is displayed and managed correctly.

site/releases/2024-oct-22/release-notes.qmd

github-actions · 2024-10-19T01:34:31Z

A PR preview is available: Preview URL

site/releases/2024-oct-22/release-notes.qmd

github-actions · 2024-10-21T15:47:08Z

A PR preview is available: Preview URL

github-actions · 2024-10-21T16:23:06Z

A PR preview is available: Preview URL

github-actions · 2024-10-21T16:33:06Z

A PR preview is available: Preview URL

nrichers · 2024-10-21T16:54:58Z

@validbeck, great feedback, thank you. Will address where you haven't already. Some comments:

We've being using target="_blank for CTA links out of release notes, FYI. Can you please add these?

I don't think this should be true globally. We should open links in new tabs ONLY if the links are external and this is already configured in _quarto.yml via link-external-newwindow: true. Links to other pages in the docs should not be spawning tabs for the docs site all over the place, especially given how many links we now have. Less is more.

(For reference, at my previous gig we had a rule that internal docs links could never spawn another tab, as we had >300k pages and it was just messy otherwise. You can always open links in a new tab if this is your preference.)

We need some links to the metric test reference pages!
Needs a link to the DatasetDescription reference.

I'm on the fence about these but with a preference not to add too many links — release notes should tell you how to get started with something after you read them and that should be the notebooks, not some reference page. Again, less is more.

validbeck · 2024-10-21T17:01:46Z

Forgot to mention, I like the new rounded corners!

I don't think this should be true globally. We should open links in new tabs ONLY if the links are external and this is already configured in _quarto.yml via link-external-newwindow: true. Links to other pages in the docs should not be spawning tabs for the docs site all over the place, especially given how many links we now have. Less is more.

This is how the buttons were when I grabbed them to do the first round of links — I will adjust in my PR now that I know this wasn't intentional.

I'm on the fence about these but with a preference not to add too many links — release notes should tell you how to get started with something after you read them and that should be the notebooks, not some reference page. Again, less is more.

I understand what you're arguing, but in this case I vote for them. As a developer, I would expect to see the reference for the function or test you're talking about.

nrichers · 2024-10-21T18:32:01Z

This is how the buttons were when I grabbed them to do the first round of links — I will adjust in my PR now that I know this wasn't intentional.

I think that was likely a side effect of which button you grabbed to create others which just goes to show that there is an issue with the old HTML button approach. Thank you for fixing!

I understand what you're arguing, but in this case I vote for them. As a developer, I would expect to see the reference for the function or test you're talking about.

Will add for now but we should discuss the meaning of the tip of the iceberg that is release notes ... Reference content on its own likely is not enough to do something, so why not just link to something you can actually use out of the box?

General thoughts on links in release notes, for further discussion:

The user journey into a feature should guide you clearly — It shouldn't start with a bunch of doors you have to choose from at the beginning. That means fewer links with a clear path, not more options.
Progressive disclosure tells us to provide reference links in the context where they matter — i.e., in the notebook itself, where we show you how to use a function or metric.
Release notes should feel clean and simple — Linking maximalism is the antithesis of that. (This might partly be a repeat comment on a recent PR where we added scores of new buttons. I think they add clutter, increase authoring cost, and most will likely never be clicked.)
Treat release notes as if they had an expiration date — Related to the previous point, release notes are like water under the bridge: once something has been published, it receives fewer and fewer eyeballs over time. The real value lies in the core documentation for a feature that sticks around and where people will look for information.
Don't forget the authoring cost — Release notes are expensive to author to begin with, and we should limit how much time we spend on them. Looking up reference content links is fairly low on the bang-for-the-buck scale (and if the release notes are the only place where they can find a link, that's a bug and not a feature).

All of which goes to say: use links sparingly, including in release notes.

github-actions · 2024-10-21T19:05:45Z

A PR preview is available: Preview URL

nrichers · 2024-10-21T19:28:20Z

Forgot to mention, I like the new rounded corners!

@validbeck I rounded the corners a bit more via df602de which also happens to solve the issue we've been having with corner artefacts from screenshots on macOS.

Before	After

github-actions · 2024-10-21T19:34:06Z

A PR preview is available: Preview URL

github-actions · 2024-10-21T20:57:04Z

A PR preview is available: Preview URL

sc-6860/release-notes-to-sprint-60

nrichers · 2024-10-21T22:19:21Z

GLOBAL COMMENTS

We've being using target="_blank for CTA links out of release notes, FYI. Can you please add these?

Removed from the style guide, where you had added target="_blank as guidance.

I don't like the "Try it..." or "Learn about..." especially without the name of the link, and as we indicate in our style guide, nebulous/clumsy links should be avoided. For example, this is what I ended up doing in the clean-up:
(If there is a user guide, just use the name of the guide! No need for a lead-in.)

I replaced all the docs links with the actual topic title and I think that is a generally good rule to follow, thank you.

I do NOT agree with adding lead-in text for a button that describes what the button does ... That's just a simpler button replaced with text + button. As a general rule, we should be mindful about avoiding unnecessary verbiage — less is more!

Also, we have some notebooks that have very long titles that themselves need to be edited and that would make for some pretty painful buttons:

Here's how the buttons now look — doc links use the exact title and notebook links remain as before:

I also don't like the FA icons, too busy. Please remove them.

Agreed, removed.

Video thumbnails could be bigger. You can steal the dimensions from this release: https://docs-demo.vm.validmind.ai/pr_previews/nrichers/sc-6860/release-notes-to-sprint-60/releases/2024-may-22/release-notes.html#new-quickstart-video

They're the exact same size as in the docs ... If people want a bigger view, they can just watch the videos on YouTube.

I would break up some of the more chunkier paragraphs into some bullet points.

I scanned the text and searched for paragraphs of >200 words. I find release notes often look fragmented anyway, so I made no changes here but if there are specific sections where you think this would help, go ahead.

The _how-to-upgrade.qmd includes is in main, so you might need to pull in to retrieve it so you can append it at the end.

Yup, that fixed it! Not sure why it was missing when I created the release notes initially but the file is there now.

Section comments

Support for ongoing monitoring

The screenshot here needs to be cropped ;)

I would normally agree but in this case there is a new "Ongoing Monitoring" option in the navigation on the left that you would not see if you cropped the screenshot. (FWIW, the docs for ongoing monitoring use only a very cropped version that has been reduced to the graph.)

I'd also do a two column for the image with all the text up to "The monitoring template for your model automatically populates with data as your code runs..." and see what that looks like.

Since the screenshot is not being cropped, I'm going to leave this as is. I think it's fine to have a full-width screenshot here as ongoing monitoring is basically our halo feature in these release notes.

Support for bias and fairness tests

We need some links to the metric test reference pages!

Links added.

New Metric Over Time content block

The three screenshots are two cramped. I'd try two rows:

Text | Adding a new block & CTA link (button should be centered)

Selecting the block | Metric over time graph

Agreed, looks better in two columns, with a bit of padding between them.

Coincidentally, I noticed that you've been adding this in some places:

::: {.w-10-ns}
<!-- FILLER COLUMN DO NOT REMOVE -->
:::

These spacer columns should not be necessary with Tachyons CSS, you can use .pr1 or .pl1 to pad columns as needed, see https://tachyons.io/docs/layout/spacing/.

Filtering columns when running tests

Needs a link to the DatasetDescription reference.

Link added.

Enhanced custom field management with Python support

This video needs the styling we apply to all videos.

Ugh, done. I sense another CSS class ... manually adding video styling is getting old fast!

Rich text editor for validation guidelines

The columns here are squished.

As in there's not enough margin? I did add .pr4 to the left column but if there's something else you meant, let me know.

Updated training content

Let's use the listing standard here to make the training stand out a bit instead of the random screenshot link: https://docs-demo.vm.validmind.ai/pr_previews/nrichers/sc-6860/release-notes-to-sprint-60/releases/2024-may-22/release-notes.html#sandbox-getting-started

I don't know what this comment means? It's not a screenshot, it's the live training page embedded in the release notes. I did look at the example, but it's just a small callout. I'd prefer to give more emphasis to our training page here.

The column widths could also be adjusted to be a bit neater.

Fixed.

github-actions · 2024-10-21T22:20:04Z

A PR preview is available: Preview URL

github-actions · 2024-10-21T23:11:42Z

A PR preview is available: Preview URL

github-actions · 2024-10-21T23:16:05Z

A PR preview is available: Preview URL

github-actions · 2024-10-21T23:57:07Z

A PR preview is available: Preview URL

github-actions · 2024-10-22T15:38:34Z

A PR preview is available: Preview URL

* Edits up until Releases * Edits for 2024-aug-13 releases * Adjusted figcaption to center text * Edits for 2024-jul-22 releases * Edits for 2024-jun-10 releases * Edits to overview guides * Releases tweaks up to June 10 2024 * Edits to march 27 2024 wip * Final tweaks to march 27 2024 * Edited feb 14 2024 * Edited jan 26 2024 * Edited jan 18 2024 * May 22 2024 releases WIP * Edited may 22 2024 * Added _how-to-upgrade.qmd single-source * Added _how-to-upgrade.qmd to newer releases missing it * Edited nov 09 2023 * Edited oct 25 2023 * WIP Sep 27 2023 * Edited Sep 27 2023 * Tweaks to variable in buttons & links * Edited aug 15 2023 * Edited jul 24 2023 * Edited jun 22 2023 * Edited may 30 2023 * Visual tweaks * More cleanup * Some more tweaks * Final? edits * Feedback from Nik * Maybe the python docs need to be updated on remote? * Nope, just a docs-demo preview issue * Release notes to sprint 60 (#483) * Generate initial release notes * Fix LUA error * WIP edits * Initial edit pass & add images * Add new button to styles.css * Clean up styles.css * WIP more edits * WIP more edits * WIP more edits * Update button style * WIP edits * Second edit pass complete * Add button info to style guide * Minor edits * Fix broken link * Run make get-source * Final tweaks * Remove release notes notebook * Improved hover state for buttons * Adjusted style guide & README * Tweak to the convention example * Oops, I can't spell * Remove _blank guidance * Round screenshot corners more * Add fig-alt text * Address review comments from Beck * Add upgrade info and crop screencap * Ambiguous antecedents are a bad habit * Visual tweaks * Summary & final touches --------- Co-authored-by: Beck <164545837+validbeck@users.noreply.github.com> --------- Co-authored-by: Beck <164545837+validbeck@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

nrichers added 6 commits October 17, 2024 17:13

Generate initial release notes

b4dcd53

Fix LUA error

b769cac

WIP edits

9f116ca

Initial edit pass & add images

7933814

Add new button to styles.css

e04bbc0

Clean up styles.css

2e62a46

nrichers added the internal Not to be externalized in the release notes label Oct 18, 2024

nrichers added 10 commits October 18, 2024 14:38

WIP more edits

36d9291

WIP more edits

242df36

WIP more edits

65e0fe8

Update button style

b38bac7

WIP edits

5a682db

Second edit pass complete

831b9bd

Add button info to style guide

f8f2222

Minor edits

6c172af

Fix broken link

53de6d9

Run make get-source

a1e2587

nrichers added 2 commits October 18, 2024 18:19

Final tweaks

5694d99

Remove release notes notebook

53c45a6

nrichers requested review from validbeck and mehdi0501 October 19, 2024 01:28

nrichers commented Oct 19, 2024

View reviewed changes

site/releases/2024-oct-22/release-notes.qmd Outdated Show resolved Hide resolved

nrichers commented Oct 21, 2024

View reviewed changes

site/releases/2024-oct-22/release-notes.qmd Show resolved Hide resolved

nrichers commented Oct 21, 2024

View reviewed changes

site/releases/2024-oct-22/release-notes.qmd Outdated Show resolved Hide resolved

Improved hover state for buttons

3de7135

Tweak to the convention example

76d0299

Oops, I can't spell

86869aa

Remove _blank guidance

1b51f72

Round screenshot corners more

df602de

Add fig-alt text

629fbe8

nrichers added 2 commits October 21, 2024 15:13

Address review comments from Beck

aa47c66

Merge branch 'main' of github.com:validmind/documentation into nrichers/

9a42104

sc-6860/release-notes-to-sprint-60

nrichers and others added 2 commits October 21, 2024 16:03

Add upgrade info and crop screencap

c2a90aa

Ambiguous antecedents are a bad habit

3420dd4

Visual tweaks

3b0c4cf

validbeck approved these changes Oct 22, 2024

View reviewed changes

Summary & final touches

6fe4465

nrichers merged commit 3798d37 into main Oct 22, 2024
5 checks passed

nrichers deleted the nrichers/sc-6860/release-notes-to-sprint-60 branch October 22, 2024 23:07

Release notes to sprint 60 #483

Release notes to sprint 60 #483

Uh oh!

Conversation

nrichers commented Oct 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Internal Notes for Reviewers

Frontend

Developer framework

Documentation

Other improvements

New button class for Markdown links

Rounded edges for screenshots

External Release Notes

Uh oh!

github-actions bot commented Oct 19, 2024

Uh oh!

github-actions bot commented Oct 19, 2024

PR Summary

Key Enhancements

Bug Fixes

Documentation Updates

Test Suggestions

Uh oh!

github-actions bot commented Oct 19, 2024

PR Summary

Key Enhancements

Bug Fixes

Documentation Updates

Test Suggestions

Uh oh!

Uh oh!

github-actions bot commented Oct 19, 2024

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 21, 2024

Uh oh!

github-actions bot commented Oct 21, 2024

Uh oh!

github-actions bot commented Oct 21, 2024

Uh oh!

nrichers commented Oct 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

validbeck commented Oct 21, 2024

Uh oh!

nrichers commented Oct 21, 2024

Uh oh!

github-actions bot commented Oct 21, 2024

Uh oh!

nrichers commented Oct 21, 2024

Uh oh!

github-actions bot commented Oct 21, 2024

Uh oh!

github-actions bot commented Oct 21, 2024

Uh oh!

nrichers commented Oct 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

GLOBAL COMMENTS

Section comments

Support for ongoing monitoring

Support for bias and fairness tests

New Metric Over Time content block

Filtering columns when running tests

Enhanced custom field management with Python support

Rich text editor for validation guidelines

Updated training content

Uh oh!

github-actions bot commented Oct 21, 2024

Uh oh!

github-actions bot commented Oct 21, 2024

Uh oh!

github-actions bot commented Oct 21, 2024

Uh oh!

github-actions bot commented Oct 21, 2024

Uh oh!

github-actions bot commented Oct 22, 2024

Uh oh!

Uh oh!

nrichers commented Oct 18, 2024 •

edited

Loading

nrichers commented Oct 21, 2024 •

edited

Loading

nrichers commented Oct 21, 2024 •

edited

Loading