Skip to content

Commit 74a2fed

Browse files
validbeckvalidbeck
authored
Pulling in latest from developer-framework PR#195 (#438)
* Pulling in latest from developer-framework * Added .screenshot class to model inventory fields
1 parent 62b5cfb commit 74a2fed

File tree

255 files changed

+1998
-15943
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

255 files changed

+1998
-15943
lines changed

‎site/notebooks.zip

8.01 KB
Binary file not shown.

‎site/notebooks/code_samples/credit_risk/application_scorecard_demo.ipynb

Lines changed: 60 additions & 92 deletions
Original file line numberDiff line numberDiff line change
@@ -86,9 +86,9 @@
8686
"### New to ValidMind?\n",
8787
"If you haven't already seen our [Get started with the ValidMind Developer Framework](https://docs.validmind.ai/developer/get-started-developer-framework.html), we recommend you begin by exploring the available resources in this section. There, you can learn more about documenting models, find code samples, or read our developer reference.\n",
8888
"\n",
89-
"<div class=\"alert alert-block alert-info\" style=\"background-color: #f7e4ee; color: #222425; border: 1px solid #222425;\">For access to all features available in this notebook, create a free ValidMind account.\n",
90-
"\n",
91-
"Signing up is FREE — <a href=\"https://app.prod.validmind.ai\"><b>Sign up now</b></a></div>"
89+
"<div class=\"alert alert-block alert-info\" style=\"background-color: #f7e4ee; color: black; border: 1px solid black;\"><b>For access to all features available in this notebook, create a free ValidMind account.</b>\n",
90+
"<br></br>\n",
91+
"Signing up is FREE — <a href=\"https://docs.validmind.ai/guide/configuration/register-with-validmind.html\"><b>Register with ValidMind</b></a></div>\n"
9292
]
9393
},
9494
{
@@ -155,7 +155,7 @@
155155
"\n",
156156
"Get your code snippet:\n",
157157
"\n",
158-
"1. In a browser, log into the [Platform UI](https://app.prod.validmind.ai).\n",
158+
"1. In a browser, [log in to ValidMind](https://docs.validmind.ai/guide/configuration/log-in-to-validmind.html).\n",
159159
"\n",
160160
"2. In the left sidebar, navigate to **Model Inventory** and click **+ Register new model**.\n",
161161
"\n",
@@ -595,8 +595,8 @@
595595
"source": [
596596
"test = vm.tests.run_test(\n",
597597
" \"validmind.data_validation.TabularDescriptionTables:raw_dataset\",\n",
598-
" inputs={\n",
599-
" \"dataset\": vm_raw_dataset,\n",
598+
" input_grid={\n",
599+
" \"dataset\": [vm_raw_dataset],\n",
600600
" },\n",
601601
")\n",
602602
"test.log()"
@@ -610,8 +610,8 @@
610610
"source": [
611611
"test = vm.tests.run_test(\n",
612612
" \"validmind.data_validation.MissingValuesBarPlot:raw_dataset\",\n",
613-
" inputs={\n",
614-
" \"dataset\": vm_raw_dataset,\n",
613+
" input_grid={\n",
614+
" \"dataset\": [vm_raw_dataset]\n",
615615
" },\n",
616616
")\n",
617617
"test.log()"
@@ -646,8 +646,8 @@
646646
"source": [
647647
"test = vm.tests.run_test(\n",
648648
" \"validmind.data_validation.TabularDescriptionTables:preprocess_dataset\",\n",
649-
" inputs={\n",
650-
" \"dataset\": vm_preprocess_dataset,\n",
649+
" input_grid={\n",
650+
" \"dataset\": [vm_preprocess_dataset]\n",
651651
" },\n",
652652
")\n",
653653
"test.log()"
@@ -661,8 +661,8 @@
661661
"source": [
662662
"test = vm.tests.run_test(\n",
663663
" \"validmind.data_validation.IQROutliersTable\",\n",
664-
" inputs={\n",
665-
" \"dataset\": vm_preprocess_dataset,\n",
664+
" input_grid={\n",
665+
" \"dataset\": [vm_preprocess_dataset]\n",
666666
" },\n",
667667
")\n",
668668
"test.log()"
@@ -676,8 +676,8 @@
676676
"source": [
677677
"test = vm.tests.run_test(\n",
678678
" \"validmind.data_validation.ClassImbalance\",\n",
679-
" inputs={\n",
680-
" \"dataset\": vm_preprocess_dataset,\n",
679+
" input_grid={\n",
680+
" \"dataset\": [vm_preprocess_dataset]\n",
681681
" },\n",
682682
")\n",
683683
"test.log()"
@@ -691,8 +691,8 @@
691691
"source": [
692692
"test = vm.tests.run_test(\n",
693693
" \"validmind.data_validation.TabularNumericalHistograms\",\n",
694-
" inputs={\n",
695-
" \"dataset\": vm_preprocess_dataset,\n",
694+
" input_grid={\n",
695+
" \"dataset\": [vm_preprocess_dataset]\n",
696696
" },\n",
697697
")\n",
698698
"test.log()"
@@ -706,8 +706,8 @@
706706
"source": [
707707
"test = vm.tests.run_test(\n",
708708
" \"validmind.data_validation.TabularCategoricalBarPlots\",\n",
709-
" inputs={\n",
710-
" \"dataset\": vm_preprocess_dataset,\n",
709+
" input_grid={\n",
710+
" \"dataset\": [vm_preprocess_dataset]\n",
711711
" },\n",
712712
")\n",
713713
"test.log()"
@@ -721,8 +721,8 @@
721721
"source": [
722722
"test = vm.tests.run_test(\n",
723723
" \"validmind.data_validation.TargetRateBarPlots\",\n",
724-
" inputs={\n",
725-
" \"dataset\": vm_preprocess_dataset,\n",
724+
" input_grid={\n",
725+
" \"dataset\": [vm_preprocess_dataset]\n",
726726
" },\n",
727727
" params={\n",
728728
" \"default_column\": lending_club.target_column,\n",
@@ -740,8 +740,8 @@
740740
"source": [
741741
"test = vm.tests.run_test(\n",
742742
" \"validmind.data_validation.PearsonCorrelationMatrix\",\n",
743-
" inputs={\n",
744-
" \"dataset\": vm_preprocess_dataset,\n",
743+
" input_grid={\n",
744+
" \"dataset\": [vm_preprocess_dataset]\n",
745745
" },\n",
746746
")\n",
747747
"test.log()"
@@ -755,8 +755,8 @@
755755
"source": [
756756
"test = vm.tests.run_test(\n",
757757
" \"validmind.data_validation.HighPearsonCorrelation\",\n",
758-
" inputs={\n",
759-
" \"dataset\": vm_preprocess_dataset,\n",
758+
" input_grid={\n",
759+
" \"dataset\": [vm_preprocess_dataset]\n",
760760
" },\n",
761761
")\n",
762762
"test.log()"
@@ -784,8 +784,8 @@
784784
"source": [
785785
"test = vm.tests.run_test(\n",
786786
" \"validmind.data_validation.WOEBinTable\",\n",
787-
" inputs={\n",
788-
" \"dataset\": vm_preprocess_dataset,\n",
787+
" input_grid={\n",
788+
" \"dataset\": [vm_preprocess_dataset]\n",
789789
" },\n",
790790
" params={\n",
791791
" \"breaks_adj\": lending_club.breaks_adj,\n",
@@ -802,8 +802,8 @@
802802
"source": [
803803
"test = vm.tests.run_test(\n",
804804
" \"validmind.data_validation.WOEBinPlots\",\n",
805-
" inputs={\n",
806-
" \"dataset\": vm_preprocess_dataset,\n",
805+
" input_grid={\n",
806+
" \"dataset\": [vm_preprocess_dataset]\n",
807807
" },\n",
808808
" params={\n",
809809
" \"breaks_adj\": lending_club.breaks_adj,\n",
@@ -866,24 +866,9 @@
866866
"outputs": [],
867867
"source": [
868868
"test = vm.tests.run_test(\n",
869-
" \"validmind.model_validation.statsmodels.RegressionCoeffsPlot\",\n",
870-
" inputs={\n",
871-
" \"models\": [vm_model],\n",
872-
" },\n",
873-
")\n",
874-
"test.log()"
875-
]
876-
},
877-
{
878-
"cell_type": "code",
879-
"execution_count": null,
880-
"metadata": {},
881-
"outputs": [],
882-
"source": [
883-
"test = vm.tests.run_test(\n",
884-
" \"validmind.model_validation.statsmodels.RegressionModelsCoeffs\",\n",
885-
" inputs={\n",
886-
" \"models\": [vm_model],\n",
869+
" \"validmind.model_validation.statsmodels.RegressionCoeffs\",\n",
870+
" input_grid={\n",
871+
" \"model\": [vm_model]\n",
887872
" },\n",
888873
")\n",
889874
"test.log()"
@@ -917,26 +902,10 @@
917902
"outputs": [],
918903
"source": [
919904
"test = vm.tests.run_test(\n",
920-
" \"validmind.model_validation.sklearn.ClassifierPerformance:train_dataset\",\n",
921-
" inputs={\n",
922-
" \"dataset\": vm_train_ds,\n",
923-
" \"model\": vm_model,\n",
924-
" },\n",
925-
")\n",
926-
"test.log()"
927-
]
928-
},
929-
{
930-
"cell_type": "code",
931-
"execution_count": null,
932-
"metadata": {},
933-
"outputs": [],
934-
"source": [
935-
"test = vm.tests.run_test(\n",
936-
" \"validmind.model_validation.sklearn.ClassifierPerformance:test_dataset\",\n",
937-
" inputs={\n",
938-
" \"dataset\": vm_test_ds,\n",
939-
" \"model\": vm_model,\n",
905+
" \"validmind.model_validation.sklearn.ClassifierPerformance\",\n",
906+
" input_grid={\n",
907+
" \"dataset\": [vm_train_ds, vm_test_ds],\n",
908+
" \"model\": [vm_model],\n",
940909
" },\n",
941910
")\n",
942911
"test.log()"
@@ -950,9 +919,9 @@
950919
"source": [
951920
"test = vm.tests.run_test(\n",
952921
" \"validmind.model_validation.statsmodels.GINITable\",\n",
953-
" inputs={\n",
954-
" \"datasets\": [vm_train_ds, vm_test_ds],\n",
955-
" \"model\": vm_model,\n",
922+
" input_grid={\n",
923+
" \"dataset\": [vm_train_ds, vm_test_ds],\n",
924+
" \"model\": [vm_model],\n",
956925
" },\n",
957926
")\n",
958927
"test.log()"
@@ -966,9 +935,9 @@
966935
"source": [
967936
"test = vm.tests.run_test(\n",
968937
" \"validmind.model_validation.sklearn.ConfusionMatrix\",\n",
969-
" inputs={\n",
970-
" \"dataset\": vm_test_ds,\n",
971-
" \"model\": vm_model,\n",
938+
" input_grid={\n",
939+
" \"dataset\": [vm_train_ds, vm_test_ds],\n",
940+
" \"model\": [vm_model],\n",
972941
" },\n",
973942
")\n",
974943
"test.log()"
@@ -982,9 +951,9 @@
982951
"source": [
983952
"test = vm.tests.run_test(\n",
984953
" \"validmind.model_validation.sklearn.ROCCurve\",\n",
985-
" inputs={\n",
986-
" \"model\": vm_model,\n",
987-
" \"dataset\": vm_test_ds,\n",
954+
" input_grid={\n",
955+
" \"dataset\": [vm_train_ds, vm_test_ds],\n",
956+
" \"model\": [vm_model],\n",
988957
" },\n",
989958
")\n",
990959
"test.log()"
@@ -998,9 +967,9 @@
998967
"source": [
999968
"test = vm.tests.run_test(\n",
1000969
" \"validmind.model_validation.statsmodels.PredictionProbabilitiesHistogram\",\n",
1001-
" inputs={\n",
1002-
" \"model\": vm_model,\n",
1003-
" \"datasets\": [vm_train_ds, vm_test_ds],\n",
970+
" input_grid={\n",
971+
" \"dataset\": [vm_train_ds, vm_test_ds],\n",
972+
" \"model\": [vm_model],\n",
1004973
" },\n",
1005974
")\n",
1006975
"test.log()"
@@ -1014,9 +983,9 @@
1014983
"source": [
1015984
"test = vm.tests.run_test(\n",
1016985
" \"validmind.model_validation.statsmodels.CumulativePredictionProbabilities\",\n",
1017-
" inputs={\n",
1018-
" \"model\": vm_model,\n",
1019-
" \"datasets\": [vm_train_ds, vm_test_ds],\n",
986+
" input_grid={\n",
987+
" \"model\": [vm_model],\n",
988+
" \"dataset\": [vm_train_ds, vm_test_ds],\n",
1020989
" },\n",
1021990
")\n",
1022991
"test.log()"
@@ -1030,9 +999,8 @@
1030999
"source": [
10311000
"test = vm.tests.run_test(\n",
10321001
" \"validmind.model_validation.statsmodels.ScorecardHistogram\",\n",
1033-
" inputs={\n",
1034-
" \"model\": vm_model,\n",
1035-
" \"datasets\": [vm_train_ds, vm_test_ds],\n",
1002+
" input_grid={\n",
1003+
" \"dataset\": [vm_train_ds, vm_test_ds],\n",
10361004
" },\n",
10371005
" params={\n",
10381006
" \"score_column\": \"glm_scores\",\n",
@@ -1066,9 +1034,9 @@
10661034
"source": [
10671035
"test = vm.tests.run_test(\n",
10681036
" \"validmind.model_validation.statsmodels.RegressionPermutationFeatureImportance\",\n",
1069-
" inputs={\n",
1070-
" \"model\": vm_model,\n",
1071-
" \"dataset\": vm_test_ds,\n",
1037+
" input_grid={\n",
1038+
" \"model\": [vm_model],\n",
1039+
" \"dataset\": [vm_train_ds, vm_test_ds],\n",
10721040
" },\n",
10731041
")\n",
10741042
"test.log()"
@@ -1082,9 +1050,9 @@
10821050
"source": [
10831051
"test = vm.tests.run_test(\n",
10841052
" \"validmind.model_validation.FeaturesAUC\",\n",
1085-
" inputs={\n",
1086-
" \"model\": vm_model,\n",
1087-
" \"dataset\": vm_test_ds,\n",
1053+
" input_grid={\n",
1054+
" \"model\": [vm_model],\n",
1055+
" \"dataset\": [vm_train_ds, vm_test_ds],\n",
10881056
" },\n",
10891057
")\n",
10901058
"test.log()"
@@ -1104,7 +1072,7 @@
11041072
"\n",
11051073
"### Work with your model documentation\n",
11061074
"\n",
1107-
"1. In the [Platform UI](https://app.prod.validmind.ai), go to the **Documentation** page for the model you registered earlier.\n",
1075+
"1. In the ValidMind Platform UI, go to the **Documentation** page for the model you registered earlier. ([Need more help?](https://docs.validmind.ai/guide/model-documentation/working-with-model-documentation.html))\n",
11081076
"\n",
11091077
"2. Expand the following sections and take a look around:\n",
11101078
"\n",
@@ -1142,7 +1110,7 @@
11421110
"name": "python",
11431111
"nbconvert_exporter": "python",
11441112
"pygments_lexer": "ipython3",
1145-
"version": "3.10.13"
1113+
"version": "3.10.14"
11461114
}
11471115
},
11481116
"nbformat": 4,

‎site/notebooks/code_samples/custom_tests/implement_custom_tests.ipynb

Lines changed: 5 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -78,9 +78,9 @@
7878
"\n",
7979
"If you haven't already seen our [Get started with the ValidMind Developer Framework](https://docs.validmind.ai/developer/get-started-developer-framework.html), we recommend you explore the available resources for developers at some point. There, you can learn more about documenting models, find code samples, or read our developer reference.\n",
8080
"\n",
81-
"<div class=\"alert alert-block alert-info\" style=\"background-color: #f7e4ee; color: #222425; border: 1px solid #222425;\">For access to all features available in this notebook, create a free ValidMind account.\n",
82-
"\n",
83-
"Signing up is FREE — <a href=\"https://app.prod.validmind.ai\"><b>Sign up now</b></a></div>\n",
81+
"<div class=\"alert alert-block alert-info\" style=\"background-color: #f7e4ee; color: black; border: 1px solid black;\"><b>For access to all features available in this notebook, create a free ValidMind account.</b>\n",
82+
"<br></br>\n",
83+
"Signing up is FREE — <a href=\"https://docs.validmind.ai/guide/configuration/register-with-validmind.html\"><b>Register with ValidMind</b></a></div>\n",
8484
"\n",
8585
"<a id='toc2_3_'></a>\n",
8686
"\n",
@@ -110,28 +110,6 @@
110110
"Example: the [`classifier_full_suite`](https://docs.validmind.ai/validmind/validmind/test_suites/classifier.html#ClassifierFullSuite) test suite runs tests from the [`tabular_dataset`](https://docs.validmind.ai/validmind/validmind/test_suites/tabular_datasets.html) and [`classifier`](https://docs.validmind.ai/validmind/validmind/test_suites/classifier.html) test suites to fully document the data and model sections for binary classification model use-cases.\n"
111111
]
112112
},
113-
{
114-
"cell_type": "markdown",
115-
"metadata": {},
116-
"source": [
117-
"<a id='toc3_'></a>\n",
118-
"\n",
119-
"## Before you begin\n",
120-
"\n",
121-
"::: {.callout-tip}\n",
122-
"\n",
123-
"<a id='toc3_1_'></a>\n",
124-
"\n",
125-
"### New to ValidMind?\n",
126-
"\n",
127-
"To access the ValidMind Platform UI, you'll need an account.\n",
128-
"\n",
129-
"Signing up is FREE — **[Create your account](https://app.prod.validmind.ai)**.\n",
130-
":::\n",
131-
"\n",
132-
"If you encounter errors due to missing modules in your Python environment, install the modules with `pip install`, and then re-run the notebook. For more help, refer to [Installing Python Modules](https://docs.python.org/3/installing/index.html).\n"
133-
]
134-
},
135113
{
136114
"cell_type": "markdown",
137115
"metadata": {},
@@ -164,7 +142,7 @@
164142
"\n",
165143
"Get your code snippet:\n",
166144
"\n",
167-
"1. In a browser, log into the [Platform UI](https://app.prod.validmind.ai).\n",
145+
"1. In a browser, [log in to ValidMind](https://docs.validmind.ai/guide/configuration/log-in-to-validmind.html).\n",
168146
"\n",
169147
"2. In the left sidebar, navigate to **Model Inventory** and click **+ Register new model**.\n",
170148
"\n",
@@ -837,7 +815,7 @@
837815
"\n",
838816
"### Work with your model documentation\n",
839817
"\n",
840-
"1. From the [**Model Inventory**](https://app.prod.validmind.ai/model-inventory) in the ValidMind Platform UI, go to the model you registered earlier.\n",
818+
"1. From the **Model Inventory** in the ValidMind Platform UI, go to the model you registered earlier. ([Need more help?](https://docs.validmind.ai/guide/model-inventory/working-with-model-inventory.html))\n",
841819
"\n",
842820
"2. Click and expand the **Model Development** section.\n",
843821
"\n",

0 commit comments

Comments
 (0)