Pulling in latest changes from PR184

site/notebooks/code_samples/customization/customizing_tests_with_output_templates.ipynb

@@ -631,7 +631,10 @@
         "\n",
         "#### Add the output template to the documentation template\n",
         "\n",
-        "First, go to your project in the ValidMind UI and go to Settings > Templates. Find the Binary Classification Template that is used by the Customer Churn project. Click on the `Edit` button to bring up the template editor. Then, add the following content block below the existing `validmind.model_validation.sklearn.ClassifierPerformance` test:\n",
+        "Head to the ValidMind platform UI to [customize your documentation template](https://docs.validmind.ai/guide/model-documentation/customize-documentation-templates.html). \n",
+        "\n",
+        "- Find the Binary Classification Template that is used by the Customer Churn model\n",
+        "- Add the following content block below the existing `validmind.model_validation.sklearn.ClassifierPerformance` test:\n",
         "\n",
         "```yaml\n",
         "- content_type: metric\n",

site/notebooks/code_samples/nlp_and_llm/foundation_models_integration_demo.ipynb

@@ -277,7 +277,7 @@
             "cell_type": "markdown",
             "metadata": {},
             "source": [
-                "Next, use the ValidMind Developer Framework to run validation tests on the model. The `vm.run_documentation_tests` function analyzes the current project's documentation template and collects all the tests associated with it into a test suite.\n",
+                "Next, use the ValidMind Developer Framework to run validation tests on the model. The `vm.run_documentation_tests` function analyzes the current model's documentation template and collects all the tests associated with it into a test suite.\n",
                 "\n",
                 "The function then runs the test suite, logs the results to the ValidMind API and displays them to you.\n"
             ]

site/notebooks/code_samples/nlp_and_llm/rag_documentation_demo.ipynb

@@ -81,7 +81,7 @@
    "source": [
     "### ValidMind Initialization\n",
     "\n",
-    "Now we will import and initialize the ValidMind framework so we can connect to our project in the ValidMind platform. This will let us log inputs, plots, and test results to our model documentation."
+    "Now we will import and initialize the ValidMind framework so we can connect to our model in the ValidMind platform. This will let us log inputs, plots, and test results to our model documentation."
    ]
   },
   {
@@ -138,7 +138,7 @@
    "source": [
     "# Dataset Loader\n",
     "\n",
-    "Great, now that we have all of our dependencies installed, the developer framework initialized and connected to our model documentation project and our OpenAI API key setup, we can go ahead and load our datasets. We will use the synthetic `RFP` dataset included with ValidMind for this notebook. This dataset contains a variety of RFP questions and ground truth answers that we can use both as the source where our Retriever will search for similar question-answer pairs as well as our test set for evaluating the performance of our RAG model. To do this, we just have to load it and call the preprocess function to get a split of the data into train and test sets."
+    "Great, now that we have all of our dependencies installed, the developer framework initialized and connected to our model and our OpenAI API key setup, we can go ahead and load our datasets. We will use the synthetic `RFP` dataset included with ValidMind for this notebook. This dataset contains a variety of RFP questions and ground truth answers that we can use both as the source where our Retriever will search for similar question-answer pairs as well as our test set for evaluating the performance of our RAG model. To do this, we just have to load it and call the preprocess function to get a split of the data into train and test sets."
    ]
   },
   {
@@ -399,7 +399,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "What we have done here is to initialize the `OpenAIEmbeddings` class so it uses OpenAI's `text-embedding-3-small` model. We then created an `embed` function that takes in an `input` dictionary and uses the `embed_query` method of the embedding client to compute the embeddings of the `question`. We use an `embed` function since that is how ValidMind supports any custom model. We will use this strategy for the retrieval and generator models as well but you could also use, say, a HuggingFace model directly. See the documentation for more information on which model types are directly supported - [ValidMind Documentation](https://docs.validmind.ai/validmind/validmind.html)... Finally, we use the `init_model` function from the ValidMind framework to create a `VMModel` object that can be used in ValidMind tests. This also logs the model to our model documentation project and any test that uses the model will be linked to the logged model and its metadata."
+    "What we have done here is to initialize the `OpenAIEmbeddings` class so it uses OpenAI's `text-embedding-3-small` model. We then created an `embed` function that takes in an `input` dictionary and uses the `embed_query` method of the embedding client to compute the embeddings of the `question`. We use an `embed` function since that is how ValidMind supports any custom model. We will use this strategy for the retrieval and generator models as well but you could also use, say, a HuggingFace model directly. See the documentation for more information on which model types are directly supported - [ValidMind Documentation](https://docs.validmind.ai/validmind/validmind.html)... Finally, we use the `init_model` function from the ValidMind framework to create a `VMModel` object that can be used in ValidMind tests. This also logs the model to our model documentation and any test that uses the model will be linked to the logged model and its metadata."
    ]
   },
   {

site/notebooks/how_to/load_datasets_predictions.ipynb

@@ -188,7 +188,7 @@
                 "\n",
                 "### Preview the documentation template\n",
                 "\n",
-                "A template predefines sections for your documentation project and provides a general outline to follow, making the documentation process much easier.\n",
+                "A template predefines sections for your model documentation and provides a general outline to follow, making the documentation process much easier.\n",
                 "\n",
                 "You will upload documentation and test results into this template later on. For now, take a look at the structure that the template provides with the `vm.preview_template()` function from the ValidMind library and note the empty sections:\n"
             ]

site/notebooks/how_to/run_documentation_tests_with_config.ipynb

@@ -187,7 +187,7 @@
                 "\n",
                 "### Preview the documentation template\n",
                 "\n",
-                "A template predefines sections for your documentation project and provides a general outline to follow, making the documentation process much easier.\n",
+                "A template predefines sections for your model documentation and provides a general outline to follow, making the documentation process much easier.\n",
                 "\n",
                 "You will upload documentation and test results into this template later on. For now, take a look at the structure that the template provides with the `vm.preview_template()` function from the ValidMind library and note the empty sections:\n"
             ]

site/notebooks/how_to/run_tests_that_require_multiple_datasets.ipynb

@@ -185,7 +185,7 @@
                 "\n",
                 "### Preview the documentation template\n",
                 "\n",
-                "A template predefines sections for your documentation project and provides a general outline to follow, making the documentation process much easier.\n",
+                "A template predefines sections for your model documentation and provides a general outline to follow, making the documentation process much easier.\n",
                 "\n",
                 "You will upload documentation and test results into this template later on. For now, take a look at the structure that the template provides with the `vm.preview_template()` function from the ValidMind library and note the empty sections:\n"
             ]

site/python-docs/validmind.html

@@ -150,7 +150,7 @@ <h1 class="modulename">
                 <section id="__version__">
                     <div class="attr variable">
             <span class="name">__version__</span>        =
-<span class="default_value">&#39;2.5.7&#39;</span>
+<span class="default_value">&#39;2.5.13&#39;</span>
 
 
     </div>
@@ -685,7 +685,7 @@ <h6 id="arguments">Arguments:</h6>
 
 <ul>
 <li><strong>results (list):</strong>  A list of ThresholdTestResults objects</li>
-<li><strong>inputs (list):</strong>  A list of input keys (names) that were used to run the test</li>
+<li><strong>inputs (list):</strong>  A list of input IDs that were used to run the test</li>
 </ul>
 
 <h6 id="raises">Raises:</h6>

site/python-docs/validmind/tests/data_validation.html

@@ -34,13 +34,10 @@ <h3>Submodules</h3>
             <ul>
                     <li><a href="data_validation/ACFandPACFPlot.html">ACFandPACFPlot</a></li>
                     <li><a href="data_validation/ADF.html">ADF</a></li>
-                    <li><a href="data_validation/ANOVAOneWayTable.html">ANOVAOneWayTable</a></li>
                     <li><a href="data_validation/AutoAR.html">AutoAR</a></li>
                     <li><a href="data_validation/AutoMA.html">AutoMA</a></li>
                     <li><a href="data_validation/AutoSeasonality.html">AutoSeasonality</a></li>
                     <li><a href="data_validation/AutoStationarity.html">AutoStationarity</a></li>
-                    <li><a href="data_validation/BivariateFeaturesBarPlots.html">BivariateFeaturesBarPlots</a></li>
-                    <li><a href="data_validation/BivariateHistograms.html">BivariateHistograms</a></li>
                     <li><a href="data_validation/BivariateScatterPlots.html">BivariateScatterPlots</a></li>
                     <li><a href="data_validation/ChiSquaredFeaturesTable.html">ChiSquaredFeaturesTable</a></li>
                     <li><a href="data_validation/ClassImbalance.html">ClassImbalance</a></li>
@@ -51,7 +48,6 @@ <h3>Submodules</h3>
                     <li><a href="data_validation/Duplicates.html">Duplicates</a></li>
                     <li><a href="data_validation/EngleGrangerCoint.html">EngleGrangerCoint</a></li>
                     <li><a href="data_validation/FeatureTargetCorrelationPlot.html">FeatureTargetCorrelationPlot</a></li>
-                    <li><a href="data_validation/HeatmapFeatureCorrelations.html">HeatmapFeatureCorrelations</a></li>
                     <li><a href="data_validation/HighCardinality.html">HighCardinality</a></li>
                     <li><a href="data_validation/HighPearsonCorrelation.html">HighPearsonCorrelation</a></li>
                     <li><a href="data_validation/IQROutliersBarPlot.html">IQROutliersBarPlot</a></li>
@@ -61,7 +57,6 @@ <h3>Submodules</h3>
                     <li><a href="data_validation/LaggedCorrelationHeatmap.html">LaggedCorrelationHeatmap</a></li>
                     <li><a href="data_validation/MissingValues.html">MissingValues</a></li>
                     <li><a href="data_validation/MissingValuesBarPlot.html">MissingValuesBarPlot</a></li>
-                    <li><a href="data_validation/MissingValuesRisk.html">MissingValuesRisk</a></li>
                     <li><a href="data_validation/PearsonCorrelationMatrix.html">PearsonCorrelationMatrix</a></li>
                     <li><a href="data_validation/PhillipsPerronArch.html">PhillipsPerronArch</a></li>
                     <li><a href="data_validation/RollingStatsPlot.html">RollingStatsPlot</a></li>

site/python-docs/validmind/tests/data_validation/ACFandPACFPlot.html

@@ -75,27 +75,29 @@ <h1 class="modulename">
             <div class="docstring"><p>Analyzes time series data using Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF) plots to
 reveal trends and correlations.</p>
 
-<p><strong>Purpose</strong>: The ACF (Autocorrelation Function) and PACF (Partial Autocorrelation Function) plot test is employed
-to analyze time series data in machine learning models. It illuminates the correlation of the data over time by
-plotting the correlation of the series with its own lags (ACF), and the correlations after removing effects already
-accounted for by earlier lags (PACF). This information can identify trends, such as seasonality, degrees of
-autocorrelation, and inform the selection of order parameters for AutoRegressive Integrated Moving Average (ARIMA)
-models.</p>
+<h3 id="purpose">Purpose</h3>
 
-<p><strong>Test Mechanism</strong>: The <code><a href="#ACFandPACFPlot">ACFandPACFPlot</a></code> test accepts a dataset with a time-based index. It first confirms the
-index is of a datetime type, then handles any NaN values. The test subsequently generates ACF and PACF plots for
-each column in the dataset, producing a subplot for each. If the dataset doesn't include key columns, an error is
-returned.</p>
+<p>The ACF (Autocorrelation Function) and PACF (Partial Autocorrelation Function) plot test is employed to analyze
+time series data in machine learning models. It illuminates the correlation of the data over time by plotting the
+correlation of the series with its own lags (ACF), and the correlations after removing effects already accounted
+for by earlier lags (PACF). This information can identify trends, such as seasonality, degrees of autocorrelation,
+and inform the selection of order parameters for AutoRegressive Integrated Moving Average (ARIMA) models.</p>
 
-<p><strong>Signs of High Risk</strong>:</p>
+<h3 id="test-mechanism">Test Mechanism</h3>
+
+<p>The <code><a href="#ACFandPACFPlot">ACFandPACFPlot</a></code> test accepts a dataset with a time-based index. It first confirms the index is of a datetime
+type, then handles any NaN values. The test subsequently generates ACF and PACF plots for each column in the
+dataset, producing a subplot for each. If the dataset doesn't include key columns, an error is returned.</p>
+
+<h3 id="signs-of-high-risk">Signs of High Risk</h3>
 
 <ul>
 <li>Sudden drops in the correlation at a specific lag might signal a model at high risk.</li>
 <li>Consistent high correlation across multiple lags could also indicate non-stationarity in the data, which may
 suggest that a model estimated on this data won't generalize well to future, unknown data.</li>
 </ul>
 
-<p><strong>Strengths</strong>:</p>
+<h3 id="strengths">Strengths</h3>
 
 <ul>
 <li>ACF and PACF plots offer clear graphical representations of the correlations in time series data.</li>
@@ -105,7 +107,7 @@ <h1 class="modulename">
 parameters.</li>
 </ul>
 
-<p><strong>Limitations</strong>:</p>
+<h3 id="limitations">Limitations</h3>
 
 <ul>
 <li>ACF and PACF plots are exclusively for time series data and hence, can't be applied to all ML models.</li>

site/python-docs/validmind/tests/data_validation/ADF.html

@@ -81,39 +81,43 @@ <h1 class="modulename">
 
             <div class="docstring"><p>Assesses the stationarity of a time series dataset using the Augmented Dickey-Fuller (ADF) test.</p>
 
-<p><strong>Purpose</strong>: The Augmented Dickey-Fuller (ADF) test metric is used here to determine the order of integration,
-i.e., the stationarity of a given time series data. The stationary property of data is pivotal in many machine
-learning models as it impacts the reliability and effectiveness of predictions and forecasts.</p>
+<h3 id="purpose">Purpose</h3>
 
-<p><strong>Test Mechanism</strong>: The ADF test starts by executing the ADF function from the statsmodels library on every feature
-of the dataset. Multiple outputs are generated for each run, including the ADF test statistic and p-value, count of
-lags used, the number of observations factored into the test, critical values at various confidence levels, and the
-maximized information criterion. These results are stored for each feature for subsequent analysis.</p>
+<p>The Augmented Dickey-Fuller (ADF) test metric is used to determine the order of integration, i.e., the stationarity
+of a given time series dataset. The stationary property of data is pivotal in many machine learning models as it
+impacts the reliability and effectiveness of predictions and forecasts.</p>
 
-<p><strong>Signs of High Risk</strong>:</p>
+<h3 id="test-mechanism">Test Mechanism</h3>
+
+<p>The ADF test is executed using the <code>adfuller</code> function from the <code>statsmodels</code> library on each feature of the
+dataset. Multiple outputs are generated for each run, including the ADF test statistic and p-value, count of lags
+used, the number of observations considered in the test, critical values at various confidence levels, and the
+information criterion. These results are stored for each feature for subsequent analysis.</p>
+
+<h3 id="signs-of-high-risk">Signs of High Risk</h3>
 
 <ul>
-<li>An inflated ADF statistic and high p-value (generally above 0.05) insinuate a high risk to the model's
-performance due to the presence of a unit root indicating non-stationarity.</li>
-<li>Such non-stationarity might result in untrustworthy or insufficient forecasts.</li>
+<li>An inflated ADF statistic and high p-value (generally above 0.05) indicate a high risk to the model's performance
+due to the presence of a unit root indicating non-stationarity.</li>
+<li>Non-stationarity might result in untrustworthy or insufficient forecasts.</li>
 </ul>
 
-<p><strong>Strengths</strong>:</p>
+<h3 id="strengths">Strengths</h3>
 
 <ul>
-<li>The ADF test is robust to more sophisticated correlation within the data, which empowers it to be deployed in
-settings where data might display complex stochastic behavior.</li>
-<li>The ADF test provides explicit outputs like test statistics, critical values, and information criterion, thereby
-enhancing our understanding and transparency of the model validation process.</li>
+<li>The ADF test is robust to sophisticated correlations within the data, making it suitable for settings where data
+displays complex stochastic behavior.</li>
+<li>It provides explicit outputs like test statistics, critical values, and information criterion, enhancing
+understanding and transparency in the model validation process.</li>
 </ul>
 
-<p><strong>Limitations</strong>:</p>
+<h3 id="limitations">Limitations</h3>
 
 <ul>
 <li>The ADF test might demonstrate low statistical power, making it challenging to differentiate between a unit root
-and near-unit-root processes causing false negatives.</li>
-<li>The test assumes the data follows an autoregressive process, which might not be the case all the time.</li>
-<li>The ADF test finds it demanding to manage time series data with structural breaks.</li>
+and near-unit-root processes, potentially causing false negatives.</li>
+<li>It assumes the data follows an autoregressive process, which might not always be the case.</li>
+<li>The test struggles with time series data that have structural breaks.</li>
 </ul>
 </div>