Skip to content

Removed documentation projects from JN #184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Original file line number Diff line number Diff line change
Expand Up @@ -631,7 +631,10 @@
"\n",
"#### Add the output template to the documentation template\n",
"\n",
"First, go to your project in the ValidMind UI and go to Settings > Templates. Find the Binary Classification Template that is used by the Customer Churn project. Click on the `Edit` button to bring up the template editor. Then, add the following content block below the existing `validmind.model_validation.sklearn.ClassifierPerformance` test:\n",
"Head to the ValidMind platform UI to [customize your documentation template](https://docs.validmind.ai/guide/model-documentation/customize-documentation-templates.html). \n",
"\n",
"- Find the Binary Classification Template that is used by the Customer Churn model\n",
"- Add the following content block below the existing `validmind.model_validation.sklearn.ClassifierPerformance` test:\n",
"\n",
"```yaml\n",
"- content_type: metric\n",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, use the ValidMind Developer Framework to run validation tests on the model. The `vm.run_documentation_tests` function analyzes the current project's documentation template and collects all the tests associated with it into a test suite.\n",
"Next, use the ValidMind Developer Framework to run validation tests on the model. The `vm.run_documentation_tests` function analyzes the current model's documentation template and collects all the tests associated with it into a test suite.\n",
"\n",
"The function then runs the test suite, logs the results to the ValidMind API and displays them to you.\n"
]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@
"source": [
"### ValidMind Initialization\n",
"\n",
"Now we will import and initialize the ValidMind framework so we can connect to our project in the ValidMind platform. This will let us log inputs, plots, and test results to our model documentation."
"Now we will import and initialize the ValidMind framework so we can connect to our model in the ValidMind platform. This will let us log inputs, plots, and test results to our model documentation."
]
},
{
Expand Down Expand Up @@ -138,7 +138,7 @@
"source": [
"# Dataset Loader\n",
"\n",
"Great, now that we have all of our dependencies installed, the developer framework initialized and connected to our model documentation project and our OpenAI API key setup, we can go ahead and load our datasets. We will use the synthetic `RFP` dataset included with ValidMind for this notebook. This dataset contains a variety of RFP questions and ground truth answers that we can use both as the source where our Retriever will search for similar question-answer pairs as well as our test set for evaluating the performance of our RAG model. To do this, we just have to load it and call the preprocess function to get a split of the data into train and test sets."
"Great, now that we have all of our dependencies installed, the developer framework initialized and connected to our model and our OpenAI API key setup, we can go ahead and load our datasets. We will use the synthetic `RFP` dataset included with ValidMind for this notebook. This dataset contains a variety of RFP questions and ground truth answers that we can use both as the source where our Retriever will search for similar question-answer pairs as well as our test set for evaluating the performance of our RAG model. To do this, we just have to load it and call the preprocess function to get a split of the data into train and test sets."
]
},
{
Expand Down Expand Up @@ -399,7 +399,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"What we have done here is to initialize the `OpenAIEmbeddings` class so it uses OpenAI's `text-embedding-3-small` model. We then created an `embed` function that takes in an `input` dictionary and uses the `embed_query` method of the embedding client to compute the embeddings of the `question`. We use an `embed` function since that is how ValidMind supports any custom model. We will use this strategy for the retrieval and generator models as well but you could also use, say, a HuggingFace model directly. See the documentation for more information on which model types are directly supported - [ValidMind Documentation](https://docs.validmind.ai/validmind/validmind.html)... Finally, we use the `init_model` function from the ValidMind framework to create a `VMModel` object that can be used in ValidMind tests. This also logs the model to our model documentation project and any test that uses the model will be linked to the logged model and its metadata."
"What we have done here is to initialize the `OpenAIEmbeddings` class so it uses OpenAI's `text-embedding-3-small` model. We then created an `embed` function that takes in an `input` dictionary and uses the `embed_query` method of the embedding client to compute the embeddings of the `question`. We use an `embed` function since that is how ValidMind supports any custom model. We will use this strategy for the retrieval and generator models as well but you could also use, say, a HuggingFace model directly. See the documentation for more information on which model types are directly supported - [ValidMind Documentation](https://docs.validmind.ai/validmind/validmind.html)... Finally, we use the `init_model` function from the ValidMind framework to create a `VMModel` object that can be used in ValidMind tests. This also logs the model to our model documentation and any test that uses the model will be linked to the logged model and its metadata."
]
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@
"source": [
"### Preview the documentation template\n",
"\n",
"A template predefines sections for your documentation project and provides a general outline to follow, making the documentation process much easier.\n",
"A template predefines sections for your model documentation and provides a general outline to follow, making the documentation process much easier.\n",
"\n",
"You will upload documentation and test results into this template later on. For now, take a look at the structure that the template provides with the `vm.preview_template()` function from the ValidMind library and note the empty sections:"
]
Expand Down
Loading
Loading