Skip to content

Updated workflows/actions to dynamically generate an .env file #552

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
OPENAI_API_KEY=ADD_YOUR_KEY_HERE

# API INFO TO RUN /notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb
PLATFORM_API_KEY=ADD_YOUR_API_KEY_HERE
PLATFORM_API_SECRET=ADD_YOUR_API_SECRET_HERE
VM_API_HOST=<api_host>
VM_API_KEY=<api_key>
VM_API_SECRET=<api_secret>
VM_API_MODEL=<model>
16 changes: 15 additions & 1 deletion .github/actions/demo-notebook/action.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
name: Execute demo notebook
description: Installs python3, validmind, checks dependencies then executes ONLY the Intro for Model Developers notebook with development heap tracking

inputs: {}
inputs:
env_file:
description: "Load the created .env file"
required: true

runs:
using: "composite"
Expand All @@ -26,10 +29,21 @@ runs:
pip install pycairo
pip check

- name: Ensure .env file is available
shell: bash
id: find_env
run: |
if [ ! -f "${{ inputs.env_file }}" ]; then
echo "Error: .env file not found at ${{ inputs.env_file }}"
exit 1
fi

- name: Execute ONLY the Intro for Model Developers notebook with heap development
shell: bash
if: ${{ steps.find_env.outcome == 'success' }}
run: |
cd site
source ../${{ inputs.env_file }}
quarto render --profile exe-demo notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb &> render_errors.log || {
echo "Execute for intro_for_model_developers_EXECUTED.ipynb failed";
cat render_errors.log;
Expand Down
16 changes: 15 additions & 1 deletion .github/actions/prod-notebook/action.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
name: Execute prod notebook
description: Installs python3, validmind, checks dependencies then executes ONLY the Intro for Model Developers notebook with production heap tracking

inputs: {}
inputs:
env_file:
description: "Load the created .env file"
required: true

runs:
using: "composite"
Expand All @@ -26,10 +29,21 @@ runs:
pip install pycairo
pip check

- name: Ensure .env file is available
shell: bash
id: find_env
run: |
if [ ! -f "${{ inputs.env_file }}" ]; then
echo "Error: .env file not found at ${{ inputs.env_file }}"
exit 1
fi

- name: Execute ONLY the Intro for Model Developers notebook with heap production
shell: bash
if: ${{ steps.find_env.outcome == 'success' }}
run: |
cd site
source ../${{ inputs.env_file }}
quarto render --profile exe-prod notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb &> render_errors.log || {
echo "Execute for intro_for_model_developers_EXECUTED.ipynb failed";
cat render_errors.log;
Expand Down
16 changes: 15 additions & 1 deletion .github/actions/staging-notebook/action.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
name: Execute staging notebook
description: Installs python3, validmind, checks dependencies then executes ONLY the Intro for Model Developers notebook with staging heap tracking

inputs: {}
inputs:
env_file:
description: "Load the created .env file"
required: true

runs:
using: "composite"
Expand All @@ -26,10 +29,21 @@ runs:
pip install pycairo
pip check

- name: Ensure .env file is available
shell: bash
id: find_env
run: |
if [ ! -f "${{ inputs.env_file }}" ]; then
echo "Error: .env file not found at ${{ inputs.env_file }}"
exit 1
fi

- name: Execute ONLY the Intro for Model Developers notebook with heap staging
shell: bash
if: ${{ steps.find_env.outcome == 'success' }}
run: |
cd site
source ../${{ inputs.env_file }}
quarto render --profile exe-staging notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb &> render_errors.log || {
echo "Execute for intro_for_model_developers_EXECUTED.ipynb failed";
cat render_errors.log;
Expand Down
23 changes: 18 additions & 5 deletions .github/workflows/deploy-docs-prod.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,18 +35,31 @@ jobs:
uses: dorny/paths-filter@v2
id: filter
with:
base: prod
head: staging
filters: |
notebooks:
- 'site/notebooks/**'

# If yes then execute the prod notebook
- name: Execute prod Intro for Model Developers notebook
# If yes then create the .env file for use in execution step
- name: Create .env file
if: steps.filter.outputs.notebooks == 'true'
id: create_env
run: |
touch .env
echo VM_API_HOST=${{ secrets.PLATFORM_API_HOST }} >> .env
echo VM_API_KEY=${{ secrets.PLATFORM_API_KEY }} >> .env
echo VM_API_SECRET=${{ secrets.PLATFORM_API_SECRET }} >> .env
echo VM_API_MODEL=${{ secrets.PLATFORM_DEV_MODEL }} >> .env
cat .env

# Only execute the prod notebook if .env file is created
- name: Execute prod Intro for Model Developers notebook
if: ${{ steps.create_env.outcome == 'success' }}
uses: ./.github/actions/prod-notebook
id: execute-prod-notebook
env:
PLATFORM_API_KEY: ${{ secrets.PLATFORM_API_KEY }}
PLATFORM_API_SECRET: ${{ secrets.PLATFORM_API_SECRET }}
with:
env_file: .env

# Prod bucket is in us-east-1
- name: Configure AWS credentials
Expand Down
23 changes: 18 additions & 5 deletions .github/workflows/deploy-docs-staging.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,18 +35,31 @@ jobs:
uses: dorny/paths-filter@v2
id: filter
with:
base: staging
head: main
filters: |
notebooks:
- 'site/notebooks/**'

# If yes then execute the staging notebook
- name: Execute staging Intro for Model Developers notebook
# If yes then create the .env file for use in execution step
- name: Create .env file
if: steps.filter.outputs.notebooks == 'true'
id: create_env
run: |
touch .env
echo VM_API_HOST=${{ secrets.PLATFORM_API_HOST }} >> .env
echo VM_API_KEY=${{ secrets.PLATFORM_API_KEY }} >> .env
echo VM_API_SECRET=${{ secrets.PLATFORM_API_SECRET }} >> .env
echo VM_API_MODEL=${{ secrets.PLATFORM_DEV_MODEL }} >> .env
cat .env

# Only execute the staging notebook if .env file is created
- name: Execute staging Intro for Model Developers notebook
if: ${{ steps.create_env.outcome == 'success' }}
uses: ./.github/actions/staging-notebook
id: execute-staging-notebook
env:
PLATFORM_API_KEY: ${{ secrets.PLATFORM_API_KEY }}
PLATFORM_API_SECRET: ${{ secrets.PLATFORM_API_SECRET }}
with:
env_file: .env

# Staging bucket is in us-west-2
- name: Configure AWS credentials
Expand Down
23 changes: 18 additions & 5 deletions .github/workflows/validate-docs-site.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,18 +34,31 @@ jobs:
uses: dorny/paths-filter@v2
id: filter
with:
base: main
head: ${{ github.head_ref }}
filters: |
notebooks:
- 'site/notebooks/**'

# If yes then execute the demo notebook
- name: Execute demo Intro for Model Developers notebook
# If yes then create the .env file for use in execution step
- name: Create .env file
if: steps.filter.outputs.notebooks == 'true'
id: create_env
run: |
touch .env
echo VM_API_HOST=${{ secrets.PLATFORM_API_HOST }} >> .env
echo VM_API_KEY=${{ secrets.PLATFORM_API_KEY }} >> .env
echo VM_API_SECRET=${{ secrets.PLATFORM_API_SECRET }} >> .env
echo VM_API_MODEL=${{ secrets.PLATFORM_DEV_MODEL }} >> .env
cat .env

# Only execute the demo notebook if .env file is created
- name: Execute demo Intro for Model Developers notebook
if: ${{ steps.create_env.outcome == 'success' }}
uses: ./.github/actions/demo-notebook
id: execute-demo-notebook
env:
PLATFORM_API_KEY: ${{ secrets.PLATFORM_API_KEY }}
PLATFORM_API_SECRET: ${{ secrets.PLATFORM_API_SECRET }}
with:
env_file: .env

- name: Test for warnings or errors
run: |
Expand Down
17 changes: 12 additions & 5 deletions site/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@ DEST_DIR_NB := notebooks
DEST_DIR_PYTHON := _site/validmind
DEST_DIR_TESTS := tests
GIT_BRANCH := $(shell git rev-parse --abbrev-ref HEAD)
PROFILE := exe-demo
FILE_PATH := notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb

# Define .PHONY target for help section
.PHONY: help clean clone notebooks python-docs docs-site deploy-demo deploy-demo-branch delete-demo-branch deploy-prod deploy-staging release-notes
.PHONY: help clean clone notebooks python-docs docs-site deploy-demo deploy-demo-branch delete-demo-branch deploy-prod deploy-staging release-notes execute

# Help section
help:
Expand All @@ -24,6 +26,7 @@ help:
@echo " deploy-staging Deploy docs staging site to s3://docs-ci-cd-staging/site/"
@echo " help Display this help message (default target)"
@echo " release-notes Generate release notes from pull requests since latest tag and update _quarto.yml"
@echo " execute Execute a Jupyter Notebook (must have valid .env credentials set up in root)"

# Clean up source directory
clean:
Expand All @@ -41,8 +44,8 @@ notebooks:
@rm -f notebooks.zip
@rm -rf $(DEST_DIR_NB)/ && mkdir -p $(DEST_DIR_NB)
@cp -r $(SRC_DIR)/notebooks/. $(DEST_DIR_NB)/
@echo "Checking out intro_for_model_developers_EXECUTED.ipynb from main ..."
@git checkout main -- notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb
@echo "Duplicating notebooks/tutorials/intro_for_model_developers.ipynb for execution"
@cp notebooks/tutorials/intro_for_model_developers.ipynb notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb
@echo "Copying LICENSE into notebooks ..."
@cp -r $(SRC_DIR)/LICENSE $(DEST_DIR_NB)/
@rm -rf $(DEST_DIR_NB)/code_sharing
Expand Down Expand Up @@ -77,8 +80,6 @@ get-source: clean clone notebooks python-docs test-descriptions
# Get all source files
docs-site: clean clone notebooks python-docs test-descriptions
quarto render --profile development
# quarto render notebooks/how_to/explore_tests.ipynb --execute
# quarto render notebooks/how_to/explore_test_suites.ipynb --execute

# Deployment to https://docs-demo.vm.validmind.ai/
deploy-demo:
Expand Down Expand Up @@ -120,3 +121,9 @@ deploy-staging:
# Generate release notes
release-notes:
@python ../release-scripts/generate_release_objects.py

# Execute a Jupyter Notebook
# Will default to `exe-demo` profile & the `notebooks/tutorials/intro_for_model_developers_EXECUTED.ipynb` if no input provided
# To override: make execute PROFILE=select-profile FILE_PATH=notebooks/notebook-path-here.ipynb
execute:
quarto render --profile $(PROFILE) $(FILE_PATH)
Original file line number Diff line number Diff line change
Expand Up @@ -31,26 +31,58 @@ Keeing model credentials in a separate file also allows for precise access contr

This is a hidden file, so you may need to change your settings to view it.

2. Locate the code snippet for your model.
2. Locate the code snippet for your model:

- These credentials can be found on the **{{< fa rocket >}} Getting Started** page for models registered in the model inventory.
- Copy the values from this page and paste them into your `.env` file in the following format:
a. In the left sidebar, click **{{< fa cubes >}} Model Inventory**.
b. Select a model by clicking on it or find your model by applying a filter or searching for it.[^3]
c. In the left sidebar that appears for your model, click **{{< fa rocket >}} Getting Started**.
d. Locate the code snippet and click **{{< fa regular copy >}} Copy snippet to clipboard**.

3. Paste the credentials into your `.env` file in the following format:

```yaml
VM_API_MODEL=<Model Identifier>
VM_API_HOST=<API Host URL>
VM_API_KEY=<API Key>s
VM_API_SECRET=<API Secret>
VM_API_HOST=<api_host>
VM_API_KEY=<api_key>
VM_API_SECRET=<api_secret>
VM_API_MODEL=<model>
```

3. Insert this code snippet above your model identifier credentials:
::: {.column-margin}
For example, if your credentials look like this:

```python
import validmind as vm

vm.init(
api_host = "https://api.prod.validmind.ai/api/v1/tracking",
api_key = "key123",
api_secret = "secret456",
model = "model789"
)
```

Then your `.env` file should look like this:

```bash
VM_API_HOST=https://api.prod.validmind.ai/api/v1/tracking
VM_API_KEY=key123
VM_API_SECRET=secret456
VM_API_MODEL=model789
```

:::


4. Insert this code snippet above your model identifier credentials:

```python
%load_ext dotenv
%dotenv dev.env
```

The updated notebook should look like this:
5. Remove the inline credentials from `vm.init()`.[^4]

The updated code cell should now look like this:

```python
%load_ext dotenv
Expand All @@ -59,19 +91,27 @@ Keeing model credentials in a separate file also allows for precise access contr
import validmind as vm

vm.init(
api_host = "http://localhost:3000/api/v1/tracking",
model = "..."
)
```

::: {.column-margin}
To enable monitoring when you are storing credentials in an `.env` file:^[[Enable monitoring](/guide/monitoring/enable-monitoring.qmd)]

4. Run the cell. Instead of using inline credentials, this cell will now load your model credentials from a `.env` file.
```python
%load_ext dotenv
%dotenv .env

<!---
## Troubleshooting
import validmind as vm

vm.init(
monitoring=True
)
```

:::

6. Run the cell. Instead of using inline credentials, this cell will now load your model credentials from a `.env` file.

[Include any common issues or errors that may arise during the task and how to resolve them.]
--->

## What's next

Expand All @@ -85,3 +125,5 @@ Keeing model credentials in a separate file also allows for precise access contr

[^2]: [Manage permissions](/guide/configuration/manage-permissions.qmd)

[^3]: [Working with the model inventory](/guide/model-inventory/working-with-model-inventory.qmd#search-filter-and-sort-models)

2 changes: 1 addition & 1 deletion site/guide/model-inventory/archive-delete-models.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ Model deletion is also permanent and cannot be undone.

2. Select a model or find your model by applying a filter or searching for it.[^7]

3. First, archive your model.[^6]
3. First, archive your model.[^8]

4. Then, under the [model stage]{.smallcaps} section, select **Deleted**.

Expand Down
Loading
Loading