diff --git a/notebooks/community/model_garden/model_garden_pytorch_biomedclip.ipynb b/notebooks/community/model_garden/model_garden_pytorch_biomedclip.ipynb index 46f284f8d..106fd9fce 100644 --- a/notebooks/community/model_garden/model_garden_pytorch_biomedclip.ipynb +++ b/notebooks/community/model_garden/model_garden_pytorch_biomedclip.ipynb @@ -3,13 +3,13 @@ { "cell_type": "code", "execution_count": null, - "id": "7d9bbf86da5e", "metadata": { + "cellView": "form", "id": "7d9bbf86da5e" }, "outputs": [], "source": [ - "# Copyright 2023 Google LLC\n", + "# Copyright 2025 Google LLC\n", "#\n", "# Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", @@ -26,37 +26,28 @@ }, { "cell_type": "markdown", - "id": "99c1c3fc2ca5", "metadata": { "id": "99c1c3fc2ca5" }, "source": [ "# Vertex AI Model Garden - BiomedCLIP\n", "\n", - "\n", - "
\n", - " \n", - " \"Colab Run in Colab\n", + "\n", + " \n", - " \n", - " \n", - "
\n", + " \n", + " \"Google
Run in Colab Enterprise\n", " \n", " 		\n", + " \n", " \n", - " \"GitHub\n", - " View on GitHub\n", + " \"GitHub
View on GitHub\n", " \n", " 		\n", - " \n", - " \"Vertex\n", - "Open in Vertex AI Workbench\n", - " (A Python-3 CPU notebook is recommended)\n", - "
" + "
" ] }, { "cell_type": "markdown", - "id": "-ThBX_vbxEJD", "metadata": { "id": "-ThBX_vbxEJD" }, @@ -71,7 +62,6 @@ "- Serve BiomedCLIP using Vertex AI\n", "- Run zero-shot image classification with BiomedCLIP\n", "\n", - "\n", "### Costs\n", "\n", "This tutorial uses billable components of Google Cloud:\n", @@ -79,12 +69,11 @@ "* Vertex AI\n", "* Cloud Storage\n", "\n", - "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage pricing](https://cloud.google.com/storage/pricing), and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage." + "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing), [Cloud Storage pricing](https://cloud.google.com/storage/pricing), and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage." ] }, { "cell_type": "markdown", - "id": "264c07757582", "metadata": { "id": "264c07757582" }, @@ -94,191 +83,144 @@ "**NOTE**: Jupyter runs lines prefixed with `!` as shell commands, and it interpolates Python variables prefixed with `$` into these commands." ] }, - { - "cell_type": "markdown", - "id": "ioensNKM8ned", - "metadata": { - "id": "ioensNKM8ned" - }, - "source": [ - "### Colab only\n", - "Run the following commands for Colab and skip this section if you are using Workbench." - ] - }, { "cell_type": "code", "execution_count": null, - "id": "2707b02ef5df", "metadata": { - "id": "2707b02ef5df" + "cellView": "form", + "id": "0172207da484" }, "outputs": [], "source": [ - "import sys\n", + "# @title Setup Google Cloud project\n", "\n", - "if \"google.colab\" in sys.modules:\n", - " ! pip3 install --upgrade google-cloud-aiplatform\n", - " from google.colab import auth as google_auth\n", + "# @markdown 1. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n", "\n", - " google_auth.authenticate_user()\n", + "# @markdown 2. **[Optional]** [Create a Cloud Storage bucket](https://cloud.google.com/storage/docs/creating-buckets) for storing experiment outputs. Set the BUCKET_URI for the experiment environment. The specified Cloud Storage bucket (`BUCKET_URI`) should be located in the same region as where the notebook was launched. Note that a multi-region bucket (eg. \"us\") is not considered a match for a single region covered by the multi-region range (eg. \"us-central1\"). If not set, a unique GCS bucket will be created instead.\n", "\n", - " # Restart the notebook kernel after installs.\n", - " import IPython\n", + "BUCKET_URI = \"gs://\" # @param {type:\"string\"}\n", "\n", - " app = IPython.Application.instance()\n", - " app.kernel.do_shutdown(True)" - ] - }, - { - "cell_type": "markdown", - "id": "bb7adab99e41", - "metadata": { - "id": "bb7adab99e41" - }, - "source": [ - "### Setup Google Cloud project\n", + "# @markdown 3. **[Optional]** Set region. If not set, the region will be set automatically according to Colab Enterprise environment.\n", "\n", - "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n", + "REGION = \"\" # @param {type:\"string\"}\n", "\n", - "1. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n", + "# @markdown 4. If you want to run predictions with A100 80GB or H100 GPUs, we recommend using the regions listed below. **NOTE:** Make sure you have associated quota in selected regions. Click the links to see your current quota for each GPU type: [Nvidia A100 80GB](https://console.cloud.google.com/iam-admin/quotas?metric=aiplatform.googleapis.com%2Fcustom_model_serving_nvidia_a100_80gb_gpus), [Nvidia H100 80GB](https://console.cloud.google.com/iam-admin/quotas?metric=aiplatform.googleapis.com%2Fcustom_model_serving_nvidia_h100_gpus).\n", "\n", - "1. [Enable the Vertex AI API and Compute Engine API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,compute_component).\n", + "# @markdown > | Machine Type | Accelerator Type | Recommended Regions |\n", + "# @markdown | ----------- | ----------- | ----------- |\n", + "# @markdown | a2-ultragpu-1g | 1 NVIDIA_A100_80GB | us-central1, us-east4, europe-west4, asia-southeast1, us-east4 |\n", + "# @markdown | a3-highgpu-2g | 2 NVIDIA_H100_80GB | us-west1, asia-southeast1, europe-west4 |\n", + "# @markdown | a3-highgpu-4g | 4 NVIDIA_H100_80GB | us-west1, asia-southeast1, europe-west4 |\n", + "# @markdown | a3-highgpu-8g | 8 NVIDIA_H100_80GB | us-central1, us-east5, europe-west4, us-west1, asia-southeast1 |\n", "\n", - "1. [Create a Cloud Storage bucket](https://cloud.google.com/storage/docs/creating-buckets) for storing experiment outputs.\n", + "# Import the necessary packages\n", "\n", - "1. [Create a service account](https://cloud.google.com/iam/docs/service-accounts-create#iam-service-accounts-create-console) with `Vertex AI User` and `Storage Object Admin` roles for deploying fine tuned model to Vertex AI endpoint." - ] - }, - { - "cell_type": "markdown", - "id": "6c460088b873", - "metadata": { - "id": "6c460088b873" - }, - "source": [ - "Fill following variables for experiments environment:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "855d6b96f291", - "metadata": { - "id": "855d6b96f291" - }, - "outputs": [], - "source": [ - "# Cloud project id.\n", - "PROJECT_ID = \"\" # @param {type:\"string\"}\n", + "# Upgrade Vertex AI SDK.\n", + "! pip3 install --upgrade --quiet 'google-cloud-aiplatform>=1.64.0'\n", + "! git clone https://github.com/GoogleCloudPlatform/vertex-ai-samples.git\n", "\n", - "# The region you want to launch jobs in.\n", - "REGION = \"us-central1\" # @param {type:\"string\"}\n", + "import datetime\n", + "import importlib\n", + "import os\n", + "import uuid\n", + "from typing import Tuple\n", "\n", - "# The Cloud Storage bucket for storing experiments output.\n", - "BUCKET_URI = \"gs://your_bucket_uri\" # @param {type:\"string\"}\n", + "import numpy as np\n", + "from google.cloud import aiplatform\n", "\n", - "! gcloud config set project $PROJECT_ID\n", + "common_util = importlib.import_module(\n", + " \"vertex-ai-samples.community-content.vertex_model_garden.model_oss.notebook_util.common_util\"\n", + ")\n", "\n", - "# The service account looks like:\n", - "# '@.iam.gserviceaccount.com'\n", - "# Please go to https://cloud.google.com/iam/docs/service-accounts-create#iam-service-accounts-create-console\n", - "# and create service account with `Vertex AI User` and `Storage Object Admin` roles.\n", - "# The service account for deploying fine tuned model.\n", - "SERVICE_ACCOUNT = \"\" # @param {type:\"string\"}\n", + "models, endpoints = {}, {}\n", "\n", - "# The serving port.\n", - "SERVE_PORT = 7080" - ] - }, - { - "cell_type": "markdown", - "id": "e828eb320337", - "metadata": { - "id": "e828eb320337" - }, - "source": [ - "### Initialize Vertex AI API" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "12cd25839741", - "metadata": { - "id": "12cd25839741" - }, - "outputs": [], - "source": [ - "from google.cloud import aiplatform\n", + "# Get the default cloud project id.\n", + "PROJECT_ID = os.environ[\"GOOGLE_CLOUD_PROJECT\"]\n", "\n", - "aiplatform.init(project=PROJECT_ID, location=REGION, staging_bucket=BUCKET_URI)" - ] - }, - { - "cell_type": "markdown", - "id": "2cc825514deb", - "metadata": { - "id": "2cc825514deb" - }, - "source": [ - "### Define constants" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0172207da484", - "metadata": { - "id": "0172207da484" - }, - "outputs": [], - "source": [ - "# The pre-built serving docker image.\n", - "SERVE_DOCKER_URI = \"us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/pytorch-open-clip-serve\"" - ] - }, - { - "cell_type": "markdown", - "id": "0c250872074f", - "metadata": { - "id": "0c250872074f" - }, - "source": [ - "### Define utility functions" + "# Get the default region for launching jobs.\n", + "if not REGION:\n", + " REGION = os.environ[\"GOOGLE_CLOUD_REGION\"]\n", + "\n", + "# Enable the Vertex AI API and Compute Engine API, if not already.\n", + "print(\"Enabling Vertex AI API and Compute Engine API.\")\n", + "! gcloud services enable aiplatform.googleapis.com compute.googleapis.com\n", + "\n", + "# Cloud Storage bucket for storing the experiment artifacts.\n", + "# A unique GCS bucket will be created for the purpose of this notebook. If you\n", + "# prefer using your own GCS bucket, change the value yourself below.\n", + "now = datetime.datetime.now().strftime(\"%Y%m%d%H%M%S\")\n", + "BUCKET_NAME = \"/\".join(BUCKET_URI.split(\"/\")[:3])\n", + "\n", + "if BUCKET_URI is None or BUCKET_URI.strip() == \"\" or BUCKET_URI == \"gs://\":\n", + " BUCKET_URI = f\"gs://{PROJECT_ID}-tmp-{now}-{str(uuid.uuid4())[:4]}\"\n", + " BUCKET_NAME = \"/\".join(BUCKET_URI.split(\"/\")[:3])\n", + " ! gsutil mb -l {REGION} {BUCKET_URI}\n", + "else:\n", + " assert BUCKET_URI.startswith(\"gs://\"), \"BUCKET_URI must start with `gs://`.\"\n", + " shell_output = ! gsutil ls -Lb {BUCKET_NAME} | grep \"Location constraint:\" | sed \"s/Location constraint://\"\n", + " bucket_region = shell_output[0].strip().lower()\n", + " if bucket_region != REGION:\n", + " raise ValueError(\n", + " \"Bucket region %s is different from notebook region %s\"\n", + " % (bucket_region, REGION)\n", + " )\n", + "print(f\"Using this GCS Bucket: {BUCKET_URI}\")\n", + "\n", + "STAGING_BUCKET = os.path.join(BUCKET_URI, \"temporal\")\n", + "MODEL_BUCKET = os.path.join(BUCKET_URI, \"biomedclip\")\n", + "\n", + "\n", + "# Initialize Vertex AI API.\n", + "print(\"Initializing Vertex AI API.\")\n", + "aiplatform.init(project=PROJECT_ID, location=REGION, staging_bucket=STAGING_BUCKET)\n", + "\n", + "# Gets the default SERVICE_ACCOUNT.\n", + "shell_output = ! gcloud projects describe $PROJECT_ID\n", + "project_number = shell_output[-1].split(\":\")[1].strip().replace(\"'\", \"\")\n", + "SERVICE_ACCOUNT = f\"{project_number}-compute@developer.gserviceaccount.com\"\n", + "print(\"Using this default Service Account:\", SERVICE_ACCOUNT)\n", + "\n", + "\n", + "# Provision permissions to the SERVICE_ACCOUNT with the GCS bucket\n", + "! gsutil iam ch serviceAccount:{SERVICE_ACCOUNT}:roles/storage.admin $BUCKET_NAME\n", + "\n", + "! gcloud config set project $PROJECT_ID\n", + "! gcloud projects add-iam-policy-binding --no-user-output-enabled {PROJECT_ID} --member=serviceAccount:{SERVICE_ACCOUNT} --role=\"roles/storage.admin\"\n", + "! gcloud projects add-iam-policy-binding --no-user-output-enabled {PROJECT_ID} --member=serviceAccount:{SERVICE_ACCOUNT} --role=\"roles/aiplatform.user\"" ] }, { "cell_type": "code", "execution_count": null, - "id": "354da31189dc", "metadata": { + "cellView": "form", "id": "354da31189dc" }, "outputs": [], "source": [ - "from datetime import datetime\n", - "from typing import Tuple\n", + "# @title Deploy\n", "\n", + "# @markdown This section uploads the model to Model Registry and deploys it on the Endpoint.\n", "\n", - "def get_job_name_with_datetime(prefix: str) -> str:\n", - " \"\"\"Gets the job name with date time when triggering training or deployment\n", - " jobs in Vertex AI.\n", - " \"\"\"\n", - " return prefix + datetime.now().strftime(\"_%Y%m%d_%H%M%S\")\n", + "# @markdown The model deployment step will take ~10 minutes to complete.\n", "\n", + "model_id = \"hf-hub:microsoft/BiomedCLIP-PubMedBERT_256-vit_base_patch16_224\"\n", + "serving_accelerator_type = \"NVIDIA_L4\" # @param [\"NVIDIA_TESLA_V100\",\"NVIDIA_L4\"]\n", + "TASK = \"zero-shot-image-classification\"\n", + "PRECISION = \"amp\"\n", + "SERVE_PORT = 7080\n", "\n", - "def download_image(url: str) -> str:\n", - " \"\"\"Downloads an image from the given URL.\n", - "\n", - " Args:\n", - " url: The URL of the image to download.\n", - "\n", - " Returns:\n", - " base64 encoded image.\n", - " \"\"\"\n", - " !wget -O image.jpg $url\n", - " !base64 image.jpg > image.txt\n", - " return open(\"image.txt\").read()\n", + "# The pre-built serving docker image.\n", + "SERVE_DOCKER_URI = \"us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/pytorch-open-clip-serve:20250128_1414_RC00\"\n", + "\n", + "if serving_accelerator_type == \"NVIDIA_L4\":\n", + " serving_machine_type = \"g2-standard-8\"\n", + "elif serving_accelerator_type == \"NVIDIA_TESLA_V100\":\n", + " serving_machine_type = \"n1-standard-8\"\n", + "else:\n", + " raise ValueError(\n", + " f\"Recommended GPU setting not found for: {serving_accelerator_type}\"\n", + " )\n", "\n", "\n", "def deploy_model(\n", @@ -287,9 +229,9 @@ " service_account: str,\n", " task: str,\n", " precision: str,\n", - " machine_type=\"n1-standard-8\",\n", - " accelerator_type=\"NVIDIA_TESLA_V100\",\n", - " accelerator_count=1,\n", + " machine_type: str,\n", + " accelerator_type: str,\n", + " accelerator_count: int,\n", ") -> Tuple[aiplatform.Model, aiplatform.Endpoint]:\n", " \"\"\"Deploys trained models into Vertex AI.\"\"\"\n", " endpoint = aiplatform.Endpoint.create(display_name=f\"{model_name}-endpoint\")\n", @@ -317,60 +259,33 @@ " accelerator_count=accelerator_count,\n", " deploy_request_timeout=1800,\n", " service_account=service_account,\n", - " system_labels={\n", - " \"NOTEBOOK_NAME\": \"model_garden_pytorch_biomedclip.ipynb\"\n", - " },\n", + " system_labels={\"NOTEBOOK_NAME\": \"model_garden_pytorch_biomedclip.ipynb\"},\n", " )\n", - " return model, endpoint" - ] - }, - { - "cell_type": "markdown", - "id": "jqmCtkGnhDmp", - "metadata": { - "id": "jqmCtkGnhDmp" - }, - "source": [ - "### Run inferences with serving images\n", - "This section uploads the model to Model Registry and deploys it on the Endpoint.\n", + " return model, endpoint\n", "\n", - "The model deployment step will take ~10 minutes to complete." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "4f79683ccd37", - "metadata": { - "id": "4f79683ccd37" - }, - "outputs": [], - "source": [ - "model_id = \"hf-hub:microsoft/BiomedCLIP-PubMedBERT_256-vit_base_patch16_224\"" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "bf55e38815dc", - "metadata": { - "id": "bf55e38815dc" - }, - "outputs": [], - "source": [ - "model, endpoint = deploy_model(\n", - " model_name=get_job_name_with_datetime(prefix=\"biomedclip-serve\"),\n", + "\n", + "common_util.check_quota(\n", + " project_id=PROJECT_ID,\n", + " region=REGION,\n", + " accelerator_type=serving_accelerator_type,\n", + " accelerator_count=1,\n", + " is_for_training=False,\n", + ")\n", + "\n", + "models[\"biomedclip_model\"], endpoints[\"biomedclip_endpoint\"] = deploy_model(\n", + " model_name=common_util.get_job_name_with_datetime(prefix=\"biomedclip-serve\"),\n", " model_id=model_id,\n", " service_account=SERVICE_ACCOUNT,\n", - " task=\"zero-shot-image-classification\",\n", - " precision=\"amp\",\n", - ")\n", - "print(\"endpoint_name:\", endpoint.name)" + " task=TASK,\n", + " precision=PRECISION,\n", + " machine_type=serving_machine_type,\n", + " accelerator_type=serving_accelerator_type,\n", + " accelerator_count=1,\n", + ")" ] }, { "cell_type": "markdown", - "id": "80b3fd2ace09", "metadata": { "id": "80b3fd2ace09" }, @@ -383,21 +298,36 @@ { "cell_type": "code", "execution_count": null, - "id": "4ab04da3ec9a", "metadata": { + "cellView": "form", "id": "4ab04da3ec9a" }, "outputs": [], "source": [ - "import numpy as np\n", + "# @title Predict\n", "\n", "# # Loads an existing endpoint as below.\n", - "# endpoint_name = endpoint.name\n", + "# endpoint_name = endpoints[\"biomedclip_endpoint\"].name\n", "# aip_endpoint_name = (\n", "# f\"projects/{PROJECT_ID}/locations/{REGION}/endpoints/{endpoint_name}\"\n", "# )\n", "# endpoint = aiplatform.Endpoint(aip_endpoint_name)\n", "\n", + "\n", + "def download_image(url: str) -> str:\n", + " \"\"\"Downloads an image from the given URL.\n", + "\n", + " Args:\n", + " url: The URL of the image to download.\n", + "\n", + " Returns:\n", + " base64 encoded image.\n", + " \"\"\"\n", + " !wget -O image.jpg $url\n", + " !base64 image.jpg > image.txt\n", + " return open(\"image.txt\").read()\n", + "\n", + "\n", "instances = [\n", " {\n", " \"text\": \"This is a photo of adenocarcinoma histopathology\",\n", @@ -414,7 +344,7 @@ " {\"text\": \"This is a photo of hematoxylin and eosin histopathology\"},\n", " {\"text\": \"This is a photo of pie chart\"},\n", "]\n", - "response = endpoint.predict(instances=instances)\n", + "response = endpoints[\"biomedclip_endpoint\"].predict(instances=instances)\n", "\n", "print(response.predictions)\n", "\n", @@ -422,30 +352,30 @@ "print(f\"Selected class: {instances[selected_idx]['text']}\")" ] }, - { - "cell_type": "markdown", - "id": "af21a3cff1e0", - "metadata": { - "id": "af21a3cff1e0" - }, - "source": [ - "### Clean up resources" - ] - }, { "cell_type": "code", "execution_count": null, - "id": "911406c1561e", "metadata": { + "cellView": "form", "id": "911406c1561e" }, "outputs": [], "source": [ + "# @title Delete the models and endpoints\n", + "# @markdown Delete the experiment models and endpoints to recycle the resources\n", + "# @markdown and avoid unnecessary continuous charges that may incur.\n", + "\n", "# Undeploy model and delete endpoint.\n", - "endpoint.delete(force=True)\n", + "for endpoint in endpoints.values():\n", + " endpoint.delete(force=True)\n", "\n", "# Delete models.\n", - "model.delete()" + "for model in models.values():\n", + " model.delete()\n", + "\n", + "delete_bucket = False # @param {type:\"boolean\"}\n", + "if delete_bucket:\n", + " ! gsutil -m rm -r $BUCKET_NAME" ] } ],