feat: add support for partial failures sink in import rag files.

PiperOrigin-RevId: 675672254

.kokoro/release.sh

@@ -23,7 +23,7 @@ python3 -m releasetool publish-reporter-script > /tmp/publisher-script; source /
 export PYTHONUNBUFFERED=1
 
 # Move into the `google-cloud-aiplatform` package, build the distribution and upload.
-GCA_TWINE_PASSWORD=$(cat "${KOKORO_KEYSTORE_DIR}/73713_google-cloud-pypi-token-keystore-2")
+GCA_TWINE_PASSWORD=$(cat "${KOKORO_KEYSTORE_DIR}/73713_google-cloud-pypi-token-keystore-1")
 cd github/python-aiplatform
 python3 setup.py sdist bdist_wheel
 twine upload --username __token__ --password "${GCA_TWINE_PASSWORD}" dist/*

.kokoro/release/common.cfg

@@ -28,7 +28,7 @@ before_action {
   fetch_keystore {
     keystore_resource {
       keystore_config_id: 73713
-      keyname: "google-cloud-pypi-token-keystore-2"
+      keyname: "google-cloud-pypi-token-keystore-1"
     }
   }
 }

.release-please-manifest.json

@@ -1,3 +1,3 @@
 {
-    ".": "1.67.1"
+    ".": "1.66.0"
 }

CHANGELOG.md

@@ -1,35 +1,5 @@
 # Changelog
 
-## [1.67.1](https://github.com/googleapis/python-aiplatform/compare/v1.67.0...v1.67.1) (2024-09-18)
-
-
-### Bug Fixes
-
-* Fix rag corpus creation error ([6083b93](https://github.com/googleapis/python-aiplatform/commit/6083b93848be6ec975c95cd072c8d86357339d02))
-
-## [1.67.0](https://github.com/googleapis/python-aiplatform/compare/v1.66.0...v1.67.0) (2024-09-17)
-
-
-### Features
-
-* Add support for partial failures sink in import rag files. ([07e471e](https://github.com/googleapis/python-aiplatform/commit/07e471e0a069551f2c855e167e549fa92ac6af95))
-* Adding Feature Store Vector DB option for RAG corpuses to SDK ([cfc3421](https://github.com/googleapis/python-aiplatform/commit/cfc3421fe8a883d459b66ed8c9f39697ded23f20))
-* Adding Pinecone Vector DB option for RAG corpuses to SDK ([f78b953](https://github.com/googleapis/python-aiplatform/commit/f78b953f561b8697d07a530e89c7e727db1161ed))
-* Adding Vertex Vector Search Vector DB option for RAG corpuses to SDK ([f882657](https://github.com/googleapis/python-aiplatform/commit/f882657183e34c8e07baa4b8dc9f45ed8bca9db7))
-* Allow customizing pipeline caching options for model evaluation jobs. ([73490b2](https://github.com/googleapis/python-aiplatform/commit/73490b22a239cb1a3c31349f8db6cfbc5232e231))
-* GenAI - Switched the GA version of the `generative_models` classes to use the v1  service APIs instead of v1beta1 ([66d84af](https://github.com/googleapis/python-aiplatform/commit/66d84afdd5b20f70b3ff62f25cc32ac0b324d5d5))
-
-
-### Bug Fixes
-
-* GenAI - Fixed `GenerativeModel.compute_tokens` for v1 API ([0de2987](https://github.com/googleapis/python-aiplatform/commit/0de298786c43427cb1a20b91cbabd1ce921c16da))
-* Tensorboard - Fix error in tensorboard batch upload of nested dirs ([ab157c8](https://github.com/googleapis/python-aiplatform/commit/ab157c8ead718b2a1a1d13306c1256c1cb2561f1))
-
-
-### Documentation
-
-* Manually add summary overview page. ([0bc608a](https://github.com/googleapis/python-aiplatform/commit/0bc608a9c045007f12325231ed7f0069a40f469b))
-
 ## [1.66.0](https://github.com/googleapis/python-aiplatform/compare/v1.65.0...v1.66.0) (2024-09-11)
 
 

README.rst

@@ -3,10 +3,6 @@ Vertex AI SDK for Python
 
 |GA| |pypi| |versions| |unit-tests| |system-tests| |sample-tests|
 
-.. note::
-
-   For Gemini API and Generative AI on Vertex AI, please reference `Vertex Generative AI SDK for Python`_
-.. _Vertex Generative AI SDK for Python: https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest
 
 `Vertex AI`_: Google Vertex AI is an integrated suite of machine learning tools and services for building and using ML models with AutoML or custom code. It offers both novices and experts the best workbench for the entire machine learning development lifecycle.
 
@@ -117,6 +113,18 @@ Vertex AI SDK resource based functionality can be used by importing the followin
 
     from google.cloud import aiplatform
 
+Vertex AI SDK preview functionality can be used by importing the following namespace:
+
+.. code-block:: Python
+
+    from vertexai import preview
+
+Vertex AI SDK general availability (GA) functionality can be used by importing the following namespace:
+
+.. code-block:: Python
+
+    import vertexai
+
 Initialization
 ^^^^^^^^^^^^^^
 Initialize the SDK to store common configurations that you use with the SDK.

docs/README.rst

@@ -1,12 +1,7 @@
 Vertex AI SDK for Python
 =================================================
 
-|GA| |pypi| |versions| |unit-tests| |system-tests| |sample-tests|
-
-.. note::
-
-   For Gemini API and Generative AI on Vertex AI, please reference `Vertex Generative AI SDK for Python`_
-.. _Vertex Generative AI SDK for Python: https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest
+|GA| |pypi| |versions|
 
 `Vertex AI`_: Google Vertex AI is an integrated suite of machine learning tools and services for building and using ML models with AutoML or custom code. It offers both novices and experts the best workbench for the entire machine learning development lifecycle.
 
@@ -19,16 +14,11 @@ Vertex AI SDK for Python
    :target: https://pypi.org/project/google-cloud-aiplatform/
 .. |versions| image:: https://img.shields.io/pypi/pyversions/google-cloud-aiplatform.svg
    :target: https://pypi.org/project/google-cloud-aiplatform/
-.. |unit-tests| image:: https://storage.googleapis.com/cloud-devrel-public/python-aiplatform/badges/sdk-unit-tests.svg
-   :target: https://storage.googleapis.com/cloud-devrel-public/python-aiplatform/badges/sdk-unit-tests.html
-.. |system-tests| image:: https://storage.googleapis.com/cloud-devrel-public/python-aiplatform/badges/sdk-system-tests.svg
-   :target: https://storage.googleapis.com/cloud-devrel-public/python-aiplatform/badges/sdk-system-tests.html
-.. |sample-tests| image:: https://storage.googleapis.com/cloud-devrel-public/python-aiplatform/badges/sdk-sample-tests.svg
-   :target: https://storage.googleapis.com/cloud-devrel-public/python-aiplatform/badges/sdk-sample-tests.html
 .. _Vertex AI: https://cloud.google.com/vertex-ai/docs
-.. _Client Library Documentation: https://cloud.google.com/python/docs/reference/aiplatform/latest
+.. _Client Library Documentation: https://cloud.google.com/python/docs/reference/aiplatform/latest/summary_overview
 .. _Product Documentation:  https://cloud.google.com/vertex-ai/docs
 
+
 Quick Start
 -----------
 
@@ -69,6 +59,7 @@ Mac/Linux
     <your-env>/bin/pip install google-cloud-aiplatform
 
 
+
 Windows
 ^^^^^^^
 
@@ -79,7 +70,6 @@ Windows
     <your-env>\Scripts\activate
     <your-env>\Scripts\pip.exe install google-cloud-aiplatform
 
-
 Supported Python Versions
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 Python >= 3.8
@@ -117,6 +107,18 @@ Vertex AI SDK resource based functionality can be used by importing the followin
 
     from google.cloud import aiplatform
 
+Vertex AI SDK preview functionality can be used by importing the following namespace:
+
+.. code-block:: Python
+
+    from vertexai import preview
+
+Vertex AI SDK general availability (GA) functionality can be used by importing the following namespace:
+
+.. code-block:: Python
+
+    import vertexai
+
 Initialization
 ^^^^^^^^^^^^^^
 Initialize the SDK to store common configurations that you use with the SDK.
@@ -173,8 +175,8 @@ You can also create and import a dataset in separate steps:
     my_dataset = aiplatform.TextDataset.create(
         display_name="my-dataset")
 
-    my_dataset.import_data(
-        gcs_source=['gs://path/to/my/dataset.csv'],
+    my_dataset.import(
+        gcs_source=['gs://path/to/my/dataset.csv']
         import_schema_uri=aiplatform.schema.dataset.ioformat.text.multi_label_classification
     )
 
@@ -230,6 +232,7 @@ It must write the model artifact to the environment variable populated by the tr
       container_uri="us-docker.pkg.dev/vertex-ai/training/tf-cpu.2-2:latest",
       requirements=["gcsfs==0.7.1"],
       model_serving_container_image_uri="us-docker.pkg.dev/vertex-ai/prediction/tf2-cpu.2-2:latest",
+
   )
 
   model = job.run(my_dataset,
@@ -309,6 +312,7 @@ Please visit `Importing models to Vertex AI`_ for a detailed overview:
 
 .. _Importing models to Vertex AI: https://cloud.google.com/vertex-ai/docs/general/import-model
 
+
 Model Evaluation
 ----------------
 
@@ -413,6 +417,7 @@ To get predictions from endpoints:
 
   endpoint.predict(instances=[[6.7, 3.1, 4.7, 1.5], [4.6, 3.1, 1.5, 0.2]])
 
+
 To undeploy models from an endpoint:
 
 .. code-block:: Python
@@ -535,6 +540,7 @@ To use Explanation Metadata in endpoint deployment and model upload:
   aiplatform.Model.upload(..., explanation_metadata=explanation_metadata)
 
 
+
 Cloud Profiler
 ----------------------------
 

gemini_docs/README.md

@@ -1,6 +1,7 @@
 # Vertex Generative AI SDK for Python
 The Vertex Generative AI SDK helps developers use Google's generative AI
 [Gemini models](http://cloud.google.com/vertex-ai/docs/generative-ai/multimodal/overview)
+and [PaLM language models](http://cloud.google.com/vertex-ai/docs/generative-ai/language-model-overview)
 to build AI-powered features and applications.
 The SDKs support use cases like the following:
 
@@ -24,13 +25,7 @@ For detailed instructions, see [quickstart](http://cloud.google.com/vertex-ai/do
 
 #### Imports:
 ```python
-import vertexai
-```
-
-#### Initialization:
-
-```python
-vertexai.init(project='my-project', location='us-central1')
+from vertexai.generative_models import GenerativeModel, Image, Content, Part, Tool, FunctionDeclaration, GenerationConfig
 ```
 
 #### Basic generation:
@@ -59,7 +54,7 @@ print(vision_model.generate_content(["What is in the video? ", video_part]))
 ```
 
 #### Chat
-```python
+```
 from vertexai.generative_models import GenerativeModel, Image
 vision_model = GenerativeModel("gemini-ultra-vision")
 vision_chat = vision_model.start_chat()
@@ -69,7 +64,7 @@ print(vision_chat.send_message("What things do I like?."))
 ```
 
 #### System instructions
-```python
+```
 from vertexai.generative_models import GenerativeModel
 model = GenerativeModel(
     "gemini-1.0-pro",
@@ -83,7 +78,7 @@ print(model.generate_content("Why is sky blue?"))
 
 #### Function calling
 
-```python
+```
 # First, create tools that the model is can use to answer your questions.
 # Describe a function by specifying it's schema (JsonSchema format)
 get_current_weather_func = generative_models.FunctionDeclaration(
@@ -137,7 +132,7 @@ print(chat.send_message(
 
 #### Automatic Function calling
 
-```python
+```
 from vertexai.preview.generative_models import GenerativeModel, Tool, FunctionDeclaration, AutomaticFunctionCallingResponder
 
 # First, create functions that the model can use to answer your questions.
@@ -184,7 +179,7 @@ print(chat.send_message("What is the weather like in Boston?"))
 
 -  To perform bring-your-own-response(BYOR) evaluation, provide the model responses in the `response` column in the dataset. If a pairwise metric is used for BYOR evaluation, provide the baseline model responses in the `baseline_model_response` column.
 
-```python
+```
 import pandas as pd
 from vertexai.evaluation import EvalTask, MetricPromptTemplateExamples
 
@@ -208,7 +203,7 @@ eval_result = eval_task.evaluate(experiment_run_name="eval-experiment-run")
 ```
 -  To perform evaluation with Gemini model inference, specify the `model` parameter with a `GenerativeModel` instance.  The input column name to the model is `prompt` and must be present in the dataset.
 
-```python
+```
 from vertexai.evaluation import EvalTask
 from vertexai.generative_models import GenerativeModel
 
@@ -228,7 +223,7 @@ result = EvalTask(
 
 - If a `prompt_template` is specified, the `prompt` column is not required. Prompts can be assembled from the evaluation dataset, and all prompt template variable names must be present in the dataset columns.
 
-```python
+```
 import pandas as pd
 from vertexai.evaluation import EvalTask, MetricPromptTemplateExamples
 from vertexai.generative_models import GenerativeModel
@@ -250,7 +245,7 @@ result = EvalTask(
 parameter with a custom inference function. The input column name to the
 custom inference function is `prompt` and must be present in the dataset.
 
-```python
+```
 from openai import OpenAI
 from vertexai.evaluation import EvalTask, MetricPromptTemplateExamples
 
@@ -284,7 +279,7 @@ the `baseline_model` input to a `PairwiseMetric` instance and the candidate
 `model` input to the `EvalTask.evaluate()` function. The input column name
 to both models is `prompt` and must be present in the dataset.
 
-```python
+```
 import pandas as pd
 from vertexai.evaluation import EvalTask, MetricPromptTemplateExamples, PairwiseMetric
 from vertexai.generative_models import GenerativeModel

google/cloud/aiplatform/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.67.1"  # {x-release-please-version}
+__version__ = "1.66.0"  # {x-release-please-version}

google/cloud/aiplatform/tensorboard/uploader.py

@@ -386,7 +386,6 @@ def _pre_create_runs_and_time_series(self):
                 if (run_name and run_name != ".")
                 else uploader_utils.DEFAULT_RUN_NAME
             )
-            run_name = uploader_utils.reformat_run_name(run_name)
             run_names.append(run_name)
             for event in events:
                 _filter_graph_defs(event)

google/cloud/aiplatform/v1/schema/predict/instance/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.67.1"  # {x-release-please-version}
+__version__ = "1.66.0"  # {x-release-please-version}

google/cloud/aiplatform/v1/schema/predict/instance_v1/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.67.1"  # {x-release-please-version}
+__version__ = "1.66.0"  # {x-release-please-version}

google/cloud/aiplatform/v1/schema/predict/params/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.67.1"  # {x-release-please-version}
+__version__ = "1.66.0"  # {x-release-please-version}

google/cloud/aiplatform/v1/schema/predict/params_v1/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.67.1"  # {x-release-please-version}
+__version__ = "1.66.0"  # {x-release-please-version}

google/cloud/aiplatform/v1/schema/predict/prediction/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.67.1"  # {x-release-please-version}
+__version__ = "1.66.0"  # {x-release-please-version}

google/cloud/aiplatform/v1/schema/predict/prediction_v1/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.67.1"  # {x-release-please-version}
+__version__ = "1.66.0"  # {x-release-please-version}

google/cloud/aiplatform/v1/schema/trainingjob/definition/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.67.1"  # {x-release-please-version}
+__version__ = "1.66.0"  # {x-release-please-version}

google/cloud/aiplatform/v1/schema/trainingjob/definition_v1/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.67.1"  # {x-release-please-version}
+__version__ = "1.66.0"  # {x-release-please-version}

google/cloud/aiplatform/v1beta1/schema/predict/instance/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.67.1"  # {x-release-please-version}
+__version__ = "1.66.0"  # {x-release-please-version}

google/cloud/aiplatform/v1beta1/schema/predict/instance_v1beta1/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.67.1"  # {x-release-please-version}
+__version__ = "1.66.0"  # {x-release-please-version}

google/cloud/aiplatform/v1beta1/schema/predict/params/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.67.1"  # {x-release-please-version}
+__version__ = "1.66.0"  # {x-release-please-version}

google/cloud/aiplatform/v1beta1/schema/predict/params_v1beta1/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.67.1"  # {x-release-please-version}
+__version__ = "1.66.0"  # {x-release-please-version}

google/cloud/aiplatform/v1beta1/schema/predict/prediction/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.67.1"  # {x-release-please-version}
+__version__ = "1.66.0"  # {x-release-please-version}