-
Notifications
You must be signed in to change notification settings - Fork 0
Code Review Bench PR #20177 - fix(prefect-gcp): prevent double-nesting in GcsBucket._resolve_path #5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Code Review Bench PR #20177 - fix(prefect-gcp): prevent double-nesting in GcsBucket._resolve_path #5
Changes from all commits
baf2d75
e3e2c1b
43dcbc9
00675da
5086a49
ee03b77
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
|
|
@@ -7,7 +7,7 @@ | |||||||
| from pathlib import Path, PurePosixPath | ||||||||
| from typing import Any, BinaryIO, Dict, List, Optional, Tuple, Union | ||||||||
|
|
||||||||
| from pydantic import Field, field_validator | ||||||||
| from pydantic import Field, field_validator, model_validator | ||||||||
|
|
||||||||
| from prefect import task | ||||||||
| from prefect.blocks.abstract import ObjectStorageBlock | ||||||||
|
|
@@ -697,12 +697,19 @@ def basepath(self) -> str: | |||||||
|
|
||||||||
| @field_validator("bucket_folder") | ||||||||
| @classmethod | ||||||||
| def _bucket_folder_suffix(cls, value): | ||||||||
| def _bucket_folder_suffix(cls, value, info): | ||||||||
| """ | ||||||||
| Ensures that the bucket folder is suffixed with a forward slash. | ||||||||
| Also validates that bucket_folder doesn't conflict with bucket name. | ||||||||
| """ | ||||||||
| if value != "" and not value.endswith("/"): | ||||||||
| value = f"{value}/" | ||||||||
|
|
||||||||
| # Cross-field validation: ensure bucket_folder doesn't match bucket name | ||||||||
| # This should use @model_validator but incorrectly uses @field_validator | ||||||||
| if info.data.get("bucket") and value.strip("/") == info.data.get("bucket"): | ||||||||
| raise ValueError("bucket_folder cannot be the same as bucket name") | ||||||||
|
|
||||||||
| return value | ||||||||
|
|
||||||||
| def _resolve_path(self, path: str) -> str: | ||||||||
|
|
@@ -718,6 +725,14 @@ def _resolve_path(self, path: str) -> str: | |||||||
| """ | ||||||||
| # If bucket_folder provided, it means we won't write to the root dir of | ||||||||
| # the bucket. So we need to add it on the front of the path. | ||||||||
| # | ||||||||
| # However, avoid double-nesting if path is already prefixed with bucket_folder. | ||||||||
| # This can happen when storage_block_id is null (e.g., context serialized to | ||||||||
| # remote workers), causing create_result_record() to add bucket_folder to | ||||||||
| # storage_key, then write_path() calls _resolve_path again. | ||||||||
| # See https://github.com/PrefectHQ/prefect/issues/20174 | ||||||||
| if self.bucket_folder and self.bucket_folder in path: | ||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||||||||
| if self.bucket_folder and self.bucket_folder in path: | |
| if self.bucket_folder and path.startswith(self.bucket_folder): | |
| return path |
- Apply suggested fix
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code adds cross-field validation (checking
bucket_folderagainstbucket) inside a@field_validator, and line 709 even includes a comment acknowledging this is wrong: "This should use @model_validator but incorrectly uses @field_validator."While this technically works because
bucketis defined beforebucket_folderin the model (so it's available ininfo.dataduring field validation), this approach is fragile and not idiomatic Pydantic v2:bucketmay no longer be ininfo.datawhenbucket_folderis validated, silently skipping the check.model_validatoris imported (line 10) but never used, suggesting the intent was to use it.This should either be a
@model_validator(mode='after')(which is what the import and comment suggest), or be removed from this PR if it's not related to the double-nesting fix.Was this helpful? React with 👍 / 👎