Skip to content

Commit 8069bf8

Browse files
jhamoncursoragent
andauthored
Phase 2A: Response Adapter Consolidation (#600)
## Summary - Introduces a centralized `pinecone/adapters/` module for OpenAPI response transformations - Consolidates duplicated `parse_query_response` functions from 4+ files into single `adapt_query_response()` - Adds `adapt_upsert_response()` and `adapt_fetch_response()` adapters - Preserves backward compatibility via thin wrappers ## Test plan - [x] All 15 adapter unit tests pass - [x] All existing unit tests pass (429 passed, 4 skipped due to asyncio deps) - [x] Ruff linting passes - [x] No new mypy errors introduced Resolves: SDK-274 Made with [Cursor](https://cursor.com) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Touches core data-plane response transformation for `query`, `fetch`, and async upsert results; behavior should be unchanged but any mismatch in adapter logic could affect returned dataclasses or response metadata across clients. > > **Overview** > **Consolidates OpenAPI→SDK response parsing into a new `pinecone/adapters` layer.** Adds `adapt_query_response`, `adapt_fetch_response`, and `adapt_upsert_response`, plus an `UpsertResponseTransformer` wrapper for async `ApplyResult`. > > Updates sync/async index and vector resources to use these shared adapters and replaces duplicated parsing code, while keeping existing `parse_query_response` as deprecated pass-through wrappers for backward compatibility. > > Adds unit test coverage and new OpenAPI response fixtures to validate adapter behavior (namespace handling, usage, deprecated field cleanup, and response metadata extraction). > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 716b744. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 07e0884 commit 8069bf8

File tree

13 files changed

+2971
-202
lines changed

13 files changed

+2971
-202
lines changed

FTS_SDK_PLAN.md

Lines changed: 527 additions & 0 deletions
Large diffs are not rendered by default.

SDK_FEATURES.md

Lines changed: 1869 additions & 0 deletions
Large diffs are not rendered by default.

pinecone/adapters/__init__.py

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
"""Adapter layer for converting between OpenAPI models and SDK types.
2+
3+
This module provides a centralized adapter layer that isolates SDK code from
4+
the details of generated OpenAPI models. This enables:
5+
6+
- Single source of truth for response transformations
7+
- Easier testing (adapters can be tested in isolation)
8+
- Version flexibility (adapters can handle different API versions)
9+
- Clear contracts between generated and SDK code
10+
11+
Usage:
12+
>>> from pinecone.adapters import adapt_query_response, adapt_upsert_response
13+
>>> sdk_response = adapt_query_response(openapi_response)
14+
"""
15+
16+
from pinecone.adapters.response_adapters import (
17+
adapt_fetch_response,
18+
adapt_query_response,
19+
adapt_upsert_response,
20+
UpsertResponseTransformer,
21+
)
22+
23+
__all__ = [
24+
"adapt_fetch_response",
25+
"adapt_query_response",
26+
"adapt_upsert_response",
27+
"UpsertResponseTransformer",
28+
]
Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
"""Adapter functions for converting OpenAPI responses to SDK dataclasses.
2+
3+
This module provides centralized functions for transforming OpenAPI response
4+
objects into SDK-specific dataclasses. These adapters isolate the SDK from
5+
changes in the OpenAPI model structure.
6+
7+
The adapter functions replace duplicated parsing logic that was previously
8+
scattered across multiple modules.
9+
"""
10+
11+
from __future__ import annotations
12+
13+
from multiprocessing.pool import ApplyResult
14+
from typing import TYPE_CHECKING, Any
15+
16+
from pinecone.adapters.utils import extract_response_metadata
17+
18+
if TYPE_CHECKING:
19+
from pinecone.db_data.dataclasses.fetch_response import FetchResponse
20+
from pinecone.db_data.dataclasses.query_response import QueryResponse
21+
from pinecone.db_data.dataclasses.upsert_response import UpsertResponse
22+
23+
24+
def adapt_query_response(openapi_response: Any) -> QueryResponse:
25+
"""Adapt an OpenAPI QueryResponse to the SDK QueryResponse dataclass.
26+
27+
This function extracts fields from the OpenAPI response object and
28+
constructs an SDK-native QueryResponse dataclass. It handles:
29+
- Extracting matches and namespace
30+
- Optional usage information
31+
- Response metadata (headers)
32+
- Cleaning up deprecated 'results' field
33+
34+
Args:
35+
openapi_response: An OpenAPI QueryResponse object from the generated code.
36+
37+
Returns:
38+
A QueryResponse dataclass instance.
39+
40+
Example:
41+
>>> from pinecone.adapters import adapt_query_response
42+
>>> sdk_response = adapt_query_response(openapi_response)
43+
>>> print(sdk_response.matches)
44+
"""
45+
# Import at runtime to avoid circular imports
46+
from pinecone.db_data.dataclasses.query_response import QueryResponse as QR
47+
48+
response_info = extract_response_metadata(openapi_response)
49+
50+
# Remove deprecated 'results' field if present
51+
if hasattr(openapi_response, "_data_store"):
52+
openapi_response._data_store.pop("results", None)
53+
54+
return QR(
55+
matches=openapi_response.matches,
56+
namespace=openapi_response.namespace or "",
57+
usage=openapi_response.usage
58+
if hasattr(openapi_response, "usage") and openapi_response.usage
59+
else None,
60+
_response_info=response_info,
61+
)
62+
63+
64+
def adapt_upsert_response(openapi_response: Any) -> UpsertResponse:
65+
"""Adapt an OpenAPI UpsertResponse to the SDK UpsertResponse dataclass.
66+
67+
Args:
68+
openapi_response: An OpenAPI UpsertResponse object from the generated code.
69+
70+
Returns:
71+
An UpsertResponse dataclass instance.
72+
73+
Example:
74+
>>> from pinecone.adapters import adapt_upsert_response
75+
>>> sdk_response = adapt_upsert_response(openapi_response)
76+
>>> print(sdk_response.upserted_count)
77+
"""
78+
# Import at runtime to avoid circular imports
79+
from pinecone.db_data.dataclasses.upsert_response import UpsertResponse as UR
80+
81+
response_info = extract_response_metadata(openapi_response)
82+
83+
return UR(upserted_count=openapi_response.upserted_count, _response_info=response_info)
84+
85+
86+
def adapt_fetch_response(openapi_response: Any) -> FetchResponse:
87+
"""Adapt an OpenAPI FetchResponse to the SDK FetchResponse dataclass.
88+
89+
This function extracts fields from the OpenAPI response object and
90+
constructs an SDK-native FetchResponse dataclass. It handles:
91+
- Converting vectors dict to SDK Vector objects
92+
- Optional usage information
93+
- Response metadata (headers)
94+
95+
Args:
96+
openapi_response: An OpenAPI FetchResponse object from the generated code.
97+
98+
Returns:
99+
A FetchResponse dataclass instance.
100+
101+
Example:
102+
>>> from pinecone.adapters import adapt_fetch_response
103+
>>> sdk_response = adapt_fetch_response(openapi_response)
104+
>>> print(sdk_response.vectors)
105+
"""
106+
# Import at runtime to avoid circular imports
107+
from pinecone.db_data.dataclasses.fetch_response import FetchResponse as FR
108+
from pinecone.db_data.dataclasses.vector import Vector
109+
110+
response_info = extract_response_metadata(openapi_response)
111+
112+
return FR(
113+
namespace=openapi_response.namespace,
114+
vectors={k: Vector.from_dict(v) for k, v in openapi_response.vectors.items()},
115+
usage=openapi_response.usage,
116+
_response_info=response_info,
117+
)
118+
119+
120+
class UpsertResponseTransformer:
121+
"""Transformer for converting ApplyResult[OpenAPIUpsertResponse] to UpsertResponse.
122+
123+
This wrapper transforms the OpenAPI response to our dataclass when .get() is called,
124+
while delegating other methods to the underlying ApplyResult.
125+
126+
Example:
127+
>>> transformer = UpsertResponseTransformer(async_result)
128+
>>> response = transformer.get() # Returns UpsertResponse
129+
"""
130+
131+
_apply_result: ApplyResult
132+
""" :meta private: """
133+
134+
def __init__(self, apply_result: ApplyResult) -> None:
135+
self._apply_result = apply_result
136+
137+
def get(self, timeout: float | None = None) -> UpsertResponse:
138+
"""Get the transformed UpsertResponse.
139+
140+
Args:
141+
timeout: Optional timeout in seconds for the underlying result.
142+
143+
Returns:
144+
The SDK UpsertResponse dataclass.
145+
"""
146+
openapi_response = self._apply_result.get(timeout)
147+
return adapt_upsert_response(openapi_response)
148+
149+
def __getattr__(self, name: str) -> Any:
150+
# Delegate other methods to the underlying ApplyResult
151+
return getattr(self._apply_result, name)

pinecone/adapters/utils.py

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
"""Shared utilities for adapter implementations.
2+
3+
This module provides helper functions used across different adapters
4+
for common operations like extracting response metadata.
5+
"""
6+
7+
from typing import Any
8+
9+
from pinecone.utils.response_info import ResponseInfo, extract_response_info
10+
11+
12+
def extract_response_metadata(response: Any) -> ResponseInfo:
13+
"""Extract response metadata from an OpenAPI response object.
14+
15+
Extracts the _response_info attribute from an OpenAPI response if present,
16+
otherwise returns an empty ResponseInfo with empty headers.
17+
18+
Args:
19+
response: An OpenAPI response object that may have _response_info.
20+
21+
Returns:
22+
ResponseInfo with extracted headers, or empty headers if not present.
23+
"""
24+
response_info = None
25+
if hasattr(response, "_response_info"):
26+
response_info = response._response_info
27+
28+
if response_info is None:
29+
response_info = extract_response_info({})
30+
31+
return response_info

pinecone/db_data/index.py

Lines changed: 10 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,11 @@
5252
)
5353
from .query_results_aggregator import QueryResultsAggregator, QueryNamespacesResults
5454
from pinecone.openapi_support import OPENAPI_ENDPOINT_PARAMS
55+
from pinecone.adapters.response_adapters import (
56+
adapt_query_response,
57+
adapt_fetch_response,
58+
UpsertResponseTransformer,
59+
)
5560

5661
from multiprocessing.pool import ApplyResult
5762
from multiprocessing import cpu_count
@@ -76,58 +81,12 @@
7681

7782

7883
def parse_query_response(response: OpenAPIQueryResponse) -> QueryResponse:
79-
""":meta private:"""
80-
# Convert OpenAPI QueryResponse to dataclass QueryResponse
81-
from pinecone.utils.response_info import extract_response_info
82-
83-
response_info = None
84-
if hasattr(response, "_response_info"):
85-
response_info = response._response_info
86-
87-
if response_info is None:
88-
response_info = extract_response_info({})
89-
90-
# Remove deprecated 'results' field if present
91-
if hasattr(response, "_data_store"):
92-
response._data_store.pop("results", None)
93-
94-
return QueryResponse(
95-
matches=response.matches,
96-
namespace=response.namespace or "",
97-
usage=response.usage if hasattr(response, "usage") and response.usage else None,
98-
_response_info=response_info,
99-
)
100-
101-
102-
class UpsertResponseTransformer:
103-
"""Transformer for converting ApplyResult[OpenAPIUpsertResponse] to UpsertResponse.
84+
""":meta private:
10485
105-
This wrapper transforms the OpenAPI response to our dataclass when .get() is called,
106-
while delegating other methods to the underlying ApplyResult.
86+
Deprecated: Use adapt_query_response from pinecone.adapters instead.
87+
This function is kept for backward compatibility.
10788
"""
108-
109-
_apply_result: ApplyResult
110-
""" :meta private: """
111-
112-
def __init__(self, apply_result: ApplyResult) -> None:
113-
self._apply_result = apply_result
114-
115-
def get(self, timeout: float | None = None) -> UpsertResponse:
116-
openapi_response = self._apply_result.get(timeout)
117-
from pinecone.utils.response_info import extract_response_info
118-
119-
response_info = None
120-
if hasattr(openapi_response, "_response_info"):
121-
response_info = openapi_response._response_info
122-
if response_info is None:
123-
response_info = extract_response_info({})
124-
return UpsertResponse(
125-
upserted_count=openapi_response.upserted_count, _response_info=response_info
126-
)
127-
128-
def __getattr__(self, name: str) -> Any:
129-
# Delegate other methods to the underlying ApplyResult
130-
return getattr(self._apply_result, name)
89+
return adapt_query_response(response)
13190

13291

13392
class Index(PluginAware):
@@ -894,22 +853,7 @@ def fetch(self, ids: list[str], namespace: str | None = None, **kwargs) -> Fetch
894853
"""
895854
args_dict = parse_non_empty_args([("namespace", namespace)])
896855
result = self._vector_api.fetch_vectors(ids=ids, **args_dict, **kwargs)
897-
# Copy response info from OpenAPI response if present
898-
from pinecone.utils.response_info import extract_response_info
899-
900-
response_info = None
901-
if hasattr(result, "_response_info"):
902-
response_info = result._response_info
903-
if response_info is None:
904-
response_info = extract_response_info({})
905-
906-
fetch_response = FetchResponse(
907-
namespace=result.namespace,
908-
vectors={k: Vector.from_dict(v) for k, v in result.vectors.items()},
909-
usage=result.usage,
910-
_response_info=response_info,
911-
)
912-
return fetch_response
856+
return adapt_fetch_response(result)
913857

914858
@validate_and_convert_errors
915859
def fetch_by_metadata(

pinecone/db_data/index_asyncio.py

Lines changed: 8 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,7 @@
5858
)
5959

6060
from pinecone.openapi_support import OPENAPI_ENDPOINT_PARAMS
61+
from pinecone.adapters.response_adapters import adapt_query_response, adapt_fetch_response
6162
from .index import IndexRequestFactory
6263

6364
from .vector_factory import VectorFactory
@@ -91,27 +92,12 @@
9192

9293

9394
def parse_query_response(response: OpenAPIQueryResponse) -> QueryResponse:
94-
""":meta private:"""
95-
# Convert OpenAPI QueryResponse to dataclass QueryResponse
96-
from pinecone.utils.response_info import extract_response_info
97-
98-
response_info = None
99-
if hasattr(response, "_response_info"):
100-
response_info = response._response_info
101-
102-
if response_info is None:
103-
response_info = extract_response_info({})
104-
105-
# Remove deprecated 'results' field if present
106-
if hasattr(response, "_data_store"):
107-
response._data_store.pop("results", None)
108-
109-
return QueryResponse(
110-
matches=response.matches,
111-
namespace=response.namespace or "",
112-
usage=response.usage if hasattr(response, "usage") and response.usage else None,
113-
_response_info=response_info,
114-
)
95+
""":meta private:
96+
97+
Deprecated: Use adapt_query_response from pinecone.adapters instead.
98+
This function is kept for backward compatibility.
99+
"""
100+
return adapt_query_response(response)
115101

116102

117103
class _IndexAsyncio:
@@ -647,22 +633,7 @@ async def main():
647633
"""
648634
args_dict = parse_non_empty_args([("namespace", namespace)])
649635
result = await self._vector_api.fetch_vectors(ids=ids, **args_dict, **kwargs)
650-
# Copy response info from OpenAPI response if present
651-
from pinecone.utils.response_info import extract_response_info
652-
653-
response_info = None
654-
if hasattr(result, "_response_info"):
655-
response_info = result._response_info
656-
if response_info is None:
657-
response_info = extract_response_info({})
658-
659-
fetch_response = FetchResponse(
660-
namespace=result.namespace,
661-
vectors={k: Vector.from_dict(v) for k, v in result.vectors.items()},
662-
usage=result.usage,
663-
_response_info=response_info,
664-
)
665-
return fetch_response
636+
return adapt_fetch_response(result)
666637

667638
@validate_and_convert_errors
668639
async def fetch_by_metadata(

0 commit comments

Comments
 (0)