Skip to content

Remove XCom API endpoint full deserialization option #27740

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bolkedebruin wants to merge 1 commit into from
Closed

Remove XCom API endpoint full deserialization option #27740

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 1 addition & 12 deletions airflow/api_connexion/endpoints/xcom_endpoint.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,14 +88,10 @@ def get_xcom_entry(
task_id: str,
dag_run_id: str,
xcom_key: str,
deserialize: bool = False,
session: Session = NEW_SESSION,
) -> APIResponse:
"""Get an XCom entry"""
if deserialize:
query = session.query(XCom, XCom.value)
else:
query = session.query(XCom)
query = session.query(XCom)

query = query.filter(XCom.dag_id == dag_id, XCom.task_id == task_id, XCom.key == xcom_key)
query = query.join(DR, and_(XCom.dag_id == DR.dag_id, XCom.run_id == DR.run_id))
Expand All @@ -105,11 +101,4 @@ def get_xcom_entry(
if item is None:
raise NotFound("XCom entry not found")

if deserialize:
xcom, value = item
stub = copy.copy(xcom)
stub.value = value
stub.value = XCom.deserialize_value(stub)
item = stub

return xcom_schema.dump(item)
17 changes: 0 additions & 17 deletions airflow/api_connexion/openapi/v1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1588,23 +1588,6 @@ paths:
x-openapi-router-controller: airflow.api_connexion.endpoints.xcom_endpoint
operationId: get_xcom_entry
tags: [XCom]
parameters:
- in: query
name: deserialize
schema:
type: boolean
default: false
required: false
description: |
Whether to deserialize an XCom value when using a custom XCom backend.

The XCom API endpoint calls `orm_deserialize_value` by default since an XCom may contain value
that is potentially expensive to deserialize in the web server. Setting this to true overrides
the consideration, and calls `deserialize_value` instead.

This parameter is not meaningful when using the default XCom backend.

*New in version 2.4.0*
responses:
'200':
description: Success.
Expand Down
16 changes: 1 addition & 15 deletions airflow/www/static/js/types/api-generated.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3711,20 +3711,6 @@ export interface operations {
/** The XCom key. */
xcom_key: components["parameters"]["XComKey"];
};
query: {
/**
* Whether to deserialize an XCom value when using a custom XCom backend.
*
* The XCom API endpoint calls `orm_deserialize_value` by default since an XCom may contain value
* that is potentially expensive to deserialize in the web server. Setting this to true overrides
* the consideration, and calls `deserialize_value` instead.
*
* This parameter is not meaningful when using the default XCom backend.
*
* *New in version 2.4.0*
*/
deserialize?: boolean;
};
};
responses: {
/** Success. */
Expand Down Expand Up @@ -4543,7 +4529,7 @@ export type GetVariableVariables = CamelCasedPropertiesDeep<operations['get_vari
export type DeleteVariableVariables = CamelCasedPropertiesDeep<operations['delete_variable']['parameters']['path']>;
export type PatchVariableVariables = CamelCasedPropertiesDeep<operations['patch_variable']['parameters']['path'] & operations['patch_variable']['parameters']['query'] & operations['patch_variable']['requestBody']['content']['application/json']>;
export type GetXcomEntriesVariables = CamelCasedPropertiesDeep<operations['get_xcom_entries']['parameters']['path'] & operations['get_xcom_entries']['parameters']['query']>;
export type GetXcomEntryVariables = CamelCasedPropertiesDeep<operations['get_xcom_entry']['parameters']['path'] & operations['get_xcom_entry']['parameters']['query']>;
export type GetXcomEntryVariables = CamelCasedPropertiesDeep<operations['get_xcom_entry']['parameters']['path']>;
export type GetExtraLinksVariables = CamelCasedPropertiesDeep<operations['get_extra_links']['parameters']['path']>;
export type GetLogVariables = CamelCasedPropertiesDeep<operations['get_log']['parameters']['path'] & operations['get_log']['parameters']['query']>;
export type GetDagDetailsVariables = CamelCasedPropertiesDeep<operations['get_dag_details']['parameters']['path']>;
Expand Down
6 changes: 6 additions & 0 deletions newsfragments/27740.significant.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
XCom API endpoint no longer supports ``deserialize`` option

For security reasons the XCom API no longer supports the ``deserialize`` option
to deserialize arbitrary XCom values in the webserver. Custom XCom backend can
implement their own ``orm_deserialize_value`` if required. However it is strongly
suggested to deserialize on the client side.
2 changes: 0 additions & 2 deletions tests/api_connexion/endpoints/test_xcom_endpoint.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -185,8 +185,6 @@ def _create_xcom_entry(self, dag_id, run_id, execution_date, task_id, xcom_key,
@pytest.mark.parametrize(
"query, expected_value",
[
pytest.param("?deserialize=true", "real deserialized TEST_VALUE", id="true"),
pytest.param("?deserialize=false", "orm deserialized TEST_VALUE", id="false"),
pytest.param("", "orm deserialized TEST_VALUE", id="default"),
],
)
Expand Down