Skip to content

Commit 642c4bb

Browse files
committed
[SPARK-48639][CONNECT][PYTHON] Add Origin to RelationCommon
### What changes were proposed in this pull request? This PR proposes to add `Origin` (from #46789) to `Relation.RelationCommon` This is a revert of the revert. ### Why are the changes needed? To have the common protobuf message to keep the source code info. ```diff - // TODO(SPARK-48639): Add origin like Expression.ExpressionCommon - - // (Required) Shared relation metadata. - string source_info = 1; + // (Optional) Shared relation metadata. + reserved 1; ``` This is considered as a breaking change, and we should fix up all other branches down to avoid, which isn't really worthwhile. ### Does this PR introduce _any_ user-facing change? No. This is not used. ### How was this patch tested? CI should validate protobuf definition, and exiting tests should pass. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #47115 from HyukjinKwon/SPARK-48639-followup. Authored-by: Hyukjin Kwon <gurwls223@apache.org> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
1 parent 5943905 commit 642c4bb

File tree

3 files changed

+178
-159
lines changed

3 files changed

+178
-159
lines changed

connector/connect/common/src/main/protobuf/spark/connect/relations.proto

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ import "google/protobuf/any.proto";
2323
import "spark/connect/expressions.proto";
2424
import "spark/connect/types.proto";
2525
import "spark/connect/catalog.proto";
26+
import "spark/connect/common.proto";
2627

2728
option java_multiple_files = true;
2829
option java_package = "org.apache.spark.connect.proto";
@@ -106,13 +107,14 @@ message Unknown {}
106107

107108
// Common metadata of all relations.
108109
message RelationCommon {
109-
// TODO(SPARK-48639): Add origin like Expression.ExpressionCommon
110-
111110
// (Required) Shared relation metadata.
112-
string source_info = 1;
111+
string source_info = 1 [deprecated=true];
113112

114113
// (Optional) A per-client globally unique id for a given connect plan.
115114
optional int64 plan_id = 2;
115+
116+
// (Optional) Keep the information of the origin for this expression such as stacktrace.
117+
Origin origin = 3;
116118
}
117119

118120
// Relation that uses a SQL query to generate the output.

0 commit comments

Comments
 (0)