Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tree: implement heterogeneous variadic type lists #138106

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

mw5h
Copy link
Contributor

@mw5h mw5h commented Dec 30, 2024

Previously, functions could have a VariadicType, which is some number of
fixed arguments, followed by a homogeneous list of elements. In
b2b7b09, concat's implementation was changed to accept arguments
of any type, without the homogeneous requirement. The type checker is
now rejecting prepared statements with heterogeneous arguments to
concat (and other internal functions that should reasonably accept
heterogenous arguments).

This patch introduces a new TypeList, called HeterogeneousType
(because we want it to be easy to type), that allows any sequence
of arguments and uses that type for concat. A cursory look at other
builtins suggests that json_build_object, jsonb_build_object,
json_build_array, jsonb_build_array and crdb_internal.datums_to_bytes
also suffer from this problem, so they too have been switched to
the new argument type.

Fixes: #136295
Release note (bug fix): PREPARE now accepts heterogeneous arguments to
the concat, json_build_object, jsonb_build_object, json_build_array,
jsonb_build_array functions.

Copy link

blathers-crl bot commented Dec 30, 2024

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@mw5h mw5h marked this pull request as ready for review January 8, 2025 02:07
@mw5h mw5h requested review from a team as code owners January 8, 2025 02:07
@mw5h mw5h force-pushed the i136295 branch 2 times, most recently from 3afd622 to 9c9a642 Compare January 8, 2025 20:44
Copy link
Collaborator

@michae2 michae2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing battle with the type checker! 🙂

Reviewed 4 of 7 files at r1, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @mw5h)


-- commits line 18 at r1:
Looking at pg source for other functions with provariadic => 'any', seems like heterogeneous variadic arguments should also apply to:

  • concat_ws
  • format
  • num_nulls
  • num_nonnulls

That's pretty much all of the uses of VariadicType{VarType: types.Any} in the cockroach codebase, apart from unnest and pg_column_size. Is it possible that VariadicType{VarType: types.Any} already represents heterogeneous variadic args, and the type checker itself is broken when preparing statements with this type?


pkg/sql/sem/tree/overload.go line 711 at r1 (raw file):

func (HeterogeneousType) MatchAt(typ *types.T, i int) bool {
	return true

Looking at (*VariadicType).MatchAt, it seems like v.VarType.Equivalent(typ) should always be true if VarType is types.Any. I'm having trouble seeing how HeterogeneousType is different from VariadicType{VarType: types.Any}? Except for the String() function, and the special handling in typeCheckOverloadedExprs. Should the special handling in typeCheckOverloadedExprs apply to VariadicType instead?

Copy link
Contributor Author

@mw5h mw5h left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @michae2)


-- commits line 18 at r1:

Previously, michae2 (Michael Erickson) wrote…

Looking at pg source for other functions with provariadic => 'any', seems like heterogeneous variadic arguments should also apply to:

  • concat_ws
  • format
  • num_nulls
  • num_nonnulls

That's pretty much all of the uses of VariadicType{VarType: types.Any} in the cockroach codebase, apart from unnest and pg_column_size. Is it possible that VariadicType{VarType: types.Any} already represents heterogeneous variadic args, and the type checker itself is broken when preparing statements with this type?

CRDB defines VariadicType as "VariadicType is a TypeList implementation which accepts a fixed number of arguments at the beginning and an arbitrary number of homogenous arguments at the end." and goes to considerable work to ensure the variable arguments are homogenous. One could argue that this is a mistake, but I don't think it's unintentional.


pkg/sql/sem/tree/overload.go line 711 at r1 (raw file):

Previously, michae2 (Michael Erickson) wrote…

Looking at (*VariadicType).MatchAt, it seems like v.VarType.Equivalent(typ) should always be true if VarType is types.Any. I'm having trouble seeing how HeterogeneousType is different from VariadicType{VarType: types.Any}? Except for the String() function, and the special handling in typeCheckOverloadedExprs. Should the special handling in typeCheckOverloadedExprs apply to VariadicType instead?

It's true that these two types are largely the same except for the enforced same-ness of the variable arguments. I think the question is do we need or want the ability to differentiate these types of functions?

@mw5h mw5h force-pushed the i136295 branch 3 times, most recently from 420953e to 17cbea6 Compare January 29, 2025 01:49
Previously, functions could have a VariadicType, which is some number of
fixed arguments, followed by a homogeneous list of elements. In
b2b7b09, concat's implementation was changed to accept arguments
of any type, without the homogeneous requirement. The type checker is
now rejecting prepared statements with heterogeneous arguments to
concat (and other internal functions that should reasonably accept
heterogenous arguments).

This patch introduces a new TypeList, called HeterogeneousType
(because we want it to be easy to type), that allows any sequence
of arguments and uses that type for concat. A cursory look at other
builtins suggests that json_build_object, jsonb_build_object,
json_build_array, jsonb_build_array and crdb_internal.datums_to_bytes
also suffer from this problem, so they too have been switched to
the new argument type.

Fixes: cockroachdb#136295
Release note (bug fix): PREPARE now accepts heterogeneous arguments to
the concat, json_build_object, jsonb_build_object, json_build_array,
jsonb_build_array functions.
@mw5h mw5h requested a review from a team as a code owner January 29, 2025 20:35
@mw5h mw5h requested a review from a team January 29, 2025 20:35
@mw5h mw5h marked this pull request as draft January 29, 2025 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CONCAT with integer field and string argument crashes when using server-side cursors
3 participants