UDT #168

PiotrJunior · 2025-03-25T09:58:21Z

Adds UserDefinedType.
Currently not all Integration tests are enabled, as UDT don't support typeHints for now and resultMetadata is not yet supported.

wprzytula · 2025-03-25T11:51:29Z

Just FYI @PiotrJunior: I'm not going to review this as long as it has conflicts with the base branch.

adespawn · 2025-04-15T08:44:36Z

It this PR ready for the review?

wprzytula · 2025-04-16T08:19:12Z

It this PR ready for the review?

I doubt it is. It has conflicts and the CI is failing.

src/types/type_wrappers.rs

wprzytula · 2025-05-22T12:13:06Z

src/types/type_wrappers.rs

+                //
+                // This will be fixed at a later time, as it requires more investigation into how UDT works.


❓ When? As UDT support is added in this PR, I believe you should be now able to solve the problem.

This comment is regarding typeGuessing and I think @adespawn can elaborate on this topic.

This will be fixed in next PR.

src/requests/parameter_wrappers.rs

examples/DataStax/udt/udt-insert-select.js

src/types/type_wrappers.rs

Removes CqlTypeClass, to reduce the overhead from it's type. This means, that array will be treated as ambiguous types.

Copilot

Pull Request Overview

This PR adds support for a new UserDefinedType (UDT) across multiple parts of the codebase including unit tests, integration tests, type wrappers, and request handling. Key changes include:

Adding new test cases for UDT in unit and integration tests.
Enhancing Rust type wrappers to support UDT with additional fields and associated methods.
Updating JavaScript utilities and driver code to properly encode, decode, and wrap UDTs.

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
test/unit/query-parameter-tests.js	Added UDT test case to verify proper parameter encoding.
test/unit/cql-value-wrapper-tests.js	Introduced test for retrieving UDT type values from the N-API.
test/integration/supported/client-execute-tests.js	Adjusted and commented out UDT/tuple tests; added TODOs for missing features.
test/integration/supported/client-execute-prepared-tests.js	Updated handling of UDT/tuple tests in prepared statement flows.
src/types/type_wrappers.rs	Extended ComplexType with UDT-specific fields and helper methods.
src/tests/result_tests.rs	Added test function for UDT value wrapper functionality.
src/tests/request_values_tests.rs	Added UDT mapping and sample UDT value in tests.
src/result.rs	Implemented N-API conversion for UDT in CqlValueWrapper.
src/requests/parameter_wrappers.rs	Added function to convert UDT parameters from JS arrays to CqlValue.
lib/types/cql-utils.js	Enhanced UDT encoding logic for driver parameter wrapping.
examples/DataStax/udt/udt-insert-select.js	Updated example to use dynamic unique key and replication strategy changes.

Comments suppressed due to low confidence (2)

test/integration/supported/client-execute-tests.js:1183

Instead of commenting out tests with block comments, consider using a test skip function (e.g., 'xit' or an equivalent) to clearly indicate that the test is intentionally disabled until UDT support is complete.

            // TODO: Add support for column information retrieval

test/integration/supported/client-execute-prepared-tests.js:1408

To enhance clarity and maintainability, use a formal test skip mechanism to disable tests temporarily rather than commenting them out.

        // Test for UDT and Tuple support

examples/DataStax/udt/udt-insert-select.js

lib/types/cql-utils.js

src/requests/parameter_wrappers.rs

src/tests/result_tests.rs

…-result UUID result

src/types/type_wrappers.rs

src/tests/result_tests.rs

wprzytula · 2025-06-04T08:53:08Z

lib/types/cql-utils.js

+ * @param {Object} value
+ * @param {rust.ComplexType} type
+ * @returns {Array<rust.ComplexType|Array<Array<String|any>>>} Returns tuple: [rust.ComplexType, [[fieldName, fieldValue]]]
+ */
+function encodeUdt(value, type) {


❓

Returns tuple: [rust.ComplexType, [[fieldName, fieldValue]]]

Really? Double Array? Why?

Because, this function returns array of tuples of field name and value, but in JS tuple is represented as array

Please explain this thoroughly in a comment.

examples/DataStax/udt/udt-insert-select.js

wprzytula · 2025-06-04T11:08:32Z

test/integration/supported/client-execute-tests.js

+            // TODO: Add support for column information retrieval
+            /* vit("2.1", "should retrieve column information", function (done) {
                const client = setupInfo.client;
                client.execute(
                    util.format(selectQuery, sampleId),


❓ @adespawn @ZuzaOsa @PiotrJunior Do described limitations (unimplemented parts of the driver's API) include missing fields/properties (not only methods)?

wprzytula · 2025-06-04T11:09:22Z

test/integration/supported/client-execute-tests.js

+            // TODO: Add type guessing for UDT and Tuple
+            /* vit(


❓ Why isn't this implemented in this PR yet?
❓ When is it going to be implemented?
🔧 If there's no issue for this missing feature yet, please open it!

src/types/type_wrappers.rs

wprzytula · 2025-06-04T11:10:33Z

src/types/type_wrappers.rs

+    Uuid = 0x000C,
+    Varint = 0x000E,
+    Custom = 0x0000,
+    Unprovided = 0x0070,


❓ How can you be sure that there's no type that uses this value? Where did you take it from?

those codes are taken from CQL protocol specification

I'm specifically asking about the Unprovided variant.

Installing packages with `pip` using the `--user` flag may be the way to avoid permission issues. Let's try this out.

…ci-ccm-permission-errors CI: attempt to fix ccm permission errors

Expand ComplexType to include UDTs.

Adds support for converting udt responses to a JavaScript object.

Add tests for converting CQL values to JavaScript values

Adds support for converting JS UDT type into Rust UDT type.

Adds tests for converting JS UDT into Rust CQL values.

Add proper type IDs to the type wrappers. IDs match those in CQL protocol. This is will be useful for testing metadata.

adespawn · 2025-06-14T17:51:13Z

lib/types/cql-utils.js

+        }
+        const element = getWrapped(
+            types[i],
+            value[names[i]] !== undefined ? value[names[i]] : null, // Undefineds should be treated as nulls, so we convert them to nulls.


Suggested change

value[names[i]] !== undefined ? value[names[i]] : null, // Undefineds should be treated as nulls, so we convert them to nulls.

value[names[i]] !== undefined ? value[names[i]] : null, // Undefined values should be treated as null value.

adespawn · 2025-06-14T17:56:13Z

src/requests/parameter_wrappers.rs

+        match pair.get::<Undefined>(1) {
+            Ok(Some(_)) | Ok(None) => res.push((name, None)),
+            _ => {
+                let value = cql_value_from_napi_value(&typ, &pair, 1)?;
+                res.push((name, Some(value)));
+            }
+        }


Why do we attempt to get undefined, if we can only receive null?

adespawn · 2025-06-14T17:59:14Z

src/result.rs

+                keyspace: _,
+                name: _,


Do we loose this information, or did DSX not provide those information in result?

adespawn · 2025-06-14T18:02:15Z

src/requests/parameter_wrappers.rs

+                .udt_metadata
+                .as_ref()
+                .map(|udt| udt.keyspace.clone())
+                .expect("Expected keyspace for UDT")
+                .clone(),
+            name: typ
+                .udt_metadata
+                .as_ref()
+                .map(|udt| udt.name.clone())
+                .expect("Expected name for UDT")
+                .clone(),


Did DataStax provide any possibility to have UDT in unprepared queries?

From what I can see expecting keyspace and name limits it's usage only to prepared queries

adespawn · 2025-06-14T18:04:01Z

src/requests/parameter_wrappers.rs

+                .udt_metadata
+                .as_ref()
+                .map(|udt| udt.keyspace.clone())
+                .expect("Expected keyspace for UDT")
+                .clone(),


Shouldn't you use helper function you added: get_udt_keyspace?

The base branch was changed.

PiotrJunior force-pushed the tuple-type branch from e1cc160 to 7a30313 Compare March 25, 2025 10:26

PiotrJunior force-pushed the udt branch from 8f33a0e to 73febf9 Compare March 25, 2025 10:29

PiotrJunior added this to the Iteration 2 milestone Mar 25, 2025

PiotrJunior requested review from adespawn, ZuzaOsa and wprzytula March 25, 2025 10:50

PiotrJunior force-pushed the tuple-type branch from 7a30313 to ec23954 Compare March 25, 2025 11:49

PiotrJunior force-pushed the tuple-type branch 3 times, most recently from 1810f26 to b396da9 Compare April 14, 2025 15:37

PiotrJunior force-pushed the udt branch 2 times, most recently from 49dcad6 to 02cd0aa Compare April 14, 2025 16:51

PiotrJunior force-pushed the tuple-type branch from 118e151 to ef0dd0d Compare April 14, 2025 21:56

PiotrJunior force-pushed the udt branch 3 times, most recently from b9092ef to 725fd64 Compare April 15, 2025 07:46

PiotrJunior marked this pull request as draft April 15, 2025 07:47

PiotrJunior marked this pull request as ready for review April 15, 2025 07:47

PiotrJunior force-pushed the tuple-type branch from 027a138 to 96c7e40 Compare April 15, 2025 07:53

PiotrJunior force-pushed the udt branch from 725fd64 to 4e42666 Compare April 15, 2025 07:54

PiotrJunior force-pushed the tuple-type branch 2 times, most recently from 85d6171 to d8ed718 Compare April 15, 2025 11:21

PiotrJunior force-pushed the udt branch from 4e42666 to 4e7f697 Compare April 15, 2025 11:28

wprzytula removed request for ZuzaOsa, adespawn and wprzytula April 16, 2025 08:19

wprzytula assigned PiotrJunior Apr 16, 2025

wprzytula requested changes May 25, 2025

View reviewed changes

adespawn added 2 commits May 29, 2025 12:50

Use raw bytes for UUID in results

e3df1fc

Change array and type representation

6268876

Removes CqlTypeClass, to reduce the overhead from it's type. This means, that array will be treated as ambiguous types.

PiotrJunior force-pushed the udt branch from b25f091 to 6b7b046 Compare June 3, 2025 07:39

PiotrJunior requested a review from wprzytula June 3, 2025 07:47

ZuzaOsa requested a review from Copilot June 3, 2025 10:46

Copilot AI reviewed Jun 3, 2025

View reviewed changes

Stapox35 reviewed Jun 3, 2025

View reviewed changes

Merge pull request #225 from scylladb-zpp-2024-javascript-driver/uuid…

f70fbd9

…-result UUID result

wprzytula reviewed Jun 4, 2025

View reviewed changes

src/types/type_wrappers.rs Outdated Show resolved Hide resolved

src/types/type_wrappers.rs Outdated Show resolved Hide resolved

src/types/type_wrappers.rs Outdated Show resolved Hide resolved

wprzytula requested changes Jun 4, 2025

View reviewed changes

wprzytula and others added 3 commits June 4, 2025 14:55

ci: fight CI flakiness by using pip --user

e1ae3d6

Installing packages with `pip` using the `--user` flag may be the way to avoid permission issues. Let's try this out.

Merge pull request #230 from scylladb-zpp-2024-javascript-driver/fix-…

080844f

…ci-ccm-permission-errors CI: attempt to fix ccm permission errors

UDT ComplexType

7962fa9

Expand ComplexType to include UDTs.

PiotrJunior force-pushed the udt branch 2 times, most recently from 97c4090 to 9b001dd Compare June 10, 2025 09:52

PiotrJunior added 7 commits June 10, 2025 12:44

UDT result

a2cb8d0

Adds support for converting udt responses to a JavaScript object.

Add UDT result tests

f472979

Add tests for converting CQL values to JavaScript values

UDT request

018fe93

Adds support for converting JS UDT type into Rust UDT type.

Add UDT request tests

920d8b6

Adds tests for converting JS UDT into Rust CQL values.

Enable UDT examples

ede476e

Enable integration tests for UDT and Tuple

c8d41f5

Add proper type IDs

12c394b

Add proper type IDs to the type wrappers. IDs match those in CQL protocol. This is will be useful for testing metadata.

PiotrJunior force-pushed the udt branch from 9b001dd to 12c394b Compare June 10, 2025 10:44

adespawn requested changes Jun 14, 2025

View reviewed changes

adespawn force-pushed the second-iteration branch 2 times, most recently from 16128e9 to 6300c7f Compare June 17, 2025 09:58

adespawn mentioned this pull request Jun 17, 2025

Invalid column types #238

Open

Base automatically changed from second-iteration to main June 25, 2025 12:49

		//
		// This will be fixed at a later time, as it requires more investigation into how UDT works.

	value[names[i]] !== undefined ? value[names[i]] : null, // Undefineds should be treated as nulls, so we convert them to nulls.
	value[names[i]] !== undefined ? value[names[i]] : null, // Undefined values should be treated as null value.

UDT #168

Are you sure you want to change the base?

UDT #168

Uh oh!

Conversation

PiotrJunior commented Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wprzytula commented Mar 25, 2025

Uh oh!

adespawn commented Apr 15, 2025

Uh oh!

wprzytula commented Apr 16, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

PiotrJunior commented Mar 25, 2025 •

edited

Loading