Skip to content

Commit

Permalink
sql: Add unsigned int types (MaterializeInc#14304)
Browse files Browse the repository at this point in the history
This commit adds 16-bit, 32-bit, and 64-bit unsigned integer types to
Materialize. In addition it also adds various casts to and from the
unsigned int types.

The binary format of these types is big endian, which is the same as
every other numeric type. These types have separate OIDs from the
signed integer types so psql and other drivers don't confuse these for
signed integers.

No arithmetic operators are added for unsigned integers in this commit.
Those can be added in a follow up commit.

Works towards resolving #7629
  • Loading branch information
jkosh44 authored Aug 24, 2022
1 parent 4510ba7 commit a8131a5
Show file tree
Hide file tree
Showing 40 changed files with 2,451 additions and 92 deletions.
206 changes: 128 additions & 78 deletions doc/user/content/sql/functions/cast.md

Large diffs are not rendered by default.

6 changes: 5 additions & 1 deletion doc/user/content/sql/types/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Materialize's type system consists of two classes of types:
## Built-in types

Type | Aliases | Use | Size (bytes) | Catalog name | Syntax
-----|---------|-----|--------------|----------------|-----
-----|-------|-----|--------------|----------------|-----
[`bigint`](integer) | `int8` | Large signed integer | 8 | Named | `123`
[`boolean`](boolean) | `bool` | State of `TRUE` or `FALSE` | 1 | Named | `TRUE`, `FALSE`
[`bytea`](bytea) | `bytea` | Unicode string | Variable | Named | `'\xDEADBEEF'` or `'\\000'`
Expand All @@ -33,8 +33,12 @@ Type | Aliases | Use | Size (bytes) | Catalog name | Syntax
[`oid`](oid) | | PostgreSQL object identifier | 4 | Named | `123`
[`real`](float) | `float4` | Single precision floating-point number | 4 | Named | `1.23`
[`record`](record) | | Tuple with arbitrary contents | Variable | Unnameable | `ROW($expr, ...)`
[`smallint`](integer) | `int2` | Small signed integer | 2 | Named | `123`
[`text`](text) | `string` | Unicode string | Variable | Named | `'foo'`
[`time`](time) | | Time without date | 4 | Named | `TIME '01:23:45'`
[`uint2`](uint) | | Small unsigned integer | 2 | Named | `123`
[`uint4`](uint) | `uint` | Unsigned integer | 4 | Named | `123`
[`uint8`](uint) | | Large unsigned integer | 8 | Named | `123`
[`timestamp`](timestamp) | | Date and time | 8 | Named | `TIMESTAMP '2007-02-01 15:04:05'`
[`timestamp with time zone`](timestamp) | `timestamp with time zone` | Date and time with timezone | 8 | Named | `TIMESTAMPTZ '2007-02-01 15:04:05+06'`
[Arrays](array) (`[]`) | | Multidimensional array | Variable | Named | `ARRAY[...]`
Expand Down
2 changes: 2 additions & 0 deletions doc/user/content/sql/types/integer.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ To | Required context
[`oid`](../oid) | Implicit
[`real`/`double precision`](../float) | Implicit
[`text`](../text) | Assignment
[`uint2`/`uint4`/`uint8`](../uint) | Depends on specific cast

#### To `integer` or `bigint`

Expand All @@ -85,6 +86,7 @@ From | Required context
[`numeric`](../numeric) | Assignment
[`real`/`double precision`](../float) | Assignment
[`text`](../text) | Explicit
[`uint2`/`uint4`/`uint8`](../uint) | Depends on specific cast

## Examples

Expand Down
103 changes: 103 additions & 0 deletions doc/user/content/sql/types/uint.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
---
title: "Unsigned Integer types"
description: "Express unsigned integers"
menu:
main:
parent: 'sql-types'
aliases:
- /sql/types/uint2
- /sql/types/uint4
- /sql/types/uint8
---

## `uint2` info

Detail | Info
-------|------
**Size** | 2 bytes
**Catalog name** | `mz_catalog.uint2`
**OID** | 16,460
**Range** | [0, 65,535]

## `uint4` info

Detail | Info
-------|------
**Size** | 4 bytes
**Catalog name** | `mz_catalog.uint4`
**OID** | 16,462
**Range** | [0, 4,294,967,295]

## `uint8` info

Detail | Info
-------|------
**Size** | 8 bytes
**Catalog name** | `mz_catalog.uint8`
**OID** | 14,464
**Range** | [0, 18,446,744,073,709,551,615]

## Details

### Valid casts

For details about casting, including contexts, see [Functions:
Cast](../../functions/cast).

#### Between unsigned integer types

From | To | Required context
--------|---------|--------
`uint2` | `uint4` | Implicit
`uint2` | `uint8` | Implicit
`uint4` | `uint2` | Assignment
`uint4` | `uint8` | Implicit
`uint8` | `uint2` | Assignment
`uint8` | `uint4` | Assignment

#### From unsigned integer types

You can cast unsigned integer types to:

To | Required context
---|--------
[`numeric`](../numeric) | Implicit
[`real`/`double precision`](../float) | Implicit
[`text`](../text) | Assignment
[`smallint`/`integer`/`bigint`](../integer) | Depends on specific cast

#### To `uint4` or `uint8`

You can cast the following types to unsigned integer types:

From | Required context
---|--------
[`boolean`](../boolean) (`integer` only) | Explicit
[`jsonb`](../jsonb) | Explicit
[`oid`](../oid) (`integer` and `bigint` only) | Assignment
[`numeric`](../numeric) | Assignment
[`real`/`double precision`](../float) | Assignment
[`text`](../text) | Explicit
[`smallint`/`integer`/`bigint`](../integer) | Depends on specific cast

## Examples

```sql
SELECT 123::uint4 AS int_v;
```
```nofmt
int_v
-------
123
```

<hr/>

```sql
SELECT 1.23::uint4 AS int_v;
```
```nofmt
int_v
-------
1
```
6 changes: 6 additions & 0 deletions src/adapter/src/catalog.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2271,6 +2271,9 @@ impl<S: Append> Catalog<S> {
CatalogType::Int16 => CatalogType::Int16,
CatalogType::Int32 => CatalogType::Int32,
CatalogType::Int64 => CatalogType::Int64,
CatalogType::UInt16 => CatalogType::UInt16,
CatalogType::UInt32 => CatalogType::UInt32,
CatalogType::UInt64 => CatalogType::UInt64,
CatalogType::Interval => CatalogType::Interval,
CatalogType::Jsonb => CatalogType::Jsonb,
CatalogType::Numeric => CatalogType::Numeric,
Expand Down Expand Up @@ -4553,6 +4556,9 @@ impl ExprHumanizer for ConnCatalog<'_> {
.join(",")
),
PgLegacyChar => "\"char\"".into(),
UInt16 => "uint2".into(),
UInt32 => "uint4".into(),
UInt64 => "uint8".into(),
ty => {
let pgrepr_type = mz_pgrepr::Type::from(ty);
let pg_catalog_schema =
Expand Down
72 changes: 72 additions & 0 deletions src/adapter/src/catalog/builtin.rs
Original file line number Diff line number Diff line change
Expand Up @@ -837,6 +837,72 @@ pub const TYPE_ANYCOMPATIBLEMAP: BuiltinType<NameReference> = BuiltinType {
},
};

pub const TYPE_UINT2: BuiltinType<NameReference> = BuiltinType {
name: "uint2",
schema: MZ_CATALOG_SCHEMA,
oid: mz_pgrepr::oid::TYPE_UINT2_OID,
details: CatalogTypeDetails {
typ: CatalogType::UInt16,
array_id: None,
},
};

pub const TYPE_UINT2_ARRAY: BuiltinType<NameReference> = BuiltinType {
name: "_uint2",
schema: MZ_CATALOG_SCHEMA,
oid: mz_pgrepr::oid::TYPE_UINT2_ARRAY_OID,
details: CatalogTypeDetails {
typ: CatalogType::Array {
element_reference: TYPE_UINT2.name,
},
array_id: None,
},
};

pub const TYPE_UINT4: BuiltinType<NameReference> = BuiltinType {
name: "uint4",
schema: MZ_CATALOG_SCHEMA,
oid: mz_pgrepr::oid::TYPE_UINT4_OID,
details: CatalogTypeDetails {
typ: CatalogType::UInt32,
array_id: None,
},
};

pub const TYPE_UINT4_ARRAY: BuiltinType<NameReference> = BuiltinType {
name: "_uint4",
schema: MZ_CATALOG_SCHEMA,
oid: mz_pgrepr::oid::TYPE_UINT4_ARRAY_OID,
details: CatalogTypeDetails {
typ: CatalogType::Array {
element_reference: TYPE_UINT4.name,
},
array_id: None,
},
};

pub const TYPE_UINT8: BuiltinType<NameReference> = BuiltinType {
name: "uint8",
schema: MZ_CATALOG_SCHEMA,
oid: mz_pgrepr::oid::TYPE_UINT8_OID,
details: CatalogTypeDetails {
typ: CatalogType::UInt64,
array_id: None,
},
};

pub const TYPE_UINT8_ARRAY: BuiltinType<NameReference> = BuiltinType {
name: "_uint8",
schema: MZ_CATALOG_SCHEMA,
oid: mz_pgrepr::oid::TYPE_UINT8_ARRAY_OID,
details: CatalogTypeDetails {
typ: CatalogType::Array {
element_reference: TYPE_UINT8.name,
},
array_id: None,
},
};

pub const MZ_DATAFLOW_OPERATORS: BuiltinLog = BuiltinLog {
name: "mz_dataflow_operators",
schema: MZ_CATALOG_SCHEMA,
Expand Down Expand Up @@ -2146,6 +2212,12 @@ pub static BUILTINS_STATIC: Lazy<Vec<Builtin<NameReference>>> = Lazy::new(|| {
Builtin::Type(&TYPE_ANYCOMPATIBLENONARRAY),
Builtin::Type(&TYPE_ANYCOMPATIBLELIST),
Builtin::Type(&TYPE_ANYCOMPATIBLEMAP),
Builtin::Type(&TYPE_UINT2),
Builtin::Type(&TYPE_UINT2_ARRAY),
Builtin::Type(&TYPE_UINT4),
Builtin::Type(&TYPE_UINT4_ARRAY),
Builtin::Type(&TYPE_UINT8),
Builtin::Type(&TYPE_UINT8_ARRAY),
];
for (schema, funcs) in &[
(PG_CATALOG_SCHEMA, &*mz_sql::func::PG_CATALOG_BUILTINS),
Expand Down
3 changes: 3 additions & 0 deletions src/adapter/src/coord/timestamp_selection.rs
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,9 @@ impl<S: Append + 'static> Coordinator<S> {
ScalarType::Int16 => evaled.unwrap_int16().try_into()?,
ScalarType::Int32 => evaled.unwrap_int32().try_into()?,
ScalarType::Int64 => evaled.unwrap_int64().try_into()?,
ScalarType::UInt16 => evaled.unwrap_uint16().into(),
ScalarType::UInt32 => evaled.unwrap_uint32().into(),
ScalarType::UInt64 => evaled.unwrap_uint64(),
ScalarType::TimestampTz => {
evaled.unwrap_timestamptz().timestamp_millis().try_into()?
}
Expand Down
48 changes: 48 additions & 0 deletions src/expr/src/scalar.proto
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,51 @@ message ProtoUnaryFunc {
google.protobuf.Empty trunc_float32 = 216;
google.protobuf.Empty trunc_float64 = 217;
google.protobuf.Empty trunc_numeric = 218;
google.protobuf.Empty cast_int16_to_uint16 = 219;
google.protobuf.Empty cast_int16_to_uint32 = 220;
google.protobuf.Empty cast_int16_to_uint64 = 221;
google.protobuf.Empty cast_int32_to_uint16 = 222;
google.protobuf.Empty cast_int32_to_uint32 = 223;
google.protobuf.Empty cast_int32_to_uint64 = 224;
google.protobuf.Empty cast_int64_to_uint16 = 225;
google.protobuf.Empty cast_int64_to_uint32 = 226;
google.protobuf.Empty cast_int64_to_uint64 = 227;
google.protobuf.Empty cast_numeric_to_uint16 = 228;
google.protobuf.Empty cast_numeric_to_uint32 = 229;
google.protobuf.Empty cast_numeric_to_uint64 = 230;
google.protobuf.Empty cast_float32_to_uint16 = 231;
google.protobuf.Empty cast_float32_to_uint32 = 232;
google.protobuf.Empty cast_float32_to_uint64 = 233;
google.protobuf.Empty cast_float64_to_uint16 = 234;
google.protobuf.Empty cast_float64_to_uint32 = 235;
google.protobuf.Empty cast_float64_to_uint64 = 236;
google.protobuf.Empty cast_string_to_uint16 = 237;
google.protobuf.Empty cast_string_to_uint32 = 238;
google.protobuf.Empty cast_string_to_uint64 = 239;
google.protobuf.Empty cast_uint16_to_float32 = 240;
google.protobuf.Empty cast_uint16_to_float64 = 241;
google.protobuf.Empty cast_uint16_to_int32 = 242;
google.protobuf.Empty cast_uint16_to_uint32 = 243;
google.protobuf.Empty cast_uint16_to_int64 = 244;
google.protobuf.Empty cast_uint16_to_uint64 = 245;
google.protobuf.Empty cast_uint16_to_string = 246;
mz_repr.adt.numeric.ProtoOptionalNumericMaxScale cast_uint16_to_numeric = 247;
google.protobuf.Empty cast_uint32_to_float32 = 248;
google.protobuf.Empty cast_uint32_to_float64 = 249;
google.protobuf.Empty cast_uint32_to_uint16 = 250;
google.protobuf.Empty cast_uint32_to_int32 = 251;
google.protobuf.Empty cast_uint32_to_int64 = 252;
google.protobuf.Empty cast_uint32_to_uint64 = 253;
google.protobuf.Empty cast_uint32_to_string = 254;
mz_repr.adt.numeric.ProtoOptionalNumericMaxScale cast_uint32_to_numeric = 255;
google.protobuf.Empty cast_uint64_to_float32 = 256;
google.protobuf.Empty cast_uint64_to_float64 = 257;
google.protobuf.Empty cast_uint64_to_uint16 = 258;
google.protobuf.Empty cast_uint64_to_int32 = 259;
google.protobuf.Empty cast_uint64_to_uint32 = 260;
google.protobuf.Empty cast_uint64_to_int64 = 261;
google.protobuf.Empty cast_uint64_to_string = 262;
mz_repr.adt.numeric.ProtoOptionalNumericMaxScale cast_uint64_to_numeric = 263;
}
}

Expand Down Expand Up @@ -621,5 +666,8 @@ message ProtoEvalError {
ProtoIncompatibleArrayDimensions incompatible_array_dimensions = 52;
string type_from_oid = 53;
ProtoIndexOutOfRange index_out_of_range = 54;
google.protobuf.Empty uint16_out_of_range = 55;
google.protobuf.Empty uint32_out_of_range = 56;
google.protobuf.Empty uint64_out_of_range = 57;
}
}
Loading

0 comments on commit a8131a5

Please sign in to comment.