Skip to content

Docs: update description of serial #19443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ The `tools dump` command dumps the schema objects to the client file system in t
- `database`: A fully consistent dump, with one snapshot taken before starting the dump. Applied by default.
- `table`: Consistency within each dumped table, taking individual independent snapshots for each table. Might run faster and have less impact on the current workload processing in the database.

- `--avoid-copy`: Do not create a snapshot before dumping. The default consistency snapshot might be inapplicable in some cases (for example, for tables with external blobs).
- `--avoid-copy`: Do not create a snapshot before dumping. The default consistency snapshot might be inapplicable in some cases (for example, for tables with external blobs).{% if feature_serial %} For correct export of tables with [serial](../../../../yql/reference/types/serial.md) types, this parameter must not be set. Otherwise, the current value of the sequence generator will not be copied, and new values will start from the initial value, which may lead to primary key conflicts.{% endif %}

- `--save-partial-result`: Retain the result of a partial dump. Without this option, dumps that terminate with an error are deleted.

Expand Down
60 changes: 60 additions & 0 deletions ydb/docs/en/core/yql/reference/syntax/alter-sequence.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# ALTER SEQUENCE

Modifies the parameters of an existing `Sequence` object associated with a [Serial](../types/serial.md) column.

## Syntax

```yql
ALTER SEQUENCE [ IF EXISTS ] path_to_sequence
[ INCREMENT [ BY ] increment ]
[ START [ WITH ] start_value ]
[ RESTART [ [ WITH ] restart_value ]];
```

## Parameters

* `path_to_sequence` — the absolute path to the sequence object.

The path is constructed as `<path_to_table>/_serial_column_<column_name>`,
where `<path_to_table>` is the absolute path to the table, and `<column_name>` is the name of the column with the `Serial` data type.
For example, for the column `user_id` in the table `/local/users`, the corresponding `Sequence` path will be `/local/users/_serial_column_user_id`.

* `IF EXISTS` — when used, the statement does not return an error if the sequence does not exist at the specified path.

* `INCREMENT [ BY ] increment` — sets the increment step for the sequence. Default: 1.

* `START [ WITH ] start_value` — sets a new start value for the sequence. Changing this parameter with `ALTER SEQUENCE` does not affect the current value; the new start value is used with `ALTER SEQUENCE RESTART` if no value is specified. Default: 1.

* `RESTART [ [ WITH ] restart_value ]` — sets the current sequence value to the specified `restart_value`. If no value is specified, it sets the current value to the start value.

## Examples

```yql
CREATE TABLE users (
user_hash Uint64,
user_id Serial,
name Utf8,
email Utf8,
PRIMARY KEY (user_hash, user_id)
);
```

Change the increment step for `user_id` and set the current value to 1000:

```yql
ALTER SEQUENCE `/Root/users/_serial_column_user_id`
INCREMENT BY 5
RESTART 1000;
```

An alternative way to change the current value is to first set a new start value, and then `RESTART` the `Sequence`. After this, subsequent calls to `RESTART` without an explicit value will set the current value to 1000:

```yql
ALTER SEQUENCE `/Root/users/_serial_column_user_id` INCREMENT BY 5 START WITH 1000;
ALTER SEQUENCE `/Root/users/_serial_column_user_id` RESTART;
```

## See also

* [{#T}](create-table.md)
* [{#T}](../types/serial.md)
4 changes: 2 additions & 2 deletions ydb/docs/en/core/yql/reference/syntax/create_table/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,11 +95,11 @@ By default, if the `STORE` parameter is not specified, a row-oriented table is c

{% if feature_column_container_type == true %}

For non-key columns, any data types are allowed, whereas for key columns only [primitive](../../types/primitive.md) types are permitted. When specifying complex types (for example, List<String>), the type should be enclosed in double quotes.
For non-key columns, any data types are allowed, whereas for key columns only [primitive](../../types/primitive.md){% if feature_serial %} and [serial](../../types/serial.md){% endif %} types are permitted. When specifying complex types (for example, List<String>), the type should be enclosed in double quotes.

{% else %}

For both key and non-key columns, only [primitive](../../types/primitive.md) data types are allowed.
For both key and non-key columns, only [primitive](../../types/primitive.md){% if feature_serial %} and [serial](../../types/serial.md){% endif %} data types are allowed.

{% endif %}

Expand Down
6 changes: 6 additions & 0 deletions ydb/docs/en/core/yql/reference/syntax/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,3 +97,9 @@

{% endif %}

{% if feature_serial %}

* [ALTER SEQUENCE](alter-sequence.md)

{% endif %}

1 change: 1 addition & 0 deletions ydb/docs/en/core/yql/reference/syntax/toc_i.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ items:
- { name: ALTER VIEW, href: alter-view.md, when: feature_view }
- { name: ALTER TOPIC, href: alter-topic.md, when: feature_topic_control_plane }
- { name: ALTER USER, href: alter-user.md, when: feature_user_and_group }
- { name: ALTER SEQUENCE, href: alter-sequence.md, when: feature_serial }
- { name: ANALYZE, href: analyze.md, when: backend_name == "YDB" }
- { name: CREATE ASYNC REPLICATION, href: create-async-replication.md, when: feature_async_replication }
- { name: CREATE GROUP, href: create-group.md, when: feature_user_and_group }
Expand Down
1 change: 1 addition & 0 deletions ydb/docs/en/core/yql/reference/types/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
This section contains articles on YQL data types:

- [Simple/Primitive types](primitive.md)
{% if feature_serial %}- [Serial types](serial.md){% endif %}
- [Optional types](optional.md)
- [Containers](containers.md)
- [Special types](special.md)
Expand Down
95 changes: 95 additions & 0 deletions ydb/docs/en/core/yql/reference/types/serial.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Serial Types

Serial types are integer data types with an associated value-generation mechanism. They create auto-increment columns: each new row inserted into a table automatically generates a unique value for this column (similar to the [SERIAL](https://www.postgresql.org/docs/current/datatype-numeric.html#DATATYPE-SERIAL) type in PostgreSQL or the [AUTO_INCREMENT](https://dev.mysql.com/doc/refman/9.0/en/example-auto-increment.html) property in MySQL).

## Description
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This header doesn't bring any extra value to the reader. Instead, split the text to some meaningful sections, for example: automatic sequence creation, underlying data types, working with such columns after creation, etc.


When a column of a serial type is defined, a separate schema object called a `Sequence` is created and bound to this column. This object is a private sequence generator and it is hidden from the user. The `Sequence` will be destroyed together with the table.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add sequence to glossary.md, link to its definition on first usage.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, below is enough content to make a concepts/ article about sequences. Let's structure it similarly to coordination nodes (as another uncommon schema object), for example:

  1. Brief description in glossary.md like https://ydb.tech/docs/en/concepts/glossary#coordination-node
  2. A separate article explaining parameters, use cases, etc., like https://ydb.tech/docs/en/concepts/datamodel/coordination-node

Then, a large chunk of this goes away, and you can just say that a sequence (link) is automatically created for each table column of the Serial data type.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "hidden from the user" part doesn't seem to be true, given we now have the ALTER SEQUENCE query that's supposed to interact with them.


The `Sequence` object supports several parameters that determine its behavior. These parameters can be altered after creation using the [ALTER SEQUENCE](../syntax/alter-sequence.md) command.

By default, values generated by the `Sequence` start from one, are incremented by one with each new value, and are limited according to the chosen type.

{% note info %}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we move away sequence-related content to a separate article, this is roughly when the serial-related content would start; thus, there is no need for it to be in a note block.


Serial columns are supported both for columns included in the primary key and for non-key columns.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to double-check: is this true for both row-oriented and column-oriented tables? If not, this needs to be clarified.


However, such columns cannot be [altered](../syntax/alter_table/family#mod-column-groups) or [dropped](../syntax/alter_table/columns.md) from the table — attempting to perform these operations will result in an error.

{% endnote %}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing header and explanation for the table

| Type | Maximum Value | Underlying Type |
|-------------|----------------------|----------|
| `SmallSerial` | $2^15–1$ | `Int16` |
| `Serial2` | $2^15–1$ | `Int16` |
| `Serial` | $2^31–1$ | `Int32` |
| `Serial4` | $2^31–1$ | `Int32` |
| `Serial8` | $2^63–1$ | `Int64` |
| `BigSerial` | $2^63–1$ | `Int64` |

If a sequence reaches its maximum value, insertion results in an error:

```text
Error: Failed to get next val for sequence: /dev/test/users/_serial_column_user_id, status: SCHEME_ERROR
<main>: Error: sequence [OwnerId: <some>, LocalPathId: <some>] doesn't have any more values available, code: 200503
```

{% note info %}

The next value is allocated by the generator before the actual insertion into the table and is considered used even if the row is not successfully inserted (for example, in case of transaction rollback).
As a result, the values in such a column may have gaps and may not form a continuous sequence.

{% endnote %}

Tables with `Serial` columns support [copy](../../../reference/ydb-cli/tools-copy.md), [rename](../../../reference/ydb-cli/commands/tools/rename.md), [dump](../../../reference/ydb-cli/export-import/tools-dump.md), [restore](../../../reference/ydb-cli/export-import/import-file.md), and [import](../../../reference/ydb-cli/export-import/import-s3.md)/[export](../../../reference/ydb-cli/export-import/export-s3.md) CLI operations.

## Usage Example

Carefully choose the columns for your [PRIMARY KEY](../../../dev/primary-key/row-oriented.md). To ensure scalability and high performance, avoid using monotonically increasing primary keys; this pattern directs all inserts to the last partition, concentrating load on a single server.

Instead, use a hash of the entire primary key (or a portion of it) as the first key element to distribute data evenly across cluster partitions.

```yql
CREATE TABLE users (
user_hash Uint64,
user_id Serial,
name Utf8,
email Utf8,
PRIMARY KEY (user_hash, user_id)
);
```

The `user_hash` field can be calculated on the application side, for example, by applying a hash function to the `email` column.

``` yql
UPSERT INTO users (user_hash, name, email) VALUES (123456789, 'Alice', 'alice@example.com');
INSERT INTO users (user_hash, name, email) VALUES (987654321, 'Bob', 'bob@example.com');
REPLACE INTO users (user_hash, name, email) VALUES (111111111, 'John', 'john@example.com');
```

Result (example `user_hash` values are used):

| user_hash | email | name | user_id |
|-------------|-----------------------|-------|---------|
| 123456789 | `alice@example.com` | Alice | 1 |
| 987654321 | `bob@example.com` | Bob | 2 |
| 111111111 | `john@example.com` | John | 3 |

You can also explicitly specify a value for a column of type `Serial` during insertion, for example, when restoring data. In this case, the insertion behaves like a regular integer column, and the sequence remains unaffected:

``` yql
UPSERT INTO users (user_hash, user_id, name, email) VALUES (222222222, 10, 'Peter', 'peter@example.com');
```

### Suboptimal Schema Example

```yql
CREATE TABLE users_bad (
user_id Serial,
name Utf8,
email Utf8,
PRIMARY KEY (user_id)
);
```

In this example, the auto-increment column is the sole component of the primary key, resulting in an uneven load and creating a bottleneck on the last partition.
3 changes: 3 additions & 0 deletions ydb/docs/en/core/yql/reference/types/toc_i.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ items:
href: index.md
- name: Simple
href: primitive.md
- name: Serial
href: serial.md
when: feature_serial
- name: Optional
href: optional.md
- name: Containers
Expand Down
51 changes: 51 additions & 0 deletions ydb/docs/ru/core/yql/reference/syntax/alter-sequence.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# ALTER SEQUENCE

Изменяет параметры уже существующего объекта `Sequence`, привязанного к колонке [Serial](../types/serial.md) типа.

## Синтаксис

```yql
ALTER SEQUENCE [ IF EXISTS ] path_to_sequence
[ INCREMENT [ BY ] increment ]
[ START [ WITH ] start_value ]
[ RESTART [ [ WITH ] restart_value ]];
```

## Параметры

* `path_to_sequence` - абсолютный путь до объекта `Sequence`.

Путь формируется как `<path_to_table>/_serial_column_{column_name}`,
где `<path_to_table>` — абсолютный путь до таблицы, a `{column_name}` — имя колонки типа `Serial`.
Например, для таблицы с путём `/local/users` и колонки `user_id` путь к соответствующему `Sequence` будет `/local/users/_serial_column_user_id`.
* `IF EXISTS` - при использовании этой конструкции, выражение не возвращает ошибку, если не существует `Sequence` по указаному пути.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* `IF EXISTS` - при использовании этой конструкции, выражение не возвращает ошибку, если не существует `Sequence` по указаному пути.
* `IF EXISTS` - при использовании этой конструкции, вызов не возвращает ошибку, если не существует `Sequence` по указаному пути.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://ydb.tech/docs/ru/yql/reference/syntax/drop-view?version=main тут тоже написано про выражение

* `INCREMENT [ BY ] increment` - задает шаг изменения последовательности. Значение по умолчанию: 1.
* `START [ WITH ] start_value` - устанавливает новое стартовое значение для последовательности. Изменение этого параметра через `ALTER SEQUENCE` не влияет на текущее значение последовательности, но будет использовано, если выполнить `ALTER SEQUENCE RESTART` без указания значения. Значение по умолчанию: 1.
* `RESTART [ [ WITH ] restart_value ]` - изменяет текущее значение последовательности на указанное в `restart_value`. Если значение не указано, текущее значение последовательности будет установлено в текущее стартовое значение.

## Примеры

``` yql
CREATE TABLE users (
user_hash Uint64,
user_id Serial,
name Utf8,
email Utf8,
PRIMARY KEY (user_hash, user_id)
);
```

Изменить шаг последовательности для `user_id` и установить текущее значение равным 1000:

```yql
ALTER SEQUENCE `/Root/users/_serial_column_user_id`
INCREMENT BY 5
RESTART 1000;
```

Альтернативный способ изменить текущее значение — сначала изменить стартовое значение, а затем выполнить `RESTART` (после этого последующие сбросы через `RESTART` без указания `restart_value` будут устанавливать текущее значение в 1000):

```yql
ALTER SEQUENCE `/Root/users/_serial_column_user_id` INCREMENT BY 5 START WITH 1000;
ALTER SEQUENCE `/Root/users/_serial_column_user_id` RESTART;
```
12 changes: 2 additions & 10 deletions ydb/docs/ru/core/yql/reference/syntax/create_table/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,19 +96,11 @@ WITH (

{% if feature_column_container_type == true %}

Для неключевых колонок допускаются любые типы данных{% if feature_serial %} , кроме [серийных](../../types/serial.md) {% endif %}, для ключевых - только [примитивные](../../types/primitive.md){% if feature_serial %} и [серийные](../../types/serial.md){% endif %}. При указании сложных типов (например, `List<String>`) тип заключается в двойные кавычки.
Для неключевых колонок допускаются любые типы данных, для ключевых - только [примитивные](../../types/primitive.md){% if feature_serial %} и [серийные](../../types/serial.md){% endif %}. При указании сложных типов (например, `List<String>`) тип заключается в двойные кавычки.

{% else %}

{% if feature_serial %}

Для ключевых колонок допускаются только [примитивные](../../types/primitive.md) и [серийные](../../types/serial.md) типы данных, для неключевых колонок допускаются только [примитивные](../../types/primitive.md).

{% else %}

Для ключевых и неключевых колонок допускаются только [примитивные](../../types/primitive.md) типы данных.

{% endif %}
Для ключевых и неключевых колонок допускаются только [примитивные](../../types/primitive.md){% if feature_serial %} и [серийные](../../types/serial.md){% endif %} типы данных.

{% endif %}

Expand Down
6 changes: 6 additions & 0 deletions ydb/docs/ru/core/yql/reference/syntax/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,3 +104,9 @@
* [DROP ASYNC REPLICATION](drop-async-replication.md)

{% endif %}

{% if feature_serial %}

* [ALTER SEQUENCE](alter-sequence.md)

{% endif %}
1 change: 1 addition & 0 deletions ydb/docs/ru/core/yql/reference/syntax/toc_i.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ items:
- { name: ALTER RESOURCE POOL CLASSIFIER, href: alter-resource-pool-classifier.md, when: feature_resource_pool_classifier }
- { name: ALTER TOPIC, href: alter-topic.md, when: feature_topic_control_plane }
- { name: ALTER USER, href: alter-user.md, when: feature_user_and_group }
- { name: ALTER SEQUENCE, href: alter-sequence.md, when: feature_serial }
- { name: ANALYZE, href: analyze.md, when: backend_name == "YDB" }
- { name: CREATE ASYNC REPLICATION, href: create-async-replication.md, when: feature_async_replication }
- { name: CREATE GROUP, href: create-group.md, when: feature_user_and_group }
Expand Down
Loading
Loading