Skip to content

Commit 4559ca8

Browse files
Vladislav Gogovazevaykin
Vladislav Gogov
authored and
azevaykin
committed
Docs: Compression for Column Table (ydb-platform#11458)
1 parent f6f5176 commit 4559ca8

File tree

21 files changed

+270
-67
lines changed

21 files changed

+270
-67
lines changed

‎ydb/docs/en/core/_includes/codec_zstd_allow_for_olap_note.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{% note alert %}
2+
3+
{% include [codec_zstd_allow_for_olap_text](codec_zstd_allow_for_olap_text.md) %}
4+
5+
{% endnote %}

‎ydb/docs/en/core/_includes/codec_zstd_allow_for_olap_text.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Codec `"zstd"` is supported only for [column-oriented](../concepts/datamodel/table.md#column-oriented-tables) tables.

‎ydb/docs/en/core/_includes/only_allow_for_olap_note.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{% if oss == true and backend_name == "YDB" %}
2+
3+
{% note alert %}
4+
5+
{% include [only_allow_for_olap_text](only_allow_for_olap_text.md) %}
6+
7+
{% endnote %}
8+
9+
{% endif %}

‎ydb/docs/en/core/_includes/only_allow_for_olap_text.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Supported only for [column-oriented](../concepts/datamodel/table.md#column-oriented-tables) tables.

‎ydb/docs/en/core/_includes/only_allow_for_oltp_note.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{% if oss == true and backend_name == "YDB" %}
2+
3+
{% note alert %}
4+
5+
{% include [only_allow_for_oltp_text](only_allow_for_oltp_text.md) %}
6+
7+
{% endnote %}
8+
9+
{% endif %}

‎ydb/docs/en/core/_includes/only_allow_for_oltp_text.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Supported only for [row-oriented](../concepts/datamodel/table.md#row-oriented-tables) tables.

‎ydb/docs/en/core/concepts/datamodel/_includes/table.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -185,6 +185,8 @@ In most cases, working with {{ ydb-short-name }} column-oriented tables is simil
185185
+ Available in both the primary key and other columns: `Date`, `Datetime`, `Timestamp`, `Int32`, `Int64`, `Uint8`, `Uint16`, `Uint32`, `Uint64`, `Utf8`, `String`;
186186
+ Available only in columns not included in the primary key: `Bool`, `Decimal`, `Double`, `Float`, `Int8`, `Int16`, `Interval`, `JsonDocument`, `Json`, `Uuid`, `Yson`.
187187

188+
* Column-oriented tables support column groups, but only for compression settings.
189+
188190
Let's recreate the "article" table, this time in column-oriented format, using the following YQL command:
189191

190192
```yql

‎ydb/docs/en/core/yql/reference/yql-core/syntax/alter_table/family.md‎

Lines changed: 34 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,5 @@
11
# Changing column groups
22

3-
{% if oss == true and backend_name == "YDB" %}
4-
5-
{% include [OLAP_not_allow_note](../../../../_includes/not_allow_for_olap_note.md) %}
6-
7-
{% endif %}
8-
93
The mechanism of [column groups](../../../../concepts/datamodel/table.md#column-groups) allows for improved performance of partial row read operations by dividing the storage of table columns into several groups. The most commonly used scenario is to organize the storage of infrequently used attributes into a separate column group.
104

115
## Creating column groups {#creating-column-groups}
@@ -38,7 +32,18 @@ ALTER TABLE series_with_families
3832
ALTER COLUMN release_date SET FAMILY family_small;
3933
```
4034

41-
Using the `ALTER FAMILY` command, you can change the parameters of the column group. The code below changes the storage type to `hdd` for the `default` column group in the `series_with_families` table:
35+
Using the `ALTER FAMILY` command, you can change the parameters of the column group.
36+
37+
38+
### Changing storage type
39+
40+
{% if oss == true and backend_name == "YDB" %}
41+
42+
{% include [OLTP_only_allow_note](../../../../_includes/only_allow_for_oltp_note.md) %}
43+
44+
{% endif %}
45+
46+
The code below changes the storage type to `hdd` for the `default` column group in the `series_with_families` table:
4247

4348
```yql
4449
ALTER TABLE series_with_families ALTER FAMILY default SET DATA "hdd";
@@ -50,4 +55,26 @@ Available types of storage devices depend on the {{ ydb-short-name }} cluster co
5055

5156
{% endnote %}
5257

58+
### Changing compression codec
59+
60+
The code below changes the compression codec to `lz4` for the `default` column group in the `series_with_families` table:
61+
62+
```yql
63+
ALTER TABLE series_with_families ALTER FAMILY default SET COMPRESSION "lz4";
64+
```
65+
66+
### Changing compression level of codec
67+
68+
{% if oss == true and backend_name == "YDB" %}
69+
70+
{% include [OLAP_only_allow_note](../../../../_includes/only_allow_for_olap_note.md) %}
71+
72+
{% endif %}
73+
74+
The code below changes the compression level of codec if it supports different compression levels for the `default` column group in the `series_with_families` table:
75+
76+
```yql
77+
ALTER TABLE series_with_families ALTER FAMILY default SET COMPRESSION_LEVEL 5;
78+
```
79+
5380
You can specify any parameters of a group of columns from the [`CREATE TABLE`](../create_table/index.md) command.

‎ydb/docs/en/core/yql/reference/yql-core/syntax/create_table/family.md‎

Lines changed: 63 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,39 +1,80 @@
11
# Column groups
22

3+
Columns of the same table can be grouped to set the following parameters:
4+
5+
* `DATA`: A storage device type for the data in this column group. Acceptable values: `ssd`, `rot`.
6+
37
{% if oss == true and backend_name == "YDB" %}
48

5-
{% include [not_allow_for_olap](../../../../_includes/not_allow_for_olap_note.md) %}
9+
{% include [OLTP_only_allow_note](../../../../_includes/only_allow_for_oltp_note.md) %}
610

711
{% endif %}
812

9-
Columns of the same table can be grouped to set the following parameters:
13+
* `COMPRESSION`: A data compression codec. Acceptable values: `off`, `lz4`, `zstd`.
1014

11-
* `DATA`: A storage device type for the data in this column group. Acceptable values: `ssd`, `rot`.
12-
* `COMPRESSION`: A data compression codec. Acceptable values: `off`, `lz4`.
15+
{% if oss == true and backend_name == "YDB" %}
16+
17+
{% include [codec_zstd_allow_for_olap_note](../../../../_includes/codec_zstd_allow_for_olap_note.md) %}
1318

14-
By default, all columns are in the same group named `default`. If necessary, the parameters of this group can also be redefined.
19+
{% endif %}
20+
21+
* `COMPRESSION_LEVEL` — compression level of codec if it supports different compression levels.
22+
23+
{% if oss == true and backend_name == "YDB" %}
24+
25+
{% include [OLAP_only_allow_note](../../../../_includes/only_allow_for_olap_note.md) %}
26+
27+
{% endif %}
28+
29+
By default, all columns are in the same group named `default`. If necessary, the parameters of this group can also be redefined, if they are not redefined, then predefined values are applied.
1530

1631
## Example
1732

1833
In the example below, for the created table, the `family_large` group of columns is added and set for the `series_info` column, and the parameters for the default group, which is set by `default` for all other columns, are also redefined.
1934

20-
```yql
21-
CREATE TABLE series_with_families (
22-
series_id Uint64,
23-
title Utf8,
24-
series_info Utf8 FAMILY family_large,
25-
release_date Uint64,
26-
PRIMARY KEY (series_id),
27-
FAMILY default (
28-
DATA = "ssd",
29-
COMPRESSION = "off"
30-
),
31-
FAMILY family_large (
32-
DATA = "rot",
33-
COMPRESSION = "lz4"
34-
)
35-
);
36-
```
35+
{% list tabs %}
36+
37+
- Creating a row-oriented table
38+
39+
```sql
40+
CREATE TABLE series_with_families (
41+
series_id Uint64,
42+
title Utf8,
43+
series_info Utf8 FAMILY family_large,
44+
release_date Uint64,
45+
PRIMARY KEY (series_id),
46+
FAMILY default (
47+
DATA = "ssd",
48+
COMPRESSION = "off"
49+
),
50+
FAMILY family_large (
51+
DATA = "rot",
52+
COMPRESSION = "lz4"
53+
)
54+
);
55+
```
56+
57+
- Creating a column-oriented table
58+
59+
```sql
60+
CREATE TABLE series_with_families (
61+
series_id Uint64,
62+
title Utf8,
63+
series_info Utf8 FAMILY family_large,
64+
release_date Uint64,
65+
PRIMARY KEY (series_id),
66+
FAMILY default (
67+
COMPRESSION = "lz4"
68+
),
69+
FAMILY family_large (
70+
COMPRESSION = "zstd",
71+
COMPRESSION_LEVEL = 5
72+
)
73+
)
74+
WITH (STORE = COLUMN);
75+
```
76+
77+
{% endlist %}
3778

3879
{% note info %}
3980

‎ydb/docs/en/core/yql/reference/yql-core/syntax/create_table/index.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -234,6 +234,9 @@ When creating row-oriented tables, it is possible to specify:
234234
* [Column groups](family.md).
235235
* [Additional parameters](with.md).
236236

237-
For column-oriented tables, only [additional parameters](with.md) can be specified during creation.
237+
When creating column-oriented tables, it is possible to specify:
238+
239+
* [Column groups](family.md).
240+
* [Additional parameters](with.md).
238241

239242
{% endif %}

0 commit comments

Comments
 (0)