Skip to content

Commit 06a92a6

Browse files
authored
Merge ea2a849 into 0e5d23d
2 parents 0e5d23d + ea2a849 commit 06a92a6

File tree

31 files changed

+936
-582
lines changed

31 files changed

+936
-582
lines changed
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{% if oss == true and backend_name == "YDB" %}
2+
3+
{% note warning %}
4+
5+
{% include [OLTP_not_allow_text](not_allow_for_oltp_text.md) %}
6+
7+
{% endnote %}
8+
9+
{% endif %}
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Supported only for [column-oriented](../concepts/datamodel/table.md#column-oriented-tables) tables. Support for [row-oriented](../concepts/datamodel/table.md#row-oriented-tables) tables is currently under development.
Lines changed: 12 additions & 275 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,22 @@
1-
# Time to Live (TTL)
1+
# Time to Live (TTL) and eviction to external storage
22

3-
This section describes how the TTL mechanism works and what its limits are. It also gives examples of commands and code snippets that can be used to enable, configure, and disable TTL.
3+
This section describes how the TTL mechanism works and what its limits are.
44

55
## How it works {#how-it-works}
66

7-
{{ ydb-short-name }} allows you to specify a TTL column in both [row-oriented](../datamodel/table.md#row-oriented-tables) and [column-oriented](../datamodel/table.md#column-oriented-tables) tables. Values in TTL columns determine the table rows lifetime.
7+
The table's TTL is a sequence of storage tiers. Each tier contains an expression (TTL expression) and an action. When the expression is triggered, that tier is assigned to the row. When a tier is assigned to a row, the specified action is automatically performed: moving the row to external storage or deleting it. External storage is represented by the [external data source](../datamodel/external_data_source.md) object.
88

9-
{% note warning %}
10-
11-
An item with the `NULL` value in the TTL column is never deleted.
12-
13-
{% endnote %}
9+
{{ ydb-short-name }} allows you to specify a column (TTL column) whose values are used in TTL expressions. The expression is triggered when the specified number of seconds has passed since the time recorded in the TTL column. For rows with `NULL` value in TTL column, the expression is not triggered.
1410

1511
The timestamp for deleting a table item is determined by the formula:
1612

1713
```text
18-
expiration_time = valueof(ttl_column) + expire_after_seconds
14+
eviction_time = valueof(ttl_column) + evict_after_seconds
1915
```
2016

2117
{% note info %}
2218

23-
TTL doesn't guarantee that the item will be deleted exactly at `expiration_time`, it might happen later. If it's important to exclude logically obsolete but not yet physically deleted items from the selection, use query-level filtering.
19+
TTL doesn't guarantee that the item will be deleted exactly at `eviction_time`, it might happen later. If it's important to exclude logically obsolete but not yet physically deleted items from the selection, use query-level filtering.
2420

2521
{% endnote %}
2622

@@ -58,273 +54,14 @@ The *BRO* has the following properties:
5854
* Nanoseconds.
5955

6056
* You can't specify multiple TTL columns.
61-
* You can't delete the TTL column. However, if this is required, you should first [disable TTL](#disable) for the table.
57+
* You can't delete the TTL column. However, if this is required, you should first [disable TTL](../../recipes/yql/ttl.md#disable) for the table.
58+
* Only {{ objstorage-name }} is supported as external storage.
59+
* The delete action can only be specified for the last tier.
6260

6361
## Setup {#setting}
6462

6563
Currently, you can manage TTL settings using:
6664

67-
* [YQL](../../yql/reference/index.md).
68-
* [{{ ydb-short-name }} console client](../../reference/ydb-cli/index.md).
69-
* {{ ydb-short-name }} {% if oss %}C++, {% endif %}Go and Python [SDK](../../reference/ydb-sdk/index.md).
70-
71-
### Enabling TTL for an existing table {#enable-on-existent-table}
72-
73-
In the example below, the items of the `mytable` table will be deleted an hour after the time set in the `created_at` column:
74-
75-
{% list tabs group=tool %}
76-
77-
- YQL
78-
79-
```yql
80-
ALTER TABLE `mytable` SET (TTL = Interval("PT1H") ON created_at);
81-
```
82-
83-
- CLI
84-
85-
```bash
86-
$ {{ ydb-cli }} -e <endpoint> -d <database> table ttl set --column created_at --expire-after 3600 mytable
87-
```
88-
89-
{% if oss == true %}
90-
91-
- C++
92-
93-
```c++
94-
session.AlterTable(
95-
"mytable",
96-
TAlterTableSettings()
97-
.BeginAlterTtlSettings()
98-
.Set("created_at", TDuration::Hours(1))
99-
.EndAlterTtlSettings()
100-
);
101-
```
102-
103-
{% endif %}
104-
105-
- Go
106-
107-
```go
108-
err := session.AlterTable(ctx, "mytable",
109-
options.WithSetTimeToLiveSettings(
110-
options.NewTTLSettings().ColumnDateType("created_at").ExpireAfter(time.Hour),
111-
),
112-
)
113-
```
114-
115-
- Python
116-
117-
```python
118-
session.alter_table('mytable', set_ttl_settings=ydb.TtlSettings().with_date_type_column('created_at', 3600))
119-
```
120-
121-
{% endlist %}
122-
123-
{% note tip %}
124-
125-
When setting up TTL using YQL, an `Interval` is created from a string literal in [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) format with [some restrictions](../../yql/reference/builtins/basic#data-type-literals).
126-
127-
{% endnote %}
128-
129-
The example below shows how to use the `modified_at` column with a numeric type (`Uint32`) as a TTL column. The column value is interpreted as the number of seconds since the Unix epoch:
130-
131-
{% list tabs group=tool %}
132-
133-
- YQL
134-
135-
```yql
136-
ALTER TABLE `mytable` SET (TTL = Interval("PT1H") ON modified_at AS SECONDS);
137-
```
138-
139-
- CLI
140-
141-
```bash
142-
$ {{ ydb-cli }} -e <endpoint> -d <database> table ttl set --column modified_at --expire-after 3600 --unit seconds mytable
143-
```
144-
145-
{% if oss == true %}
146-
147-
- C++
148-
149-
```c++
150-
session.AlterTable(
151-
"mytable",
152-
TAlterTableSettings()
153-
.BeginAlterTtlSettings()
154-
.Set("modified_at", TTtlSettings::EUnit::Seconds, TDuration::Hours(1))
155-
.EndAlterTtlSettings()
156-
);
157-
```
158-
159-
{% endif %}
160-
161-
- Go
162-
163-
```go
164-
err := session.AlterTable(ctx, "mytable",
165-
options.WithSetTimeToLiveSettings(
166-
options.NewTTLSettings().ColumnSeconds("modified_at").ExpireAfter(time.Hour),
167-
),
168-
)
169-
```
170-
171-
- Python
172-
173-
```python
174-
session.alter_table('mytable', set_ttl_settings=ydb.TtlSettings().with_value_since_unix_epoch('modified_at', UNIT_SECONDS, 3600))
175-
```
176-
177-
{% endlist %}
178-
179-
### Enabling TTL for a newly created table {#enable-for-new-table}
180-
181-
For a newly created table, you can pass TTL settings along with the table description:
182-
183-
{% list tabs group=tool %}
184-
185-
- YQL
186-
187-
```yql
188-
CREATE TABLE `mytable` (
189-
id Uint64,
190-
expire_at Timestamp,
191-
PRIMARY KEY (id)
192-
) WITH (
193-
TTL = Interval("PT0S") ON expire_at
194-
);
195-
```
196-
197-
{% if oss == true %}
198-
199-
- C++
200-
201-
```c++
202-
session.CreateTable(
203-
"mytable",
204-
TTableBuilder()
205-
.AddNullableColumn("id", EPrimitiveType::Uint64)
206-
.AddNullableColumn("expire_at", EPrimitiveType::Timestamp)
207-
.SetPrimaryKeyColumn("id")
208-
.SetTtlSettings("expire_at")
209-
.Build()
210-
);
211-
```
212-
213-
{% endif %}
214-
215-
- Go
216-
217-
```go
218-
err := session.CreateTable(ctx, "mytable",
219-
options.WithColumn("id", types.Optional(types.TypeUint64)),
220-
options.WithColumn("expire_at", types.Optional(types.TypeTimestamp)),
221-
options.WithTimeToLiveSettings(
222-
options.NewTTLSettings().ColumnDateType("expire_at"),
223-
),
224-
)
225-
```
226-
227-
- Python
228-
229-
```python
230-
session.create_table(
231-
'mytable',
232-
ydb.TableDescription()
233-
.with_column(ydb.Column('id', ydb.OptionalType(ydb.DataType.Uint64)))
234-
.with_column(ydb.Column('expire_at', ydb.OptionalType(ydb.DataType.Timestamp)))
235-
.with_primary_key('id')
236-
.with_ttl(ydb.TtlSettings().with_date_type_column('expire_at'))
237-
)
238-
```
239-
240-
{% endlist %}
241-
242-
### Disabling TTL {#disable}
243-
244-
{% list tabs group=tool %}
245-
246-
- YQL
247-
248-
```yql
249-
ALTER TABLE `mytable` RESET (TTL);
250-
```
251-
252-
- CLI
253-
254-
```bash
255-
$ {{ ydb-cli }} -e <endpoint> -d <database> table ttl reset mytable
256-
```
257-
258-
{% if oss == true %}
259-
260-
- C++
261-
262-
```c++
263-
session.AlterTable(
264-
"mytable",
265-
TAlterTableSettings()
266-
.BeginAlterTtlSettings()
267-
.Drop()
268-
.EndAlterTtlSettings()
269-
);
270-
```
271-
272-
{% endif %}
273-
274-
- Go
275-
276-
```go
277-
err := session.AlterTable(ctx, "mytable",
278-
options.WithDropTimeToLive(),
279-
)
280-
```
281-
282-
- Python
283-
284-
```python
285-
session.alter_table('mytable', drop_ttl_settings=True)
286-
```
287-
288-
{% endlist %}
289-
290-
### Getting TTL settings {#describe}
291-
292-
The current TTL settings can be obtained from the table description:
293-
294-
{% list tabs group=tool %}
295-
296-
- CLI
297-
298-
```bash
299-
$ {{ ydb-cli }} -e <endpoint> -d <database> scheme describe mytable
300-
```
301-
302-
{% if oss == true %}
303-
304-
- C++
305-
306-
```c++
307-
auto desc = session.DescribeTable("mytable").GetValueSync().GetTableDescription();
308-
auto ttl = desc.GetTtlSettings();
309-
```
310-
311-
{% endif %}
312-
313-
- Go
314-
315-
```go
316-
desc, err := session.DescribeTable(ctx, "mytable")
317-
if err != nil {
318-
// process error
319-
}
320-
ttl := desc.TimeToLiveSettings
321-
```
322-
323-
- Python
324-
325-
```python
326-
desc = session.describe_table('mytable')
327-
ttl = desc.ttl_settings
328-
```
329-
330-
{% endlist %}
65+
* [YQL](../../recipes/yql/ttl.md).
66+
* [{{ ydb-short-name }} console client](../../recipes/ydb-cli/ttl.md).
67+
* {{ ydb-short-name }} {% if oss %}C++, {% endif %}Go and Python [SDK](../../recipes/ydb-sdk/ttl.md).

ydb/docs/en/core/concepts/datamodel/_includes/table.md

Lines changed: 2 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -134,16 +134,9 @@ If there are multiple followers, their delay from the leader may vary: although
134134

135135
| Parameter name | Type | Acceptable values | Update capability | Reset capability |
136136
| ------------- | --- | ------------------- | --------------------- | ------------------ |
137-
| `TTL` | Expression | `Interval("<literal>") ON <column> [AS <unit>]` | Yes | Yes |
137+
| `TTL` | Expression | `Interval("<literal>") ON <column> [AS <unit>]` or `Interval("literal1") action1, ..., Interval("literal1") action1 ON <column> [AS <unit>]` | Yes | Yes |
138138

139-
Where `<unit>` is a unit of measurement, specified only for column with a [numeric type](../../../concepts/ttl.md#restrictions):
140-
141-
* `SECONDS`
142-
* `MILLISECONDS`
143-
* `MICROSECONDS`
144-
* `NANOSECONDS`
145-
146-
For more information about deleting expired data, see [Time to Live (TTL)](../../../concepts/ttl.md).
139+
Syntax of TTL value is described in the article [{#T}](../../../yql/reference/syntax/create_table/with.md#time-to-live). For more information about deleting expired data, see [Time to Live (TTL)](../../../concepts/ttl.md).
147140

148141
### Renaming a table {#rename}
149142

ydb/docs/en/core/concepts/toc_i.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ items:
1919
- name: Change Data Capture (CDC)
2020
href: cdc.md
2121
when: feature_changefeed
22-
- name: Time to Live (TTL)
22+
- name: Time to live and eviction
2323
href: ttl.md
2424
- name: Scan queries
2525
href: scan_query.md

ydb/docs/en/core/recipes/ydb-cli/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ Table of contents:
66

77
* [{#T}](convert-table-type.md)
88
* [{#T}](benchmarks.md)
9+
* [{#T}](ttl.md)
910

1011
See also:
1112

ydb/docs/en/core/recipes/ydb-cli/toc_p.yaml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,6 @@ items:
22
- name: Convert table type
33
href: convert-table-type.md
44
- name: Benchmarks
5-
href: benchmarks.md
5+
href: benchmarks.md
6+
- name: Time to live (TTL)
7+
href: ttl.md

0 commit comments

Comments
 (0)