Skip to content

Commit 9232821

Browse files
author
DataUI VCS Robot
committed
Release 17.02.2022
1 parent 7004f38 commit 9232821

File tree

284 files changed

+12552
-2520
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

284 files changed

+12552
-2520
lines changed

‎en/_assets/data-transfer/lifecycle/copy-and-replication.svg

Lines changed: 44 additions & 0 deletions
Loading

‎en/_assets/data-transfer/lifecycle/copy.svg

Lines changed: 37 additions & 0 deletions
Loading

‎en/_assets/data-transfer/lifecycle/replication.svg

Lines changed: 37 additions & 0 deletions
Loading

‎en/_includes/before-begin.md

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
## Before you start
1+
## Before you start {#before-you-begin}
22

3-
Go to the Yandex.Cloud [management console]({{ link-console-main }}) and select the folder where you want to perform the operations. If there is no such folder, create one:
3+
Go to the {{ yandex-cloud }} [management console]({{ link-console-main }}) and select the folder where you want to perform the operations. If that folder doesn't exist, create it:
44

55
{% list tabs %}
66

@@ -14,32 +14,32 @@ Go to the Yandex.Cloud [management console]({{ link-console-main }}) and select
1414

1515
1. See a description of the create folder command:
1616

17-
```
18-
$ yc resource-manager folder create --help
17+
```bash
18+
yc resource-manager folder create --help
1919
```
2020

21-
2. Create a new folder:
21+
1. Create a new folder:
2222

2323
* with a name and without a description:
2424

25-
```
26-
$ yc resource-manager folder create \
27-
--name new-folder
25+
```bash
26+
yc resource-manager folder create \
27+
--name new-folder
2828
```
2929

3030
{% include [name-format](name-format.md) %}
3131

3232
* with a name and description:
3333

34-
```
35-
$ yc resource-manager folder create \
36-
--name new-folder \
37-
--description "my first folder with description"
34+
```bash
35+
yc resource-manager folder create \
36+
--name new-folder \
37+
--description "my first folder with description"
3838
```
3939

4040
- API
4141

42-
To create a folder, use the [create](../resource-manager/api-ref/Folder/create.md) method for the [Folder](../resource-manager/api-ref/Folder/index.md) resource of {{ resmgr-full-name }}.
42+
Use the [create](../resource-manager/api-ref/Folder/create.md) method for the [Folder](../resource-manager/api-ref/Folder/index.md) resource of the {{ resmgr-full-name }} service.
4343

4444
{% endlist %}
4545

‎en/_includes/create-folder.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
1. Click **Create folder** in the [Home page]({{ link-console-main }}) of the management console.
22

3-
2. Enter the folder name.
3+
1. Enter the folder name.
44

55
{% include [name-format](name-format.md) %}
66

7-
1. Select **Create a default network**. A [network](../vpc/concepts/network.md#network) is created with subnets in each availability zone. A [default security group](../vpc/concepts/security-groups.md#default-security-group) will also be created in this network, inside which all network traffic is allowed.
7+
1. Select **Create a default network**. A [network](../vpc/concepts/network.md#network) is created with subnets in each availability zone. A [default security group](../vpc/concepts/security-groups.md#default-security-group), where all network traffic is allowed inside, will also be created in this network.
88

99
1. Click **Create**.
1010

‎en/_includes/data-transfer/data-transfer-limits.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
#### Quotas {#dataproc-quotas}
2+
3+
| Limits | Value |
4+
| -------------------------------- | ----: |
5+
| Number of active transfers | 10 |
6+
7+
#### Limits {#dataproc-limits}
8+
9+
| Limits | Value |
10+
| -------------------------------- | ----: |
11+
| Number of active transfers | 1000 |
12+

‎en/_includes/data-transfer/delete-endpoint.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{% note warning %}
2+
3+
Before deleting an endpoint, [delete](../../data-transfer/operations/transfer.md#delete) all transfers that it is a part of.
4+
5+
{% endnote %}
6+
7+
To delete an endpoint:
8+
9+
{% list tabs %}
10+
11+
- Management console
12+
1. Go to the folder page and select **{{ data-transfer-name }}**.
13+
1. Select the endpoint to delete.
14+
1. Click **Delete** on the panel above.
15+
1. In the window that opens, click **Delete**.
16+
17+
{% endlist %}
18+

‎en/_includes/data-transfer/features-of-work.md

Lines changed: 0 additions & 2 deletions
This file was deleted.

‎en/_includes/data-transfer/fields/cleanup-policy-disabled-drop-truncate.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
**Cleanup policy**: Select a way to clean up data in the target database before the transfer:
2+
3+
* `Disabled`: Do not clean.
4+
5+
Select this option if only replication without copying data is performed.
6+
7+
* `Drop`: Fully delete tables included in the transfer (default).
8+
9+
Use this option so that the latest version of the table schema is always transferred to the target database from the source whenever the transfer is activated.
10+
11+
* `Truncate`: Delete only the data from the tables included in the transfer but leave the schema.
12+
13+
Use this option if the schema in the target database differs from the one that would have been transferred from the source during the transfer.
14+

‎en/_includes/data-transfer/fields/cleanup-policy-disabled-drop.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
**Cleanup policy**: Select a way to clean up data in the target database before the transfer:
2+
3+
* `Disabled`: Do not clean.
4+
5+
Select this option if only replication without copying data is performed.
6+
7+
* `Drop`: Fully delete tables included in the transfer (default).
8+
9+
Use this option so that the latest version of the table schema is always transferred to the target database from the source whenever the transfer is activated.
10+

‎en/_includes/data-transfer/fields/cluster-id.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
**MDB cluster ID**: Select the cluster to connect to.
2+

‎en/_includes/data-transfer/fields/database-name.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
**Database name**: Specify the name of the database in the selected cluster.
2+

‎en/_includes/data-transfer/fields/password.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
**Password**: Enter the user's password to the database.
2+

‎en/_includes/data-transfer/fields/subnet-id.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
**Subnet ID**: Select or [create](../../../vpc/operations/subnet-create.md) a subnet in the desired [availability zone](../../../overview/concepts/geo-scope.md).
2+
3+
If the source and target are geographically close, connecting over the selected subnet speeds up the transfer.
4+

‎en/_includes/data-transfer/fields/timezone.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
**DB time zone**: Specify the [IANA Time Zone Database](https://www.iana.org/time-zones) identifier. Defaults to UTC+0.
2+

‎en/_includes/data-transfer/fields/topic.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
**Topic**: Specify the name of the topic to connect to.
2+

‎en/_includes/data-transfer/fields/username.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
**Username**: Enter the name of the database user that {{ data-transfer-name }} will use for the connection.
2+

‎en/_includes/data-transfer/kf-yds-additional-settings.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
{% note warning %}
2+
3+
Data is processed in the following order:
4+
5+
1. Transformation.
6+
1. Conversion.
7+
8+
{% endnote %}
9+
10+
* **Transformation rules**:
11+
12+
* **Processing function**: Select one of the functions created in [{{ sf-full-name }}](../../functions/).
13+
* **SA Account**: Select or [create](../../iam/operations/sa/create.md) [a service account](../../iam/concepts/users/service-accounts.md) that the processing function will start under.
14+
* **Number of attempts**: Set the number of attempts to invoke the processing function.
15+
* **Buffer size to send**: Set the size of the buffer (in bytes) which when full data will be transferred to the processing function.
16+
17+
The maximum buffer size is 3.5 MB. For more information about restrictions that apply when working with functions in {{ sf-name }}, see the [corresponding section](../../functions/concepts/limits.md).
18+
19+
* **Sending interval**: Set the duration of the interval (in seconds) after the expiration of which the data from the stream should be transferred to the processing function.
20+
21+
{% note info %}
22+
23+
If the buffer becomes full or the sending interval expires, the data is transferred to the processing function.
24+
25+
{% endnote %}
26+
27+
* **Call timeout**: Set the allowed timeout of the response from the processing function (in seconds).
28+
29+
{% note warning %}
30+
31+
Values in the **Sending interval** and **Call timeout** fields are specified with the `s` postfix, for example, `10s`.
32+
33+
{% endnote %}
34+
35+
* **Conversion rules**:
36+
* **Data format**: Select one of the available formats:
37+
* `Json`
38+
* `CSV`
39+
* **Data schema**: Specify the schema as a list of fields or upload a file with a description of the schema in JSON format.
40+

‎en/_includes/data-transfer/necessary-settings/clickhouse.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
{% list tabs %}
2+
3+
* MDB cluster
4+
5+
Connecting to a database with the cluster ID specified in {{ yandex-cloud }}. Available only for clusters deployed in [{{ mch-name }}](../../../managed-clickhouse/).
6+
7+
To connect, set the required parameters:
8+
9+
* {% include [Field Cluster ID](../fields/cluster-id.md) %}
10+
* {% include [Field Database name](../fields/database-name.md) %}
11+
* {% include [Field Username](../fields/username.md) %}
12+
* {% include [Field Password](../fields/password.md) %}
13+
14+
* OnPremise cluster
15+
16+
Connecting to the database with explicitly specified host network addresses and ports.
17+
18+
To connect, set the required parameters:
19+
20+
* **Shards**.
21+
* **ID**: Specify a row that will allow the service to distinguish shards from each other.
22+
* **Hosts**: Specify FQDNs or IP addresses of the hosts in the shard.
23+
24+
* **Connection via SSL**: Enable if the cluster allows only encrypted connections.
25+
26+
* **PEM certificate**: If transmitted data needs to be encrypted, for example, to meet the requirements of [PCI DSS]({{ link-pci-dss-en }}), upload the [certificate](../../../managed-clickhouse/operations/connect#get-ssl-cert) file or add its contents as text.
27+
28+
* **HTTP port**: Set the number of the port that {{ data-transfer-name }} will use for the connection.
29+
30+
When connecting via the HTTP port:
31+
* For optional fields, default values are used (if any).
32+
* Recording complex types is supported (such as `array` and `tuple`).
33+
34+
* **Native port**: Set the number of the native port that {{ data-transfer-name }} will use for the connection.
35+
* {% include [Field Database name](../fields/database-name.md) %}
36+
* {% include [Field Username](../fields/username.md) %}
37+
* {% include [Field Password](../fields/password.md) %}
38+
39+
{% endlist %}
40+

‎en/_includes/data-transfer/necessary-settings/kafka.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
{% list tabs %}
2+
3+
* Managed Kafka
4+
5+
Connecting to the database with the cluster ID specified in {{ yandex-cloud }}. Available only for clusters deployed in [{{ mkf-name }}](../../../managed-kafka/).
6+
7+
To connect, set the required parameters:
8+
* **Managed Kafka**: Select the cluster to connect to.
9+
* **Authentication**: Specify the name and password of the account that {{ data-transfer-name }} will connect to the topic as.
10+
* {% include [Field Topic](../fields/topic.md) %}
11+
12+
* On Premise
13+
14+
Connecting to the topic with explicitly specified broker host network addresses.
15+
16+
To connect, set the required parameters:
17+
18+
* **On Premise**:
19+
20+
* **Broker addresses**: Specify the IP addresses or FQDNs of the broker hosts.
21+
22+
If the {{ KF }} port number differs from the standard one, specify it with a colon after the host name:
23+
24+
```text
25+
<broker host IP or FQDN>:<port number>
26+
```
27+
28+
* **SSL**: Use encryption to protect the connection.
29+
* **PEM certificate**: If transmitted data needs to be encrypted, for example, to meet the requirements of [PCI DSS]({{ link-pci-dss-en }}), upload the [certificate](../../../managed-kafka/operations/connect.md#get-ssl-cert) file or add its contents as text.
30+
31+
* **Authentication**: If you select `SASL`, specify the account name, password, and hashing mechanism.
32+
* {% include [Field Topic](../fields/topic.md) %}
33+
34+
{% endlist %}
35+

‎en/_includes/data-transfer/necessary-settings/mongodb.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
{% list tabs %}
2+
3+
* Custom installation
4+
5+
Connecting to the database with an explicitly specified network address and port.
6+
7+
To connect, set the required parameters:
8+
9+
* **Host list**: Specify the IP addresses or FQDN of the hosts to connect to.
10+
* **Port**: Set the port on which the service will connect to the cluster.
11+
* **CA Certificate**: If the transmitted data needs to be encrypted, for example, to meet the requirements of [PCI DSS]({{ link-pci-dss-en }}), upload the [certificate](../../../managed-mongodb/operations/connect.md#get-ssl-cert) or add its contents as text.
12+
* **Authentication source**: The name of the database associated with the user and password.
13+
* {% include [Field Username](../fields/username.md) %}
14+
* {% include [Field Password](../fields/password.md) %}
15+
* **Replica set**: Specify the name of the replica set.
16+
17+
* MDB cluster
18+
19+
Connecting to the database with the cluster ID specified in {{ yandex-cloud }}. Available only for clusters deployed in [{{ mmg-full-name }}](../../../managed-mongodb/).
20+
21+
To connect, set the required parameters:
22+
23+
* {% include [Field Cluster ID](../fields/cluster-id.md) %}
24+
* **Authentication source**: Specify the name of the database associated with the user and password.
25+
* {% include [Field Username](../fields/username.md) %}
26+
* {% include [Field Password](../fields/password.md) %}
27+
28+
{% endlist %}
29+

‎en/_includes/data-transfer/necessary-settings/mysql.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
{% list tabs %}
2+
3+
* Custom installation
4+
5+
Connecting to the database with an explicitly specified network address and port.
6+
7+
To connect, set the required parameters:
8+
* **Host IP or FQDN**: Enter the IP address or FQDN of the master host to connect to.
9+
* **Database port**: Set the number of the port that {{ data-transfer-name }} will use for the connection.
10+
* **CA Certificate**: If the transmitted data needs to be encrypted, for example, to meet the requirements of [PCI DSS]({{ link-pci-dss-en }}), upload the [certificate](../../../managed-mysql/operations/connect.md#get-ssl-cert) file or add its contents as text.
11+
* {% include [Field Database name](../fields/database-name.md) %}
12+
* {% include [Field Username](../fields/username.md) %}
13+
* {% include [Field Password](../fields/password.md) %}
14+
15+
* MDB cluster
16+
17+
Connecting to the database with the cluster ID specified in {{ yandex-cloud }}. Available only for clusters deployed in [{{ mmy-name }}](../../../managed-mysql/).
18+
19+
To connect, set the required parameters:
20+
* {% include [Field Cluster ID](../fields/cluster-id.md) %}
21+
* {% include [Field Database name](../fields/database-name.md) %}
22+
* {% include [Field Username](../fields/username.md) %}
23+
* {% include [Field Password](../fields/password.md) %}
24+
25+
{% endlist %}
26+

‎en/_includes/data-transfer/necessary-settings/postgresql.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
{% list tabs %}
2+
3+
* Custom installation
4+
5+
Connecting to the database with an explicitly specified network address and port.
6+
7+
To connect, set the required parameters:
8+
9+
* **Host IP or FQDN**: Specify the IP address or FQDN of the master host.
10+
* **Database port**: Set the number of the port that {{ data-transfer-name }} will use for the connection.
11+
* **CA Certificate**: If the transmitted data needs to be encrypted, for example, to meet the requirements of [PCI DSS]({{ link-pci-dss-en }}), upload the [certificate](../../../managed-postgresql/operations/connect.md#get-ssl-cert) or add its contents as text.
12+
* {% include [Field Database name](../fields/database-name.md) %}
13+
* {% include [Field Username](../fields/username.md) %}
14+
* {% include [Field Password](../fields/password.md) %}
15+
16+
* MDB cluster
17+
18+
Connecting to the database with the cluster ID specified in {{ yandex-cloud }}. Available only for clusters deployed in [{{ mpg-name }}](../../../managed-postgresql/).
19+
20+
To connect, set the required parameters:
21+
22+
* {% include [Field Cluster ID](../fields/cluster-id.md) %}
23+
* {% include [Field Database name](../fields/database-name.md) %}
24+
* {% include [Field Username](../fields/username.md) %}
25+
* {% include [Field Password](../fields/password.md) %}
26+
27+
{% endlist %}
28+

‎en/_includes/data-transfer/necessary-settings/ydb.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
* **Database name**: Enter the name of the database.
2+
* **Service account ID**: Select or [create](../../../iam/operations/sa/create.md) [a service account](../../../iam/concepts/users/service-accounts.md) that {{ data-transfer-name }} will connect to the database as.
3+

0 commit comments

Comments
 (0)