YDBDOCS-691: move out recipes out of SDK reference + add a couple extra YQL recipes

ydb/docs/en/core/concepts/_includes/transactions.md

@@ -18,7 +18,7 @@ If consistency or freshness requirement for data read by a transaction can be re
 * *Stale Read-Only*: Read operations within a transaction may return results that are slightly out-of-date (lagging by fractions of a second). Each individual read returns consistent data, but no consistency between different reads is guaranteed.
 * *Snapshot Read-Only*: All the read operations within a transaction access the database snapshot. All the data reads are consistent. The snapshot is taken when the transaction begins, meaning the transaction sees all changes committed before it began.
 
-The transaction execution mode is specified in its settings when creating the transaction. See the examples for the {{ ydb-short-name }} SDK in the [{#T}](../../reference/ydb-sdk/recipes/tx-control.md).
+The transaction execution mode is specified in its settings when creating the transaction. See the examples for the {{ ydb-short-name }} SDK in the [{#T}](../../recipes/ydb-sdk/tx-control.md).
 
 ## YQL language {#language-yql}
 

ydb/docs/en/core/dev/batch-upload.md

@@ -19,7 +19,7 @@ To accelerate data uploads, consider the following recommendations:
 * If you need to push data to a table with a synchronous secondary index, we recommend that you first push data to a table and, when done, build a secondary index.
 * You should avoid writing data sequentially in ascending or descending order of the primary key.
    Writing data to a table with a monotonically increasing key causes all new data to be written to the end of the table since all tables in YDB are sorted by ascending primary key. As YDB splits table data into shards based on key ranges, inserts are always processed by the same server responsible for the "last" shard. Concentrating the load on a single server will result in slow data uploading and inefficient use of a distributed system.
-* Some use cases require writing the initial data (often large amounts) to a table before enabling OLTP workloads. In this case, transactionality at the level of individual queries is not required, and you can use `BulkUpsert` calls in the [CLI](../reference/ydb-cli/export-import/tools-restore.md), [SDK](../reference/ydb-sdk/recipes/bulk-upsert.md) and API. Since no transactionality is used, this approach has a much lower overhead than YQL queries. In case of a successful response to the query, the `BulkUpsert` method guarantees that all data added within this query is committed.
+* Some use cases require writing the initial data (often large amounts) to a table before enabling OLTP workloads. In this case, transactionality at the level of individual queries is not required, and you can use `BulkUpsert` calls in the [CLI](../reference/ydb-cli/export-import/tools-restore.md), [SDK](../recipes/ydb-sdk/bulk-upsert.md) and API. Since no transactionality is used, this approach has a much lower overhead than YQL queries. In case of a successful response to the query, the `BulkUpsert` method guarantees that all data added within this query is committed.
 
 {% note warning %}
 

ydb/docs/en/core/dev/example-app/java/index.md

@@ -27,8 +27,8 @@ Next, from the same working directory, run the command to start the test app. Th
 
 Main driver initialization parameters
 * A connection string containing details about an [endpoint](../../../concepts/connect.md#endpoint) and [database](../../../concepts/connect.md#database). This is the only parameter that is required.
-* [Authentication](../../../reference/ydb-sdk/recipes/auth.md#auth-provider) provider. Unless explicitly specified, an [anonymous connection](../../../concepts/auth.md) is used.
-* [Session pool](../../../reference/ydb-sdk/recipes/session-pool-limit.md) settings
+* [Authentication](../../../recipes/ydb-sdk/auth.md#auth-provider) provider. Unless explicitly specified, an [anonymous connection](../../../concepts/auth.md) is used.
+* [Session pool](../../../recipes/ydb-sdk/session-pool-limit.md) settings
 
 App code snippet for driver initialization:
 

ydb/docs/en/core/dev/getting-started.md

@@ -32,7 +32,7 @@ Additionally, you'd want to set up at least one of the available ways to run ad-
 
 - Go through [YQL tutorial](yql-tutorial/index.md) to get familiar with {{ ydb-short-name }}'s SQL dialect.
 - Explore [example applications](example-app/index.md) to see how working with SDK's looks like.
-- Check out [SDK recipies](../reference/ydb-sdk/recipes/index.md) for typical SDK use cases, which you can refer to later.
+- Check out [SDK recipies](../recipes/ydb-sdk/index.md) for typical SDK use cases, which you can refer to later.
 - Leverage your IDE capabilities to navigate the SDK code.
 
 ### For PostgreSQL-compatibility route

ydb/docs/en/core/recipes/index.md

@@ -0,0 +1,6 @@
+# Recipes for working with {{ ydb-short-name }}
+
+This section of {{ ydb-short-name }} documentation contains ready-to-use recipes for various aspects of interacting with {{ ydb-short-name }}. They are grouped into the following categories:
+
+* [{#T}](ydb-sdk/index.md)
+* [{#T}](yql/index.md)

ydb/docs/en/core/recipes/toc_p.yaml

@@ -0,0 +1,11 @@
+items:
+- name: Overview
+  href: index.md
+- name: YDB SDK
+  include:
+    mode: link
+    path: ydb-sdk/toc_p.yaml
+- name: YQL
+  include:
+    mode: link
+    path: yql/toc_p.yaml

ydb/docs/en/core/reference/ydb-sdk/recipes/auth-env.md → ydb/docs/en/core/recipes/ydb-sdk/auth-env.md

@@ -7,7 +7,7 @@ description: "The section describes examples of the authentication code using en
 
 {% include [work in progress message](_includes/addition.md) %}
 
-When using this method, the authentication mode and its parameters are defined by the environment that an application is run in, [as described here](../auth.md#env).
+When using this method, the authentication mode and its parameters are defined by the environment that an application is run in, [as described here](../../reference/ydb-sdk/auth.md#env).
 
 By setting one of the following environment variables, you can control the authentication method:
 

ydb/docs/en/core/reference/ydb-sdk/recipes/index.md → ydb/docs/en/core/recipes/ydb-sdk/index.md

@@ -1,16 +1,11 @@
----
-title: "Overview of code recipes when working with the {{ ydb-short-name }} SDK."
-description: "This section contains code recipes in different programming languages for a variety of tasks that are common when working with the {{ ydb-short-name }} SDK."
----
-
-# Code recipes
+# {{ ydb-short-name }} SDK code recipes
 
 {% include [work in progress message](_includes/addition.md) %}
 
 This section contains code recipes in different programming languages for a variety of tasks that are common when working with the {{ ydb-short-name }} SDK.
 
 Table of contents:
-- [Overview](index.md)
+
 - [Initializing the driver](init.md)
 - [Authentication](auth.md)
    - [Using a token](auth-access-token.md)
@@ -36,3 +31,9 @@ Table of contents:
    - [Enable logging](debug-logs.md)
    - [Enable metrics in Prometheus](debug-prometheus.md)
    - [Enable tracing in Jaeger](debug-jaeger.md)
+
+See also:
+
+- [{#T}](../../dev/index.md)
+- [{#T}](../../dev/example-app/index.md)
+- [{#T}](../../reference/ydb-sdk/index.md)

ydb/docs/en/core/reference/ydb-sdk/recipes/init.md → ydb/docs/en/core/recipes/ydb-sdk/init.md

@@ -7,7 +7,7 @@ description: "The article describes the examples of the code for connecting to {
 
 {% include [work in progress message](_includes/addition.md) %}
 
-To connect to {{ ydb-short-name }}, you need to specify the required and additional parameters that define the driver's behavior (learn more in [Connecting to the {{ ydb-short-name }} server](../../../concepts/connect.md)).
+To connect to {{ ydb-short-name }}, you need to specify the required and additional parameters that define the driver's behavior (learn more in [Connecting to the {{ ydb-short-name }} server](../../concepts/connect.md)).
 
 Below are examples of the code for connecting to {{ ydb-short-name }} (driver creation) in different {{ ydb-short-name }} SDKs.
 

ydb/docs/en/core/reference/ydb-sdk/recipes/retry.md → ydb/docs/en/core/recipes/ydb-sdk/retry.md

@@ -260,8 +260,8 @@ Below are code examples showing the {{ ydb-short-name }} SDK built-in tools for
    * `CompletableFuture<Result<T>> supplyResult`: Executing the operation that returns data. As an argument, it accepts the lambda `Function<Session, CompletableFuture<Result<T>>> fn`
 
    When using the `SessionRetryContext` class, make sure that the operation will be retried in the following cases
-   * The lambda function returned a [retryable](../error_handling.md) error code
-   * The lambda function invoked an `UnexpectedResultException` with a [retryable](../error_handling.md) error code
+   * The lambda function returned a [retryable](../../reference/ydb-sdk/error_handling.md) error code
+   * The lambda function invoked an `UnexpectedResultException` with a [retryable](../../reference/ydb-sdk/error_handling.md) error code
 
 
 

ydb/docs/en/core/reference/ydb-sdk/recipes/tx-control.md → ydb/docs/en/core/recipes/ydb-sdk/tx-control.md

@@ -5,7 +5,7 @@ description: "In this article, you will learn how to set up the transaction exec
 
 # Setting up the transaction execution mode
 
-To run your queries, first you need to specify the [transaction execution mode](../../../concepts/transactions.md#modes) in the {{ ydb-short-name }} SDK.
+To run your queries, first you need to specify the [transaction execution mode](../../concepts/transactions.md#modes) in the {{ ydb-short-name }} SDK.
 
 Below are code examples showing the {{ ydb-short-name }} SDK built-in tools to create an object for the *transaction execution mode*.
 

ydb/docs/en/core/recipes/yql/accessing-json.md

@@ -0,0 +1,103 @@
+# Accessing values inside JSON with YQL
+
+YQL provides two main ways to retrieve values from JSON:
+
+- Using [**JSON functions from the SQL standard**](../../yql/reference/builtins/json.md). This approach is recommended for simple cases and for teams that are familiar with them from other DBMSs.
+- Using [**Yson UDF**](../../yql/reference/udf/list/yson.md), [list](../../yql/reference/builtins/list.md) and [dict](../../yql/reference/builtins/dict.md) builtins, and [lambdas](../../yql/reference/syntax/expressions.md#lambda). This approach is more flexible and tightly integrated with {{ ydb-short-name }}'s data type system, thus recommended for complex cases.
+
+Below are the recipes that will use the same input JSON to demonstrate how to use each option to check whether a key exists, get a specific value, and retrieve a subtree.
+
+## JSON functions
+
+```yql
+$json = @@{
+    "friends": [
+        {
+            "name": "James Holden",
+            "age": 35
+        },
+        {
+            "name": "Naomi Nagata",
+            "age": 30
+        }
+    ]
+}@@j;
+
+SELECT
+    JSON_EXISTS($json, "$.friends[*].name"), -- True
+    CAST(JSON_VALUE($json, "$.friends[0].age") AS Int32), -- 35
+    JSON_QUERY($json, "$.friends[0]"); -- {"name": "James Holden", "age": 35}
+```
+
+`JSON_*` functions expect the `Json` data type as an input to run. In this example, the string literal has the suffix `j`, marking it as `Json`. In tables, data could be stored in either JSON format or as a string representation. To convert data from `String` to `JSON` data type, use the `CAST` function, such as `CAST(my_string AS JSON)`.
+
+## Yson UDF
+
+This approach typically combines multiple functions and expressions, so a query might leverage different specific strategies. 
+
+### Convert the whole JSON to YQL containers
+
+```yql
+$json = @@{
+    "friends": [
+        {
+            "name": "James Holden",
+            "age": 35
+        },
+        {
+            "name": "Naomi Nagata",
+            "age": 30
+        }
+    ]
+}@@j;
+
+$containers = Yson::ConvertTo($json, Struct<friends:List<Struct<name:String?,age:Int32?>>>);
+$has_name = ListAny(
+    ListMap($containers.friends, ($friend) -> {
+        return $friend.name IS NOT NULL;
+    })
+);
+$get_age = $containers.friends[0].age;
+$get_first_friend = Yson::SerializeJson(Yson::From($containers.friends[0]));
+
+SELECT 
+    $has_name, -- True
+    $get_age, -- 35
+    $get_first_friend; -- {"name": "James Holden", "age": 35}
+```
+
+It is **not** necessary to convert the whole JSON object to a structured combination of containers. Some fields can be omitted if not used, while some subtrees could be left in an unstructured data type like `Json`.
+
+### Work with in-memory representation
+
+```yql
+$json = @@{
+    "friends": [
+        {
+            "name": "James Holden",
+            "age": 35
+        },
+        {
+            "name": "Naomi Nagata",
+            "age": 30
+        }
+    ]
+}@@j;
+
+$has_name = ListAny(
+    ListMap(Yson::ConvertToList($json.friends), ($friend) -> {
+        return Yson::Contains($friend, "name");
+    })
+);
+$get_age = Yson::ConvertToInt64($json.friends[0].age);
+$get_first_friend = Yson::SerializeJson($json.friends[0]);
+
+SELECT 
+    $has_name, -- True
+    $get_age, -- 35
+    $get_first_friend; -- {"name": "James Holden", "age": 35}
+```
+
+## See also
+
+- [{#T}](modifying-json.md)

ydb/docs/en/core/recipes/yql/index.md

@@ -0,0 +1,13 @@
+# YQL recipes
+
+This section contains query recipes for various tasks that can be solved with YQL, {{ ydb-short-name }}'s SQL dialect.
+
+Table of contents:
+
+* [{#T}](accessing-json.md)
+* [{#T}](modifying-json.md)
+
+See also:
+
+- [{#T}](../../yql/reference/index.md)
+- [{#T}](../../dev/index.md)

ydb/docs/en/core/recipes/yql/modifying-json.md

@@ -0,0 +1,23 @@
+# Modifying JSON with YQL
+
+In memory, YQL operates on immutable values. Thus, when a query needs to change something inside a JSON value, the mindset should be about constructing a new value from pieces of the old one.
+
+This example query takes an input JSON named `$fields`, parses it, substitutes key `a` with 0, drops key `d`, and adds a key `c` with value 3:
+
+```yql
+$fields = '{"a": 1, "b": 2, "d": 4}'j;
+$pairs = DictItems(Yson::ConvertToInt64Dict($fields));
+$result_pairs = ListExtend(ListNotNull(ListMap($pairs, ($item) -> {
+    $item = if ($item.0 == "a", ("a", 0), $item);
+    return if ($item.0 == "d", null, $item);
+})), [("c", 3)]);
+$result_dict = ToDict($result_pairs);
+SELECT Yson::SerializeJson(Yson::From($result_dict));
+```
+
+## See also
+
+- [{#T}](../../yql/reference/udf/list/yson.md)
+- [{#T}](../../yql/reference/builtins/list.md)
+- [{#T}](../../yql/reference/builtins/dict.md)
+- [{#T}](accessing-json.md)

ydb/docs/en/core/recipes/yql/toc_p.yaml

@@ -0,0 +1,7 @@
+items:
+- name: Overview
+  href: index.md
+- name: Accessing JSON
+  href: accessing-json.md
+- name: Modifying JSON
+  href: modifying-json.md

ydb/docs/en/core/reference/ydb-sdk/_includes/auth.md

@@ -13,7 +13,7 @@ If the token generation object is not defined, the driver won't add any authenti
 
 ## Methods for creating token generation objects {#auth-provider}
 
-You can click any of the methods below to go to the source code of an example in the repository. You can also learn about the [authentication code recipes](../recipes/auth.md).
+You can click any of the methods below to go to the source code of an example in the repository. You can also learn about the [authentication code recipes](../../../recipes/ydb-sdk/auth.md).
 
 {% list tabs %}
 

ydb/docs/en/core/reference/ydb-sdk/_includes/index.md

@@ -1,8 +1,8 @@
-# YDB SDK
+# {{ ydb-short-name }} SDK reference
 
 {% include [index_intro_overlay.md](index_intro_overlay.md) %}
 
-OpenSource SDKs in the following programming languages are available to work with YDB:
+OpenSource SDKs in the following programming languages are available to work with {{ ydb-short-name }}:
 
 {% if oss %}
 - C++ [https://github.com/ydb-platform/ydb/tree/main/ydb/public/sdk/cpp](https://github.com/ydb-platform/ydb/tree/main/ydb/public/sdk/cpp)
@@ -19,7 +19,7 @@ The SDK documentation contains the following sections:
 
 - [Installation](../install.md)
 - [Authentication](../auth.md)
-- [Code recipes](../recipes/index.md)
+- [Code recipes](../../../recipes/ydb-sdk/index.md)
 - [Comparison of SDK features](../feature-parity.md)
 
 See also:

ydb/docs/en/core/reference/ydb-sdk/toc_i.yaml

@@ -19,7 +19,5 @@ items:
       href: grpc-headers.md
     - name: Health Check API
       href: health-check-api.md
-  - name: Code recipes
-    include: { mode: link, path: recipes/toc_p.yaml }
   - name: Comparison of SDK features
     href: feature-parity.md

ydb/docs/en/core/toc_i.yaml

@@ -27,18 +27,22 @@ items:
   include:
     mode: link
     path: reference/toc_p.yaml
-- name: Questions and answers
+- name: Recipes
   include:
     mode: link
-    path: faq/toc_p.yaml
-- name: Downloads
+    path: recipes/toc_p.yaml
+- name: Questions and answers
   include:
     mode: link
-    path: downloads/toc_p.yaml
+    path: faq/toc_p.yaml
 - name: Public materials
   include:
     mode: link
     path: public-materials/toc_p.yaml
+- name: Downloads
+  include:
+    mode: link
+    path: downloads/toc_p.yaml
 - name: Changelog
   include:
     mode: link

ydb/docs/en/core/yql/reference/yql-core/builtins/_includes/json.md

@@ -19,32 +19,6 @@ Despite the fact that the `age` field in the first object is of the `Number` typ
 
 To work with JSON, YQL implements a subset of the [SQL support for JavaScript Object Notation (JSON)](https://www.iso.org/standard/67367.html) standard, which is part of the common ANSI SQL standard.
 
-## Cookbook
-
-```
-$json = CAST(@@{
-    "friends": [
-        {
-            "name": "James Holden",
-            "age": 35
-        },
-        {
-            "name": "Naomi Nagata",
-            "age": 30
-        }
-    ]
-}@@ AS Json);
-
-SELECT
-    JSON_EXISTS($json, "$.friends[*].name"), -- True,
-    JSON_VALUE($json, "$.friends[0].age"), -- "35" (type Utf8?)
-    JSON_QUERY($json, "$.friends[0]"); -- {"name": "James Holden", "age": 35}
-```
-
-### Accessing a field in the database
-
-In tables, data can be stored in JSON format or as a string representation. JSON_* functions expect JSON as an input to run. To convert `String->JSON`, use the `CAST` function, such as `CAST (my_string AS JSON)`.
-
 ## JsonPath
 
 Values inside JSON objects are accessed using a query language called JsonPath. All functions for JSON accept a JsonPath query as an argument.
@@ -1174,3 +1148,7 @@ SELECT
     JSON_QUERY($json, "$.friends.name" WITH CONDITIONAL WRAPPER); -- ["James Holden", "Naomi Nagata"]
 ```
 
+## See also
+
+* [{#T}](../../../../recipes/yql/accessing-json.md)
+* [{#T}](../../../../recipes/yql/modifying-json.md)

ydb/docs/en/core/yql/reference/yql-core/udf/list/yson.md

@@ -293,3 +293,7 @@ SELECT Yson::ConvertToDoubleDict($yson, Yson::Options(false as Strict)); --- { "
 
 If you need to use the same Yson library settings throughout the query, it's more convenient to use [PRAGMA yson.AutoConvert;](../../syntax/pragma.md#yson.autoconvert) and/or [PRAGMA yson.Strict;](../../syntax/pragma.md#yson.strict). Only with these `PRAGMA` you can affect implicit calls to the Yson library occurring when you work with Yson/Json data types.
 
+## See also
+
+* [{#T}](../../../../recipes/yql/accessing-json.md)
+* [{#T}](../../../../recipes/yql/modifying-json.md)