Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] Add docs of warning of Docker env 1.3 of JDK 11 and some doc of FE config. #5628

Merged
merged 2 commits into from
Apr 13, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 39 additions & 11 deletions docs/en/administrator-guide/config/fe_config.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ There are two ways to view the configuration items of FE:
1. FE web page

Open the FE web page `http://fe_host:fe_http_port/variable` in the browser. You can see the currently effective FE configuration items in `Configure Info`.
    

2. View by command

After the FE is started, you can view the configuration items of the FE in the MySQL client with the following command:
Expand All @@ -66,7 +66,7 @@ There are two ways to configure FE configuration items:
1. Static configuration

Add and set configuration items in the `conf/fe.conf` file. The configuration items in `fe.conf` will be read when the FE process starts. Configuration items not in `fe.conf` will use default values.
    

2. Dynamic configuration via MySQL protocol

After the FE starts, you can set the configuration items dynamically through the following commands. This command requires administrator privilege.
Expand Down Expand Up @@ -96,7 +96,7 @@ There are two ways to configure FE configuration items:
`async_pending_load_task_pool_size = 20`

Then restart the FE process to take effect the configuration.
    

2. Modify `dynamic_partition_enable`

Through `ADMIN SHOW FRONTEND CONFIG;` you can see that the configuration item can be dynamically configured (`IsMutable` is true). And it is the unique configuration of Master FE. Then first we can connect to any FE and execute the following command to modify the configuration:
Expand Down Expand Up @@ -127,7 +127,7 @@ There are two ways to configure FE configuration items:
This configuration will decide whether to resend agent task when create_time for agent_task is set, only when current_time - create_time > agent_task_resend_wait_time_ms can ReportHandler do resend agent task.

This configuration is currently mainly used to solve the problem of repeated sending of `PUBLISH_VERSION` agent tasks. The current default value of this configuration is 5000, which is an experimental value.

Because there is a certain time delay between submitting agent tasks to AgentTaskQueue and submitting to be, Increasing the value of this configuration can effectively solve the problem of repeated sending of agent tasks,

But at the same time, it will cause the submission of failed or failed execution of the agent task to be executed again for an extended period of time.
Expand Down Expand Up @@ -266,8 +266,6 @@ This configuration is used to set the time interval for updating the value of th

### `disable_colocate_join`

### `disable_colocate_join`

### `disable_colocate_relocate`

### `disable_hadoop_load`
Expand Down Expand Up @@ -358,6 +356,11 @@ This variable is a dynamic configuration, and users can modify the configuration

### `frontend_address`

Status: Deprecated, not recommended use. This parameter may be deleted later
Type: string
Description: Explicitly set the IP address of FE instead of using *InetAddress.getByName* to get the IP address. Usually in *InetAddress.getByName* When the expected results cannot be obtained. Only IP address is supported, not hostname.
Default value: 0.0.0.0

### `hadoop_load_default_timeout_second`

### `heartbeat_mgr_blocking_queue_size`
Expand Down Expand Up @@ -435,7 +438,7 @@ The max size of allowed HTTP headers. The unit of this configuration is BYTE. De
### `max_agent_task_threads_num`

### `max_allowed_in_element_num_of_delete`

This configuration is used to limit element num of InPredicate in delete statement. The default value is 1024.

### `max_allowed_packet`
Expand Down Expand Up @@ -471,8 +474,6 @@ Can cooperate with `mix_clone_task_timeout_sec` to control the maximum and minim

### `max_load_timeout_second`

### `max_mysql_service_task_threads_num`

### `max_query_retry_time`

### `max_routine_load_job_num`
Expand Down Expand Up @@ -519,6 +520,15 @@ This configuration is specifically used to limit timeout setting for stream load

### `meta_dir`

Type: string
Description: Doris meta data will be saved here.The storage of this dir is highly recommended as to be:

* High write performance (SSD)

* Safe (RAID)

Default value: DORIS_HOME_DIR + "/doris-meta";

### `meta_publish_timeout_ms`

### `min_bytes_per_broker_scanner`
Expand All @@ -536,9 +546,23 @@ See the description of `max_clone_task_timeout_sec`.

### `min_load_timeout_second`

### `mysql_service_nio_enabled`

Type: bool
Description: Whether FE starts the MySQL server based on NiO model. It is recommended to turn off this option when the query connection is less than 1000 or the concurrency scenario is not high.
Default value: true

### `mysql_service_io_threads_num`

### `mysql_service_nio_enabled`
Type: int
Description: When FeEstarts the MySQL server based on NIO model, the number of threads responsible for IO events. Only `mysql_service_nio_enabled` is true takes effect.
Default value: 4

### `max_mysql_service_task_threads_num`

Type: int
Description: When FeEstarts the MySQL server based on NIO model, the number of threads responsible for Task events. Only `mysql_service_nio_enabled` is true takes effect.
Default value: 4096

### `net_buffer_length`

Expand Down Expand Up @@ -578,6 +602,10 @@ See the description of `max_clone_task_timeout_sec`.

### `query_port`

Type: int
Description: FE MySQL server port
Default value: 9030

### `query_timeout`

### `remote_fragment_exec_timeout_ms`
Expand Down Expand Up @@ -661,7 +689,7 @@ This variable is a session variable, and the session level takes effect.
### `thrift_client_timeout_ms`

The connection timeout and socket timeout config for thrift server.

The value for thrift_client_timeout_ms is set to be larger than zero to prevent some hang up problems in java.net.SocketInputStream.socketRead0.

### `thrift_server_max_worker_threads`
Expand Down
6 changes: 3 additions & 3 deletions docs/en/installing/compilation.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Note: For different versions of Oris, you need to download the corresponding mir
| apachedoris/doris-dev:build-env-1.2 | [4ef5a8c](https://github.com/apache/incubator-doris/commit/4ef5a8c8560351d7fff7ff8fd51c4c7a75e006a8) or later | 0.12.x or later |
| apachedoris/doris-dev:build-env-1.3 | [ad67dd3](https://github.com/apache/incubator-doris/commit/ad67dd34a04c1ca960cff38e5b335b30fc7d559f) or later | 0.14.x or later |

Doris 0.14.0 will use apachedoris/doris-dev:build-env-1.2
Warning: Doris 0.14.0 still used apachedoris/doris-dev:build-env-1.2 to compile. After thie version, the code will use apachedoris/doris-dev:build-env-1.3 to compile . **In the docker image of build-env-1.3, the default JDK version is upgraded to 11. So FE will use OPENJDK 11 to compile. If the docker image after build-env-1.3 is used for compilation of FE, the Java version of FE running env also needs to be upgraded to JDK11 or above. Otherwise unexpected running errors may be caused. **

2. Running Mirror

Expand Down Expand Up @@ -119,7 +119,7 @@ You can try to compile Doris directly in your own Linux environment.
sudo add-apt-repository ppa:ubuntu-toolchain-r/ppa
sudo apt update
sudo apt install gcc-10 g++-10
```
```
If you are using CentOS you can use the following command to install the dependencies

```
Expand All @@ -136,7 +136,7 @@ You can try to compile Doris directly in your own Linux environment.
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1
```
```
After installation, set environment variables `PATH`, `JAVA_HOME`, etc.
Doris 0.14.0 will use gcc7 env to compile.

Expand Down
57 changes: 49 additions & 8 deletions docs/zh-CN/administrator-guide/config/fe_config.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ FE 的配置项有两种方式进行配置:
### `agent_task_resend_wait_time_ms`

当代理任务的创建时间被设置的时候,此配置将决定是否重新发送代理任务, 当且仅当当前时间减去创建时间大于 `agent_task_task_resend_wait_time_ms` 时,ReportHandler可以重新发送代理任务。

该配置目前主要用来解决`PUBLISH_VERSION`代理任务的重复发送问题, 目前该配置的默认值是5000,是个实验值,由于把代理任务提交到代理任务队列和提交到be存在一定的时间延迟,所以调大该配置的值可以有效解决代理任务的重复发送问题,

但同时会导致提交失败或者执行失败的代理任务再次被执行的时间延长。
Expand Down Expand Up @@ -264,8 +264,6 @@ FE 的配置项有两种方式进行配置:

### `disable_colocate_join`

### `disable_colocate_join`

### `disable_colocate_relocate`

### `disable_hadoop_load`
Expand Down Expand Up @@ -356,6 +354,14 @@ FE 的配置项有两种方式进行配置:

### `frontend_address`

状态:已废弃,不建议使用,后续可能删除该参数

类型:string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个参数可以废弃了,直接标注为废弃吧

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok


说明:显式的设置FE的IP地址, 代替使用 *InetAddress.getByName*获取IP地址。通常在*InetAddress.getByName*无法获取预期结果的情况下使用。仅支持IP地址,不支持hostname。

默认值:0.0.0.0

### `hadoop_load_default_timeout_second`

### `heartbeat_mgr_blocking_queue_size`
Expand Down Expand Up @@ -431,7 +437,7 @@ HTTP服务允许接收请求的Header的最大长度,单位为比特,默认
### `max_agent_task_threads_num`

### `max_allowed_in_element_num_of_delete`

该配置被用于限制delete语句中谓词in的元素数量。默认值为1024。

### `max_allowed_packet`
Expand Down Expand Up @@ -465,8 +471,6 @@ HTTP服务允许接收请求的Header的最大长度,单位为比特,默认

### `max_load_timeout_second`

### `max_mysql_service_task_threads_num`

### `max_query_retry_time`

### `max_routine_load_job_num`
Expand Down Expand Up @@ -513,6 +517,15 @@ current running txns on db xxx is xx, larger than limit xx

### `meta_dir`

类型:string

说明:FE元数据的存储目录,强烈建议该目录应该为:

* 高性能 (如SSD设备)
* 安全(如RAID)

默认值:DORIS_HOME_DIR + "/doris-meta";

### `meta_publish_timeout_ms`

### `min_bytes_per_broker_scanner`
Expand All @@ -530,9 +543,31 @@ current running txns on db xxx is xx, larger than limit xx

### `min_load_timeout_second`

### `mysql_service_nio_enabled`

类型:bool
说明:FE是否启动基于NIO模型的MySQL服务端。建议在查询连接低于1000或并发场景不高的场景可以关闭该选项。
默认值:true

### `mysql_service_io_threads_num`

### `mysql_service_nio_enabled`
类型:int

说明:当FE启动基于NIO模型的MySQL服务端时,负责IO Event的线程数量,仅在

`mysql_service_nio_enabled`为true时生效。

默认值:4

### `max_mysql_service_task_threads_num`

类型:int

说明:当FE启动基于NIO模型的MySQL服务端时,负责查询Task Event的线程数量,仅在

`mysql_service_nio_enabled`为true时生效。

默认值:4096

### `net_buffer_length`

Expand Down Expand Up @@ -572,6 +607,12 @@ current running txns on db xxx is xx, larger than limit xx

### `query_port`

类型:int

说明:FE负责监听MySQL请求的端口。

默认值:9030

### `query_timeout`

### `remote_fragment_exec_timeout_ms`
Expand Down Expand Up @@ -655,7 +696,7 @@ current running txns on db xxx is xx, larger than limit xx
### `thrift_client_timeout_ms`

这是 thrift 服务端的关于连接超时和socket读取数据超时的配置。

thrift_client_timeout_ms 的值被设置为大于0来避免线程卡在java.net.SocketInputStream.socketRead0的问题.

### `thrift_server_max_worker_threads`
Expand Down
3 changes: 2 additions & 1 deletion docs/zh-CN/installing/compilation.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ under the License.
apachedoris/doris-dev build-env f8bc5d4024e0 21 hours ago 3.28GB
```


注: 针对不同的 Doris 版本,需要下载对应的镜像版本

| 镜像版本 | commit id | doris 版本 |
Expand All @@ -53,7 +54,7 @@ under the License.
| apachedoris/doris-dev:build-env-1.2 | [4ef5a8c](https://github.com/apache/incubator-doris/commit/4ef5a8c8560351d7fff7ff8fd51c4c7a75e006a8) | 0.12.x, 0.13 |
| apachedoris/doris-dev:build-env-1.3 | [ad67dd3](https://github.com/apache/incubator-doris/commit/ad67dd34a04c1ca960cff38e5b335b30fc7d559f) | 0.14.x 或更新版本 |

注意: doris 0.14.0 版本仍然使用apachedoris/doris-dev:build-env-1.2 编译,之后的代码将使用apachedoris/doris-dev:build-env-1.3
注意: doris 0.14.0 版本仍然使用apachedoris/doris-dev:build-env-1.2 编译,之后的代码将使用apachedoris/doris-dev:build-env-1.3。**在build-env-1.3的docker镜像中,默认的JDK的版本升级到了11,所以FE将会使用OpenJDK 11进行编译。如果使用build-env-1.3之后的docker镜像进行FE编译的话,后续运行FE的Java版本也需要同时升级到JDK11以上,否则可能导致非预期的运行错误。**

2. 运行镜像

Expand Down
4 changes: 3 additions & 1 deletion fe/fe-core/src/main/java/org/apache/doris/common/Config.java
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,9 @@ public class Config extends ConfigBase {
* Default is "0.0.0.0", which means not set.
* CAN NOT set this as a hostname, only IP.
*/
@ConfField public static String frontend_address = "0.0.0.0";
@Deprecated
@ConfField
public static String frontend_address = "0.0.0.0";

/**
* Declare a selection strategy for those servers have many ips.
Expand Down