Skip to content

Commit

Permalink
Node options from cfg file for production (#62468) (#71463)
Browse files Browse the repository at this point in the history
* chore(NA): load NODE_OPTIONS from options files across environments

* chore(NA): move node.ci.options to config folder

* docs(NA): update docs to explain how to set node options from the cfg fil

* chore(NA): removed test npm scripts

* fix(NA): typo on setup script for CI

* chore(NA): add debug info

* chore(NA): export options on CI

* chore(NA): remove debug info

* chore(NA): support for configurable config folder using env var

* chore(NA): add node.options file into docker img

* fix(NA): use calculated config dir on node options for ci

* chore(NA): node bin scripts bootstrap and node_with_options implementation for bash

* chore(NA): complete node_with_options scripts with bat version

* chore(NA): add bin/node dev script and remove cli for run_with_node_options

* chore(NA): increase default maxBuffer

* chore(NA): remove run with options script from package.json

* chore(NA): include kbn-node script and underlying usage of it

* chore(NA): remove change on eslint

* chore(NA): correct typo on kbn node script comment

Co-authored-by: Tyler Smalley <tylersmalley@me.com>

* chore(NA): correct typo on kbn node script comment

Co-authored-by: Tyler Smalley <tylersmalley@me.com>

* chore(NA): add line to describe each option should be specified in a separated line

* chore(NA): remove node options from dev and ci env

* chore(NA): remove changes from package.json

* chore(NA): fix docker image build

* chore(NA): change value for example of --max-old-space-size in the node.options file

Co-authored-by: Tyler Smalley <tylersmalley@me.com>

* chore(NA): remove --no-warnings from node.options and force it in the bin scripts

* chore(NA): prevent 'The system cannot find the file' error message

* chore(NA): introduce slash when building path for %DIR%

* chore(NA): read options from file only if it exists

Co-authored-by: Jonathan Budzenski <jbudz@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Tyler Smalley <tylersmalley@me.com>

Co-authored-by: Jonathan Budzenski <jbudz@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Tyler Smalley <tylersmalley@me.com>
  • Loading branch information
4 people authored Jul 13, 2020
1 parent 81b4f82 commit bf08e8a
Show file tree
Hide file tree
Showing 11 changed files with 87 additions and 12 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ disabledPlugins
webpackstats.json
/config/*
!/config/kibana.yml
!/config/node.options
coverage
selenium
.babel_register_cache.json
Expand Down
6 changes: 6 additions & 0 deletions config/node.options
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
## Node command line options
## See `node --help` and `node --v8-options` for available options
## Please note you should specify one option per line

## max size of old space in megabytes
#--max-old-space-size=4096
4 changes: 2 additions & 2 deletions docs/setup/production.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -167,9 +167,9 @@ These can be used to automatically update the list of hosts as a cluster is resi
Kibana has a default maximum memory limit of 1.4 GB, and in most cases, we recommend leaving this unconfigured. In some scenarios, such as large reporting jobs,
it may make sense to tweak limits to meet more specific requirements.

You can modify this limit by setting `--max-old-space-size` in the `NODE_OPTIONS` environment variable. For deb and rpm, packages this is passed in via `/etc/default/kibana` and can be appended to the bottom of the file.
You can modify this limit by setting `--max-old-space-size` in the `node.options` config file that can be found inside `kibana/config` folder or any other configured with the environment variable `KIBANA_PATH_CONF` (for example in debian based system would be `/etc/kibana`).

The option accepts a limit in MB:
--------
NODE_OPTIONS="--max-old-space-size=2048" bin/kibana
--max-old-space-size=2048
--------
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@
"uiFramework:documentComponent": "cd packages/kbn-ui-framework && yarn documentComponent",
"kbn:watch": "node scripts/kibana --dev --logging.json=false",
"build:types": "tsc --p tsconfig.types.json",
"docs:acceptApiChanges": "node --max-old-space-size=6144 scripts/check_published_api_changes.js --accept",
"docs:acceptApiChanges": "node --max-old-space-size=6144 scripts/check_published_api_changes.js --accept",
"kbn:bootstrap": "node scripts/register_git_hook",
"spec_to_console": "node scripts/spec_to_console",
"storybook": "node scripts/storybook"
Expand Down
7 changes: 6 additions & 1 deletion src/dev/build/tasks/bin/scripts/kibana
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,16 @@ while [ -h "$SCRIPT" ] ; do
done

DIR="$(dirname "${SCRIPT}")/.."
CONFIG_DIR=${KIBANA_PATH_CONF:-"$DIR/config"}
NODE="${DIR}/node/bin/node"
test -x "$NODE"
if [ ! -x "$NODE" ]; then
echo "unable to find usable node.js executable."
exit 1
fi

NODE_OPTIONS="--no-warnings --max-http-header-size=65536 ${NODE_OPTIONS}" NODE_ENV=production exec "${NODE}" "${DIR}/src/cli" ${@}
if [ -f "${CONFIG_DIR}/node.options" ]; then
KBN_NODE_OPTS="$(grep -v ^# < ${CONFIG_DIR}/node.options | xargs)"
fi
NODE_OPTIONS="--no-warnings --max-http-header-size=65536 $KBN_NODE_OPTS $NODE_OPTIONS" NODE_ENV=production exec "${NODE}" "${DIR}/src/cli" ${@}
7 changes: 6 additions & 1 deletion src/dev/build/tasks/bin/scripts/kibana-keystore
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,16 @@ while [ -h "$SCRIPT" ] ; do
done

DIR="$(dirname "${SCRIPT}")/.."
CONFIG_DIR=${KIBANA_PATH_CONF:-"$DIR/config"}
NODE="${DIR}/node/bin/node"
test -x "$NODE"
if [ ! -x "$NODE" ]; then
echo "unable to find usable node.js executable."
exit 1
fi

"${NODE}" "${DIR}/src/cli_keystore" "$@"
if [ -f "${CONFIG_DIR}/node.options" ]; then
KBN_NODE_OPTS="$(grep -v ^# < ${CONFIG_DIR}/node.options | xargs)"
fi
NODE_OPTIONS="$KBN_NODE_OPTS $NODE_OPTIONS" "${NODE}" "${DIR}/src/cli_keystore" "$@"
17 changes: 16 additions & 1 deletion src/dev/build/tasks/bin/scripts/kibana-keystore.bat
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
@echo off

SETLOCAL
SETLOCAL ENABLEDELAYEDEXPANSION

set SCRIPT_DIR=%~dp0
for %%I in ("%SCRIPT_DIR%..") do set DIR=%%~dpfI
Expand All @@ -12,6 +12,21 @@ If Not Exist "%NODE%" (
Exit /B 1
)

set CONFIG_DIR=%KIBANA_PATH_CONF%
If [%KIBANA_PATH_CONF%] == [] (
set CONFIG_DIR=%DIR%\config
)

IF EXIST "%CONFIG_DIR%\node.options" (
for /F "eol=# tokens=*" %%i in (%CONFIG_DIR%\node.options) do (
If [!NODE_OPTIONS!] == [] (
set "NODE_OPTIONS=%%i"
) Else (
set "NODE_OPTIONS=!NODE_OPTIONS! %%i"
)
)
)

TITLE Kibana Keystore
"%NODE%" "%DIR%\src\cli_keystore" %*

Expand Down
7 changes: 6 additions & 1 deletion src/dev/build/tasks/bin/scripts/kibana-plugin
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,16 @@ while [ -h "$SCRIPT" ] ; do
done

DIR="$(dirname "${SCRIPT}")/.."
CONFIG_DIR=${KIBANA_PATH_CONF:-"$DIR/config"}
NODE="${DIR}/node/bin/node"
test -x "$NODE"
if [ ! -x "$NODE" ]; then
echo "unable to find usable node.js executable."
exit 1
fi

NODE_OPTIONS="--no-warnings ${NODE_OPTIONS}" NODE_ENV=production exec "${NODE}" "${DIR}/src/cli_plugin" "$@"
if [ -f "${CONFIG_DIR}/node.options" ]; then
KBN_NODE_OPTS="$(grep -v ^# < ${CONFIG_DIR}/node.options | xargs)"
fi
NODE_OPTIONS="--no-warnings $KBN_NODE_OPTS $NODE_OPTIONS" NODE_ENV=production exec "${NODE}" "${DIR}/src/cli_plugin" "$@"
23 changes: 20 additions & 3 deletions src/dev/build/tasks/bin/scripts/kibana-plugin.bat
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
@echo off

SETLOCAL
SETLOCAL ENABLEDELAYEDEXPANSION

set SCRIPT_DIR=%~dp0
for %%I in ("%SCRIPT_DIR%..") do set DIR=%%~dpfI
Expand All @@ -13,9 +13,26 @@ If Not Exist "%NODE%" (
Exit /B 1
)

TITLE Kibana Server
set CONFIG_DIR=%KIBANA_PATH_CONF%
If [%KIBANA_PATH_CONF%] == [] (
set CONFIG_DIR=%DIR%\config
)

IF EXIST "%CONFIG_DIR%\node.options" (
for /F "eol=# tokens=*" %%i in (%CONFIG_DIR%\node.options) do (
If [!NODE_OPTIONS!] == [] (
set "NODE_OPTIONS=%%i"
) Else (
set "NODE_OPTIONS=!NODE_OPTIONS! %%i"
)
)
)

:: Include pre-defined node option
set "NODE_OPTIONS=--no-warnings %NODE_OPTIONS%"

set "NODE_OPTIONS=--no-warnings %NODE_OPTIONS%" && "%NODE%" "%DIR%\src\cli_plugin" %*
TITLE Kibana Server
"%NODE%" "%DIR%\src\cli_plugin" %*

:finally

Expand Down
24 changes: 22 additions & 2 deletions src/dev/build/tasks/bin/scripts/kibana.bat
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
@echo off

SETLOCAL
SETLOCAL ENABLEDELAYEDEXPANSION

set SCRIPT_DIR=%~dp0
for %%I in ("%SCRIPT_DIR%..") do set DIR=%%~dpfI
Expand All @@ -14,7 +14,27 @@ If Not Exist "%NODE%" (
Exit /B 1
)

set "NODE_OPTIONS=--no-warnings --max-http-header-size=65536 %NODE_OPTIONS%" && "%NODE%" "%DIR%\src\cli" %*
set CONFIG_DIR=%KIBANA_PATH_CONF%
If [%KIBANA_PATH_CONF%] == [] (
set CONFIG_DIR=%DIR%\config
)

IF EXIST "%CONFIG_DIR%\node.options" (
for /F "eol=# tokens=*" %%i in (%CONFIG_DIR%\node.options) do (
If [!NODE_OPTIONS!] == [] (
set "NODE_OPTIONS=%%i"
) Else (
set "NODE_OPTIONS=!NODE_OPTIONS! %%i"
)
)
)

:: Include pre-defined node option
set "NODE_OPTIONS=--no-warnings --max-http-header-size=65536 %NODE_OPTIONS%"

:: This should run independently as the last instruction
:: as we need NODE_OPTIONS previously set to expand
"%NODE%" "%DIR%\src\cli" %*

:finally

Expand Down
1 change: 1 addition & 0 deletions src/dev/build/tasks/copy_source_task.js
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ export const CopySourceTask = {
'typings/**',
'webpackShims/**',
'config/kibana.yml',
'config/node.options',
'tsconfig*.json',
'.i18nrc.json',
'kibana.d.ts',
Expand Down

0 comments on commit bf08e8a

Please sign in to comment.