Skip to content

Commit 404d9dd

Browse files
authored
Add CM server installation role (#226)
Signed-off-by: Webster Mudge <wmudge@cloudera.com>
1 parent 9351e16 commit 404d9dd

File tree

13 files changed

+1100
-0
lines changed

13 files changed

+1100
-0
lines changed

roles/cm_server/README.md

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# cm_server
2+
3+
Install Cloudera Manager server.
4+
5+
This role automates the installation of Cloudera Manager server packages, configures its command-line arguments and Java options, and prepares its database connection. It supports both embedded and external database configurations. When using an external database, the database server and its respective Java client libraries must be pre-installed and configured on the Cloudera Manager server host.
6+
7+
The role will:
8+
- Install the specified Cloudera Manager server packages.
9+
- Configure command-line arguments for the Cloudera Manager server process.
10+
- Set Java Virtual Machine (JVM) options for the Cloudera Manager server.
11+
- Configure the database connection parameters for the Cloudera Manager server's metadata.
12+
- Optionally, install packages for and prepare the Cloudera Manager server's embedded database if selected.
13+
- Prepare the Cloudera Manager server database (schema creation, etc.).
14+
15+
# Requirements
16+
17+
- A valid Java Development Kit (JDK) must be installed on the target host.
18+
- A valid Cloudera Manager package repository must be configured and accessible on the target host.
19+
- **For external databases:** The external database server (PostgreSQL, Oracle, or MySQL) must be installed, configured, and accessible from the Cloudera Manager server host. Additionally, the corresponding JDBC driver (Java client libraries) must be installed on the Cloudera Manager server host prior to running this role.
20+
- **For embedded database:** PostgreSQL 10 is required and is *not* installed by this role; the `cloudera-manager-server-db-2` package pulls it as a dependency.
21+
22+
# Dependencies
23+
24+
None.
25+
26+
# Parameters
27+
28+
| Variable | Type | Required | Default | Description |
29+
| --- | --- | --- | --- | --- |
30+
| `cloudera_manager_server_packages` | `list` of `str` | `False` | `["cloudera-manager-server"]` | List of packages to install for the Cloudera Manager server. |
31+
| `cloudera_manager_cmf_server_args` | `str` | `False` | | Cloudera Manager server command line arguments (e.g., for custom flags). |
32+
| `cloudera_manager_cmf_java_opts` | `str` | `False` | `-Xmx4G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp` | Cloudera Manager server Java options for JVM tuning. |
33+
| `cloudera_manager_server_embedded_db_packages` | `list` of `str` | `False` | `["cloudera-manager-server-db-2"]` | List of packages to install specifically for the Cloudera Manager server embedded database. Only relevant if `cloudera_manager_database_type` is `embedded`. |
34+
| `cloudera_manager_database_type` | `str` | `False` | `postgresql` | Database type for the Cloudera Manager server. If not `embedded`, the external database must be configured prior to running this role. Valid choices are `postgresql`, `oracle`, `mysql`, `embedded`. |
35+
| `cloudera_manager_database_host` | `str` | `True` if `cloudera_manager_database_type != embedded` | | Database hostname for the Cloudera Manager server. |
36+
| `cloudera_manager_database_port` | `int` | `True` if `cloudera_manager_database_type != embedded` | | Database port for the Cloudera Manager server. |
37+
| `cloudera_manager_database_name` | `str` | `False` | `scm` | Database name for the Cloudera Manager server. |
38+
| `cloudera_manager_database_user` | `str` | `False` | `scm` | Database username for the Cloudera Manager server. |
39+
| `cloudera_manager_database_password` | `str` | `True` if `cloudera_manager_database_type != embedded` | | Database password for the Cloudera Manager server. |
40+
41+
# Example Playbook
42+
43+
```yaml
44+
- hosts: cm_server_host
45+
tasks:
46+
- name: Install Cloudera Manager server with embedded database
47+
ansible.builtin.import_role:
48+
name: cloudera.exe.cm_server
49+
vars:
50+
cloudera_manager_database_type: embedded
51+
# The 'cloudera_manager_server_embedded_db_packages' default will be used.
52+
# No database connection details (host, port, user, password) are needed for embedded.
53+
54+
- name: Install Cloudera Manager server with external PostgreSQL database
55+
ansible.builtin.import_role:
56+
name: cloudera.exe.cm_server
57+
vars:
58+
cloudera_manager_database_type: postgresql
59+
cloudera_manager_database_host: "db.example.com"
60+
cloudera_manager_database_port: 5432
61+
cloudera_manager_database_name: "scm_prod"
62+
cloudera_manager_database_user: "scm_user"
63+
cloudera_manager_database_password: "super_secure_password"
64+
cloudera_manager_cmf_java_opts: "-Xmx8G -XX:+UseG1GC" # Custom Java opts
65+
```
66+
67+
## License
68+
69+
```
70+
Copyright 2024 Cloudera, Inc.
71+
72+
Licensed under the Apache License, Version 2.0 (the "License");
73+
you may not use this file except in compliance with the License.
74+
You may obtain a copy of the License at
75+
76+
https://www.apache.org/licenses/LICENSE-2.0
77+
78+
Unless required by applicable law or agreed to in writing, software
79+
distributed under the License is distributed on an "AS IS" BASIS,
80+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
81+
See the License for the specific language governing permissions and
82+
limitations under the License.
83+
```

roles/cm_server/defaults/main.yml

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Copyright 2024 Cloudera, Inc.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
cloudera_manager_server_packages:
16+
- cloudera-manager-server
17+
18+
# cloudera_manager_cmf_server_args:
19+
cloudera_manager_cmf_java_opts: "-Xmx4G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp"
20+
21+
cloudera_manager_server_embedded_db_packages:
22+
- cloudera-manager-server-db-2
23+
24+
cloudera_manager_database_type: postgresql
25+
# cloudera_manager_database_host:
26+
# cloudera_manager_database_port:
27+
cloudera_manager_database_name: scm
28+
cloudera_manager_database_user: scm
29+
# cloudera_manager_database_password:

roles/cm_server/handlers/main.yml

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Copyright 2024 Cloudera, Inc.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
- name: Restart Cloudera Manager server
16+
ansible.builtin.service:
17+
name: cloudera-scm-server
18+
state: restarted
19+
listen: restart cm server
20+
21+
# cloudera_manager_port is unknown when the handler runs
22+
# see https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_handlers.html#handlers-in-roles
23+
# - name: Wait for Cloudera Manager server
24+
# ansible.builtin.wait_for:
25+
# host: "{{ inventory_hostname }}"
26+
# port: "{{ cloudera_manager_port }}"
27+
# delay: 5
28+
# state: started
29+
# timeout: 300
30+
# listen: restart cm server
31+
32+
- name: Restart CM embedded server
33+
ansible.builtin.service:
34+
name: cloudera-scm-server-db
35+
state: restarted
36+
# notify:
37+
# - wait cloudera-scm-server-db
38+
39+
# - name: wait cloudera-scm-server-db
40+
# wait_for:
41+
# host: "{{ cloudera_manager_host }}"
42+
# port: 7432
43+
# delay: 5
44+
# state: started
45+
# timeout: 300
Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
# -*- coding: utf-8 -*-
2+
3+
# Copyright 2024 Cloudera, Inc.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# https://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
17+
from __future__ import annotations
18+
19+
import configparser
20+
21+
from ansible.module_utils.common.dict_transformations import recursive_diff
22+
from ansible.module_utils.common.text.converters import to_native
23+
from ansible.module_utils.basic import AnsibleModule
24+
25+
26+
class AnsibleModuleError(Exception):
27+
def __init__(self, results):
28+
self.results = results
29+
30+
31+
def main():
32+
module = AnsibleModule(
33+
argument_spec=dict(
34+
config=dict(default="/etc/cloudera-scm-server/db.properties"),
35+
script=dict(required=True),
36+
type=dict(required=True, choices=["postgresql", "oracle", "mysql"]),
37+
host=dict(required=True),
38+
port=dict(required=True, type="int"),
39+
database=dict(required=True),
40+
username=dict(required=True),
41+
password=dict(required=True, no_log=True),
42+
),
43+
)
44+
45+
# Handle parameters
46+
config = module.params["config"]
47+
script = module.params["script"]
48+
type = module.params["type"]
49+
host = module.params["host"]
50+
port = module.params["port"]
51+
database = module.params["database"]
52+
username = module.params["username"]
53+
password = module.params["password"]
54+
55+
# Run precheck
56+
try:
57+
with open(config, "r") as f:
58+
contents = "[SCM]\n" + f.read()
59+
existing = configparser.ConfigParser()
60+
existing.optionxform = str
61+
existing.read_string(contents)
62+
except Exception as e:
63+
module.fail_json(
64+
msg="Error parsing Cloudera Manager db.properties",
65+
error=to_native(e),
66+
)
67+
68+
incoming = {
69+
"com.cloudera.cmf.db.setupType": "EXTERNAL",
70+
"com.cloudera.cmf.db.type": type,
71+
"com.cloudera.cmf.db.host": host + ":" + str(port),
72+
"com.cloudera.cmf.db.name": database,
73+
"com.cloudera.cmf.db.user": username,
74+
"com.cloudera.cmf.db.password": password,
75+
}
76+
77+
diff = recursive_diff(dict(existing["SCM"]), incoming)
78+
79+
# Execute system command
80+
if diff is not None:
81+
args = f"{script} --host {host} --port {port} {type} {database} {username} {password}"
82+
83+
(rc, stdout, stderr) = module.run_command(args, use_unsafe_shell=False)
84+
85+
if rc != 0:
86+
module.fail_json(
87+
msg="Error preparing database.",
88+
rc=rc,
89+
stderr=stderr,
90+
stdout=stdout,
91+
stderr_lines=str(stderr).splitlines(),
92+
stdout_lines=str(stdout).splitlines(),
93+
)
94+
else:
95+
module.exit_json(
96+
changed=True,
97+
msg="Database preparation succeeded.",
98+
stdout=stdout,
99+
stdout_lines=str(stdout).splitlines(),
100+
)
101+
else:
102+
module.exit_json(changed=False)
103+
104+
105+
if __name__ == "__main__":
106+
main()
Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Copyright 2024 Cloudera, Inc.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
argument_specs:
16+
main:
17+
short_description: Install Cloudera Manager server
18+
description:
19+
- Install Cloudera Manager server packages.
20+
- Configure Cloudera Manager server arguments.
21+
- Configure Cloudera Manager server Java options.
22+
- Prepare Cloudera Manager server database.
23+
- Optionally, install the Cloudera Manager server embedded database. (Requires PostgreSQL 10, which is not installed.)
24+
- For external databases, the database server and the Java client libraries must be installed prior.
25+
author:
26+
- "Webster Mudge <wmudge@cloudera.com>"
27+
version_added: "5.0.0"
28+
options:
29+
cloudera_manager_server_packages:
30+
description:
31+
- List of packages to install for Cloudera Manager server.
32+
type: list
33+
elements: str
34+
default:
35+
- cloudera-manager-server
36+
cloudera_manager_cmf_server_args:
37+
description:
38+
- Cloudera Manager server command line arguments.
39+
cloudera_manager_cmf_java_opts:
40+
description:
41+
- Cloudera Manager server Java options.
42+
default: "-Xmx4G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp"
43+
cloudera_manager_server_embedded_db_packages:
44+
description:
45+
- List of packages to install for Cloudera Manager server embedded database.
46+
type: list
47+
elements: str
48+
default:
49+
- cloudera-manager-server-db-2
50+
cloudera_manager_database_type:
51+
description:
52+
- Database type for the Cloudera Manager server.
53+
- If not I(embedded), then the external database must be configured prior.
54+
choices:
55+
- postgresql
56+
- oracle
57+
- mysql
58+
- embedded
59+
default: postgreql
60+
cloudera_manager_database_host:
61+
description:
62+
- Database hostname for the Cloudera Manager server.
63+
- Required if I(cloudera_manager_database_type != embedded).
64+
cloudera_manager_database_port:
65+
description:
66+
- Database port for the Cloudera Manager server.
67+
- Required if I(cloudera_manager_database_type != embedded).
68+
type: int
69+
cloudera_manager_database_name:
70+
description:
71+
- Database name for the Cloudera Manager server.
72+
default: scm
73+
cloudera_manager_database_user:
74+
description:
75+
- Database username for the Cloudera Manager server.
76+
default: scm
77+
cloudera_manager_database_password:
78+
description:
79+
- Database password for the Cloudera Manager server.
80+
- Required if I(cloudera_manager_database_type != embedded).
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Copyright 2024 Cloudera, Inc.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
- name: Converge
16+
hosts: all
17+
gather_facts: true
18+
become: true
19+
tasks:
20+
- name: Set up Cloudera Manager server (PostgreSQL)
21+
ansible.builtin.import_role:
22+
name: cm_server
23+
vars:
24+
cloudera_manager_database_host: 127.0.0.1
25+
cloudera_manager_database_port: 5432
26+
cloudera_manager_database_password: testpassword

0 commit comments

Comments
 (0)