Skip to content

Commit

Permalink
fix(helm): harmonize REANA_HOST confugirations (reanahub#867)
Browse files Browse the repository at this point in the history 
This PR is part of harmonizing the treatment of REANA_HOSTNAME
accross all REANA components and introduces REANA_HOSTPORT Helm
value. You can refer to other PRs below.

reenahub/reana-server#717
reanahub/reana-workflow-controller#630

Closes reanahub#865
  • Loading branch information
@Alputer
Alputer committed Feb 3, 2025
1 parent b2074bc commit aabf388
Show file tree
Hide file tree
Showing 10 changed files with 84 additions and 36 deletions.
2 changes: 2 additions & 0 deletions helm/configurations/values-dev.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# REANA components pointing to `latest`, locally built master branch

reana_hostport: 30443

components:
reana_db:
image: docker.io/library/postgres:14.10
Expand Down
3 changes: 2 additions & 1 deletion helm/reana/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,8 @@ collisions.
| `notifications.email_config.smtp_port` | SMTP email server port | None |
| `notifications.enabled` | Enable REANA system events notifications. For more information, visit the [documentation page](https://docs.reana.io/administration/configuration/configuring-access/) on user sign up. | false |
| `notifications.system_status` | Cronjob pattern representing how often the system status notification should be sent. Leave it empty to deactivate it | `"0 0 * * *"` |
| `reana_hostname` | REANA hostname (e.g. reana.example.org) | None |
| `reana_hostname` | REANA hostname (e.g. reana.example.org) | localhost |
| `reana_hostport` | REANA host name port number | None |
| `namespace_runtime` | Namespace in which the REANA runtime pods (workflow engines, jobs etc...) will run | `.Release.Namespace` |
| `naming_scheme` | REANA component naming scheme | None |
| `opensearch.*` | Pass any value from [OpenSearch Helm chart values](https://github.com/opensearch-project/helm-charts/tree/main/charts/opensearch#configuration) here | - |
Expand Down
4 changes: 2 additions & 2 deletions helm/reana/templates/NOTES.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,14 @@ finalise its configuration.

4. Try to run your first REANA example:

$ firefox https://{{ default "localhost:30443" .Values.reana_hostname }}
$ firefox https://{{ .Values.reana_hostname }}:{{.Values.reana_hostport}}

Or, using command line:

$ # install REANA client
$ pip install --user reana-client
$ # set environment variables for REANA client
$ export REANA_SERVER_URL=https://{{ default "localhost:30443" .Values.reana_hostname }}
$ export REANA_SERVER_URL=https://{{ .Values.reana_hostname }}:{{.Values.reana_hostport}}
$ export REANA_ACCESS_TOKEN="$mytoken"
$ # test connection to the REANA cluster
$ reana-client ping
Expand Down
12 changes: 10 additions & 2 deletions helm/reana/templates/reana-workflow-controller.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,11 @@ spec:
{{- end }}
{{- if .Values.reana_hostname }}
- name: REANA_HOSTNAME
value: {{ .Values.reana_hostname }}
value: {{ .Values.reana_hostname | quote }}
{{- end }}
{{- if .Values.reana_hostport }}
- name: REANA_HOSTPORT
value: {{ .Values.reana_hostport | quote }}
{{- end }}
{{- if .Values.eos.enabled }}
- name: K8S_CERN_EOS_AVAILABLE
Expand Down Expand Up @@ -290,7 +294,11 @@ spec:
{{- end }}
{{- if .Values.reana_hostname }}
- name: REANA_HOSTNAME
value: {{ .Values.reana_hostname }}
value: {{ .Values.reana_hostname | quote }}
{{- end }}
{{- if .Values.reana_hostport }}
- name: REANA_HOSTPORT
value: {{ .Values.reana_hostport | quote }}
{{- end }}
{{- if .Values.debug.enabled }}
- name: WDB_SOCKET_SERVER
Expand Down
2 changes: 1 addition & 1 deletion helm/reana/templates/secrets.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ metadata:
"helm.sh/resource-policy": keep
type: kubernetes.io/tls
data:
{{- $cert := genSelfSignedCert (.Values.reana_hostname | default "localhost") nil nil 90 }}
{{- $cert := genSelfSignedCert (.Values.reana_hostname) nil nil 90 }}
tls.crt: {{ $cert.Cert | b64enc | quote }}
tls.key: {{ $cert.Key | b64enc | quote }}
{{- end }}
Expand Down
8 changes: 6 additions & 2 deletions helm/reana/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,13 @@
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

# reana_hostname: reana.cern.ch
reana_hostname: localhost
# `reana_hostname` should be set for third party integrations to work and for
# production deployments to be secure.
# production deployments to be secure. The default `localhost` value is used
# in local deployment scenarios.

reana_hostport: 443
# `reana_hostport` should be set to the port number where the REANA service will be exposed for incoming SSL connections.

debug:
enabled: false
Expand Down
72 changes: 49 additions & 23 deletions reana/reana_dev/cluster.py
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# This file is part of REANA.
# Copyright (C) 2020, 2021, 2022, 2023 CERN.
# Copyright (C) 2020, 2021, 2022, 2023, 2025 CERN.
#
# REANA is free software; you can redistribute it and/or modify it
# under the terms of the MIT License; see LICENSE file for more details.
Expand Down Expand Up @@ -66,6 +66,13 @@ def cluster_commands():
help="In which mode to run REANA cluster? (releasehelm,releasepypi,latest,debug) [default=latest]",
)
@click.option("--worker-nodes", default=0, help="How many worker nodes? [default=0]")
@click.option(
"--extra-ports",
multiple=True,
type=int,
default=(30080, 30443),
help="Extra ports to expose, format: hostPort (containerPort will be the same)",
)
@click.option(
"--disable-default-cni",
is_flag=True,
Expand All @@ -77,7 +84,7 @@ def cluster_commands():
)
@cluster_commands.command(name="cluster-create")
def cluster_create(
mounts, mode, worker_nodes, disable_default_cni, kind_node_version
mounts, mode, worker_nodes, extra_ports, disable_default_cni, kind_node_version
): # noqa: D301
"""Create new REANA cluster.
Expand All @@ -86,6 +93,7 @@ def cluster_create(
$ reana-dev cluster-create -m /var/reana:/var/reana
-m /usr/share/local/mydata:/mydata
--mode debug
--extra-ports 30080 30443 30444
"""

class literal_str(str):
Expand All @@ -99,6 +107,35 @@ def add_volume_mounts(node):

yaml.add_representer(literal_str, literal_unicode_str)

# Reserved ports mapped to their respective services
RESERVED_DEBUG_PORTS = {
"wdb": 31984,
"maildev": 32580,
"rabbitmq": 31672,
"postgresql": 30432,
}

# Get reserved port values
reserved_ports = set(RESERVED_DEBUG_PORTS.values())

# Detect conflicting ports
conflicting_ports = set(extra_ports) & reserved_ports
if conflicting_ports:
conflict_details = [
f"{port} ({service})"
for service, port in RESERVED_DEBUG_PORTS.items()
if port in conflicting_ports
]
raise click.BadParameter(
f"The following ports are reserved for debug mode and cannot be used: {', '.join(conflict_details)}"
)

# Convert extra ports into mappings
extra_port_mappings = [
{"containerPort": port, "hostPort": port, "protocol": "TCP"}
for port in extra_ports
]

control_plane = {
"role": "control-plane",
"kubeadmConfigPatches": [
Expand All @@ -109,32 +146,15 @@ def add_volume_mounts(node):
' node-labels: "ingress-ready=true"\n'
)
],
"extraPortMappings": [
{"containerPort": 30080, "hostPort": 30080, "protocol": "TCP"}, # HTTP
{"containerPort": 30443, "hostPort": 30443, "protocol": "TCP"}, # HTTPS
],
"extraPortMappings": extra_port_mappings, # Only user-specified ports
}

if mode in ("debug"):
if mode == "debug":
mounts.append({"hostPath": find_reana_srcdir(), "containerPath": "/code"})
control_plane["extraPortMappings"].extend(
[
{"containerPort": 31984, "hostPort": 31984, "protocol": "TCP"}, # wdb
{
"containerPort": 32580,
"hostPort": 32580,
"protocol": "TCP",
}, # maildev
{
"containerPort": 31672,
"hostPort": 31672,
"protocol": "TCP",
}, # rabbitmq
{
"containerPort": 30432,
"hostPort": 30432,
"protocol": "TCP",
}, # postgresql
{"containerPort": port, "hostPort": port, "protocol": "TCP"}
for port in RESERVED_DEBUG_PORTS.values()
]
)

Expand Down Expand Up @@ -283,6 +303,12 @@ def cluster_build(
callback=validate_mode_option,
help="In which mode to run REANA cluster? (releasehelm,releasepypi,latest,debug) [default=latest]",
)
@click.option(
"--hostport",
type=int,
default=30443,
help="Port number of REANA_HOSTPORT.",
)
@click.option(
"-v",
"--values",
Expand Down
13 changes: 10 additions & 3 deletions reana/reana_dev/run.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,12 @@ def run_commands():
help="Which directories from the Kubernetes nodes to mount inside the job pods? "
"cluster_node_path:job_pod_mountpath, e.g /var/reana/mydata:/mydata",
)
@click.option(
"--port",
default=30443,
type=int,
help="Port number to use for cluster creation. Defaults to 30443.",
)
@click.option("--no-cache", is_flag=True, help="Do not use Docker image layer cache.")
@click.option(
"--component",
Expand Down Expand Up @@ -241,7 +247,6 @@ def run_commands():
)
@click.option(
"--parallel",
"-p",
default=1,
type=click.IntRange(min=1),
help="Number of docker images to build in parallel.",
Expand All @@ -256,6 +261,7 @@ def run_ci(
exclude_components,
mounts,
job_mounts,
hostport,
no_cache,
component,
admin_email,
Expand Down Expand Up @@ -288,6 +294,7 @@ def run_ci(
-c r-d-helloworld
--exclude-components=r-ui,r-a-krb5,r-a-rucio,r-a-vomsproxy
--mode debug
--hostport 30443
--namespace myreana
--admin-email john.doe@example.org
--admin-password mysecretpassword
Expand All @@ -296,7 +303,7 @@ def run_ci(
components = select_components(component)
# create cluster if needed
if not is_cluster_created():
cmd = "reana-dev cluster-create --mode {}".format(mode)
cmd = f"reana-dev cluster-create --mode {mode} --extra-ports {hostport}"
for mount in mounts:
cmd += " -m {}".format(mount)
if disable_default_cni:
Expand Down Expand Up @@ -338,7 +345,7 @@ def run_ci(
cmd += " -j {}".format(job_mount)
run_command(cmd, "reana")
# run demo examples
cmd = f"eval $(reana-dev client-setup-environment -n {namespace}) && reana-dev run-example"
cmd = f"eval $(reana-dev client-setup-environment --server-hostname https://localhost:{hostport} -n {namespace}) && reana-dev run-example"
for component in components:
cmd += " -c {}".format(component)
for a_workflow_engine in workflow_engine:
Expand Down
2 changes: 1 addition & 1 deletion scripts/create-admin-user.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ kubectl -n "${kubernetes_namespace}" create secret generic "${instance_name}"-ad
# Success!
echo "Success! You may now set the following environment variables:"
echo ""
echo " $ export REANA_SERVER_URL=https://localhost:30443 # or use your URL"
echo " $ export REANA_SERVER_URL=https://localhost:30443 # or use your URL"
echo " $ export REANA_ACCESS_TOKEN=${admin_access_token}"
echo ""
echo "Please see http://docs.reana.io/getting-started/ on how to run your first REANA example."
2 changes: 1 addition & 1 deletion tests/test_cluster.py
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# This file is part of REANA
# Copyright (C) 2024 CERN.
# Copyright (C) 2024, 2025 CERN.
#
# REANA is free software; you can redistribute it and/or modify it
# under the terms of the MIT License; see LICENSE file for more details.
Expand Down

0 comments on commit aabf388

Please sign in to comment.