Skip to content

Cluster superset for platform #158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
Draft

Cluster superset for platform #158

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1bd3f99
Merge branch 'main' of https://github.com/jembi/platform into PLAT-40…
Lumiere-Lodi Oct 4, 2022
baca2bd
Clean repo
Lumiere-Lodi Oct 4, 2022
d784199
Remove duplicated config in docker compose
Lumiere-Lodi Oct 4, 2022
660b092
Fix clustering of superset as stateless package
Lumiere-Lodi Oct 5, 2022
36fac3c
Remove docker compose cluster file
Lumiere-Lodi Oct 5, 2022
5de7886
Merge branch 'main' into PLAT-401-cluster-superset-package
michaelloosen Nov 18, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .env.cluster
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,9 @@ KAFKA_INSTANCES=3
# e.g. <topic>:<partions>:<replicationFactor> -> test:3:2 (defaults to <topics>:3:1)
KAFKA_TOPICS=2xx,reprocess,3xx,metrics:3:3

# Dashboard Visualiser - Superset
SUPERSET_INSTANCES=1

# Resource limits
OPENHIM_MEMORY_LIMIT=4G
ES_MEMORY_LIMIT=20G
Expand Down
4 changes: 4 additions & 0 deletions dashboard-visualiser-superset/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@ version: '3.9'
services:
dashboard-visualiser-superset:
image: jembi/superset:latest
deploy:
placement:
max_replicas_per_node: 1
replicas: ${SUPERSET_INSTANCES}
volumes:
- superset_home:/app/superset_home
- superset:/app/superset
Expand Down
3 changes: 2 additions & 1 deletion dashboard-visualiser-superset/swarm.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ readonly ROOT_PATH
. "${ROOT_PATH}/utils/log.sh"

main() {

if [[ "${MODE}" == "dev" ]]; then
log info "Running Dashboard Visualiser Superset package in DEV mode"
superset_dev_compose_param="-c ${COMPOSE_FILE_PATH}/docker-compose.dev.yml"
Expand All @@ -27,7 +28,7 @@ main() {

if [[ "${ACTION}" == "init" ]] || [[ "${ACTION}" == "up" ]]; then
config::set_config_digests "$COMPOSE_FILE_PATH"/docker-compose.yml
try "docker stack deploy -c ${COMPOSE_FILE_PATH}/docker-compose.yml $superset_dev_compose_param instant" "Failed to deploy Dashboard Visualiser Superset"
try "docker stack deploy -c ${COMPOSE_FILE_PATH}/docker-compose.yml $superset_dev_compose_param instant" "Failed to deploy Dashboard Visualiser Superset"

docker::await_container_startup dashboard-visualiser-superset
docker::await_container_status dashboard-visualiser-superset Running
Expand Down