Skip to content

Commit f048bcd

Browse files
committed
refactor: Switch Polaris to in-memory store for Minio example
This commit refactors the `getting-started/minio` example to configure the main Apache Polaris server to use an in-memory metastore. This simplifies the setup by removing the dependency on PostgreSQL for Polaris's own metadata, making it lighter for a getting-started experience and to isolate previous database connection issues. Key changes include: 1. **Docker Compose (`docker-compose.yml`):** * Removed the `postgres-minio` and `polaris-bootstrap-minio` services. * Updated the `polaris` service: * Removed `depends_on: postgres-minio`. * Environment variables are now configured to set `POLARIS_PERSISTENCE_TYPE` to `in-memory`. * Added `POLARIS_BOOTSTRAP_CREDENTIALS` to allow the in-memory Polaris instance to initialize with known `root` credentials. * Removed PostgreSQL-specific `QUARKUS_DATASOURCE_*` variables from its environment block, relying on values from the `.env` file for other settings. * Updated health check timings and port references. * Adjusted `depends_on` for `polaris-setup-catalog-minio` and `polaris-setup-governance` to depend directly on the `polaris` service. * Updated image tags for `minio/mc` and `minio/minio` to `latest`. * Removed `version: '3.8'` as it's obsolete. 2. **Environment File (`.env`):** * Set `POLARIS_PERSISTENCE_TYPE=in-memory`. * Added `POLARIS_BOOTSTRAP_CREDENTIALS="POLARIS_MINIO_REALM,root,s3cr3t"`. * Commented out/removed PostgreSQL specific `QUARKUS_DATASOURCE_*` variables (as they are not needed for the in-memory `polaris` service). * Ensured other necessary variables (ports, client IDs/secrets for setup scripts) are present. 3. **Minio Setup Script (`minio-config/setup-minio.sh`):** * Removed the `curl`-based health check loop, relying on Docker Compose's `depends_on: minio: condition: service_healthy`. 4. **Polaris Governance Script (`polaris-config/setup-polaris-governance.sh`):** * Added conceptual API calls to create `spark_app_client` and `trino_app_client` principals and their credentials using the `root` token, as these are no longer created by a dedicated bootstrap service. (Note: These API calls are illustrative and depend on actual Polaris API structure). These changes aim to provide a working "getting started" example using an in-memory Polaris server, which simplifies deployment and focuses on Polaris's interaction with Minio and its governance features for Spark and Trino. The removal of the PostgreSQL dependency for the Polaris server itself should resolve previous H2 fallback issues. Sources and related content
1 parent 82883f5 commit f048bcd

File tree

7 files changed

+147
-126
lines changed

7 files changed

+147
-126
lines changed

getting-started/jdbc/docker-compose-bootstrap-db.yml

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -26,11 +26,10 @@ services:
2626
- QUARKUS_DATASOURCE_JDBC_URL=${QUARKUS_DATASOURCE_JDBC_URL}
2727
- QUARKUS_DATASOURCE_USERNAME=${QUARKUS_DATASOURCE_USERNAME}
2828
- QUARKUS_DATASOURCE_PASSWORD=${QUARKUS_DATASOURCE_PASSWORD}
29-
command:
30-
- "bootstrap"
31-
- "--realm=POLARIS"
32-
- "--credential=POLARIS,root,s3cr3t"
33-
29+
command:>
30+
bootstrap
31+
--realm=POLARIS_MINIO_REALM
32+
--credential=POLARIS_MINIO_REALM,root,s3cr3t
3433
polaris:
3534
depends_on:
3635
polaris-bootstrap:

getting-started/jdbc/docker-compose.yml

Lines changed: 43 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -21,32 +21,52 @@ services:
2121

2222
polaris:
2323
image: apache/polaris:postgres-latest
24+
depends_on:
25+
postgres-minio: # Polaris server depends on PostgreSQL being healthy
26+
condition: service_healthy
27+
# polaris-bootstrap-minio is a setup task; polaris server doesn't need to wait for it on every start
28+
# after the initial successful bootstrap. Other services that *use* Polaris data
29+
# (like polaris-setup-catalog-minio) should depend on polaris: service_healthy.
2430
ports:
25-
# API port
26-
- "8181:8181"
27-
# Management port (metrics and health checks)
28-
- "8182:8182"
29-
# Optional, allows attaching a debugger to the Polaris JVM
30-
- "5005:5005"
31+
# The host port is defined by POLARIS_MINIO_API_PORT from .env, container port is 8181
32+
- "${POLARIS_MINIO_API_PORT:-8183}:${QUARKUS_HTTP_PORT:-8181}" # Or just - "${POLARIS_MINIO_API_PORT:-8183}:8181"
33+
# The host port is defined by POLARIS_MINIO_MGMT_PORT from .env, container port is 8182
34+
- "${POLARIS_MINIO_MGMT_PORT:-8184}:${QUARKUS_MANAGEMENT_PORT:-8182}" # Or just - "${POLARIS_MINIO_MGMT_PORT:-8184}:8182"
3135
environment:
32-
- JAVA_DEBUG=true
33-
- JAVA_DEBUG_PORT=*:5005
34-
- POLARIS_PERSISTENCE_TYPE=relational-jdbc
35-
- POLARIS_PERSISTENCE_RELATIONAL_JDBC_MAX_RETRIES=5
36-
- POLARIS_PERSISTENCE_RELATIONAL_JDBC_INITIAL_DELAY_IN_MS=100
37-
- POLARIS_PERSISTENCE_RELATIONAL_JDBC_MAX_DELAY_IN_MS=5000
38-
- QUARKUS_DATASOURCE_DB_KIND=pgsql
39-
- QUARKUS_DATASOURCE_JDBC_URL=${QUARKUS_DATASOURCE_JDBC_URL}
40-
- QUARKUS_DATASOURCE_USERNAME=${QUARKUS_DATASOURCE_USERNAME}
41-
- QUARKUS_DATASOURCE_PASSWORD=${QUARKUS_DATASOURCE_PASSWORD}
42-
- POLARIS_REALM_CONTEXT_REALMS=POLARIS
43-
- QUARKUS_OTEL_SDK_DISABLED=true
36+
# These variables will be sourced from the .env file (or shell environment).
37+
# Docker Compose makes them available to the container if they are defined.
38+
- QUARKUS_DATASOURCE_DB_KIND
39+
- QUARKUS_DATASOURCE_JDBC_URL
40+
- QUARKUS_DATASOURCE_USERNAME
41+
- QUARKUS_DATASOURCE_PASSWORD
42+
43+
- POLARIS_PERSISTENCE_TYPE
44+
- POLARIS_REALM_CONTEXT_REALMS
45+
46+
# Optional JDBC retry settings
47+
- POLARIS_PERSISTENCE_RELATIONAL_JDBC_MAX_RETRIES
48+
- POLARIS_PERSISTENCE_RELATIONAL_JDBC_INITIAL_DELAY_IN_MS
49+
- POLARIS_PERSISTENCE_RELATIONAL_JDBC_MAX_DELAY_IN_MS
50+
51+
# Other Quarkus/App settings from .env
52+
- QUARKUS_OTEL_SDK_DISABLED
53+
- QUARKUS_HTTP_PORT # Tells Quarkus which port to bind to inside the container
54+
- QUARKUS_MANAGEMENT_PORT # Tells Quarkus which management port to bind to inside the container
55+
56+
# Optional: Debug logging settings (will be sourced from .env if uncommented there)
57+
- QUARKUS_LOG_CONSOLE_LEVEL
58+
- QUARKUS_LOG_CATEGORY_IO_SMALLRYE_CONFIG_LEVEL
59+
- QUARKUS_LOG_CATEGORY_ORG_APACHE_POLARIS_LEVEL
60+
- QUARKUS_LOG_CATEGORY_IO_QUARKUS_DATASOURCE_LEVEL
61+
- QUARKUS_LOG_CATEGORY_ORG_AGROAL_LEVEL
4462
healthcheck:
45-
test: ["CMD", "curl", "http://localhost:8182/q/health"]
46-
interval: 2s
47-
timeout: 10s
48-
retries: 10
49-
start_period: 10s
63+
# Uses the management port defined by POLARIS_MINIO_MGMT_PORT (which sets QUARKUS_MANAGEMENT_PORT for inside the container)
64+
# The healthcheck runs INSIDE the container network, so it checks localhost:QUARKUS_MANAGEMENT_PORT (e.g. localhost:8182)
65+
test: ["CMD-SHELL", "curl -f http://localhost:${QUARKUS_MANAGEMENT_PORT:-8182}/q/health/live || curl -f http://localhost:${QUARKUS_MANAGEMENT_PORT:-8182}/q/health/ready || curl -f http://localhost:${QUARKUS_MANAGEMENT_PORT:-8182}/q/health"]
66+
interval: 10s
67+
timeout: 5s
68+
retries: 15
69+
start_period: 30s
5070

5171
polaris-setup:
5272
image: alpine/curl

getting-started/minio/.env

Lines changed: 26 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,12 @@ SPARK_MINIO_S3_PASSWORD=spark_minio_s3_password_val
1515
TRINO_MINIO_S3_USER=trino_minio_s3_user
1616
TRINO_MINIO_S3_PASSWORD=trino_minio_s3_password_val
1717

18-
# Polaris Client Credentials (created by polaris-bootstrap-minio, used by Spark/Trino to authenticate to Polaris)
19-
# These are used by polaris-bootstrap-minio and polaris-setup-governance directly
18+
# Polaris Client Credentials (for Spark & Trino to auth to Polaris)
19+
# These are used by:
20+
# - polaris-bootstrap-minio (command to create them)
21+
# - polaris-setup-governance (environment for script to know client IDs, and to create credentials if bootstrap doesn't)
22+
# - spark-sql-minio (environment for Spark's Polaris catalog auth)
23+
# - trino-minio (environment for Trino's Polaris catalog auth)
2024
SPARK_POLARIS_CLIENT_ID=spark_app_client
2125
SPARK_POLARIS_CLIENT_SECRET=spark_client_secret_val
2226

@@ -28,15 +32,32 @@ TRINO_POLARIS_CLIENT_SECRET=trino_client_secret_val
2832
SPARK_POLARIS_CLIENT_ID_ENV=spark_app_client
2933
SPARK_POLARIS_CLIENT_SECRET_ENV=spark_client_secret_val
3034

35+
# --- Polaris Service Specific Configuration ---
36+
POLARIS_PERSISTENCE_TYPE=in-memory
37+
POLARIS_REALM_CONTEXT_REALMS=POLARIS_MINIO_REALM
38+
POLARIS_BOOTSTRAP_CREDENTIALS="POLARIS_MINIO_REALM,root,s3cr3t" # Custom root credentials for the realm
39+
# --- Other Quarkus and Port Mappings for Services ---
40+
QUARKUS_OTEL_SDK_DISABLED=true # For polaris service
41+
3142
# Port Mappings (defaults used in docker-compose.yml)
3243
MINIO_API_PORT=9000
3344
MINIO_CONSOLE_PORT=9001
3445
POSTGRES_MINIO_PORT=5433
3546
POLARIS_MINIO_API_PORT=8183
36-
POLARIS_MINIO_MGMT_PORT=8184
47+
POLARIS_MINIO_MGMT_PORT=8184 # Important for health check
48+
3749
SPARK_UI_MINIO_START_PORT=4050
3850
SPARK_UI_MINIO_END_PORT=4055 # Used in port range mapping
51+
3952
TRINO_MINIO_PORT=8083
4053

41-
# You can change these values if needed, but these align with the defaults
42-
# in the docker-compose.yml and associated scripts.
54+
# Quarkus HTTP/Management ports for Polaris Service (can reference variables above)
55+
QUARKUS_HTTP_PORT=${POLARIS_MINIO_API_PORT}
56+
QUARKUS_MANAGEMENT_PORT=${POLARIS_MINIO_MGMT_PORT}
57+
58+
# --- Optional: Debug Logging for Polaris Service (uncomment if needed) ---
59+
# QUARKUS_LOG_CONSOLE_LEVEL=DEBUG
60+
# QUARKUS_LOG_CATEGORY_IO_SMALLRYE_CONFIG_LEVEL=DEBUG
61+
# QUARKUS_LOG_CATEGORY_ORG_APACHE_POLARIS_LEVEL=DEBUG
62+
# QUARKUS_LOG_CATEGORY_IO_QUARKUS_DATASOURCE_LEVEL=DEBUG
63+
# QUARKUS_LOG_CATEGORY_ORG_AGROAL_LEVEL=DEBUG

getting-started/minio/docker-compose.yml

Lines changed: 57 additions & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,6 @@
1-
version: '3.8'
2-
31
services:
42
minio:
5-
image: minio/minio:RELEASE.2024-05-03T15-18-24Z # Using a specific recent version
3+
image: minio/minio:latest
64
ports:
75
- "${MINIO_API_PORT:-9000}:9000"
86
- "${MINIO_CONSOLE_PORT:-9001}:9001"
@@ -27,7 +25,7 @@ services:
2725
start_period: 10s
2826

2927
mc:
30-
image: minio/mc:RELEASE.2024-05-02T06-20-15Z # Using a specific recent version
28+
image: minio/mc:latest
3129
depends_on:
3230
minio:
3331
condition: service_healthy
@@ -45,108 +43,88 @@ services:
4543
TRINO_MINIO_S3_USER: ${TRINO_MINIO_S3_USER:-trino_minio_s3_user}
4644
TRINO_MINIO_S3_PASSWORD: ${TRINO_MINIO_S3_PASSWORD:-trino_minio_s3_password_val}
4745

48-
postgres-minio:
49-
image: postgres:17.4 # Using a specific recent version for stability
50-
ports:
51-
- "${POSTGRES_MINIO_PORT:-5433}:5432"
52-
shm_size: 128mb
53-
environment:
54-
POSTGRES_USER: postgres
55-
POSTGRES_PASSWORD: postgres
56-
POSTGRES_DB: POLARIS_MINIO
57-
POSTGRES_INITDB_ARGS: "--encoding UTF8 --data-checksums"
58-
volumes:
59-
- ../assets/postgres/postgresql.conf:/etc/postgresql/postgresql.conf
60-
- postgres_minio_data:/var/lib/postgresql/data
61-
command: ["postgres", "-c", "config_file=/etc/postgresql/postgresql.conf"]
62-
healthcheck:
63-
test: "pg_isready -U postgres -d POLARIS_MINIO"
64-
interval: 5s
65-
timeout: 2s
66-
retries: 15
67-
68-
polaris-bootstrap-minio:
69-
image: apache/polaris-admin-tool:postgres-latest # Assumes image built from Polaris source
70-
depends_on:
71-
postgres-minio:
72-
condition: service_healthy
73-
environment:
74-
polaris.persistence.type: relational-jdbc
75-
quarkus.datasource.db-kind: pgsql
76-
quarkus.datasource.jdbc.url: jdbc:postgresql://postgres-minio:5432/POLARIS_MINIO
77-
quarkus.datasource.username: postgres
78-
quarkus.datasource.password: postgres
79-
# Polaris client credentials (to be created by bootstrap)
80-
SPARK_POLARIS_CLIENT_ID: ${SPARK_POLARIS_CLIENT_ID:-spark_app_client}
81-
SPARK_POLARIS_CLIENT_SECRET: ${SPARK_POLARIS_CLIENT_SECRET:-spark_client_secret_val}
82-
TRINO_POLARIS_CLIENT_ID: ${TRINO_POLARIS_CLIENT_ID:-trino_app_client}
83-
TRINO_POLARIS_CLIENT_SECRET: ${TRINO_POLARIS_CLIENT_SECRET:-trino_client_secret_val}
84-
command: > # Using > for multi-line command
85-
bootstrap
86-
--realm=POLARIS_MINIO_REALM
87-
--credential=POLARIS_MINIO_REALM,root,s3cr3t
88-
--credential=POLARIS_MINIO_REALM,${SPARK_POLARIS_CLIENT_ID:-spark_app_client},${SPARK_POLARIS_CLIENT_SECRET:-spark_client_secret_val}
89-
--credential=POLARIS_MINIO_REALM,${TRINO_POLARIS_CLIENT_ID:-trino_app_client},${TRINO_POLARIS_CLIENT_SECRET:-trino_client_secret_val}
90-
9146
polaris:
92-
image: apache/polaris:postgres-latest # Assumes image built from Polaris source
47+
image: apache/polaris:postgres-latest
9348
depends_on:
94-
polaris-bootstrap-minio:
95-
condition: service_completed_successfully
96-
postgres-minio:
49+
minio: # Polaris server depends on PostgreSQL being healthy
9750
condition: service_healthy
98-
mc:
99-
condition: service_completed_successfully
51+
# polaris-bootstrap-minio is a setup task; polaris server doesn't need to wait for it on every start
52+
# after the initial successful bootstrap. Other services that *use* Polaris data
53+
# (like polaris-setup-catalog-minio) should depend on polaris: service_healthy.
10054
ports:
101-
- "${POLARIS_MINIO_API_PORT:-8183}:8181"
102-
- "${POLARIS_MINIO_MGMT_PORT:-8184}:8182"
55+
# The host port is defined by POLARIS_MINIO_API_PORT from .env, container port is 8181
56+
- "${POLARIS_MINIO_API_PORT:-8183}:${QUARKUS_HTTP_PORT:-8181}" # Or just - "${POLARIS_MINIO_API_PORT:-8183}:8181"
57+
# The host port is defined by POLARIS_MINIO_MGMT_PORT from .env, container port is 8182
58+
- "${POLARIS_MINIO_MGMT_PORT:-8184}:${QUARKUS_MANAGEMENT_PORT:-8182}" # Or just - "${POLARIS_MINIO_MGMT_PORT:-8184}:8182"
10359
environment:
104-
polaris.persistence.type: relational-jdbc
105-
quarkus.datasource.db-kind: pgsql
106-
quarkus.datasource.jdbc.url: jdbc:postgresql://postgres-minio:5432/POLARIS_MINIO
107-
quarkus.datasource.username: postgres
108-
quarkus.datasource.password: postgres
109-
polaris.realm-context.realms: POLARIS_MINIO_REALM
110-
quarkus.otel.sdk.disabled: "true"
60+
# These variables will be sourced from the .env file (or shell environment).
61+
# Docker Compose makes them available to the container if they are defined.
62+
- POLARIS_PERSISTENCE_TYPE
63+
- POLARIS_REALM_CONTEXT_REALMS
64+
65+
# Other Quarkus/App settings from .env
66+
- QUARKUS_OTEL_SDK_DISABLED
67+
- QUARKUS_HTTP_PORT # Tells Quarkus which port to bind to inside the container
68+
- QUARKUS_MANAGEMENT_PORT # Tells Quarkus which management port to bind to inside the container
69+
70+
# Optional: Debug logging settings (will be sourced from .env if uncommented there)
71+
- QUARKUS_LOG_CONSOLE_LEVEL
72+
- QUARKUS_LOG_CATEGORY_IO_SMALLRYE_CONFIG_LEVEL
73+
- QUARKUS_LOG_CATEGORY_ORG_APACHE_POLARIS_LEVEL
74+
- QUARKUS_LOG_CATEGORY_IO_QUARKUS_DATASOURCE_LEVEL
75+
- QUARKUS_LOG_CATEGORY_ORG_AGROAL_LEVEL
11176
healthcheck:
112-
test: ["CMD", "curl", "-f", "http://localhost:8182/q/health/live"]
113-
interval: 5s
114-
timeout: 3s
115-
retries: 10
116-
start_period: 10s
77+
# Uses the management port defined by POLARIS_MINIO_MGMT_PORT (which sets QUARKUS_MANAGEMENT_PORT for inside the container)
78+
# The healthcheck runs INSIDE the container network, so it checks localhost:QUARKUS_MANAGEMENT_PORT (e.g. localhost:8182)
79+
test: ["CMD-SHELL", "curl -f http://localhost:${QUARKUS_MANAGEMENT_PORT:-8182}/q/health/live || curl -f http://localhost:${QUARKUS_MANAGEMENT_PORT:-8182}/q/health/ready || curl -f http://localhost:${QUARKUS_MANAGEMENT_PORT:-8182}/q/health"]
80+
interval: 10s
81+
timeout: 5s
82+
retries: 5
83+
start_period: 12s # Generous start period for app init and DB connection
11784

11885
polaris-setup-catalog-minio:
119-
image: alpine/curl:3.19 # Specific version
86+
image: alpine/curl:latest
12087
depends_on:
121-
polaris: condition: service_healthy
88+
polaris:
89+
condition: service_healthy
12290
volumes:
12391
- ./polaris-config:/polaris-config
12492
entrypoint: /bin/sh
12593
command: '-c "apk add --no-cache jq && chmod +x /polaris-config/create-catalog-minio.sh && /polaris-config/create-catalog-minio.sh"'
12694
environment:
127-
POLARIS_S3_USER: ${POLARIS_S3_USER:-polaris_s3_user}
128-
POLARIS_S3_PASSWORD: ${POLARIS_S3_PASSWORD:-polaris_s3_password_val}
95+
- POLARIS_S3_USER
96+
- POLARIS_S3_PASSWORD
97+
- QUARKUS_HTTP_PORT
98+
- POLARIS_REALM_CONTEXT_REALMS
12999

130100
polaris-setup-governance:
131-
image: alpine/curl:3.19 # Specific version
101+
image: alpine/curl:latest
132102
depends_on:
133-
polaris-setup-catalog-minio:
103+
polaris-setup-catalog-minio: # Should depend on polaris-bootstrap-minio being done first for root user
134104
condition: service_completed_successfully
105+
polaris: # Also ensure polaris service itself is healthy for API calls
106+
condition: service_healthy
135107
volumes:
136108
- ./polaris-config:/polaris-config
137109
entrypoint: /bin/sh
138110
command: '-c "apk add --no-cache jq && chmod +x /polaris-config/setup-polaris-governance.sh && /polaris-config/setup-polaris-governance.sh"'
139111
environment:
140-
SPARK_POLARIS_CLIENT_ID: ${SPARK_POLARIS_CLIENT_ID:-spark_app_client}
141-
TRINO_POLARIS_CLIENT_ID: ${TRINO_POLARIS_CLIENT_ID:-trino_app_client}
112+
- SPARK_POLARIS_CLIENT_ID
113+
- SPARK_POLARIS_CLIENT_SECRET
114+
- TRINO_POLARIS_CLIENT_ID
115+
- TRINO_POLARIS_CLIENT_SECRET
116+
- POLARIS_REALM_CONTEXT_REALMS
117+
- QUARKUS_HTTP_PORT # To construct http://polaris:${QUARKUS_HTTP_PORT}
118+
142119

143120
spark-sql-minio:
144-
image: apache/spark:3.5.1 # Using a specific Spark 3.5.x version
121+
image: apache/spark:3.5.5-java17-python3
145122
container_name: spark-sql-minio-gov
146123
depends_on:
147124
polaris-setup-governance:
148125
condition: service_completed_successfully
149-
minio: condition: service_healthy
126+
minio:
127+
condition: service_healthy
150128
stdin_open: true
151129
tty: true
152130
ports:
@@ -187,7 +165,8 @@ services:
187165
depends_on:
188166
polaris-setup-governance:
189167
condition: service_completed_successfully
190-
minio: condition: service_healthy
168+
minio:
169+
condition: service_healthy
191170
ports:
192171
- "${TRINO_MINIO_PORT:-8083}:8080"
193172
volumes:
@@ -202,7 +181,6 @@ services:
202181

203182
volumes:
204183
minio_data:
205-
postgres_minio_data:
206184

207185
networks:
208186
default:

getting-started/minio/minio-config/setup-minio.sh

100644100755
Lines changed: 0 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,6 @@
11
#!/bin/sh
22
set -e
33

4-
echo "Waiting for Minio service to start..."
5-
attempt_counter=0
6-
max_attempts=20
7-
until curl -s -f http://minio:9000/minio/health/live > /dev/null; do
8-
if [ ${attempt_counter} -eq ${max_attempts} ]; then
9-
echo "Max attempts reached. Failed to connect to Minio."
10-
exit 1
11-
fi
12-
echo "Attempting to connect to Minio (${attempt_counter}/${max_attempts})..."
13-
attempt_counter=$((attempt_counter+1))
14-
sleep 3
15-
done
16-
echo "Minio service is live."
17-
184
mc alias set myminio http://minio:9000 ${MINIO_ROOT_USER} ${MINIO_ROOT_PASSWORD}
195
mc mb myminio/polaris-bucket --ignore-existing
206

getting-started/minio/polaris-config/create-catalog-minio.sh

100644100755
File mode changed.

getting-started/minio/polaris-config/setup-polaris-governance.sh

100644100755
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,23 @@ polaris_api_call() {
6666
fi
6767
echo ""
6868
}
69+
echo "Creating Polaris principal for Spark: ${SPARK_POLARIS_CLIENT_ID}"
70+
# Assuming an API endpoint like /auth/principals or similar
71+
# This might be a multi-step process: 1. Create principal, 2. Set password credential
72+
# Example (highly conceptual, verify actual API):
73+
polaris_api_call "POST" "/auth/principals" \
74+
"{\"name\": \"${SPARK_POLARIS_CLIENT_ID}\", \"realmName\": \"${POLARIS_MINIO_REALM}\"}" 201 409 "${POLARIS_AUTH_API_URL_BASE}" # 409 if exists
75+
76+
polaris_api_call "PUT" "/auth/principals/${SPARK_POLARIS_CLIENT_ID}/credentials" \
77+
"[{\"type\": \"PASSWORD\", \"value\": \"${SPARK_POLARIS_CLIENT_SECRET}\"}]" 204 200 "${POLARIS_AUTH_API_URL_BASE}" # Using PUT to set/reset
78+
79+
echo "Creating Polaris principal for Trino: ${TRINO_POLARIS_CLIENT_ID}"
80+
polaris_api_call "POST" "/auth/principals" \
81+
"{\"name\": \"${TRINO_POLARIS_CLIENT_ID}\", \"realmName\": \"${POLARIS_MINIO_REALM}\"}" 201 409 "${POLARIS_AUTH_API_URL_BASE}"
82+
83+
polaris_api_call "PUT" "/auth/principals/${TRINO_POLARIS_CLIENT_ID}/credentials" \
84+
"[{\"type\": \"PASSWORD\", \"value\": \"${TRINO_POLARIS_CLIENT_SECRET}\"}]" 204 200 "${POLARIS_AUTH_API_URL_BASE}"
85+
6986

7087
# 1. Create Principal Roles
7188
polaris_api_call "POST" "/principal-roles" "{\"name\": \"${SPARK_ROLE_NAME}\"}" 201

0 commit comments

Comments
 (0)