Skip to content

Commit 85a1d92

Browse files
ewgeniusewgenius
authored
Secret store examples for data connectors (#154)
1 parent 8f16cd2 commit 85a1d92

File tree

4 files changed

+338
-13
lines changed

4 files changed

+338
-13
lines changed

‎spiceaidocs/docs/data-connectors/databricks.md‎

Lines changed: 92 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,18 +4,107 @@ sidebar_label: 'Databricks Data Connector'
44
description: 'Databricks Data Connector Documentation'
55
---
66

7+
import Tabs from '@theme/Tabs';
8+
import TabItem from '@theme/TabItem';
9+
710
Databricks as a connector for federated SQL query against Databrick's [Delta Lake](https://docs.databricks.com/en/delta/index.html).
811

912
## Configuration
10-
### Secrets
11-
- `token`: An active personal access token for the Databricks instance (equivalent to `DATABRICKS_TOKEN`).
12-
- Other keys provided in the secret are directly passed to the underlying object store (e.g. `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` if backed by AWS S3).
1313

1414
`spice login databricks` can be used to configure secrets for the Spice runtime (including AWS object store keys).
1515

1616
### Parameters
1717
- `endpoint`: The HTTPS endpoint of the Databricks host storing the desired tables.
1818

19+
### Auth
20+
21+
An active personal access token for the Databricks instance is required (equivalent to `DATABRICKS_TOKEN`).
22+
Other keys provided in the secret are directly passed to the underlying secret store (e.g. `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` if backed by AWS S3).
23+
24+
By default Databricks connector will look for a secret named `databricks` with keys `token` and `AWS_DEFAULT_REGION`,
25+
`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`.
26+
27+
Check [Secrets Stores](/secret-stores) for more details.
28+
29+
<Tabs>
30+
<TabItem value="local" label="Local" default>
31+
```bash
32+
spice login databricks --token <access-token> --aws-region <aws-region> --aws-access-key-id <aws-access-key-id> --aws-secret-access-key <aws-secret-access-key>
33+
```
34+
35+
Learn more about [File Secret Store](/secret-stores/file).
36+
</TabItem>
37+
<TabItem value="env" label="Env">
38+
```bash
39+
SPICE_SECRET_DATABRICKS_TOKEN=<access-token> \
40+
SPICE_SECRET_DATABRICKS_AWS_DEFAULT_REGION=<aws-region> \
41+
SPICE_SECRET_DATABRICKS_AWS_ACCESS_KEY_ID=<aws-access-key-id> \
42+
SPICE_SECRET_DATABRICKS_AWS_SECRET_ACCESS_KEY=<aws-secret-access-key> \
43+
spice run
44+
```
45+
46+
`spicepod.yaml`
47+
```yaml
48+
version: v1beta1
49+
kind: Spicepod
50+
name: spice-app
51+
52+
secrets:
53+
store: env
54+
55+
# <...>
56+
```
57+
58+
Learn more about [Env Secret Store](/secret-stores/env).
59+
</TabItem>
60+
<TabItem value="k8s" label="Kubernetes">
61+
```bash
62+
kubectl create secret generic databricks \
63+
--from-literal=token='<access-token>' \
64+
--from-literal=AWS_DEFAULT_REGION='<aws-region>' \
65+
--from-literal=AWS_ACCESS_KEY_ID='<aws-access-key-id>' \
66+
--from-literal=AWS_SECRET_ACCESS_KEY='<aws-secret-access-key>'
67+
```
68+
69+
`spicepod.yaml`
70+
```yaml
71+
version: v1beta1
72+
kind: Spicepod
73+
name: spice-app
74+
75+
secrets:
76+
store: kubernetes
77+
78+
# <...>
79+
```
80+
81+
Learn more about [Kubernetes Secret Store](/secret-stores/kubernetes).
82+
</TabItem>
83+
<TabItem value="keyring" label="Keyring">
84+
Add new keychain entry (macOS), with user and password in JSON string
85+
86+
```bash
87+
security add-generic-password -l "Databricks Secret" \
88+
-a spiced -s spice_secret_databricks \
89+
-w $(echo -n '{"token": "<access-token>", "AWS_DEFAULT_REGION": "<aws-region>", "AWS_ACCESS_KEY_ID": "<aws-access-key-id>", "AWS_SECRET_ACCESS_KEY": "<aws-secret-access-key>"}')
90+
```
91+
92+
`spicepod.yaml`
93+
```yaml
94+
version: v1beta1
95+
kind: Spicepod
96+
name: spice-app
97+
98+
secrets:
99+
store: keyring
100+
101+
# <...>
102+
```
103+
104+
Learn more about [Keyring Secret Store](/secret-stores/keyring).
105+
</TabItem>
106+
</Tabs>
107+
19108
## Example
20109

21110
```yaml

‎spiceaidocs/docs/data-connectors/dremio.md‎

Lines changed: 86 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,22 +4,104 @@ sidebar_label: 'Dremio Data Connector'
44
description: 'Dremio Data Connector Documentation'
55
---
66

7+
import Tabs from '@theme/Tabs';
8+
import TabItem from '@theme/TabItem';
9+
710
[Dremio](https://www.dremio.com/) server as a connector for federated SQL queries.
811

12+
By default Dremio connector will look for a secret named `dremio` with
13+
914
## `params`
1015

1116
- `endpoint`: The endpoint used to connect to the Dremio server.
1217

13-
## `auth`
18+
## Auth
19+
20+
Username and password credentials are required to log into the Dremio Server.
21+
By default Dremio connector will look for a secret named `dremio` with keys `username` and `password`.
22+
23+
Check [Secrets Stores](/secret-stores) for more details.
24+
25+
<Tabs>
26+
<TabItem value="local" label="Local" default>
27+
```bash
28+
spice login dremio -u demo -p demo1234
29+
```
30+
31+
Learn more about [File Secret Store](/secret-stores/file).
32+
</TabItem>
33+
<TabItem value="env" label="Env">
34+
```bash
35+
SPICE_SECRET_DREMIO_USERNAME=demo \
36+
SPICE_SECRET_DREMIO_PASSWORD=demo1234 \
37+
spice run
38+
```
39+
40+
`spicepod.yaml`
41+
```yaml
42+
version: v1beta1
43+
kind: Spicepod
44+
name: spice-app
45+
46+
secrets:
47+
store: env
48+
49+
# <...>
50+
```
51+
52+
Learn more about [Env Secret Store](/secret-stores/env).
53+
</TabItem>
54+
<TabItem value="k8s" label="Kubernetes">
55+
```bash
56+
kubectl create secret generic dremio \
57+
--from-literal=username='demo' \
58+
--from-literal=password='demo1234'
59+
```
60+
61+
`spicepod.yaml`
62+
```yaml
63+
version: v1beta1
64+
kind: Spicepod
65+
name: spice-app
66+
67+
secrets:
68+
store: kubernetes
69+
70+
# <...>
71+
```
72+
73+
Learn more about [Kubernetes Secret Store](/secret-stores/kubernetes).
74+
</TabItem>
75+
<TabItem value="keyring" label="Keyring">
76+
Add new keychain entry (macOS), with user and password in JSON string
77+
78+
```bash
79+
security add-generic-password -l "Dremio Secret" \
80+
-a spiced -s spice_secret_dremio \
81+
-w $(echo -n '{"username": "demo", "password": "demo1234"}')
82+
```
83+
84+
`spicepod.yaml`
85+
```yaml
86+
version: v1beta1
87+
kind: Spicepod
88+
name: spice-app
1489

15-
Username and password credentials are required to log into the Dremio Server. `spice login dremio` can be used to configure secrets for the Spice runtime.
90+
secrets:
91+
store: keyring
92+
93+
# <...>
94+
```
1695

17-
Check [Secrets Stores](/secret-stores) for details on configuring secrets.
96+
Learn more about [Keyring Secret Store](/secret-stores/keyring).
97+
</TabItem>
98+
</Tabs>
1899

19100
## Example
20101

21102
```yaml
22103
- from: dremio:datasets.dremio_dataset
23104
name: dremio_dataset
24105
params:
25-
endpoint: grpc://127.0.0.1:32010
106+
endpoint: grpc://127.0.0.1:32010
107+
```

‎spiceaidocs/docs/data-connectors/flightsql.md‎

Lines changed: 77 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,20 +4,94 @@ sidebar_label: 'FlightSql Data Connector'
44
description: 'FlightSQL Data Connector Documentation'
55
---
66

7+
import Tabs from '@theme/Tabs';
8+
import TabItem from '@theme/TabItem';
9+
710
Connect to any FlightSQL compatible server (e.g. Influx 3.0, CnosDB, other Spice runtimes!) as a connector for federated SQL queries.
811

912
## `params`
1013

1114
- `endpoint`: The Apache Flight endpoint used to connect to the FlightSQL server.
1215

13-
## `auth`
16+
## Auth
1417

15-
Check [Secrets Stores](/secret-stores) for details on how to configure.
18+
Username and password credentials can be specified to connect to the FlightSQL server:
1619

17-
Attributes:
1820
- `username` (optional): The username to use in the underlying Apache flight Handshake Request to authenticate to the server (see [reference](https://arrow.apache.org/docs/format/Flight.html#authentication)).
1921
- `password` (optional): The password to use in the underlying Apache flight Handshake Request to authenticate to the server (see [reference](https://arrow.apache.org/docs/format/Flight.html#authentication)).
2022

23+
By default FlightSQL connector will look for a secret named `flightsql` with keys `username` and `password`.
24+
25+
Check [Secrets Stores](/secret-stores) for more details.
26+
27+
<Tabs>
28+
<TabItem value="env" label="Env">
29+
```bash
30+
SPICE_SECRET_FLIGHTSQL_USERNAME=<flight_username> \
31+
SPICE_SECRET_FLIGHTSQL_PASSWORD=<flight_password> \
32+
spice run
33+
```
34+
35+
`spicepod.yaml`
36+
```yaml
37+
version: v1beta1
38+
kind: Spicepod
39+
name: spice-app
40+
41+
secrets:
42+
store: env
43+
44+
# <...>
45+
```
46+
47+
Learn more about [Env Secret Store](/secret-stores/env).
48+
</TabItem>
49+
<TabItem value="k8s" label="Kubernetes">
50+
```bash
51+
kubectl create secret generic flightsql \
52+
--from-literal=username='<flight_username>' \
53+
--from-literal=password='<flight_password>'
54+
```
55+
56+
`spicepod.yaml`
57+
```yaml
58+
version: v1beta1
59+
kind: Spicepod
60+
name: spice-app
61+
62+
secrets:
63+
store: kubernetes
64+
65+
# <...>
66+
```
67+
68+
Learn more about [Kubernetes Secret Store](/secret-stores/kubernetes).
69+
</TabItem>
70+
<TabItem value="keyring" label="Keyring">
71+
Add new keychain entry (macOS), with user and password in JSON string
72+
73+
```bash
74+
security add-generic-password -l "FlightSQL Secret" \
75+
-a spiced -s spice_secret_flightsql \
76+
-w $(echo -n '{"username": "<flight_username>", "password": "<flight_password>"}')
77+
```
78+
79+
`spicepod.yaml`
80+
```yaml
81+
version: v1beta1
82+
kind: Spicepod
83+
name: spice-app
84+
85+
secrets:
86+
store: keyring
87+
88+
# <...>
89+
```
90+
91+
Learn more about [Keyring Secret Store](/secret-stores/keyring).
92+
</TabItem>
93+
</Tabs>
94+
2195
## Example
2296

2397
```yaml

0 commit comments

Comments
 (0)