Skip to content

feat: migrator docs #37

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions MIGRATOR.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# CipherStash Migrator

The CipherStash Migrator is a tool that can be used to migrate plaintext data in a database to its encrypted equivalent.
It works inside the CipherStash Proxy Docker container and can handle different data types such as text, JSONB, integers, booleans, floats, and dates.
By specifying the relevant columns in your table, the migrator will seamlessly encrypt the existing data and store it in designated encrypted columns.

## Prerequisites

- [CipherStash Proxy](PROXY.md)
- [Have set up EQL in your database](GETTINGSTARTED.md)
- Ensure that the columns where data will be migrated already exist.

Here’s a draft for the technical usage documentation for the CipherStash Migrator tool:

## Usage

The CipherStash Migrator allows you to specify key-value pairs where the key is the plaintext column, and the value is the corresponding encrypted column.
Multiple key-value pairs can be specified, and the tool will perform a migration for each specified column.

### Running the migrator

You will need to SSH into the CipherStash Proxy Docker container to run the migrator.

```bash
docker exec -it eql-cipherstash-proxy bash
```

Once inside the container, you have access to the migrator tool.

```bash
cipherstash-migrator --version
```

#### Flags

| Flag | Description | Required |
| --- | --- | --- |
| `--columns` | Specifies the plaintext columns and their corresponding encrypted columns. The format is `plaintext_column=encrypted_column`. | Yes |
| `--table` | Specifies the table where the data will be migrated. | Yes |
| `--database-name` | Specifies the database name. | Yes |
| `--username` | Specifies the database username. | Yes |
| `--password` | Specifies the database password. | Yes |

#### Supported data types

- Text
- JSONB
- Integer
- Boolean
- Float
- Date

### Example

The following is an example of how to run the migrator with a single column:

```bash
cipherstash-migrator --columns example_column=example_column_encrypted --table examples --database-name postgres --username postgres --password postgres
```

If you require additional data types, please [raise an issue](https://github.com/cipherstash/encrypt-query-language/issues)

### Running migrations with multiple columns

To run a migration on multiple columns at once, specify multiple key-value pairs in the `--columns` option:

```bash
cipherstash-migrator --columns test_text=encrypted_text test_jsonb=encrypted_jsonb test_int=encrypted_int test_boolean=encrypted_boolean --table examples --database-name migrator_test --username postgres --password postgres
```

## Notes

- Ensure that the corresponding encrypted columns already exist in the table before running the migration.
- Data migration operations should be tested in a development environment before being executed in production.
5 changes: 3 additions & 2 deletions PROXY.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,11 @@ Populate the following fields with your values:

## Running the Proxy

To run the proxy, you can use the `start.sh` script in this directory. This script will start the proxy using the configuration in the `cipherstash-proxy.toml` file.
To run the proxy, you can use `docker compose` to start the proxy using the configuration in the `cipherstash-proxy.toml` file.
Run the following command from the `cipherstash-proxy` directory:

```bash
./start.sh
docker compose up
```

## Using the Proxy
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# CipherStash Encrypt Query Language (EQL)

[![Why we built EQL](https://img.shields.io/badge/concept-Why%20EQL-8A2BE2)](https://github.com/cipherstash/encrypt-query-language/blob/main/WHY.md)
[![CipherStash Proxy](https://img.shields.io/badge/guide-CipherStash%20Proxy-A48CF3)](https://github.com/cipherstash/encrypt-query-language/blob/main/PROXY.md)
[![Getting started](https://img.shields.io/badge/guide-Getting%20started-008000)](https://github.com/cipherstash/encrypt-query-language/blob/main/GETTINGSTARTED.md)
[![CipherStash Proxy](https://img.shields.io/badge/guide-CipherStash%20Proxy-A48CF3)](https://github.com/cipherstash/encrypt-query-language/blob/main/PROXY.md)
[![CipherStash Migrator](https://img.shields.io/badge/guide-CipherStash%20Migrator-A48CF3)](https://github.com/cipherstash/encrypt-query-language/blob/main/MIGRATOR.md)

Encrypt Query Language (EQL) is a set of abstractions for transmitting, storing & interacting with encrypted data and indexes in PostgreSQL.

Expand Down
12 changes: 12 additions & 0 deletions cipherstash-proxy/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
name: eql
services:
cipherstash-proxy:
container_name: eql-cipherstash-proxy
ports:
- 6432:6432
environment:
- CS_STATEMENT_HANDLER=mylittleproxy
- LOG_LEVEL=debug
volumes:
- ./cipherstash-proxy.toml:/etc/cipherstash-proxy/cipherstash-proxy.toml
image: cipherstash/cipherstash-proxy:cipherstash-proxy-v0.1.1
4 changes: 0 additions & 4 deletions cipherstash-proxy/start.sh

This file was deleted.