Data Toolkit

A Rails web application for CollectionSpace data related activities.

Prerequisites

PostgreSQL Setup

This application requires PostgreSQL. The default development/test db urls are:

• postgres://toolkit:toolkit@localhost:5432/toolkit_[development|test]

To create the toolkit database user:

# adjust envvar values as appropriate
PGHOST=localhost PGUSER=admin PGPASSWORD=admin psql \
    -c "CREATE ROLE toolkit WITH LOGIN SUPERUSER PASSWORD 'toolkit';"

Using SUPERUSER allows the role to create databases, which is required for rails db:create, and is needed for tests to disable referential integrity when loading fixtures.

For production DATABASE_URL is required as an environment variable in the form:

• postgres://$username:$password@$host:$port/$db_name

Rails Setup

Initial setup and run the application:

rbenv install -s
bundle install
./bin/setup

For just running the server without redoing the setup steps:

./bin/dev

CLI tasks

# create a user
CSPACE_URL=https://anthro.collectionspace.org
EMAIL_ADDRESS=admin@anthro.collectionspace.org
PASSWORD=Administrator
./bin/rake "crud:create:user[$CSPACE_URL,$EMAIL_ADDRESS,$PASSWORD]" | jq .

# import data configs from a manifest registry
MR_URL=https://gist.githubusercontent.com/mark-cooper/0492cc97d53a47105dd29ca86799c8c7/raw/f376621f1c2e110f88fc1bdd10c5437f0abc99a5/meta-manifest2.json
./bin/rake "crud:import:manifest_registry[$MR_URL]" | jq .

# display data configs scoped to user
USER_ID=1
DATA_CFG_TYPE=record_type
DATA_CFG_RECORD_TYPE=collectionobject
./bin/rake "crud:read:data_configs[$USER_ID,$DATA_CFG_TYPE,$DATA_CFG_RECORD_TYPE]" | jq .

# create an activity
USER_ID=1
LABEL=coll1
ACT_TYPE=Activities::CreateOrUpdateRecords
DATA_CFG_ID=$(./bin/rake "crud:read:data_configs[$USER_ID,$DATA_CFG_TYPE,$DATA_CFG_RECORD_TYPE]" | jq -r '.[0].id')
FILE=test/fixtures/files/test.csv
./bin/rake "crud:create:activity[$USER_ID,$LABEL,$ACT_TYPE,$DATA_CFG_ID,$FILE]" | jq .

# list tasks for activity
./bin/rake "crud:read:tasks[1]" | jq .

Deployment

Locally with Docker.

docker compose build
docker compose up

Remote with Kamal.

# verify connections to the server
bundle exec kamal server bootstrap -d qa

# verify access to docker registry
bundle exec kamal registry login -d qa

# boot postgres
kamal accessory boot postgres -d qa
kamal accessory details postgres -d qa
kamal accessory logs postgres -d qa -f

# run the deploy process
bundle exec kamal deploy -d qa

# run a command on the container
bundle exec kamal app exec -d qa "bin/rails about"

# connect to the container
bundle exec kamal app exec -i -d qa "bin/rails console"

To run Kamal locally you must first export these envvars.

A deployment can also be made via Github:

1. Pushes to main will deploy to production (TODO).
2. Pushes to qa will deploy to qa.
3. A deployment can be triggered via the Github Actions UI.
4. A deployment can be triggered via the Github Actions CLI.

# deploy via push to qa
git checkout qa
git reset --hard $my-branch && git push --force origin qa

# deploy via gh cli
gh workflow run deploy.yml # uses the current branch
gh workflow run deploy.yml --ref qa # specify the branch to run from