Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 9 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,34 +12,23 @@ OSMT is written in Kotlin with Spring Boot, and Angular. It uses backend-instanc
OSMT uses Elasticsearch, Redis, and MySQL as back-end dependencies. OSMT also requires an OAuth2 provider. Local / non-production configurations can be stood up using only Docker. See additional notes below for [Configuration](README.md#configuration).

## Getting started
Follow the steps in the [Pre-requisites](README.md#pre-requisites) and [Running the Quickstart](README.md#running-the-quickstart) sections to bootstrap a local Quickstart OSMT instance (you can see more details in the [Quickstart Configuration](README.md#quickstart-configuration) section).
* NOTE: A Quickstart OSMT instance should be for demo purposes only. The Docker images are fixed/pinned, and the data is stored on a Docker volume. Unless your organization has taken technical steps to ensure the backing data will remain available, you should consider Quickstart data to be temporary. If you create RSDs or Collections, you should export them before shutting down the Quickstart. See the [How-To](https://osmt.io/docs.html) section of osmt.io for more information on exporting.
Follow the steps in the [Pre-requisites](README.md#pre-requisites) section.
* Please have the right technical people in your organization deploy a production OSMT instance before making any real investment in effort for building skills.
* For latest updates, reach out in the [Discussion](https://github.com/wgu-opensource/osmt/discussions) boards in GitHub.

### Using the OSMT CLI utility (`osmt_cli.sh`)
The OSMT source code includes a utility named `osmt_cli.sh`, in the project root directory. `osmt_cli.sh` simplifies setting up a local OSMT environment and doing routine tasks. It uses BASH, and works on MacOS and Linux. We have had some success with a BASH interpreter in Windows. You can run `./osmt_cli.sh -h` for the help text.
* `osmt_cli.sh` uses git to help identify directory context. If you downloaded the source code as a ZIP file, you will need to have git installed to use `osmt_cli.sh`.

### Pre-requisites
By default, OSMT is wired up for Okta as an OAuth2 provider. While you can change this, these documents will only address configuring Okta.
1. Obtain a "free developer" Okta account. This is required to log in to a local OSMT. Please follow the steps in [OAuth2 and Okta Configuration](README.md#oauth2-and-okta-configuration) below, and return here when complete. You can refer to [Environment files for Quickstart and Development Stacks](README.md#environment-files-for-quickstart-and-development-stacks) for more details.
1. Obtain a "free developer" Okta account. This is required to log in to a local OSMT. Please follow the steps in [OAuth2 and Okta Configuration](README.md#oauth2-and-okta-configuration) below, and return here when complete. You can refer to [Environment file for Development and API Tests Stack](README.md#environment-files-for-development-and-api-tests-stacks) for more details.

2. Initialize your environment files. After running this, update the OAUTH2/OIDC values in the env files (replace the `xxxxxx` values with the correct values from your Okta account)
```
./osmt_cli.sh -i
```

### Running the Quickstart
1. Start the Quickstart configuration. This will take several minutes to download resources and build the software. If you continue to loop on errors that report "Retrying in 10 seconds...", there's something wrong. Search and reach out in the [Discussion](https://github.com/wgu-opensource/osmt/discussions) boards in GitHub.
```
./osmt_cli.sh -q
```

2. Open your browser to `http://localhost:8080`.

3. You can exit the Quickstart configuration by pressing [Ctrl-C] from your terminal window.
* If you don't clean up the Docker images and volumes, Quickstart should start very quickly when re-running `osmt_cli.sh -q`.

### Running the Development configuration
1. Validate your local environment (for Docker, Java, and other SDKs / runtimes). If this command reports error, you will need to resolve them before running the Development configuration. See more in the [Requirements to Build OSMT](README.md#requirements-to-build-osmt) section
```
Expand Down Expand Up @@ -77,7 +66,7 @@ By default, OSMT is wired up for Okta as an OAuth2 provider. While you can chang
```

### Housekeeping with `osmt_cli.sh`
You can surgically clean up OSMT-related Docker images and data volumes. This step **will** delete data from local OSMT Quickstart and Development configurations. It does not remove the mysql/redis/elasticsearch Docker images, as those may be available locally for other purposes.
You can surgically clean up OSMT-related Docker images and data volumes. This step **will** delete data from Development and API Test configurations. It does not remove the mysql/redis/elasticsearch Docker images, as those may be available locally for other purposes.

```
./osmt_cli.sh -c
Expand Down Expand Up @@ -140,20 +129,7 @@ OSMT is a multi-module Maven project. pom.xml files exist in the project root, `
The [API](api/README.md) and [UI](ui/README.md) modules have their own README.md files with more specific information about those modules.

### Configuration
This project makes use of 2 similar local configurations, "Quickstart" and "Development". These both rely on docker-compose and environment files. Follow the steps in [Pre-requisites](README.md#pre-requisites) above. See [Environment files for Quickstart and Development Stacks](README.md#environment-files-for-quickstart-and-development-stacks) for more details.

### Quickstart Configuration
The Quickstart configuration uses the `docker-compose.quickstart.yml` file in the project root to stand up a non-production OSMT stack. This stack uses containers for the Spring application and the back-end dependencies (MySQL, ElasticSearch, and Redis).
* The Quickstart configuration is opinionated, in that all services and volumes are stood up on a single machine. That topology may or may not be appropriate for your organization. It could inform how to configure a production OSMT instance, but it is not intended for a production deployment.
* Quickstart uses the `osmt-quickstart.env` file to provide your OAuth2 secrets, and override default values. If you break your working copy of this env file, you can refer to `osmt-quickstart.env.example` for the default values.
* By default, the Quickstart configuration automatically imports BLS and O*NET job code metadata, and reindexes ElasticSearch. You can alter this default behavior with the `REINDEX_ELASTICSEARCH` and `SKIP_METADATA_IMPORT` environment variables (export them in your shell, or uncomment them in `osmt-quickstart.env`).
* By default, Quickstart uses an image named `wguopensource/osmt-app:latest` as the Spring application. It will pull the image tagged `latest` from DockerHub.
* You can override the tagged version number with the `OSMT_APP_IMAGE_TAG` environment variable (export it in your shell, or uncomment it in `osmt-quickstart.env`).
* You can build a new Docker image for the Spring application, and tag it with `wguopensource/osmt-app:latest`. Quickstart will use it, instead of pulling from DockerHub.
* You are not required to use `osmt_cli.sh`. You can run this command from the project root to stand up the Quickstart configuration.
```
docker-compose --file docker-compose.quickstart.yml --env-file osmt-quickstart.env --project-name osmt_quickstart up
```
This project does "Development" configuration. It relies on docker-compose and environment files. Follow the steps in [Pre-requisites](README.md#pre-requisites) above.

### Development Configuration
The Development configuration uses the `dev-stack.yml` docker-compose file in the `docker` directory, for standing up just the back-end dependencies. This facilitates doing active development in the Spring or Angular layers. You can use `osmt_cli.sh` in the steps given in [Running the Development configuration](README.md#running-the-development-configuration) to simplify starting and stopping the Docker services and Spring application.
Expand Down Expand Up @@ -199,8 +175,9 @@ When using Okta, you will use the `oauth2-okta` profile for Spring Boot, which w

If you want to enable OSMT user permissions by roles, see additional details in [Role-based Access in OSMT](README.md#role-based-access-in-osmt).

### Environment Files for Quickstart and Development Stacks
There are many ways to provide environment values to a Spring application. That said, you should never push secrets to GitHub, so you should never store secrets in source code. The OSMT project is configured to git ignore files named `osmt*.env`, and we recommend you follow this approach. The OSMT source code includes example environment files for the Quickstart and Development configurations (`osmt-quickstart.env.example` and `api/osmt-dev-stack.env.example`). Running `./osmt_cli.sh -i` will create env files for you, but you will need to replace the 'xxxxxx' values with your OAUTH2/OIDC values, following the guidance in the [OAuth2 and Okta Configuration](README.md#oauth2-and-okta-configuration) section.
### Environment Files for Development and API Tests Stacks
There are many ways to provide environment values to a Spring application. That said, you should never push secrets to GitHub, so you should never store secrets in source code. The OSMT project is configured to git ignore files named `osmt*.env`, and we recommend you follow this approach.
The OSMT source code includes example environment files for the Development configuration (`api/osmt-dev-stack.env.example`) and API Tests configuration (`test/api/osmt-apitest.env.example`). Running `./osmt_cli.sh -i` will create env files for you, but you will need to replace the 'xxxxxx' values with your OAUTH2/OIDC values, following the guidance in the [OAuth2 and Okta Configuration](README.md#oauth2-and-okta-configuration) section.

#### Alternate approaches for Providing Environment Variables
* Provide these OAuth2 values as program arguments when starting your Spring Boot app (`-Dokta.oauth2.clientId="123456qwerty"`).
Expand All @@ -215,7 +192,7 @@ There are many ways to provide environment values to a Spring application. That
```

### Post-installation (BLS, O*NET, etc)
The Quickstart configuration automatically imports the default BLS and O*NET job code metadata. For a Development configuration, you can import the metadata with this command:
For Development configuration, you can import the metadata with this command:
```
osmt_cli.sh -m
```
Expand Down
78 changes: 0 additions & 78 deletions docker-compose.quickstart.yml

This file was deleted.

22 changes: 0 additions & 22 deletions osmt-quickstart.env.example

This file was deleted.

42 changes: 3 additions & 39 deletions osmt_cli.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@

set -u

declare QUICKSTART_ENV_FILE
declare DEV_ENV_FILE
declare APITEST_ENV_FILE

Expand All @@ -23,7 +22,6 @@ declare -r OSMT_SECURITY_PROFILE="${OSMT_SECURITY_PROFILE:-}"


init_osmt_env_files() {
_init_osmt_env_file "Quickstart" "${QUICKSTART_ENV_FILE}" || return 1
_init_osmt_env_file "Development" "${DEV_ENV_FILE}" || return 1
_init_osmt_env_file "API Tests" "${APITEST_ENV_FILE}" || return 1
}
Expand All @@ -40,7 +38,6 @@ validate_osmt_dev_environment() {

echo
echo_info "Checking environment files used in local OSMT instances..."
_validate_env_file "${QUICKSTART_ENV_FILE}" || is_environment_valid+=1
_validate_env_file "${DEV_ENV_FILE}" || is_environment_valid+=1
_validate_env_file "${APITEST_ENV_FILE}" || is_environment_valid+=1

Expand All @@ -51,29 +48,6 @@ validate_osmt_dev_environment() {
fi
}

start_osmt_quickstart() {
local -i rc
_validate_git || return 1
_validate_docker_version
rc=$?
if [[ $rc -ne 0 ]]; then
echo_err "Aborting OSMT Quickstart. Exiting..."
return 1
fi

_cd_osmt_project_dir || return 1
_validate_env_file "${QUICKSTART_ENV_FILE}"
rc=$?
if [[ $rc -ne 0 ]]; then
echo_err "Aborting OSMT Quickstart. Exiting..."
return 1
fi

echo
echo_info "Starting OSMT Quickstart with docker-compose using osmt-quickstart.env"
docker-compose --file docker-compose.quickstart.yml --env-file "${QUICKSTART_ENV_FILE}" --project-name osmt_quickstart up
}

import_osmt_metadata() {
local stack_name="${1}"
echo
Expand Down Expand Up @@ -231,13 +205,12 @@ start_osmt_api_tests() {
_remove_osmt_docker_artifacts() {
remove_osmt_docker_artifacts_for_stack "osmt_dev"
remove_osmt_docker_artifacts_for_stack "osmt_api_test"
remove_osmt_docker_artifacts_for_stack "osmt_quickstart"
}

cleanup_osmt_docker_artifacts() {
local prompt_msg; prompt_msg="$(cat <<-EOF
${INDENT}Do you want to clean up OSMT-related Docker images and volumes?
${INDENT}This step will delete data from local OSMT Quickstart and Development configurations.
${INDENT}This step will delete data from Development and API Test configurations.

${INDENT}Please answer 'y' to proceed?
EOF
Expand All @@ -264,11 +237,8 @@ A command line utility to simplify onboarding with OSMT development instances. T
Usage:
osmt_cli.sh [accepts a single option]

-i Initialize environment files for Quickstart and Development configurations.
-i Initialize environment files for Development and API Test configurations.
-v Validate local environment and dependencies for development.
-q Start the Quickstart configuration. Application and services are containerized and
managed by docker-compose. The docker-compose stack will attach to the console, with containers
named "osmt_quickstart".
-d Start the backend Development Docker stack (MySQL, ElasticSearch, Redis). docker-compose stack will
be detached, with containers named "osmt_dev". You can review status with 'docker ps'.
-e Stop the detached backend Development Docker stack (MySQL, ElasticSearch, Redis).
Expand All @@ -281,8 +251,7 @@ Usage:
-r Start the local Spring app to reindex ElasticSearch.
-a Start the local API tests for OSMT. This requires a valid OSMT jar file (from a 'mvn package')
-m Import default BLS and O*NET metadata into local Development instance.
-c Surgically clean up OSMT-related Docker images and data volumes. This step will delete data from local OSMT
Quickstart and Development configurations. It does not remove the mysql/redis/elasticsearch images, as
-c Surgically clean up OSMT-related Docker images and data volumes. This step will delete data from Development and API Test configurations. It does not remove the mysql/redis/elasticsearch images, as
those may be available locally for other purposes.
-h Show this help message.

Expand All @@ -295,7 +264,6 @@ EOF
declare script_dir; script_dir=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
source "${script_dir}/bin/lib/common.sh" || exit 135

QUICKSTART_ENV_FILE="${PROJECT_DIR}/osmt-quickstart.env"
DEV_ENV_FILE="${PROJECT_DIR}/api/osmt-dev-stack.env"
APITEST_ENV_FILE="${PROJECT_DIR}/test/osmt-apitest.env"

Expand Down Expand Up @@ -325,10 +293,6 @@ while getopts "ivqdelrasmch" flag; do
validate_osmt_dev_environment || exit 135
exit 0
;;
q)
start_osmt_quickstart || exit 135
exit 0
;;
d)
start_osmt_docker_stack "${OSMT_STACK_NAME}" || exit 135
exit 0
Expand Down