From e37553455a0bfe89a02beddfba100244e790a19a Mon Sep 17 00:00:00 2001 From: Aaron Brethorst Date: Thu, 5 Sep 2024 20:42:29 -0700 Subject: [PATCH] Get OBA working in Docker Fixes https://github.com/OneBusAway/onebusaway-application-modules/issues/348 * Add Dockerfiles for building JAR and WAR files, transit data bundles, and an app server * Add scripts and config files to make everything work properly * Add a docker-compose.yml file to orchestrate everything * Update the README to explain how to use all of the new features --- .gitignore | 5 +- Dockerfile | 4 +- README.md | 61 +++--- build.sh | 2 +- docker-compose.yml | 48 ++++- docker_app_server/Dockerfile | 58 ++++++ docker_app_server/bundle/.gitkeep | 0 docker_app_server/bundle_builder/Dockerfile | 30 +++ .../bundle_builder/build_bundle.sh | 40 ++++ docker_app_server/config/context.xml | 41 ++++ .../config/host-manager_context.xml | 23 +++ .../onebusaway-api-webapp-data-sources.xml | 115 +++++++++++ ...ay-enterprise-acta-webapp-data-sources.xml | 82 ++++++++ ...it-data-federation-webapp-data-sources.xml | 64 ++++++ docker_app_server/config/server.xml | 188 ++++++++++++++++++ docker_app_server/config/tomcat-users.xml | 40 ++++ docker_app_server/copy_resources.sh | 32 +++ pom.xml | 2 + 18 files changed, 806 insertions(+), 29 deletions(-) create mode 100644 docker_app_server/Dockerfile create mode 100644 docker_app_server/bundle/.gitkeep create mode 100644 docker_app_server/bundle_builder/Dockerfile create mode 100755 docker_app_server/bundle_builder/build_bundle.sh create mode 100644 docker_app_server/config/context.xml create mode 100644 docker_app_server/config/host-manager_context.xml create mode 100644 docker_app_server/config/onebusaway-api-webapp-data-sources.xml create mode 100644 docker_app_server/config/onebusaway-enterprise-acta-webapp-data-sources.xml create mode 100644 docker_app_server/config/onebusaway-transit-data-federation-webapp-data-sources.xml create mode 100644 docker_app_server/config/server.xml create mode 100644 docker_app_server/config/tomcat-users.xml create mode 100644 docker_app_server/copy_resources.sh diff --git a/.gitignore b/.gitignore index 74b0f0c894..e668470200 100644 --- a/.gitignore +++ b/.gitignore @@ -18,4 +18,7 @@ onebusaway-enterprise-acta-webapp/overlays/ onebusaway-enterprise-webapp/overlays/ build/* -!build/.gitkeep \ No newline at end of file +!build/.gitkeep + +docker_app_server/bundle/* +!docker_app_server/bundle/.gitkeep \ No newline at end of file diff --git a/Dockerfile b/Dockerfile index b5689fe285..6cf39a8d04 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,5 +1,5 @@ # -# Copyright (C) 2012 Brian Ferris +# Copyright (C) 2024 Open Transit Software Foundation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -22,7 +22,7 @@ RUN apt-get update && apt-get install -y \ vim \ && rm -rf /var/lib/apt/lists/* -mkdir -p /root/.m2/repository +RUN mkdir -p /root/.m2/repository WORKDIR /src diff --git a/README.md b/README.md index c24fefda80..9d535a792f 100644 --- a/README.md +++ b/README.md @@ -45,25 +45,50 @@ There are two options for setting up your own OneBusAway instance: * [Configuration and Deployment Guide for v2.x](https://github.com/OneBusAway/onebusaway/wiki/Configuration-and-Deployment-Guide-for-v2.x) - Designed to provide a comprehensive deployment method for users who wish to set up a simple OneBusAway application with minimal configurations. * [onebusaway-docker (Under development)](https://github.com/OneBusAway/onebusaway-docker) - A community-supported Docker configuration for OneBusAway v2.x is currently under development. -## Docker Development Instructions +## Development Instructions with Docker -Build the image: +```sh +docker compose up builder +docker compose exec builder bash # in another shell, for now +./build.sh --help # acquaint yourself with the build.sh options +./build.sh --clean --check-updates --test -``` -docker build -t oba-app-modules . -docker exec -it onebusaway-application-modules-builder-1 /bin/bash -./build-and-test.sh +# now you have built all of the OBA artifacts: +ls build/org/onebusaway/onebusaway-application-modules + +# Download GTFS data and build an OBA transit data bundle from it +docker compose up bundle_builder + +# verify that the bundle built as expected +ls docker_app_server/bundle + +# Start up the app server +docker compose up appserver + +# enter the app server +docker compose exec appserver bash + +# run the copy_resources.sh script (it's on your path) +# to copy the necessary WAR files into /usr/local/tomcat +copy_resources.sh + +# wait a few seconds for everything to spin up... ``` +Finally, verify that everything works as expected! + +* Check out the Tomcat Web App Manager at http://localhost:8080/manager/html (user/pass: admin/admin) to verify that your OBA WARs deployed correctly +* Check out the config.json API endpoint to verify that everything built correctly: http://localhost:8080/onebusaway-api-webapp/api/where/config.json?key=test ## Status -* Latest Stable Release - `2.4.18-cs`: - * [onebusaway-transit-data-federation-webapp](https://repo.camsys-apps.com/releases/org/onebusaway/onebusaway-transit-data-federation-webapp/2.4.18-cs/onebusaway-transit-data-federation-webapp-2.4.18-cs.war) - * [onebusaway-api-webapp](https://repo.camsys-apps.com/releases/org/onebusaway/onebusaway-api-webapp/2.4.18-cs/onebusaway-api-webapp-2.4.18-cs.war) - * [onebusaway-enterprise-acta-webapp](https://repo.camsys-apps.com/releases/org/onebusaway/onebusaway-enterprise-acta-webapp/2.4.18-cs/onebusaway-enterprise-acta-webapp-2.4.18-cs.war) -* [Previous releases](https://developer.onebusaway.org/release-notes) +* Download pre-built JARs and WARs: https://developer.onebusaway.org/downloads +* Terraform/Open Tofu/Infrastructure as Code: https://github.com/onebusaway/onebusaway-deployment +* Docker + * Pre-built images: https://hub.docker.com/r/opentransitsoftwarefoundation/ + * Instructions: https://github.com/onebusAway/onebusaway-docker + ## Deployments @@ -79,14 +104,6 @@ Check out the full list on the [OneBusAway Deployments page](https://github.com/ * [Latest Stable Release](https://developer.onebusaway.org/downloads) -## Code Repository - -To browse the source online visit https://github.com/OneBusAway/onebusaway-application-modules. - -To create a local copy of the repository, use the following command: - -`$ git clone https://github.com/OneBusAway/onebusaway-application-modules.git` - ## Developer Information @@ -101,11 +118,7 @@ To create a local copy of the repository, use the following command: * Mailing lists: - [Developer discussion list](https://groups.google.com/group/onebusaway-developers) - [User discussion list](https://groups.google.com/group/onebusaway-users) - * IRC channel: - - `#onebusaway` on Freenode - - You can connect using your favorite IRC client or [chat through the web](http://webchat.freenode.net/?channels=onebusaway) (just enter a username and click *Connect*) - ## Contact Info -There are [lots of ways to get in touch with us](https://github.com/OneBusAway/onebusaway/wiki/Contact-Us). +There are [lots of ways to get in touch with us](https://developer.onebusaway.org/getting-help). diff --git a/build.sh b/build.sh index 7a0c87b759..b0c40b574b 100755 --- a/build.sh +++ b/build.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash # -# Copyright (C) 2012 Brian Ferris +# Copyright (C) 2024 Open Transit Software Foundation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/docker-compose.yml b/docker-compose.yml index 72a25c7c2d..70efab1601 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -23,4 +23,50 @@ services: - ./build:/root/.m2/repository - .:/src stdin_open: true - tty: true \ No newline at end of file + tty: true + + bundle_builder: + build: + context: ./docker_app_server/bundle_builder + volumes: + - ./build:/root/.m2/repository + - ./docker_app_server/bundle:/bundle + stdin_open: true + tty: true + + database: + image: mysql:8.3 + container_name: database + environment: + MYSQL_ROOT_PASSWORD: Ins3cure! + MYSQL_DATABASE: oba_database + MYSQL_USER: oba_user + MYSQL_PASSWORD: oba_password + ports: + - "3306:3306" + volumes: + - type: volume + source: mysql-data + target: /var/lib/mysql + restart: always + + appserver: + build: + context: docker_app_server + depends_on: + - database + ports: + - "8080:8080" + volumes: + - type: bind + source: ./docker_app_server/config/server.xml + target: /usr/local/tomcat/conf/server.xml + - type: bind + source: ./docker_app_server/config/context.xml + target: /usr/local/tomcat/conf/context.xml + - ./docker_app_server/config:/oba_config_files + - ./build:/root/.m2/repository + - ./docker_app_server/bundle:/bundle + +volumes: + mysql-data: diff --git a/docker_app_server/Dockerfile b/docker_app_server/Dockerfile new file mode 100644 index 0000000000..a48a62060b --- /dev/null +++ b/docker_app_server/Dockerfile @@ -0,0 +1,58 @@ +# +# Copyright (C) 2024 Open Transit Software Foundation +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM tomcat:8.5.98-jdk11-temurin + +ENV CATALINA_HOME /usr/local/tomcat +ENV TZ=America/Los_Angeles +ARG OBA_VERSION=2.5.13-otsf +ENV OBA_VERSION=$OBA_VERSION + +RUN apt-get update && apt-get install -y \ + jq \ + supervisor \ + tzdata \ + unzip \ + xmlstarlet \ + python3-pip \ + && apt-get clean \ + && rm -rf /var/lib/apt/lists/* + +# Set the configured time zone +RUN ln -fs /usr/share/zoneinfo/$TZ /etc/localtime && dpkg-reconfigure -f noninteractive tzdata + +# Set up the host-manager and manager webapps +RUN rm -rf $CATALINA_HOME/webapps +RUN mv $CATALINA_HOME/webapps.dist $CATALINA_HOME/webapps + +COPY ./config/tomcat-users.xml $CATALINA_HOME/conf/ + +COPY ./config/host-manager_context.xml $CATALINA_HOME/webapps/docs/META-INF/context.xml +COPY ./config/host-manager_context.xml $CATALINA_HOME/webapps/examples/META-INF/context.xml +COPY ./config/host-manager_context.xml $CATALINA_HOME/webapps/host-manager/META-INF/context.xml +COPY ./config/host-manager_context.xml $CATALINA_HOME/webapps/manager/META-INF/context.xml +COPY ./config/host-manager_context.xml $CATALINA_HOME/webapps/ROOT/META-INF/context.xml + +# MySQL Connector +WORKDIR $CATALINA_HOME/lib +RUN wget "https://cdn.mysql.com/Downloads/Connector-J/mysql-connector-j-8.4.0.tar.gz" \ + && tar -zxvf mysql-connector-j-8.4.0.tar.gz \ + && mv mysql-connector-j-8.4.0/mysql-connector-j-8.4.0.jar . \ + && rm mysql-connector-j-8.4.0.tar.gz \ + && rm -rf mysql-connector-j-8.4.0 + +COPY ./copy_resources.sh /usr/local/bin/copy_resources.sh +RUN chmod +x /usr/local/bin/copy_resources.sh diff --git a/docker_app_server/bundle/.gitkeep b/docker_app_server/bundle/.gitkeep new file mode 100644 index 0000000000..e69de29bb2 diff --git a/docker_app_server/bundle_builder/Dockerfile b/docker_app_server/bundle_builder/Dockerfile new file mode 100644 index 0000000000..7e6affad9a --- /dev/null +++ b/docker_app_server/bundle_builder/Dockerfile @@ -0,0 +1,30 @@ +# +# Copyright (C) 2024 Open Transit Software Foundation +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM tomcat:8.5.98-jdk11-temurin + +ARG OBA_VERSION=2.5.13-otsf +ENV OBA_VERSION_ENV=$OBA_VERSION + +# Start configuring OBA +WORKDIR /oba + +COPY ./build_bundle.sh . + +CMD /oba/build_bundle.sh ${OBA_VERSION_ENV} + +# for debugging +# CMD ["tail", "-f", "/dev/null"] \ No newline at end of file diff --git a/docker_app_server/bundle_builder/build_bundle.sh b/docker_app_server/bundle_builder/build_bundle.sh new file mode 100755 index 0000000000..bda6a6cde3 --- /dev/null +++ b/docker_app_server/bundle_builder/build_bundle.sh @@ -0,0 +1,40 @@ +#!/bin/bash + +# +# Copyright (C) 2024 Open Transit Software Foundation +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +OBA_VERSION="$@" + +# If GTFS_URL is not set or is empty, then use a default value: +RESOLVED_GTFS_URL=${GTFS_URL:-https://unitrans.ucdavis.edu/media/gtfs/Unitrans_GTFS.zip} + +echo "OBA Bundle Builder Starting" +echo "GTFS_URL: $GTFS_URL" +echo "Resolved GTFS_URL: $RESOLVED_GTFS_URL" +echo "OBA Version: $OBA_VERSION" + +wget -O /bundle/gtfs.zip ${RESOLVED_GTFS_URL} + +# The JAR must be executed from within the same directory +# as the bundle, or else some necessary files are not generated. + +cp /root/.m2/repository/org/onebusaway/onebusaway-transit-data-federation-builder/$OBA_VERSION/onebusaway-transit-data-federation-builder-$OBA_VERSION-withAllDependencies.jar /oba/builder.jar + +cd /bundle \ + && java -Xss4m -Xmx3g \ + -jar /oba/builder.jar \ + ./gtfs.zip \ + . diff --git a/docker_app_server/config/context.xml b/docker_app_server/config/context.xml new file mode 100644 index 0000000000..6d7f373b7d --- /dev/null +++ b/docker_app_server/config/context.xml @@ -0,0 +1,41 @@ + + + + + + + + WEB-INF/web.xml + /usr/local/tomcat/conf/web.xml + + + + + diff --git a/docker_app_server/config/host-manager_context.xml b/docker_app_server/config/host-manager_context.xml new file mode 100644 index 0000000000..2348f3c643 --- /dev/null +++ b/docker_app_server/config/host-manager_context.xml @@ -0,0 +1,23 @@ + + + + + + \ No newline at end of file diff --git a/docker_app_server/config/onebusaway-api-webapp-data-sources.xml b/docker_app_server/config/onebusaway-api-webapp-data-sources.xml new file mode 100644 index 0000000000..6450f04b39 --- /dev/null +++ b/docker_app_server/config/onebusaway-api-webapp-data-sources.xml @@ -0,0 +1,115 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + org.onebusaway.agency_metadata.model.AgencyMetadata + org.onebusaway.agency_metadata.service.AgencyMetadataDaoImpl + + + + + org.hibernate.dialect.MySQLDialect + 1 + + org.hibernate.cache.internal.NoCachingRegionFactory + true + update + 1000 + + + + + + + + org.onebusaway.api_webapp.cacheManager + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docker_app_server/config/onebusaway-enterprise-acta-webapp-data-sources.xml b/docker_app_server/config/onebusaway-enterprise-acta-webapp-data-sources.xml new file mode 100644 index 0000000000..d1adb3833c --- /dev/null +++ b/docker_app_server/config/onebusaway-enterprise-acta-webapp-data-sources.xml @@ -0,0 +1,82 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + org.onebusaway.onebusaway-enterprise-acta-webapp.cacheManager + + + + + + + + + + \ No newline at end of file diff --git a/docker_app_server/config/onebusaway-transit-data-federation-webapp-data-sources.xml b/docker_app_server/config/onebusaway-transit-data-federation-webapp-data-sources.xml new file mode 100644 index 0000000000..70e59d2b57 --- /dev/null +++ b/docker_app_server/config/onebusaway-transit-data-federation-webapp-data-sources.xml @@ -0,0 +1,64 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + /bundle + + + + diff --git a/docker_app_server/config/server.xml b/docker_app_server/config/server.xml new file mode 100644 index 0000000000..6b55ef0b2a --- /dev/null +++ b/docker_app_server/config/server.xml @@ -0,0 +1,188 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docker_app_server/config/tomcat-users.xml b/docker_app_server/config/tomcat-users.xml new file mode 100644 index 0000000000..94d007c4f2 --- /dev/null +++ b/docker_app_server/config/tomcat-users.xml @@ -0,0 +1,40 @@ + + + + + + + + + + + + + + + diff --git a/docker_app_server/copy_resources.sh b/docker_app_server/copy_resources.sh new file mode 100644 index 0000000000..90671efea3 --- /dev/null +++ b/docker_app_server/copy_resources.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash + +# +# Copyright (C) 2024 Open Transit Software Foundation +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + + +echo "Copying resources to Tomcat for OBA_VERSION $OBA_VERSION" + +cp /root/.m2/repository/org/onebusaway/onebusaway-enterprise-acta-webapp/$OBA_VERSION/onebusaway-enterprise-acta-webapp-$OBA_VERSION.war /usr/local/tomcat/webapps/onebusaway-enterprise-acta-webapp.war +cp /root/.m2/repository/org/onebusaway/onebusaway-api-webapp/$OBA_VERSION/onebusaway-api-webapp-$OBA_VERSION.war /usr/local/tomcat/webapps/onebusaway-api-webapp.war +cp /root/.m2/repository/org/onebusaway/onebusaway-transit-data-federation-webapp/$OBA_VERSION/onebusaway-transit-data-federation-webapp-$OBA_VERSION.war /usr/local/tomcat/webapps/onebusaway-transit-data-federation-webapp.war + +unzip /usr/local/tomcat/webapps/onebusaway-enterprise-acta-webapp.war -d /usr/local/tomcat/webapps/onebusaway-enterprise-acta-webapp +unzip /usr/local/tomcat/webapps/onebusaway-api-webapp.war -d /usr/local/tomcat/webapps/onebusaway-api-webapp +unzip /usr/local/tomcat/webapps/onebusaway-transit-data-federation-webapp.war -d /usr/local/tomcat/webapps/onebusaway-transit-data-federation-webapp + +cp /oba_config_files/onebusaway-enterprise-acta-webapp-data-sources.xml /usr/local/tomcat/webapps/onebusaway-enterprise-acta-webapp/WEB-INF/classes/data-sources.xml +cp /oba_config_files/onebusaway-api-webapp-data-sources.xml /usr/local/tomcat/webapps/onebusaway-api-webapp/WEB-INF/classes/data-sources.xml +cp /oba_config_files/onebusaway-transit-data-federation-webapp-data-sources.xml /usr/local/tomcat/webapps/onebusaway-transit-data-federation-webapp/WEB-INF/classes/data-sources.xml diff --git a/pom.xml b/pom.xml index 949f41d269..1d18da4f8a 100644 --- a/pom.xml +++ b/pom.xml @@ -553,6 +553,8 @@ 3.0 + **/.gitkeep + **/docker_app_server/bundle/** **/build/** **/data-sources.xml **/*.action