Datatron

This Go application automatically dumps a MySQL database using mydumper based on a configurable cron schedule.

Features

• Cron-based Scheduling: Schedule database dumps using standard cron expressions.
• mydumper Integration: Leverages mydumper for efficient and reliable MySQL backups.
• Dockerized: Easily deployable using Docker and Docker Compose.

Prerequisites

• Docker and Docker Compose installed.
• A mydumper.cnf file with your MySQL connection details and dump options.

Usage

1. Place mydumper.cnf: Ensure your mydumper.cnf file is in the root directory of this project. This file will be mounted into the Docker container.

   Example mydumper.cnf:

   [client]
   user=root
   password=root
   host=mysql
   port=3306
   
   [mydumper]
   database=testdb
   outputdir=/dump
   compress-protocol

2. Configure Cron Schedule: Set the DUMP_CRON_SCHEDULE environment variable in your docker-compose.yml file. This variable accepts a standard cron expression.

   Example docker-compose.yml snippet:

   datatron-service:
     build: .
     volumes:
       - ./mydumper.cnf:/mydumper.cnf:ro
       - ./dumps:/dump
     environment:
       DUMP_CRON_SCHEDULE: "0 2 * * *" # Dumps every day at 2 AM
     depends_on:
       - mysql

   For testing, you can use "* * * * *" to dump every minute.

3. Run with Docker Compose: Navigate to the project root directory in your terminal and run:

   docker-compose up --build -d

   This will build the datatron-service image, start the MySQL container (if not already running), and then start the datatron-service which will begin scheduling dumps.

4. Verify Dumps: Database dumps will be saved in the ./dumps directory on your host machine. You can check the logs of the datatron-service to see the execution status:

   docker-compose logs -f datatron-service

Development

To build and run the Go application locally (without Docker):

1. Install Go: Ensure you have Go (version 1.16 or higher) installed.
2. Install Dependencies:

   go mod tidy

3. Build the Application:

   go build -o mydumper-app .

4. Run the Application: You need to set the DUMP_CRON_SCHEDULE environment variable and ensure mydumper.cnf is accessible at /mydumper.cnf (or modify the Go code to look for it elsewhere).

   DUMP_CRON_SCHEDULE="* * * * *" ./mydumper-app

   Note: Running locally requires mydumper to be installed on your host system and a MySQL instance accessible from your host.