Microservices Suite 🦧

Overview

Welcome to the 📦 Microservices Suite project! This suite is a collection of Node.js microservices built using the 🦧 mono-repo strategy and leveraging the yarn workspaces concept. Each microservice runs in its isolated Docker container, and Kubernetes orchestrates the deployment, providing scalability and efficiency.

Project file structure

├─ node-microservices-suite
│  ├─ api-gateways/
|  │  ├─ app-1/
|  │  │  ├─ nginx
|  │  │  ├─ apache
|  │  │  ├─ README.md
│  ├─ graphql/
|  │  ├─ app-1/
|  │  │  ├─ apollo-server # placeholder file atm
|  │  │  ├─ README.md
│  ├─ data-persistence/
|  │  ├─ db-1/
|  │  │  ├─ Dockerfile
|  │  │  ├─ README.md
|  │  ├─ sqlite/ #for in memory or disk db for miniature devices or prototyping
|  │  ├─ ├─ db/
|  │  │  ├─ README.md
│  ├─ k8s/
|  │  ├─ service-1/
|  │  │  ├─ cluster-ip-service.yml
|  │  │  ├─ cluster-deployment.yml
|  │  │  ├─ ingress-service.yml
|  │  │  ├─ README.md
|  ├─ microservices/
|  │  ├─ service-1/
|  │  │  ├─ src
|  │  │  ├─ .env
|  │  │  ├─ .env.dev
|  │  │  ├─ app.js
|  │  │  ├─ Dockerfile
├─ │  │  ├─ Dockerfile.dev
|  │  │  ├─ ecosystem.config.js
|  │  │  ├─ index.js
|  │  │  ├─ package.json
|  │  │  ├─ task.json
|  ├─ shared/
|  │  ├─ library-1/
|  │  │  ├─ APIError.js
|  │  │  ├─ catchAsync.js
|  │  │  ├─ index.js
|  │  │  ├─ package.json
|  │  │  ├─ pick.js
|  │  │  ├─ README.md
|  │  │  ├─ validate
│  ├─ tests/
|  │  ├─ service-1/
|  │  │  ├─ e2e/
|  │  │  ├─ integration/
|  │  │  ├─ snapshot/ #if it's a micro-frontend service
|  │  │  ├─ unit/
|  │  │  ├─ README.md
|  ├─ .gitignore
|  ├─ .npmrc
|  ├─ .yarnrc.yml
|  ├─ docker-compose.yml
|  ├─ package.json
|  ├─ production.yml
|  ├─ README.md
|  ├─ Taskfile.yml
|  ├─ yarn.lock

Monorepo strategy benefits for microservices:

• Enforce DRY Principles:

  • @Japheth Obala the Prophet 😎 has done a sleek job demystifying 👉SOLID & best practice here.
  • Collocating code encourages code sharing.
  • Reduce duplication 👉read about SOLID here
  • Save time by building on existing boilerplate or reusable functionality.

• Collaboration:

  • Working within the same repository facilitates learning from peers.
  • Enables the adoption of good coding practices and the avoidance of bad ones.

• Centralized Development Chores:

  • Manage common files like .gitignore centrally from the root directory
  • Reduce unnecessary repetitions in the codebase.

• Easily Integrate Development Automation:

  • Task runner configurations and docker-compose can be managed from the root directory.
  • simplify the automation of repetitive workflows.

• Code Sharing Anywhere:

  • Publish and import organization-scoped libraries to the npm registry with

    yarn publish 
    yarn add <@microservices-suite/foo> or
    yarn @microservices-suite/<workspace-name> add <@microservices-suite/foo>

• Easily Containerize and Scale:
  • Decouple every microservice to scale individually.
  • Leverage the no-hoist yarn workspace feature and custom scripts to enable efficient packaging of microservices into isolated containers.

Technologies Used

• Yarn Workspaces:

  • Simplifies managing multiple packages within a single repository.
  • Encourage code sharing & reduce duplication
  • Make it easier to handle dependencies and scripts.

• Docker Containers:

  • Provides lightweight, portable, and self-sufficient containers for packaging and deploying microservices.

Alpine Images

• Alpine images are very minimalistic Linux minidistros that cost you only ~5MB of real estate. That is why they are used inside your favourite smart watch ⌚️.
• One objective of this project is to optimize image builds for Continuous Integration (CI) and production environments through the utilization of the slimmest possible images so that you dont bloat ⚠️ your machine in dev and we have a lean server in prod.

Key Strategies:

• Multi-Stage Builds:

  • We employ simple Docker constructs like multi-stage builds to optimize our image size.

• Hoisting and Symlinking:

  • While these practices promote the DRY (Don't Repeat Yourself) principle, they pose a threat to our objective of achieving minimized images. To mitigate this, we leverage no-hoisting and symlinked libraries
  • selectively copying only the node_modules generated by non-hoisted workspaces.
  • These are optimized to consume less disk space, ensuring that only the essential shared libraries required by a specific service are included, thereby maintaining the slim profile of our images.
  • The service itself will build her core node modules normally inside the dockerfile from her package.json

Docker & Node Best Practices

• For an in-depth guide and best practices on using Docker with Node.js, visit the Docker & Node Best Practices repository.

Kubernetes (k8s)

• Kubernetes offers automated deployment, scaling, and management of containerized applications, ensuring reliability and scalability.

Getting Started

Welcome to our project! To ensure a smooth setup and development experience, ensure you have the following tools installed on your machine:

• Docker:
  • For containerization and managing containerized applications.
  • 👉 Install docker here.
  • We love 💚 alpine images <small,simple,secure>.
  • You can read about the specific flavor here
• Task Runner Automation Tool:
  • For task automation and workflow management.
  • 👉 Install task runner here.
• Node.js:
  • As the runtime environment for executing the application code.
  • 👉 Download LTS version here
• At the project <service_root> create .env, .env.dev and .env.staging files and copy environment variables from the .env.example file

Running Services

• This project uses Task Runner Automation Tool to streamline the process of starting services in both development and production. Follow these steps to get your environment up and running:
• You can derive the service_name of a service from the workspace name found in the package.json "name": property e.g

"name": "@microservices-suite/<service_name>"

• To run a service in either modes [dev,staging,prod]:
• Action is similar to [up,down] docker compose syntax where up & down start and stop the service respectively with docker-compose <action>

task do:<service_name>:<mode>:<action>

• This command uses docker-compose to start your service(s)
• The task runner will handle the setup, ensuring your service is ready.

Running Services Without Docker

• If you prefer not to use Docker, you can use the vanilla task command to start services using node PM2 engine in production or nodemon in any other mode:

task vanilla:<service_name>:<mode>

Using Docker-Compose Directly

• Should you need to use docker-compose directly for more control over the container orchestration, you can utilize the standard commands provided by Docker:

docker-compose up
docker-compose down

Contributing

Contributions are welcome! If you'd like to contribute to the Microservices Suite project, please follow these guidelines:

1. Fork the repository and clone it to your local machine.

git clone https://github.com/microservices-suite/node-microservices-suite.git

2. Create a new branch for your feature or bug fix:

git checkout -b feat/<my-feature>

3. Make your changes and make sure that tests pass.
4. Commit your changes using the Angular commit message convention:
   • For more details, please refer to the Angular commit message convention.
5. Push to the branch:

git push origin feat/<my-feature>

6. Submit a pull request detailing your changes.
7. Please ensure that your pull request adheres to the project's code style and conventions.

License

• This project is licensed under the MIT License. Feel free to use, modify, and distribute this code for any purpose.

Acknowledgements

We would like to thank the developers and contributors to the following technologies used in this project:

• Yarn
  • Yarn workspace.
  • Also checkout yarn workspaces
  • No-hoist
• ⚓️ Docker
  • ⚓️vanilla docker (CLI)
  • docker-compose (manage a multi-container application)
• 🎡 Kubernetes
  • minikube (local kubernetes for development only!)
  • k8s
  • k8s in the cloud,GKE,EKS and more...
• Curated Resouces(by gilbertandanje@gmail.com)
  • 👉 Read Book by Sam Newman
  • 🛳️ containerization, orchestration & CICD
  • 📦 microservices
• 🦧 Monorepos
  • 👉 Monorepo is not code collocation
  • 👉 Read Medium article here
• For design patterns,best practice and DSA checkout this sheet on the Resources sheet