Cloud Application Development - ToDo's

[BenjaminBruenau]

TL:DR

Things we need to implement to pass the course and things we should implement to get a good grade

Overview

Todo's describing what needs to be done for successfully completing the Cloud Application Development Course.
This includes both the hard requirements for completing the course aswell as additional tasks to boost the possibility of completing with a better grade.
Tasks are structured into Implementation Features (functionality of the application itself) and Architecture / DevOps Tasks (how the application is structured / deployed).

Minimal Requirements

• Running on standard cloud platform
• Provides Software-as-a-Service (SaaS)
• Multi-User with authentication & authorization
• Includes five essential cloud service characteristics
  • On-demand self service
  • Broad network access (application is available over networks - accessible by different devices, e.g. Laptop, Phone)
  • resource pooling
  • rapid elasticity (resources can be claimed and freed elastically/automatically)
  • measured service (ressources are controlled and optimized automatically by the cloud system, depending on usage)
• WebUI & API
• Documentation
  • Tick 12 Factor boxes, how do we satisfy them? how not? why?

Implementation Features

• Query Builder - Backend
• Query Builder - Frontend
• Data Transformer
• Data Transformer UI (Mapping one data structure to another)
• Charts to visualize geoJson data from Data Sources (Dashboard)
• Charts to visualize Analysis Data (Dashboard)
• Analysis Service (Creating Analysis Report by aggregating Data and performing e.g. Predictions)
• Tenant Management (Project Creation, Access Control & Tenant Isolation)

Architecture / DevOps Tasks

• Cloud Native Application (how applications are created & deployed, not where) - supporting 12 Factors

  • I. Codebase
    • one codebase in git
    • factor shared code into library
  • II. Dependencies
    • dependency manager takes care of dependencies (e.g. npm/yarn or sbt)
  • III. Config
    • separate config from code
    • configuration stored in env variables (independently managed for each deployment)
  • IV. Backing services
    • backing services are treated as attached resources (can be detached and attached at will)
    • no distinction between local and external (third party, e.g. doesnt matter if local mongoDB or external service like Atlas)
  • V. Build, release, run
    • strictly separate build and run stages
    • Build Stage: convert code repo into an executable bundle (build)
    • Release Stage: combine build with the current deployment config (combination ready for execution)
    • Run Stage: run app in the execution environment
  • VI. Processes
    • stateless processes that do not share same memory / storage
  • VIII. Concurrency
    • different process types (e.g. worker process or web process), one or more processes per type
    • multi threading internally in a process still allowed, but scaling is done via the process model -> application must be able to span multiple processes running on multiple physical machines
    • processes should be managed by a process manager
  • IX. Disposability
    • maximize robustness with fast startup and graceful shutdown
    • processes can be started or stopped at moments notice
    • recover fast (on failure)
  • X. Dev/prod parity
    • keep development, staging, and production as similar as possible
    • continuous deployment -> keeping gap between development and production small
    • same backing services (e.g. Databases) in prod and dev
  • XI. Logs
    • treat logs as event streams -> stream of aggregated, time-ordered events collected from the output streams of all running processes and backing services
    • provide visibility into the behavior of the running app
    • treat log files as a ressource, managed by the cloud platform
    • centralized log stash
  • XII. Admin processes
    • run admin/management tasks as one-off processes (e.g. database migration)
    • should be run in an identical environment as the regular long-running processes -> e.g. migration scripts run in pipeline
    • treat databases as applications (never do changes manually)
  • 2. API first - Beyond 12F
    • building an API to be consumed by client applications and services
    • we e.g. provide an API to stream datasources, API to generate analysis reports, etc.
  • 14. Telemetry - Beyond 12F
    • take measurements of the application and transmit them elsewhere
    • Monitor: Application Performance, Domain-specific telemetry, Health and system logs
    • e.g. how many users are visualizing data, how many customers (how many cancel their plans)
  • 15 Authentication and Authorization - Beyond 12F
    • secure endpoints with RBAC (role-based access control)
    • every request for application resources should include who is making the request aswell as the roles the consumer belongs to

• Micro-Service Architecture

• Application deployed to Kubernetes (+ using advanced Kubernetes concepts)

• long-running asynchronous workloads (worker jobs) -> analysis service

• multi-tenancy capabilities and tenant management

• collection & visualization of application specific telemetry data (health, usage, payment)

• customization capabilities for tenants

• Terraform to setup and update the cloud infrastructure automatically

• CI/CD process for the application

• Security of the application (no unauthorized access to the application or its infrastructure)

• cost calculation model of the running costs of the application

• commercial model for the services provided by the cloud

Bonus

• Managed DB Cluster inside Kubernetes (+ Replication)
• Kubernetes Parallel Jobs (-> e.g. for Spark)
• (Multi-Region Deployment)