diff --git a/README.md b/README.md index b8c7c77..d493dcb 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,122 @@ -# Galileo Client +# Galileo Experiments + +This repository contains code that invokes the galileo shell to start experiments. +It also includes deployment files for necessary components and describes in detail which other services are +required to start an experiment. +Further, this project aims to unify different sub-projects of `edgerun` and make deployment and experiment setup easy. +The main goals and functionalities are: + +* A framework for distributed load testing experiments +* Fine-grained telemetry data collection +* HTTP Trace recording for any service +* A container orchestration adaption for ease of use + +## Who is this project for? + +For everyone that wants to see resource usage and application performance with easy configurable workload creation. +All components are tailored and suited to run on low performance devices (i.e., Raspberry Pi) but can run on default server VMs too. + +Common questions that can be answered by performing and analyzing Galileo experiments: + +* How much CPU usage does my application use? +* What is the average execution time of my application across the cluster? +* What are the differences in terms of resource usage between two nodes hosting the same application? +* What is the impact of having multiple applications running on one node? + +All these questions can be easily answered and have a simple flow in common: +1. Deploy base infrastructure +2. Deploy my application +3. Start requests +4. Analyze in Jupyter Notebooks + +In summary: simple profiling tasks. +But, this framework also targets full end-to-end tests to evaluate important cluster components (i.e, load balancer, scheduler and scaling). + +Therefore, experiments can be done to evaluate new implementations for the aforementioned components. + + +# High level overview + + + +# Kubernetes cluster setup + +![Cluster components](figures/cluster.drawio.png) + +## Overview + +### Main components +The cluster setup consists of the following main components: + +* Kubernetes +* Galileo (for clients and experiment shell) +* Telemd +* Controller (i.e., the load balancer) +* MySQL (i.e., MariaDB) +* InfluxDB v2 +* Etcd (Kubernetes requires an instance to run!) + +Kubernetes is used to host the clients (galileo running in a Pod), telemetry agents (telemd), load balancer and the applications to test. +Redis is used as a pub/sub system through which all data is sent (i.e., telemetry) and recorded by the Galileo Shell (i.e., the program that prepares and executes an experiment). +The Galileo Shell persists data in MariaDB and InfluxDB. +The provided load balancer implementation uses etcd to watch for weights for the round-robin algorithm and galileo uses redis to provide the clients with routing rules (`rtbl`). + + + +## Main interactions + +The figure above depicts all components and also highlights important interactions. +Those interactions are in short: +* Client nodes send HTTP requests to the Controller (load balancer), which forwards requests to the worker nodes which host the application pods. +* Client nodes report the results of each request (i.e., trace) via Redis. +* The Go-based load balancer implementation fetches weights and ip addresses from the etcd instance. +* The clients get the routing rules from the Redis instance (set via `rtbl` from Galileo) +* Worker nodes report resource usage (i.e., telemd) via Redis, which is saved in InfluxDB +* The Galileo Shell starts the experiments and saves metadata (i.e., the cluster hosts, misc. data) in the MariaDB + + +## Deployment + +This project provides deployment files for the following components: +* Galileo (for clients and experiment shell) +* Telemd +* Controller (i.e., the load balancer) + +Which leaves the following components to be additionally deployed: + +* Kubernetes +* MySQL (i.e., MariaDB) +* InfluxDB v2 +* Etcd (Kubernetes requires an instance to run!) + +Deployment files can be found in `deployment/kubernetes`. +Note, that we use Kubernetes node labels to schedule the workers (i.e., clients). +On nodes that should act as clients execute the following command: + + kubectl label node node-role.kubernetes.io/client=true + +The following label is used to identify nodes with hosting capabilities (i.e., workers): + + kubectl label node node-role.kubernetes.io/worker=true + +If you have multiple zones (i.e., clusters) in which you want to have seperate clients, adapt the `zone` arguments (default is `main`). +You can easily group your nodes by labelling it with the following command: + + kubectl label node ether.edgerun.io/zone=main + +## Galileo Workers + + + +# Data storage + +Galileo requires the following data components that are either deployed in the cluster or externally: + +* Redis (pub/sub for telemetry and traces) +* MySQL (i.e., MariaDB) (persistent storage for experiment metadata) +* InfluxDB v2 (stores runtime data - telemetry and traces) + -This repo contains code that invokes the galileo shell to start experiments. Environment variables ===================== diff --git a/figures/cluster.drawio b/figures/cluster.drawio new file mode 100644 index 0000000..50948f3 --- /dev/null +++ b/figures/cluster.drawio @@ -0,0 +1 @@ +7V1bd5s4EP41Pqf7EB8Q98fGSS+77ban6dluH2WQbRqCvCA3dn/9SiAwCNkmDjenuA+FkRCg75sZaTQiE232sH0bwfXqI/ZQMAGKt51oNxMAgKMC+h+T7FKJqupOKllGvsdle8Gd/wtxocKlG99DcakiwTgg/rosdHEYIpeUZDCK8GO52gIH5buu4RJVBHcuDKrSb75HVqnUNpS9/B3yl6vszqrCSx5gVpkL4hX08GNBpN1OtFmEMUmPHrYzFLDey/olve7NgdL8wSIUkjoXbH1/45Mfd/if24X1489daD1ur0wjbeYnDDb8jfnTkl3WBRHehB5irSgT7fpx5RN0t4YuK32kqFPZijwE9EylhwscEo6iqrNzPwhmOMBR0pa2MNg/Xq8gT39UHpMI36NCiZn8aAl/UBQRtD3YBWresZSSCD8gEu1oFX6BmcHG2ahl5497aDWLy1YFWIHDhZDTaZm3ve9xesA7/QkAZC9R6G/kUQbyUxyRFV7iEAa3e+l1GZF9nQ8YrzkOPxAhOw4E3BBcRikmMCKvmXpQgRvAOPbdTPzGD/bVGBgZ70EOAnvC4xDQF8KbyEVH3lzn2gyjJSJH6llySCMUQOL/LD9H4/Bodh/w0L6Mdv8WT76zxqZGdnqz5Y2nZzt+5sF4ldz30jBWewVZl9hAMyCsQ/2f9HDJDmeBz16fF9D7FMok1V8tYeAHCP+Rlc0jsbbYhsCzE4YWxuvU3y38LcNctLQeRPbClVpU10bzRWar+Q3VBsyrqpfNq6VWzastsa52a8ZV71V7Cwr7vVh2QHuHoLDGJRhl2ZilcX19cdoJtKFppzFq59O007wE7TRH7TxDOzUwMO1UzX6182kj37O1szRkblBVrbojX6NPXbVq6SqleoSDAEX19fUDhh5t+BoGMHRR1N8o2EC2p8s02QZzLYkrNO5nVaOkyQaoarKqSlQ5FzbvaXvW5bKnVdrQZa1Z9bU78rTJpfRd4a5QYY39kMSFlj8zwZ5hwjRLt4TAn1AdWEfr04P0Afb8yt/kGYGTXqZeg3MDdXn07ADIWUS6EqYEmnOcSWJ93baP1j9evR3i2XVcWu6OMsE3HN2X/Nv8lJ863ebf2ENH/d7JezTrCxeLBXClo1rPnJvGc2PsR2Lq3U4rrYuxPA07rWz8MEyvpYrWYIhuK+vC0Xx0bT74BY7gkfq2Jqo6EqJXQqj20BgBRkb0ygjRMfTPCG1kRK+M0MyhMaLWkvLIiNYYodfI9OmWEYcXLXMAEHE9OTACBE/LyCr3P1I9A1my/ndMS4NN9b8QozJ0o9L/KpAAYLYGwOFYd97RX5Dnxy8EATFWMwAEDodm8o5+Hy6CzfbmgH26PBDyPNbhoOCcRuEjjHz4ckEwtaoz6BYErZfUh8GlB2Z9PvD8QK2X5bPDaIU4RAJUynCgcnqFqpfgbwNQodCrKiMVFlSxFzTtZ6J53rpUJVfNOLEwJV5gWIbApRaCxVkvHp/lvU1Tl2rP4O5Wya3PzJd4+maNPPVBMlVD5oGpmuXMFaWdgaoly5fo1juDPozImC5R05qczIfQBGKkj88vOmJFxMBinrmTNZS+X6WhF5m/IZsmiEoh81kFRpdVQnCS545Vquwp2AlDYiYy2TNJZorkEENFdVlmmScaOsCyppDVq17rBhJIJV8jGMYLFoBUXvlTNJ2AGVNqFCDeN8lpRL1J/EeFDNQBECESGPjLkHGDAkgb1a6Zm/BdGLzmBQ++56W2E8X+LzhPmmJU4QpE2zWuJ8YNa4uay5hvK6w4JD7IKnqvTCQ6OGF3YgPuyzKFdD+r6r5yV1XaU9iW+9JB06p7+dpqir19rrZWGmpbW6trS+++fv1MJV/QfxsUMz+jvGILBwGG3qiWOW5OWS1NST49UDpVy/PWhBJNvIrw3A+bXRh6TF46PjrTeOra0HDZ1iy7NE0w+pKIoiMhl94auWosL6Weu1ZQ93fBUa0RGe7Wdx/ePbXHcT8gG6HchxFELHUJlrIl3/awrLHeFRMcIVrHS0bg9SzxbwuwkBuYD816A/jwUtoxxxtv5rEb+XPUrD8n+Ik8Gj16kV1CwohZM2GkNY+e3f+kJ/BqWYl4Bdfs0N3MWQ+eCAzP0+nfh3kugO79MhmKftqQwA8zxDwY3X+izfgkCUdOFaMsBIm0xoeB1DlUEZhUPgy0SH4VbtASRTFvX79hJbzVhuLRhiL4EUk8WjYmEKOOzTHhcAryyIQ2maCLI4remXB4CjkyoU0miLvz+2dCnfneyITmmVDZ09g7E+rOGEcmNMsE8WtX/TOhRobryIRWRowDI0KNwMM7HLNVjvdsJYH2D1D+ojBHISIsQqh8xh4vHEMSR3ciCYsMehV5IPvUR2shCaNGdu+I9POR1u0q0rlD6ATpzKJUl/FZTJF91VjE6nnf8T2eKtZNJrct7g6WRAClqWKtmVqzOjdPN3HFnXZ/N5uqxO7Xld67H1S6v/i1qg4RqPGhqTYQkDicjhGo5iek3/brVgFqfLGvje6XeYFuu78aEcrzjRWeSdypHeoiY1iEQfqFtW5hqDEJ/4CWKKw39fpdRlWV7/FL/Ln0c/yt4SibQovz4ntE3BXvXdkmkFyt4rJaCboCbMu4BTJdyWewZeyu6WSZzpbBTJkyqztTGJxglhSoiRgIUkcqTZoQazoHGraSq2m5pBFVkAFbqJvQLYsjPGyX7G+ETO/z6cfUd3HIXjN6nxzcrLHXDtGuhEi+lqX4FydqzjTbI1zimt0W12Sz9JFrF881W5na2uDYZlWniyPbXgDb6Dhnml2V5cVIMtA1Z5pFBUsDotboJluzHul28XQDwkLoILhWDQKMXHsBXNPUAXJN9qmvkWsXzzWg0KdTnP3PLlFPtnukOerR0/2fAUw3Du3/mqJ2+z8= \ No newline at end of file diff --git a/figures/cluster.drawio.png b/figures/cluster.drawio.png new file mode 100644 index 0000000..0d173f2 Binary files /dev/null and b/figures/cluster.drawio.png differ