Description

Extracted from an internal discussion document:

We can solve many of the challenges with deploying Agent on Kubernetes through a Helm chart for Elastic Agent. This allows us to improve the current experience by (mapped to priorities above that are not already met and easy):

• Deploying separate cluster-level and node-level Agents
  • This allows us to eliminate the leader-election mechanism and the associated K8s API calls. These API calls currently scale horizontally with the number of Kubernetes nodes in the cluster.
  • This also allows separate resource planning and scaling for cluster-level and node-level Agents, which need different amount of resources depending on the horizontal and vertical scale of the cluster and its nodes, respectively.
• Provide “templates” or “presets” for common integrations
  • This moves us away from our current “one-size-fits-all” k8s manifest where users have to comment/uncomment certain sections based on which integrations they want to use
  • Along with this, we can configure the minimum security privileges needed for each integration’s container and deploy separate containers for integrations that require higher-privileges
  • Examples of presets we would provide:
    • P1 A containers.logs: true option would create a DaemonSet agent for container log collection
    • P1 A deployment.metrics: true option would create a ReplicaSet agent deployed with KSM bundled in the same pod. We’d also have similar options for Services and other workload types, all which would enable inputs on the same ReplicaSet
    • P1 An autodiscovery.metrics: true option would mount the required templates for hints-based autodiscovery
    • P2 A cloudDefend.enabled: true option would add the necessary container privileges and inputs to the DaemonSet agents for Cloud Defend
    • P2 A profiling.enabled: true option would add the necessary container privileges and inputs to the DaemonSet agents for Universal Profiling
    • P2 A autoInstrumentation.enabled: true option would deploy the apm-attacher hook as a Deployment for auto instrumenting application pods with APM Agents.
• P3 Simpler sharding of kube-state-metrics (KSM)
  • For any workload metrics that depend on kube-state-metrics, we could pseudo-automate sharding by adding a simple config like ksm.shards: 2 which would shard the ReplicaSet used for the agent collecting this data, along with KSM container in the same pod.