feat: utilization collector #79
Conversation
ahmedElqutb
force-pushed
the
ashraf/utilization-collector
branch
from
4b598ae to
c864ced
Compare
|
sample metrics for the kommander namespace: |
| Memory float64 `json:"memory,omitempty" yaml:"memory,omitempty"` | ||
| } | ||
|
|
||
| type Utilisation struct { |
There was a problem hiding this comment.
nit: since this is an exported struct, I would add a short description. i.e Readings map[string] this is and map of ... which index represents ...
| if err != nil { | ||
| return nil, errors.Wrap(err, "Could not look up thanos-query pods: ") | ||
| } | ||
| if len(pods.Items) < 1 { | ||
| // Check if no thanos-query pods are found. | ||
| return nil, errors.New("Could not find any thanos-query pods.") | ||
| } |
There was a problem hiding this comment.
seems like these two if-statement can be combined?
There was a problem hiding this comment.
one checks for a failed request, the other asserts empty response
|
|
||
| // Query for CPU utilization | ||
| cpuQuery := "sum(rate(container_cpu_usage_seconds_total{container!='POD', container!=''}[1h])) by (namespace)" | ||
| start := time.Now().Add(-1 * time.Duration(c.Collector.Duration) * 24 * time.Hour) |
There was a problem hiding this comment.
what is time.Duration(c.Collector.Duration) * 24 ?
There was a problem hiding this comment.
pkg/collect/utilisation.go
Outdated
|
|
||
| // PortForward creates an asynchronous port-forward to a pod on the cluster in order to | ||
| // access it over the local network. | ||
| func PortForward(cfg *rest.Config, ns, pod string, port int, out output.Output) (localPort int, stop chan struct{}, err error) { |
There was a problem hiding this comment.
these port forwarding things shouldn't be part of this package. what do you think?
pkg/collect/utilisation.go
Outdated
| } | ||
| } | ||
|
|
||
| // Include remaining items from cpu readings |
There was a problem hiding this comment.
| // Include remaining items from cpu readings | |
| // including the remaining items from cpu readings |
| func StartPortForwarding( | ||
| ctx context.Context, | ||
| restConfig *rest.Config, | ||
| out output.Output, |
There was a problem hiding this comment.
This argument is hardcoded to non interactive output, so I would use nomral gaoling log package instead of dkp-cli-runtime as the runtime is built for printing CLI messages.
pkg/collect/utilisation.go
Outdated
|
|
||
| // PortForward creates an asynchronous port-forward to a pod on the cluster in order to | ||
| // access it over the local network. | ||
| func PortForward(cfg *rest.Config, ns, pod string, port int, out output.Output) (localPort int, stop chan struct{}, err error) { |
There was a problem hiding this comment.
Could we control port forwarding lifetime with context cancellation instead of returning back the stop channel?
ahmedElqutb
force-pushed
the
ashraf/utilization-collector
branch
from
3652635 to
67a3ea1
Compare
| // UsageReading the CPU and memory utilisation at a given timestamp | ||
| type UsageReading struct { | ||
| Timestamp time.Time `json:"timestamp,omitempty" yaml:"timestamp,omitempty"` | ||
| CPU float64 `json:"cpu,omitempty" yaml:"cpu,omitempty"` | ||
| Memory float64 `json:"memory,omitempty" yaml:"memory,omitempty"` | ||
| } |
There was a problem hiding this comment.
Have you checked if there is a some known data format for storing metrics? Maybe from prometheus itself. It would be great for interoperability and further data processing.
ahmedElqutb
force-pushed
the
ashraf/utilization-collector
branch
from
67a3ea1 to
b9991f7
Compare
Here we introduce a simple prometheus metrics collector that collects, for now, cpu and memory utilisation per namespace;
the metrics are normalized over a period of 1h, cpu usage is measured as
sum(rate(container_cpu_usage_seconds_total{container!='POD', container!=''}[1h])) by (namespace) and memory usage is measured assum(container_memory_working_set_bytes{container!='POD', container!=''}) by (namespace)`. The collector can be easily expanded to include other metrics if needed.the output is stored in the bundle as follows
and the format is as follows