Skip to content

Commit

Permalink
feat: add support for Graphite metrics provider (#1406)
Browse files Browse the repository at this point in the history
Signed-off-by: Mike Ball <mikedball@gmail.com>
  • Loading branch information
mdb authored Sep 20, 2021
1 parent d9ba36a commit 7c54090
Show file tree
Hide file tree
Showing 19 changed files with 3,401 additions and 489 deletions.
16 changes: 8 additions & 8 deletions docs/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ go get -u github.com/golangci/golangci-lint/cmd/golangci-lint
```

Brew users can quickly install the lot:

```bash
brew install go kubectl kustomize golangci-lint protobuf swagger-codegen
```
Expand Down Expand Up @@ -76,7 +76,7 @@ make test
## Running E2E tests

The end-to-end tests need to run against a kubernetes cluster with the Argo Rollouts controller
running. The rollout controller can be started with the command:
running. The rollout controller can be started with the command:

```
make start-e2e
Expand Down Expand Up @@ -106,8 +106,8 @@ make test-e2e E2E_TEST_OPTIONS="-testify.m ^TestRolloutRestart$"

3. The e2e tests are designed to run as quickly as possible, eliminating readiness and termination
delays. However, it is often desired to artificially slow down the tests for debugging purposes,
as well as to understand what the test is doing. To delay startup and termination of pods, set the
`E2E_POD_DELAY` to an integer value in seconds. This environment variable is often coupled with
as well as to understand what the test is doing. To delay startup and termination of pods, set the
`E2E_POD_DELAY` to an integer value in seconds. This environment variable is often coupled with
`E2E_TEST_OPTIONS` to debug and slow down a specific test.

```shell
Expand Down Expand Up @@ -176,14 +176,14 @@ kubectl -n argo-rollouts apply -f manifests/install.yaml
```

## Upgrading Kubernetes Libraries
Argo Rollouts has a dependency on the kubernetes/kubernetes repo for some of the functionality that has not been
pushed into the other kubernetes repositories yet. In order to import the kubernetes/kubernetes repo, all of the
associated repos have to pinned to the correct version specified by the kubernetes/kubernetes release. The
Argo Rollouts has a dependency on the kubernetes/kubernetes repo for some of the functionality that has not been
pushed into the other kubernetes repositories yet. In order to import the kubernetes/kubernetes repo, all of the
associated repos have to pinned to the correct version specified by the kubernetes/kubernetes release. The
`./hack/update-k8s-dependencies.sh` updates all the dependencies to the those correct versions.

## Documentation Changes

Modify contents in `docs/` directory.
Modify contents in `docs/` directory.

Preview changes in your browser by visiting http://localhost:8000 after running:

Expand Down
35 changes: 35 additions & 0 deletions docs/analysis/graphite.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Graphite Metrics

A [Graphite](https://graphiteapp.org/) query can be used to obtain measurements for analysis.

```yaml
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: success-rate
spec:
args:
- name: service-name
metrics:
- name: success-rate
interval: 5m
# Note that the Argo Rollouts Graphite metrics provider returns results as an array of float64s with 6 decimal places.
successCondition: results[0] >= 90.000000
failureLimit: 3
provider:
graphite:
address: http://graphite.example.com:9090
query: |
target=summarize(
asPercent(
sumSeries(
stats.timers.httpServerRequests.app.{{args.service-name}}.exception.*.method.*.outcome.{CLIENT_ERROR,INFORMATIONAL,REDIRECTION,SUCCESS}.status.*.uri.*.count
),
sumSeries(
stats.timers.httpServerRequests.app.{{args.service-name}}.exception.*.method.*.outcome.*.status.*.uri.*.count
)
),
'5min',
'avg'
)
```
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ For these reasons, in large scale high-volume production environments, a rolling
* Customizable metric queries and analysis of business KPIs
* Ingress controller integration: NGINX, ALB
* Service Mesh integration: Istio, Linkerd, SMI
* Metric provider integration: Prometheus, Wavefront, Kayenta, Web, Kubernetes Jobs, Datadog, New Relic
* Metric provider integration: Prometheus, Wavefront, Kayenta, Web, Kubernetes Jobs, Datadog, New Relic, Graphite

## Quick Start

Expand Down
7 changes: 7 additions & 0 deletions manifests/crds/analysis-run-crd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,13 @@ spec:
required:
- query
type: object
graphite:
properties:
address:
type: string
query:
type: string
type: object
job:
properties:
metadata:
Expand Down
7 changes: 7 additions & 0 deletions manifests/crds/analysis-template-crd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,13 @@ spec:
required:
- query
type: object
graphite:
properties:
address:
type: string
query:
type: string
type: object
job:
properties:
metadata:
Expand Down
7 changes: 7 additions & 0 deletions manifests/crds/cluster-analysis-template-crd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,13 @@ spec:
required:
- query
type: object
graphite:
properties:
address:
type: string
query:
type: string
type: object
job:
properties:
metadata:
Expand Down
21 changes: 21 additions & 0 deletions manifests/install.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,13 @@ spec:
required:
- query
type: object
graphite:
properties:
address:
type: string
query:
type: string
type: object
job:
properties:
metadata:
Expand Down Expand Up @@ -2711,6 +2718,13 @@ spec:
required:
- query
type: object
graphite:
properties:
address:
type: string
query:
type: string
type: object
job:
properties:
metadata:
Expand Down Expand Up @@ -5189,6 +5203,13 @@ spec:
required:
- query
type: object
graphite:
properties:
address:
type: string
query:
type: string
type: object
job:
properties:
metadata:
Expand Down
21 changes: 21 additions & 0 deletions manifests/namespace-install.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,13 @@ spec:
required:
- query
type: object
graphite:
properties:
address:
type: string
query:
type: string
type: object
job:
properties:
metadata:
Expand Down Expand Up @@ -2711,6 +2718,13 @@ spec:
required:
- query
type: object
graphite:
properties:
address:
type: string
query:
type: string
type: object
job:
properties:
metadata:
Expand Down Expand Up @@ -5189,6 +5203,13 @@ spec:
required:
- query
type: object
graphite:
properties:
address:
type: string
query:
type: string
type: object
job:
properties:
metadata:
Expand Down
142 changes: 142 additions & 0 deletions metricproviders/graphite/api.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
package graphite

import (
"encoding/json"
"fmt"
"io/ioutil"
"math"
"net/http"
"net/url"
"path"
"regexp"
"strconv"
"time"

log "github.com/sirupsen/logrus"
)

// API represents a Graphite API client
type API interface {
Query(query string) ([]dataPoint, error)
}

// APIClient is a Graphite API client
type APIClient struct {
url url.URL
client *http.Client
logCTX log.Entry
}

// Query performs a Graphite API query with the query it's passed
func (api APIClient) Query(quer string) ([]dataPoint, error) {
query := api.trimQuery(quer)
u, err := url.Parse(fmt.Sprintf("./render?%s", query))
if err != nil {
return []dataPoint{}, err
}

q := u.Query()
q.Set("format", "json")
u.RawQuery = q.Encode()

u.Path = path.Join(api.url.Path, u.Path)
u = api.url.ResolveReference(u)

req, err := http.NewRequest("GET", u.String(), nil)
if err != nil {
return []dataPoint{}, err
}

r, err := api.client.Do(req)
if err != nil {
return []dataPoint{}, err
}
defer r.Body.Close()

b, err := ioutil.ReadAll(r.Body)
if err != nil {
return []dataPoint{}, err
}

if 400 <= r.StatusCode {
return []dataPoint{}, fmt.Errorf("error response: %s", string(b))
}

var result graphiteResponse
err = json.Unmarshal(b, &result)
if err != nil {
return []dataPoint{}, err
}

return result[0].DataPoints, nil
}

func (api APIClient) trimQuery(q string) string {
space := regexp.MustCompile(`\s+`)
return space.ReplaceAllString(q, " ")
}

type dataPoint struct {
Value *float64
TimeStamp time.Time
}

func (gdp *dataPoint) UnmarshalJSON(data []byte) error {
var v []interface{}
if err := json.Unmarshal(data, &v); err != nil {
return err
}

if len(v) != 2 {
return fmt.Errorf("error unmarshaling data point: %v", v)
}

switch v[0].(type) {
case nil:
// no value
case float64:
f, _ := v[0].(float64)
gdp.Value = &f
case string:
f, err := strconv.ParseFloat(v[0].(string), 64)
if err != nil {
return err
}
gdp.Value = &f
default:
f, ok := v[0].(float64)
if !ok {
return fmt.Errorf("error unmarshaling value: %v", v[0])
}
gdp.Value = &f
}

switch v[1].(type) {
case nil:
// no value
case float64:
ts := int64(math.Round(v[1].(float64)))
gdp.TimeStamp = time.Unix(ts, 0)
case string:
ts, err := strconv.ParseInt(v[1].(string), 10, 64)
if err != nil {
return err
}
gdp.TimeStamp = time.Unix(ts, 0)
default:
ts, ok := v[1].(int64)
if !ok {
return fmt.Errorf("error unmarshaling timestamp: %v", v[0])
}
gdp.TimeStamp = time.Unix(ts, 0)
}

return nil
}

type graphiteTargetResp struct {
Target string `json:"target"`
DataPoints []dataPoint `json:"datapoints"`
}

type graphiteResponse []graphiteTargetResp
Loading

0 comments on commit 7c54090

Please sign in to comment.