Skip to content

Commit 2636690

Browse files
author
Jason Bulicek
committed
add attribution and console pages
1 parent ba8fa8c commit 2636690

File tree

4 files changed

+51
-2
lines changed

4 files changed

+51
-2
lines changed

docs/deployment/deployment_clients.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ We have an [OpenTracing](http://opentracing.io/)-compliant client libraries to f
1010
* [Go client](https://github.com/ExpediaDotCom/haystack-client-go)
1111
* [Dropwizard integration](https://github.com/ExpediaDotCom/haystack-dropwizard) that utilizes the Java client
1212
* [Spring starter](https://github.com/ExpediaDotCom/opentracing-spring-haystack-starter) that utilizes [opentracing-spring-cloud-starter](https://github.com/opentracing-contrib/java-spring-cloud)
13-
13+
* [OpenTelemetry via Zipkin](https://github.com/ExpediaDotCom/haystack-opentelemetry-example)
1414

1515
Additionally, with [Pitchfork mode of the Agent](../about/clients.html#pitchfork-agent), you are able to send spans from any Zipkin-compatible client to the agent. If you have existing client code that is written to send trace data to other systems, investigate using the [Collector subsystem](../subsystems/subsystems_collectors.html) to import that data into Haystack.
1616

docs/subsystems/subsystems_anomaly_detection.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@ title: Anomaly Detection
33
sidebar_label: Anomaly Detection
44
---
55

6-
<div class="note"><b>Note:</b> This topic describes work in progress. The released component may vary from what's described here. This topic will be updated when the code for Dependencies is released.</div>
6+
<div class="note"><i><b>Note:</b> This topic describes work in progress. The released component may vary from what's described here.</i> </div>
7+
<br>
78

89
The Anomaly Detection subsystem is responsible for identifying anomalies in services' health and triggering alerts. By default, Anomaly Detection will monitor for anomalies in duration, success percentage, and count (number of operations over time) for all operations of all services sending data to Haystack.
910

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
title: Attribution Components
3+
sidebar_label: Attribution
4+
---
5+
6+
<div class="note"><i><b>Note:</b> This topic describes work in progress. The released component may vary from what's described here.</i> </div>
7+
<br>
8+
9+
[Haystack-attribution](https://github.com/ExpediaDotCom/haystack-attribution) is a group of components that are used to monitor and manage the health of a Haystack deployment. It contains the following subsystems:
10+
11+
### Haystack-attributor Module
12+
13+
For smooth operation of a Haystack cluster, traffic should be monitored on a service-to-service basis. If a service goes berserk, it could put a severe load on the whole infrastructure and impact all other services. Attributor is the component responsible for measuring and keeping track of the volume of spans sent by each service.
14+
15+
In addition to few basic attribution fields like span count, span size, operation count; one can also do attribution on the basis of span tags.
16+
17+
For using this attributed data, there are two ways provided for persistence. One can either setup the email for getting the attribution report as an email at regular intervals. Or other way is to dump the report in s3 and use it the way user wants to use.
18+
19+
### Haystack-attribution-persistence-email Module
20+
This sub-component fetches the stats from attributor via a rest call and sends the attributed data / stats as an email. It is scheduled as a cron job in our k8s cluster to send the email (for previous calender day, currently).
21+
22+
What it does:
23+
24+
* rest call to attributor to fetch service stats for previous day's time window
25+
* creates an email template for stats
26+
* email using aws ses service / smtp to all the ids in configuration
27+
28+
### Haystack-attribution-persistence-s3 Module
29+
30+
Component which persists the attributed data to s3 for consumption by any other tool. This component is also scheduled as a cron job in our k8s cluster to persist the cost attribution details as csv in S3.
31+
32+
What it does:
33+
34+
* rest call to attributor to fetch service stats (for previous day's time window, currently)
35+
* transforms the stats data using a transformer (transformer can be overridden)
36+
* persist the transformed data to s3
37+
38+

docs/subsystems/subsystems_console.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
---
2+
title: Console
3+
sidebar_label: Console
4+
---
5+
6+
<div class="note"><i><b>Note:</b> This topic describes work in progress. The released component may vary from what's described here.</i> </div>
7+
<br>
8+
9+
[Haystack Console](https://github.com/ExpediaDotCom/haystack-console/) is a interface for managing and observing the health of a deployed Haystack cluster. The cluster works hand-in-hand with the [attributor](./subsystems_attribution.html) to facilitate changes needed to maintain a healthy deployment. More information coming soon.
10+

0 commit comments

Comments
 (0)