Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Elastic Stack deployment for flow visualization #836

Merged
merged 6 commits into from
Aug 12, 2020
Merged

Add Elastic Stack deployment for flow visualization #836

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
90a5639
Add Elastic stack deployment
zyiou Jun 15, 2020
e7692dd
address comments
zyiou Aug 11, 2020
4c8abfd
modified format
zyiou Aug 11, 2020
c3122f4
add comments for check_record
zyiou Aug 11, 2020
d3ca451
address comments
zyiou Aug 11, 2020
fab068e
address comments
zyiou Aug 12, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions docs/network-flow-visibility.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Network Flow Visibility
# ELK Flow Collector
## Purpose
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this document talks about the purpose and details of ELK flow collector, the title "Elk Flow Collector" is appropriate. This is can be a section in the document like "Flow Exporter": https://github.com/vmware-tanzu/antrea/pull/980/files#diff-a27d53d71ee3ed485cf12d6147782edb

Antrea supports sending IPFIX flow records through a flow exporter. The Elastic
Stack (ELK Stack) works as the data collector, data storage and visualization tool
Expand Down Expand Up @@ -28,7 +28,6 @@ with standard IANA fields.
| sourceNodeName | 55829 | 104 | string |
| destinationNodeName | 55829 | 105 | string |
| destinationClusterIP | 55829 | 106 | ipv4Address |
| destinationServicePort | 55829 | 107 | unsigned16 |
| destinationServicePortName| 55829 | 108 | string |

[Elasticsearch](https://www.elastic.co/elasticsearch/), as a RESTful search
Expand All @@ -48,35 +47,35 @@ Kibana dashboard is exposed as a Nodeport Service, which can be accessed via
`http://[NodeIP]: 30007`

`build/yamls/flow/kibana.ndjson` is an auto-generated reusable file containing
pre-built objects for visualizing pod-to-pod, pod-to-service and node-to-node
pre-built objects for visualizing Pod-to-Pod, Pod-to-Service and Node-to-Node
flow records. To import the dashboards into Kibana, go to
**Management -> Saved Objects** and import `build/yamls/flow/kibana.ndjson`.


## Pre-built Dashboards
The following dashboards are pre-built and recommended for Antrea flow
The following dashboards are pre-built and are recommended for Antrea flow
visualization.

### Overview
An overview of pod-based flow records information is provided.
An overview of Pod-based flow records information is provided.

<img src="https://s3-us-west-2.amazonaws.com/downloads.antrea.io/static/flow-visualization-overview.png" width="900" alt="Flow
Visualization Overview Dashboard">

### Flows
#### Pod-to-pod Traffic
Pod-to-pod Tx and Rx traffic is shown in sankey diagrams. Corresponding
source/destinationpod throughput is visualized using stacked line graph.
#### Pod-to-Pod Traffic
Pod-to-Pod Tx and Rx traffic is shown in sankey diagrams. Corresponding
source or destination Pod throughput is visualized using stacked line graph.

<img src="https://s3-us-west-2.amazonaws.com/downloads.antrea.io/static/flow-visualization-flow-1.png" width="900" alt="Flow
Visualization Flows Dashboard">

<img src="https://s3-us-west-2.amazonaws.com/downloads.antrea.io/static/flow-visualization-flow-2.png" width="900" alt="Flow
Visualization Flow Dashboard">

#### Pod-to-service Traffic
Pod-to-service traffic is presented similar to pod-to-pod traffic.
Corresponding source/destination IP addresses are shown in tooltips.
#### Pod-to-Service Traffic
Pod-to-Service traffic is presented similar to Pod-to-Pod traffic.
Corresponding source or destination IP addresses are shown in tooltips.

<img src="https://s3-us-west-2.amazonaws.com/downloads.antrea.io/static/flow-visualization-flow-3.png" width="900" alt="Flow
Visualization Flows Dashboard">
Expand All @@ -85,14 +84,15 @@ Visualization Flows Dashboard">
Visualization Flow Dashboard">

### Flow Records
Flow Records dashboard shows raw flow records over time with filter.
Flow Records dashboard shows the raw flow records over time with support
for filters.

<img src="https://s3-us-west-2.amazonaws.com/downloads.antrea.io/static/flow-visualization-flow-record.png" width="900" alt="Flow
Visualization Flow Record Dashboard">

### Node Throughput
Node Throughput dashboard visualizes inter-node and intra-node traffic
by aggregating pod traffic per node.
Node Throughput dashboard shows the visualization of inter-Node and
Copy link
Member

@srikartati srikartati Aug 12, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, here with inter-Node, intra-Node and per Node, we have to use smaller case because we are not referencing a particular node. Similarly for "all the Pod traffic", we need to use smaller because we are characterizing the traffic as pod traffic and not referencing a particular Pod.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds great. How about changing it to:
Node Throughput dashboard provides an insight in inter-Node and intra-Node traffic through aggregated Pod traffic.

intra-Node traffic by aggregating all the Pod traffic per Node.

<img src="https://s3-us-west-2.amazonaws.com/downloads.antrea.io/static/flow-visualization-node-1.png" width="900" alt="Flow
Visualization Node Throughput Dashboard">
Expand Down