Improve Kibana dashboard and the screenshots in doc

[zyiou]

This commit improves network policy diagram by grouping and color-coding flows using network policy. It shows the flows with no network policy in pie chart to provide complete information.

It also updates the screenshots of Kibana dashboard with better sankey diagrams (more variety). Example:

[codecov-io]

Codecov Report

Merging #1933 (77d50fc) into main (38c8c5e) will decrease coverage by 0.70%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #1933      +/-   ##
==========================================
- Coverage   62.46%   61.75%   -0.71%     
==========================================
  Files         201      201              
  Lines       17377    17377              
==========================================
- Hits        10854    10731     -123     
- Misses       5365     5491     +126     
+ Partials     1158     1155       -3     

Flag	Coverage Δ
kind-e2e-tests	51.30% <ø> (-1.22%)	⬇️
unit-tests	41.21% <ø> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pkg/antctl/transform/ovstracing/transform.go	0.00% <0.00%> (-50.00%)	⬇️
pkg/antctl/transform/appliedtogroup/transform.go	0.00% <0.00%> (-43.48%)	⬇️
pkg/antctl/transform/networkpolicy/transform.go	0.00% <0.00%> (-28.31%)	⬇️
...g/agent/apiserver/handlers/podinterface/handler.go	75.00% <0.00%> (-15.63%)	⬇️
pkg/agent/controller/traceflow/packetin.go	51.28% <0.00%> (-8.21%)	⬇️
pkg/agent/stats/collector.go	92.04% <0.00%> (-5.69%)	⬇️
pkg/agent/openflow/pipeline.go	62.93% <0.00%> (-4.08%)	⬇️
pkg/antctl/transform/common/transform.go	0.00% <0.00%> (-3.71%)	⬇️
...agent/controller/traceflow/traceflow_controller.go	71.93% <0.00%> (-3.17%)	⬇️
pkg/agent/apiserver/handlers/ovsflows/handler.go	73.58% <0.00%> (-2.84%)	⬇️
... and 12 more

[antoninbas]

LGTM, any noteworthy change in kibana.ndjson?

[zyiou]

LGTM, any noteworthy change in kibana.ndjson?

Thanks! As I mentioned in description. In Kibana.ndjson we add color-coding by network policies and show flows with no network policies in pie chart.

[srikartati]

In the network policy snapshots, color-coded groups are not consistent with colors on vertical ingress and egress lines. Is this expected? If expected please specify what they correspond to for avoiding any confusion.

[zyiou]

In the network policy snapshots, color-coded groups are not consistent with colors on vertical ingress and egress lines. Is this expected? If expected please specify what they correspond to for avoiding any confusion.

Yes, it is expected. That is to distinguish source and destination pod of each flow. If they are using the same color, Kibana will treat flows under the same network policy as a whole that we cannot tell the throughput of single flow.

[srikartati]

In the network policy snapshots, color-coded groups are not consistent with colors on vertical ingress and egress lines. Is this expected? If expected please specify what they correspond to for avoiding any confusion.

Yes, it is expected. That is to distinguish source and destination pod of each flow. If they are using the same color, Kibana will treat flows under the same network policy as a whole that we cannot tell the throughput of single flow.

I feel users could use Pod-to-Pod flow dashboard to figure out the throughput IMO. In network policy dashboard, if feasible, we can use the same color code for all the flows involving the same network policy even on the left vertical line. If not feasible, please add the above clarification in the document, where we add the note of network policy color coding.

[zyiou]

In the network policy snapshots, color-coded groups are not consistent with colors on vertical ingress and egress lines. Is this expected? If expected please specify what they correspond to for avoiding any confusion.

Yes, it is expected. That is to distinguish source and destination pod of each flow. If they are using the same color, Kibana will treat flows under the same network policy as a whole that we cannot tell the throughput of single flow.

I feel users could use Pod-to-Pod flow dashboard to figure out the throughput IMO. In network policy dashboard, if feasible, we can use the same color code for all the flows involving the same network policy even on the left vertical line. If not feasible, please add the above clarification in the document, where we add the note of network policy color coding.

As discussed offline, we decided to use egress network policy as source and ingress network policy as destination in sankey diagram. The updated visualization is as follows:

[srikartati]

Thanks for making the changes. Modified Sankey diagrams look good.
One clarification: the color of flow pipes is based on source and destination tuple and that is why they are unique?

[srikartati]

I think this description of color coding for network policy should be modified.

[zyiou]

Updated. Thanks!

[zyiou]

Thanks for making the changes. Modified Sankey diagrams look good.
One clarification: the color of flow pipes is based on source and destination tuple and that is why they are unique?

Thanks for pointing out. Updated the flow pipe color to be align with source. Do you think it is better?

[srikartati]

Thanks for making the changes. Modified Sankey diagrams look good.
One clarification: the color of flow pipes is based on source and destination tuple and that is why they are unique?

Thanks for pointing out. Updated the flow pipe color to be align with source. Do you think it is better?

Yes, I think this looks better and IMO users could easily distinguish network policies and flows going through them.

[srikartati]

LGTM

[srikartati]

/skip-all