Support Fetch Cilium Observe data to monitoring Cilium Service network traffic #12393
Conversation
mrproliu
added
UI
|
UI submodule seems including more changes. Could you update the changelogs of UI side? |
|
How about e2e of Cilium? |
|
Cilium integration docs should be added in the marketplace, https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-k8s-monitoring/. The docs should include
|
| public class FieldsHelper { | ||
| private static final Map<Class<?>, FieldsHelper> HELPER_MAP = new ConcurrentHashMap<>(); |
There was a problem hiding this comment.
Could you explain this? It seems to you expand the scope of this class to cover more field processes?
There was a problem hiding this comment.
Updated. Yes, Before we need to use FieldsHelper.SINGLE for getting the instance, but it only works for the ALS message. But for now, the cilium fetcher still needs this for the inflate metadata, so I make it an instance with the target inflate class. When the user needs to get the instance of FieldsHelper needs to call forClass(Class).
| // SPDX-License-Identifier: Apache-2.0 | ||
| // Copyright Authors of Cilium |
There was a problem hiding this comment.
I think we should update the LICENSE of source codes, as we copied these from a new project.
There was a problem hiding this comment.
I have update the LICENSE file, please check.
Updated. |
I will trying to create other PR for the E2E Testing. |
Thanks. I have update the documentation following your mentioned. |
| | Call CPM | Count | HTTP Request calls per minutes. | | ||
| | Duration | Nanoseconds | Total HTTP Response use duration. | | ||
| | Success CPM | Count | Total HTTP Response success(status < 500) count. | | ||
| | Status 1/2/3/4/5xx | Count | HTTP Response status code group by 1xx/2xx/3xx/4xx/5xx. | |
There was a problem hiding this comment.
Is 2xx in the same graph? 2xx will be super high.
There was a problem hiding this comment.
Yes, They are in the same graph.
There was a problem hiding this comment.
I think we should list all traffic in one graph and not-2xx in one graph. This should be more clear. Traffic and Anormal Traffic
|
Two questions
|
Hubble peer service is correlated with the cilium node, not a only pod. And Kubernetes will load balance with wich pod should be listen.
I have write a selection about this: Load Balance for Cilium Fetcher with OAP cluster. All the OAP nodes will listen bubble peer service for getting nodes. And each OAP node will calucate how many cilium node should be monitoring. |
| <plugin> | ||
| <groupId>org.apache.maven.plugins</groupId> | ||
| <artifactId>maven-dependency-plugin</artifactId> | ||
| <executions> | ||
| <execution> | ||
| <id>unpack</id> | ||
| <phase>generate-sources</phase> | ||
| <goals> | ||
| <goal>unpack</goal> | ||
| </goals> | ||
| <configuration> | ||
| <artifactItems> | ||
| <artifactItem> | ||
| <groupId>com.github.davidmoten</groupId> | ||
| <artifactId>flatbuffers-compiler</artifactId> | ||
| <version>1.12.0.1</version> | ||
| <type>${fbs.compiler.artifact.type}</type> | ||
| <classifier>distribution-${os.detected.name}</classifier> | ||
| <overWrite>true</overWrite> | ||
| <outputDirectory>${project.build.directory}</outputDirectory> | ||
| </artifactItem> | ||
| </artifactItems> | ||
| </configuration> | ||
| </execution> | ||
| </executions> | ||
| </plugin> | ||
| <plugin> | ||
| <groupId>org.codehaus.mojo</groupId> | ||
| <artifactId>build-helper-maven-plugin</artifactId> | ||
| <version>${build-helper-maven-plugin.version}</version> | ||
| <executions> | ||
| <execution> | ||
| <id>add-source</id> | ||
| <phase>generate-sources</phase> | ||
| <goals> | ||
| <goal>add-source</goal> | ||
| </goals> | ||
| <configuration> | ||
| <sources> | ||
| <source>${fbs.generated.sources}</source> | ||
| </sources> | ||
| </configuration> | ||
| </execution> | ||
| </executions> | ||
| </plugin> | ||
| </plugins> |
There was a problem hiding this comment.
Do we really need this bunch of plugins here?
There was a problem hiding this comment.
The fbs and flatbuffers can be deleted.
| }) | ||
| @ToString(of = { | ||
| "address", | ||
| "connected" |
There was a problem hiding this comment.
This field doesn’t even exist in the class
There was a problem hiding this comment.
connected can be deleted, I think I just forget to delete it when field changes.
There was a problem hiding this comment.
I think these file need to be added to the LICENSE file
Lines 215 to 226 in 87b73cb
In this PR, I mainly implemented the following features:
labelCountfunction in the OAL engine for creating count statistics based on tags.NOTE: Following cilium/cilium#32175, we found that some flow data would be missing some metadata, so we cannot get both side metrics. Then, we decide to ignore the server-side data and convert the client-side data to the server-side.
Screenshots
All Cilium Services Overview
Service Overview
Service Relation Overview
Service Instance Overview
Service Instance Relaiton Overview
Endpoint Overview
CHANGESlog.