Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ingress controller dashboards #21052

Merged
merged 1 commit into from
Sep 10, 2020

Conversation

ChrsMark
Copy link
Member

@ChrsMark ChrsMark commented Sep 10, 2020

What does this PR do?

This PR adds 2 Dashboards based ingress nginx controller fileset (implemented with #16197).

Most of the fields of ingress_controller are the same with access fileset of nginx module. In this most of the visualisations of these new dashboards are just copied from pre existing ones and modified accordingly while others are re-used. There are also some completely new ones. More spcifically:

  1. Operating systems breakdown [Filebeat Nginx] ECS, Browsers breakdown [Filebeat Nginx] ECS, Access Map [Filebeat Nginx] ECS. -> reused
  2. Top Upstreams [Filebeat Nginx Ingress Controller], Upstream Time Consumed By Path [Filebeat Nginx Ingress Controller], Request Volume By Path [Filebeat Nginx Ingress Controller] -> completely new inspired by some of https://github.com/kubernetes/ingress-nginx/tree/master/deploy/grafana/dashboards#2-request-handling-performance
  3. all others are duplicates with modified fileset.name

Why is it important?

To provide out of the box Dashboards for ingress controller service monitoring.

How to test this PR locally

  1. Setup ingress controller on minikube following https://kubernetes.io/docs/tasks/access-application-cluster/ingress-minikube/
  2. Redirect pods' logs to a temporary file: kubectl -n kube-system logs -f nginx-ingress-controller-6fc5bcc8c9-zm8zv >> /tmp/ingresspod
- module: nginx
  # Ingress-nginx controller logs. This is disabled by default. It could be used in Kubernetes environments to parse ingress-nginx logs
  ingress_controller:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths: ["/tmp/ingresspod"]
  1. Setup pipelines and dahsboards in ES
  2. Start Filebeat
  3. Produce traffic:
1. visit `http://hello-world.info/v2` and `http://hello-world.info` from different browser engines
2. use curl and wget to access the pages with different http words ie: curl -d "param1=value1&param2=value2" -X GET hello-world.info 

Related issues

Screenshots

Screenshot 2020-09-10 at 13 54 28
Screenshot 2020-09-10 at 13 52 59

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Sep 10, 2020
@ChrsMark ChrsMark self-assigned this Sep 10, 2020
@ChrsMark ChrsMark added the Team:Platforms Label for the Integrations - Platforms team label Sep 10, 2020
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Sep 10, 2020
@ChrsMark ChrsMark added :Dashboards needs_backport PR is waiting to be backported to other branches. v7.10.0 labels Sep 10, 2020
@ChrsMark ChrsMark marked this pull request as ready for review September 10, 2020 10:23
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-platforms (Team:Platforms)

@elasticmachine
Copy link
Collaborator

elasticmachine commented Sep 10, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #21052 updated]

  • Start Time: 2020-09-10T10:59:38.291+0000

  • Duration: 50 min 21 sec

Test stats 🧪

Test Results
Failed 0
Passed 5583
Skipped 825
Total 6408

Copy link
Contributor

@sorantis sorantis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

Signed-off-by: chrismark <chrismarkou92@gmail.com>
@ChrsMark ChrsMark force-pushed the add_ingress_controller_dashboards branch from 7d43636 to 1e33259 Compare September 10, 2020 10:58
@ChrsMark ChrsMark merged commit 54ea284 into elastic:master Sep 10, 2020
ChrsMark added a commit to ChrsMark/beats that referenced this pull request Sep 10, 2020
Signed-off-by: chrismark <chrismarkou92@gmail.com>
(cherry picked from commit 54ea284)
@ChrsMark ChrsMark removed the needs_backport PR is waiting to be backported to other branches. label Sep 10, 2020
v1v added a commit to v1v/beats that referenced this pull request Sep 14, 2020
* upstream/master: (362 commits)
  Add vendoring to Google Cloud Functions again (elastic#21070)
  [Elastic Agent] Add fleet.host.id for sending to endpoint. (elastic#21042)
  Do not need Google credentials before using it (elastic#21072)
  [Filebeat][New Module] Zoom webhook module (elastic#20414)
  Add support for GMT timezone offset in decode_cef (elastic#20993)
  Filebeat: Fix random error on harvester close (elastic#21048)
  Add ingress controller dashboards (elastic#21052)
  Fix loggers in composable module. (elastic#21047)
  [Ingest Manager] Increase kibana client timeout to 5 minutes (elastic#21037)
  Add changelog. (elastic#21041)
  [Elastic Agent] Add support for EQL based conditions (elastic#20994)
  Disable Kafka metricsets based on Jolokia (elastic#20989)
  Update apm agent (elastic#21031)
  Add container ECS fields in kubernetes metadata (elastic#20984)
  Sanitize event.host in Metricbeat (elastic#21022)
  Update api-keys.asciidoc - API key prerequisites (elastic#21026)
  [Filebeat][suricata] Map x509 for suricata/eve fileset (elastic#20973)
  [Filebeat][santa] Map x509 fields in santa module (elastic#20976)
  [Filebeat][fortinet] Map x509 ecs fields for fortinet fw fileset (elastic#20983)
  Bump zeek kerberos/ssl/x509 ecs version (elastic#21003)
  ...
v1v added a commit to v1v/beats that referenced this pull request Sep 14, 2020
* upstream/master: (364 commits)
  Add vendoring to Google Cloud Functions again (elastic#21070)
  [Elastic Agent] Add fleet.host.id for sending to endpoint. (elastic#21042)
  Do not need Google credentials before using it (elastic#21072)
  [Filebeat][New Module] Zoom webhook module (elastic#20414)
  Add support for GMT timezone offset in decode_cef (elastic#20993)
  Filebeat: Fix random error on harvester close (elastic#21048)
  Add ingress controller dashboards (elastic#21052)
  Fix loggers in composable module. (elastic#21047)
  [Ingest Manager] Increase kibana client timeout to 5 minutes (elastic#21037)
  Add changelog. (elastic#21041)
  [Elastic Agent] Add support for EQL based conditions (elastic#20994)
  Disable Kafka metricsets based on Jolokia (elastic#20989)
  Update apm agent (elastic#21031)
  Add container ECS fields in kubernetes metadata (elastic#20984)
  Sanitize event.host in Metricbeat (elastic#21022)
  Update api-keys.asciidoc - API key prerequisites (elastic#21026)
  [Filebeat][suricata] Map x509 for suricata/eve fileset (elastic#20973)
  [Filebeat][santa] Map x509 fields in santa module (elastic#20976)
  [Filebeat][fortinet] Map x509 ecs fields for fortinet fw fileset (elastic#20983)
  Bump zeek kerberos/ssl/x509 ecs version (elastic#21003)
  ...
v1v added a commit to v1v/beats that referenced this pull request Sep 18, 2020
…ne-2.0

* upstream/master: (44 commits)
  Update users.asciidoc (elastic#20802) (elastic#21108)
  Fix docker provider builder. (elastic#21118)
  [Elastic Agent] Add docker composable dynamic provider. (elastic#20842)
  Add new modules/filesets from rsa2elk for 7.10 (elastic#20820)
  Fix broken links to external websites (elastic#21061)
  [docs] typo in the command line (elastic#20799)
  [Filebeat] add panos type and sub_type (elastic#20912)
  Move the `compute_vm_scalset` to  a light metricset and map the cloud metadata (elastic#21038)
  [Filebeat] Add support for Cloudtrail digest files (elastic#21086)
  Add metrics collection from cost explorer into aws/billing metricset (elastic#20527)
  Add vendoring to Google Cloud Functions again (elastic#21070)
  [Elastic Agent] Add fleet.host.id for sending to endpoint. (elastic#21042)
  Do not need Google credentials before using it (elastic#21072)
  [Filebeat][New Module] Zoom webhook module (elastic#20414)
  Add support for GMT timezone offset in decode_cef (elastic#20993)
  Filebeat: Fix random error on harvester close (elastic#21048)
  Add ingress controller dashboards (elastic#21052)
  Fix loggers in composable module. (elastic#21047)
  [Ingest Manager] Increase kibana client timeout to 5 minutes (elastic#21037)
  Add changelog. (elastic#21041)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Dashboards Team:Platforms Label for the Integrations - Platforms team v7.10.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants