Skip to content

Commit

Permalink
update external ES configurations (openshift#2589)
Browse files Browse the repository at this point in the history 
* update external ES configurations

* add comment

* use http_ssl_enabled
  • Loading branch information
@QiaolingTang
QiaolingTang authored Dec 27, 2021
1 parent 31c60ba commit f467095
Show file tree
Hide file tree
Showing 44 changed files with 956 additions and 1,461 deletions.
8 changes: 4 additions & 4 deletions features/logging/cluster_log_forwarder.feature
View file
Original file line number Diff line number Diff line change
Expand Up @@ -412,10 +412,10 @@ Feature: cluster log forwarder features
And I use the "openshift-logging" project
And fluentd receiver is deployed as insecure in the "openshift-logging" project
And external elasticsearch server is deployed with:
| version | 6.8 |
| scheme | http |
| transport_ssl_enabled | false |
| project_name | openshift-logging |
| version | 6.8 |
| scheme | http |
| client_auth | false |
| project_name | openshift-logging |
# create clusterlogforwarder instace with multiple receiver
Given admin ensures "instance" cluster_log_forwarder is deleted from the "openshift-logging" project after scenario
And I obtain test data file "logging/clusterlogforwarder/multiple_receiver/clf_fluent_es.yaml"
Expand Down
126 changes: 88 additions & 38 deletions features/logging/forward_logs_to_external_elasticsearch.feature
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ Feature: Cases to test forward logs to external elasticsearch
And I have a project
And evaluation of `project` is stored in the :es_proj clipboard
Given external elasticsearch server is deployed with:
| version | 6.8 |
| scheme | <scheme> |
| transport_ssl_enabled | <transport_ssl_enabled> |
| project_name | <%= cb.es_proj.name %> |
| secret_name | pipelinesecret |
| version | 6.8 |
| scheme | <scheme> |
| client_auth | <client_auth> |
| project_name | <%= cb.es_proj.name %> |
| secret_name | pipelinesecret |

And I create a project with non-leading digit name
And evaluation of `project` is stored in the :proj clipboard
Expand Down Expand Up @@ -42,34 +42,53 @@ Feature: Cases to test forward logs to external elasticsearch
And I wait up to 300 seconds for the steps to pass:
"""
# check app logs
When I execute on the "<%= cb.log_receiver.name %>" pod:
| curl | -sk | -XGET | <scheme>://localhost:9200/*/_count?format=JSON | -H | Content-Type: application/json | -d | {"query": {"match": {"kubernetes.namespace_name": "<%= cb.proj.name %>"}}} |
When I check in the external ES pod with:
| project_name | <%= cb.es_proj.name %> |
| pod_label | app=elasticsearch-server |
| scheme | <scheme> |
| client_auth | <client_auth> |
| url_path | */_count?format=JSON |
| query | {"query": {"match": {"kubernetes.namespace_name": "<%= cb.proj.name %>"}}} |
Then the step should succeed
And the expression should be true> JSON.parse(@result[:stdout])['count'] > 0
# check journal logs
When I execute on the "<%= cb.log_receiver.name %>" pod:
| curl | -sk | -XGET | <scheme>://localhost:9200/*/_count?format=JSON | -H | Content-Type: application/json | -d | {"query": {"exists": {"field": "systemd"}}} |
When I check in the external ES pod with:
| project_name | <%= cb.es_proj.name %> |
| pod_label | app=elasticsearch-server |
| scheme | <scheme> |
| client_auth | <client_auth> |
| url_path | */_count?format=JSON |
| query | {"query": {"exists": {"field": "systemd"}}} |
Then the step should succeed
And the expression should be true> JSON.parse(@result[:stdout])['count'] > 0
# check logs in openshift* namespace
When I execute on the "<%= cb.log_receiver.name %>" pod:
| curl | -sk | -XGET | <scheme>://localhost:9200/*/_count?format=JSON | -H | Content-Type: application/json | -d | {"query": {"regexp": {"kubernetes.namespace_name": "openshift@"}}} |
Then the step should succeed
When I check in the external ES pod with:
| project_name | <%= cb.es_proj.name %> |
| pod_label | app=elasticsearch-server |
| scheme | <scheme> |
| client_auth | <client_auth> |
| url_path | */_count?format=JSON |
| query | {"query": {"regexp": {"kubernetes.namespace_name": "openshift@"}}} |
And the expression should be true> JSON.parse(@result[:stdout])['count'] > 0
# check audit logs
When I execute on the "<%= cb.log_receiver.name %>" pod:
| curl | -sk | -XGET | <scheme>://localhost:9200/*/_count?format=JSON | -H | Content-Type: application/json | -d | {"query": {"exists": {"field": "auditID"}}} |
When I check in the external ES pod with:
| project_name | <%= cb.es_proj.name %> |
| pod_label | app=elasticsearch-server |
| scheme | <scheme> |
| client_auth | <client_auth> |
| url_path | */_count?format=JSON |
| query | {"query": {"exists": {"field": "auditID"}}} |
Then the step should succeed
And the expression should be true> JSON.parse(@result[:stdout])['count'] > 0
"""

Examples:
| scheme | transport_ssl_enabled | file |
| https | true | clf-with-secret.yaml | # @case_id OCP-29845
| http | false | clf-insecure.yaml | # @case_id OCP-29846
| scheme | client_auth | file |
| https | true | clf-with-secret.yaml | # @case_id OCP-29845
| http | false | clf-insecure.yaml | # @case_id OCP-29846
# @author qitang@redhat.com
@admin
Expand All @@ -79,14 +98,14 @@ Feature: Cases to test forward logs to external elasticsearch
And I have a project
And evaluation of `project.name` is stored in the :es_proj clipboard
Given external elasticsearch server is deployed with:
| version | <version> |
| scheme | <scheme> |
| transport_ssl_enabled | <transport_ssl_enabled> |
| user_auth_enabled | true |
| project_name | <%= cb.es_proj %> |
| username | <username> |
| password | <password> |
| secret_name | <secret_name> |
| version | <version> |
| scheme | <scheme> |
| client_auth | <client_auth> |
| user_auth_enabled | true |
| project_name | <%= cb.es_proj %> |
| username | <username> |
| password | <password> |
| secret_name | <secret_name> |
And I create a project with non-leading digit name
And evaluation of `project` is stored in the :proj clipboard
Given I obtain test data file "logging/loggen/container_json_log_template.json"
Expand Down Expand Up @@ -117,33 +136,64 @@ Feature: Cases to test forward logs to external elasticsearch
And I wait up to 300 seconds for the steps to pass:
"""
# check app logs
When I execute on the pod:
| curl | -sk | -u | <username>:<password> | -XGET | <scheme>://localhost:9200/*/_count?format=JSON | -H | Content-Type: application/json | -d | {"query": {"match": {"kubernetes.namespace_name": "<%= cb.proj.name %>"}}} |
When I check in the external ES pod with:
| project_name | <%= cb.es_proj %> |
| pod_label | app=elasticsearch-server |
| scheme | <scheme> |
| client_auth | <client_auth> |
| user_auth_enabled | true |
| username | <username> |
| password | <password> |
| url_path | */_count?format=JSON |
| query | {"query": {"match": {"kubernetes.namespace_name": "<%= cb.proj.name %>"}}} |
Then the step should succeed
And the expression should be true> JSON.parse(@result[:stdout])['count'] > 0
# check journal logs
When I execute on the pod:
| curl | -sk | -u | <username>:<password> | -XGET | <scheme>://localhost:9200/*/_count?format=JSON | -H | Content-Type: application/json | -d | {"query": {"exists": {"field": "systemd"}}} |
When I check in the external ES pod with:
| project_name | <%= cb.es_proj %> |
| pod_label | app=elasticsearch-server |
| scheme | <scheme> |
| client_auth | <client_auth> |
| user_auth_enabled | true |
| username | <username> |
| password | <password> |
| url_path | */_count?format=JSON |
| query | {"query": {"exists": {"field": "systemd"}}} |
Then the step should succeed
And the expression should be true> JSON.parse(@result[:stdout])['count'] > 0
# check logs in openshift* namespace
When I execute on the pod:
| curl | -sk | -u | <username>:<password> | -XGET | <scheme>://localhost:9200/*/_count?format=JSON | -H | Content-Type: application/json | -d | {"query": {"regexp": {"kubernetes.namespace_name": "openshift@"}}} |
When I check in the external ES pod with:
| project_name | <%= cb.es_proj %> |
| pod_label | app=elasticsearch-server |
| scheme | <scheme> |
| client_auth | <client_auth> |
| user_auth_enabled | true |
| username | <username> |
| password | <password> |
| url_path | */_count?format=JSON |
| query | {"query": {"regexp": {"kubernetes.namespace_name": "openshift@"}}} |
Then the step should succeed
And the expression should be true> JSON.parse(@result[:stdout])['count'] > 0
# check audit logs
When I execute on the pod:
| curl | -sk | -u | <username>:<password> | -XGET | <scheme>://localhost:9200/*/_count?format=JSON | -H | Content-Type: application/json | -d | {"query": {"exists": {"field": "auditID"}}} |
When I check in the external ES pod with:
| project_name | <%= cb.es_proj %> |
| pod_label | app=elasticsearch-server |
| scheme | <scheme> |
| client_auth | <client_auth> |
| user_auth_enabled | true |
| username | <username> |
| password | <password> |
| url_path | */_count?format=JSON |
| query | {"query": {"exists": {"field": "auditID"}}} |
Then the step should succeed
And the expression should be true> JSON.parse(@result[:stdout])['count'] > 0
"""

Examples:
| version | scheme | transport_ssl_enabled | username | password | secret_name |
| 7.12 | https | true | test1 | redhat | test1 | #@case_id OCP-41807
| 6.8 | http | false | test2 | redhat | test2 | #@case_id OCP-41805
| 7.12 | http | true | test3 | redhat | test3 | #@case_id OCP-41808
| 6.8 | https | false | test4 | redhat | test4 | #@case_id OCP-41806
| version | scheme | client_auth | username | password | secret_name |
| 7.12 | https | true | test1 | redhat | test1 | #@case_id OCP-41807
| 6.8 | http | false | test2 | redhat | test2 | #@case_id OCP-41805
| 6.8 | https | false | test4 | redhat | test4 | #@case_id OCP-41806
Loading

0 comments on commit f467095

Please sign in to comment.