[7.x] [Security Solution][Detections] Adoption telemetry (#71102) #71504

rylnd · 2020-07-13T18:19:40Z

Backports the following commits to 7.x:

[Security Solution][Detections] Adoption telemetry ([Security Solution][Detections] Adoption telemetry #71102)

* style: sort plugin interface * WIP: UsageCollector for Security Adoption This uses ML and raw ES calls to query our ML Jobs and Rules, and parse them into a format to be consumed by telemetry. Still to come: * initialization * tests * Initialize usage collectors during plugin setup * Rename usage key The service seems to convert colons to underscores, so let's just use an underscure. * Collector is ready if we have a kibana index * Refactor collector to generate options in a function This allows us to test our adherence to the collector API, focusing particularly on the fetch function. * Refactor usage collector in anticipation of endpoint data We're going to have our usage data under one key corresponding to the app, so this nests the existing data under a 'detections' key while allowing another fetching function to be plugged into the main collector under a separate key. * Update our collector to satisfy telemetry tooling * inlines collector options * inlines schema object * makes DetectionsUsage an interface instead of a type alias * Extracts telemetry mappings via scripts/telemetry_extract * Refactor detections usage logic to perform one loop instead of two We were previously performing two loops over each set of data: one to format it down to just the data we need, and another to convert that into usage data. We now perform both steps within a single loop. * Refactor detections telemetry to be nested * Extract new nested detections telemetry mappings Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

kibanamachine · 2020-07-13T20:11:15Z

💛 Build succeeded, but was flaky

continuous-integration/kibana-ci/pull-request
Commit: 4d3ca34
Flaky suites:
- kibana-firefoxSmoke

Test Failures

Kibana Pipeline / kibana-oss-agent / Firefox UI Functional Tests.test/functional/apps/visualize/input_control_vis/chained_controls·js.visualize app input controls chained controls should disable child control when parent control is not set

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 2 times on tracked branches: https://github.com/elastic/kibana/issues/68472

[00:00:00]       │
[00:13:15]         └-: visualize app
[00:13:15]           └-> "before all" hook
[00:13:15]           └-> "before all" hook
[00:13:15]             │ debg Starting visualize before method
[00:13:15]             │ info [logstash_functional] Loading "mappings.json"
[00:13:15]             │ info [logstash_functional] Loading "data.json.gz"
[00:13:15]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] [logstash-2015.09.22] creating index, cause [api], templates [], shards [1]/[0]
[00:13:15]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[logstash-2015.09.22][0]]]).
[00:13:15]             │ info [logstash_functional] Created index "logstash-2015.09.22"
[00:13:15]             │ debg [logstash_functional] "logstash-2015.09.22" settings {"index":{"analysis":{"analyzer":{"url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:13:15]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] [logstash-2015.09.20] creating index, cause [api], templates [], shards [1]/[0]
[00:13:15]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[logstash-2015.09.20][0]]]).
[00:13:15]             │ info [logstash_functional] Created index "logstash-2015.09.20"
[00:13:15]             │ debg [logstash_functional] "logstash-2015.09.20" settings {"index":{"analysis":{"analyzer":{"url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:13:15]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] [logstash-2015.09.21] creating index, cause [api], templates [], shards [1]/[0]
[00:13:15]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[logstash-2015.09.21][0]]]).
[00:13:15]             │ info [logstash_functional] Created index "logstash-2015.09.21"
[00:13:15]             │ debg [logstash_functional] "logstash-2015.09.21" settings {"index":{"analysis":{"analyzer":{"url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:13:25]             │ info progress: 13139
[00:13:26]             │ info [logstash_functional] Indexed 4633 docs into "logstash-2015.09.22"
[00:13:26]             │ info [logstash_functional] Indexed 4757 docs into "logstash-2015.09.20"
[00:13:26]             │ info [logstash_functional] Indexed 4614 docs into "logstash-2015.09.21"
[00:13:26]             │ info [long_window_logstash] Loading "mappings.json"
[00:13:26]             │ info [long_window_logstash] Loading "data.json.gz"
[00:13:26]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] [long-window-logstash-0] creating index, cause [api], templates [], shards [1]/[0]
[00:13:26]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[long-window-logstash-0][0]]]).
[00:13:26]             │ info [long_window_logstash] Created index "long-window-logstash-0"
[00:13:26]             │ debg [long_window_logstash] "long-window-logstash-0" settings {"index":{"analysis":{"analyzer":{"makelogs_url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:13:36]             │ info progress: 10536
[00:13:40]             │ info [long_window_logstash] Indexed 14005 docs into "long-window-logstash-0"
[00:13:40]             │ info [visualize] Loading "mappings.json"
[00:13:40]             │ info [visualize] Loading "data.json"
[00:13:40]             │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] [.kibana_2/jGGnPhaHSKqfr7s1I6pNcg] deleting index
[00:13:40]             │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] [.kibana_1/XmnJ-xSuQdim5XegryYSdg] deleting index
[00:13:40]             │ info [visualize] Deleted existing index [".kibana_2",".kibana_1"]
[00:13:40]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] [.kibana] creating index, cause [api], templates [], shards [1]/[1]
[00:13:40]             │ info [visualize] Created index ".kibana"
[00:13:40]             │ debg [visualize] ".kibana" settings {"index":{"number_of_replicas":"1","number_of_shards":"1"}}
[00:13:40]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] [.kibana/CM-uyuVOTnO9Rrukls65MA] update_mapping [_doc]
[00:13:40]             │ info [visualize] Indexed 12 docs into ".kibana"
[00:13:40]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] [.kibana/CM-uyuVOTnO9Rrukls65MA] update_mapping [_doc]
[00:13:40]             │ debg Migrating saved objects
[00:13:40]             │ proc [kibana]   log   [19:00:19.207] [info][savedobjects-service] Creating index .kibana_2.
[00:13:40]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] [.kibana_2] creating index, cause [api], templates [], shards [1]/[1]
[00:13:40]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] updating number_of_replicas to [0] for indices [.kibana_2]
[00:13:40]             │ proc [kibana]   log   [19:00:19.253] [info][savedobjects-service] Reindexing .kibana to .kibana_1
[00:13:40]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] [.kibana_1] creating index, cause [api], templates [], shards [1]/[1]
[00:13:40]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] updating number_of_replicas to [0] for indices [.kibana_1]
[00:13:40]             │ info [o.e.t.LoggingTaskListener] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] 9220 finished with response BulkByScrollResponse[took=33.3ms,timed_out=false,sliceId=null,updated=0,created=12,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[00:13:40]             │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] [.kibana/CM-uyuVOTnO9Rrukls65MA] deleting index
[00:13:40]             │ proc [kibana]   log   [19:00:19.594] [info][savedobjects-service] Migrating .kibana_1 saved objects to .kibana_2
[00:13:41]             │ proc [kibana]   log   [19:00:19.609] [error][savedobjects-service] Error: Unable to migrate the corrupt Saved Object document index-pattern:test_index*. To prevent Kibana from performing a migration on every restart, please delete or fix this document by ensuring that the namespace and type in the document's id matches the values in the namespace and type fields.
[00:13:41]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] [.kibana_2/ti3gpuVWSYyEUqI6Sd-X1Q] update_mapping [_doc]
[00:13:41]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] [.kibana_2/ti3gpuVWSYyEUqI6Sd-X1Q] update_mapping [_doc]
[00:13:41]             │ proc [kibana]   log   [19:00:19.696] [info][savedobjects-service] Pointing alias .kibana to .kibana_2.
[00:13:41]             │ proc [kibana]   log   [19:00:19.754] [info][savedobjects-service] Finished in 549ms.
[00:13:41]             │ debg applying update to kibana config: {"accessibility:disableAnimations":true,"dateFormat:tz":"UTC"}
[00:13:41]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xl-1594664830414461085] [.kibana_2/ti3gpuVWSYyEUqI6Sd-X1Q] update_mapping [_doc]
[00:13:42]             │ debg replacing kibana config doc: {"defaultIndex":"logstash-*","format:bytes:defaultPattern":"0,0.[000]b"}
[00:13:43]           └-: 
[00:13:43]             └-> "before all" hook
[00:13:43]             └-: input controls
[00:13:43]               └-> "before all" hook
[00:13:43]               └-: chained controls
[00:13:43]                 └-> "before all" hook
[00:13:43]                 └-> "before all" hook
[00:13:43]                   │ debg navigating to visualize url: http://localhost:6111/app/visualize#/
[00:13:43]                   │ debg navigate to: http://localhost:6111/app/visualize#/
[00:13:43]                   │ debg browser[log] "^ A single error about an inline script not firing due to content security policy is expected!"
[00:13:43]                   │ debg ... sleep(700) start
[00:13:44]                   │ debg ... sleep(700) end
[00:13:44]                   │ debg returned from get, calling refresh
[00:13:45]                   │ debg browser[info] "INFO: 2020-07-13T19:00:24Z\n  Adding connection to http://localhost:6111/elasticsearch\n\n"
[00:13:45]                   │ERROR browser[error] (new TypeError("NetworkError when attempting to fetch resource.", ""))
[00:13:45]                   │ debg browser[log] "^ A single error about an inline script not firing due to content security policy is expected!"
[00:13:46]                   │ debg currentUrl = http://localhost:6111/app/visualize#/
[00:13:46]                   │          appUrl = http://localhost:6111/app/visualize#/
[00:13:46]                   │ debg TestSubjects.find(kibanaChrome)
[00:13:46]                   │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:13:47]                   │ debg browser[info] "INFO: 2020-07-13T19:00:26Z\n  Adding connection to http://localhost:6111/elasticsearch\n\n"
[00:13:48]                   │ debg ... sleep(501) start
[00:13:48]                   │ debg ... sleep(501) end
[00:13:48]                   │ debg in navigateTo url = http://localhost:6111/app/visualize#/?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))
[00:13:48]                   │ debg --- retry.try error: URL changed, waiting for it to settle
[00:13:49]                   │ debg ... sleep(501) start
[00:13:49]                   │ debg ... sleep(501) end
[00:13:49]                   │ debg in navigateTo url = http://localhost:6111/app/visualize#/?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))
[00:13:49]                   │ debg TestSubjects.exists(statusPageContainer)
[00:13:49]                   │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="statusPageContainer"]') with timeout=2500
[00:13:52]                   │ debg --- retry.tryForTime error: [data-test-subj="statusPageContainer"] is not displayed
[00:13:52]                   │ debg TestSubjects.click(visListingTitleLink-chained-input-control)
[00:13:52]                   │ debg Find.clickByCssSelector('[data-test-subj="visListingTitleLink-chained-input-control"]') with timeout=20000
[00:13:52]                   │ debg Find.findByCssSelector('[data-test-subj="visListingTitleLink-chained-input-control"]') with timeout=20000
[00:13:52]                   │ debg isGlobalLoadingIndicatorVisible
[00:13:52]                   │ debg TestSubjects.exists(globalLoadingIndicator)
[00:13:52]                   │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="globalLoadingIndicator"]') with timeout=1500
[00:13:53]                   │ debg --- retry.tryForTime error: [data-test-subj="globalLoadingIndicator"] is not displayed
[00:13:53]                   │ debg TestSubjects.exists(globalLoadingIndicator-hidden)
[00:13:53]                   │ debg Find.existsByCssSelector('[data-test-subj="globalLoadingIndicator-hidden"]') with timeout=100000
[00:13:54]                 └-> should disable child control when parent control is not set
[00:13:54]                   └-> "before each" hook: global before each
[00:13:54]                   │ debg comboBox.getOptionsList, comboBoxSelector: listControlSelect0
[00:13:54]                   │ debg TestSubjects.find(listControlSelect0)
[00:13:54]                   │ debg Find.findByCssSelector('[data-test-subj="listControlSelect0"]') with timeout=10000
[00:14:04]                   │ info Taking screenshot "/dev/shm/workspace/kibana/test/functional/screenshots/failure/visualize app  input controls chained controls should disable child control when parent control is not set.png"
[00:14:05]                   │ info Current URL is: http://localhost:6111/app/visualize#/edit/68305470-87bc-11e9-a991-3b492a7c3e09?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))&_a=(filters:!(),linked:!f,query:(language:kuery,query:%27%27),uiState:(),vis:(aggs:!(),params:(controls:!((fieldName:geo.src,id:%271559757816862%27,indexPattern:%27logstash-*%27,label:%27%27,options:(dynamicOptions:!t,multiselect:!t,order:desc,size:5,type:terms),parent:%27%27,type:list),(fieldName:clientip,id:%271559757836347%27,indexPattern:%27logstash-*%27,label:%27%27,options:(dynamicOptions:!t,multiselect:!t,order:desc,size:5,type:terms),parent:%271559757816862%27,type:list)),pinFilters:!f,updateFiltersOnChange:!f,useTimeFilter:!f),title:%27chained%20input%20control%27,type:input_control_vis))
[00:14:05]                   │ info Saving page source to: /dev/shm/workspace/kibana/test/functional/failure_debug/html/visualize app  input controls chained controls should disable child control when parent control is not set.html
[00:14:05]                   └- ✖ fail: visualize app  input controls chained controls should disable child control when parent control is not set
[00:14:05]                   │      TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="listControlSelect0"])
[00:14:05]                   │ Wait timed out after 10020ms
[00:14:05]                   │       at /dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
[00:14:05]                   │       at process._tickCallback (internal/process/next_tick.js:68:7)
[00:14:05]                   │ 
[00:14:05]                   │

Stack Trace

{ TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="listControlSelect0"])
Wait timed out after 10020ms
    at /dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
    at process._tickCallback (internal/process/next_tick.js:68:7) name: 'TimeoutError', remoteStacktrace: '' }

Build metrics

✅ unchanged

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

rylnd added the backport This PR is a backport of another PR label Jul 13, 2020

rylnd merged commit 6e1aaad into elastic:7.x Jul 13, 2020

rylnd deleted the backport/7.x/pr-71102 branch July 13, 2020 20:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[7.x] [Security Solution][Detections] Adoption telemetry (#71102) #71504

[7.x] [Security Solution][Detections] Adoption telemetry (#71102) #71504

Uh oh!

rylnd commented Jul 13, 2020

Uh oh!

kibanamachine commented Jul 13, 2020

Standard Out

Stack Trace

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

[7.x] [Security Solution][Detections] Adoption telemetry (#71102) #71504

[7.x] [Security Solution][Detections] Adoption telemetry (#71102) #71504

Uh oh!

Conversation

rylnd commented Jul 13, 2020

Uh oh!

kibanamachine commented Jul 13, 2020

💛 Build succeeded, but was flaky

Test Failures

Standard Out

Stack Trace

Build metrics

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants