Closed
Description
Found in 7.0.0-alpha1
The following errors were spotted in Elasticsearch's logs for the gallery1 job:
[2018-07-16T11:32:43,808][ERROR][o.e.x.m.j.p.l.CppLogMessageHandler] [gallery1_20180716-1131_700-alpha1_2527] [autodetect/7377] [CMultimodalPriorUtils.h@236] Unable to bracket left percentile = -11.2898, (a,b) = (-1.79769e+308,-1.79769e+308), (f(a),f(b)) = (7.61041,7.61041)
...
[2018-07-16T11:34:02,104][ERROR][o.e.x.m.j.p.l.CppLogMessageHandler] [gallery1_20180716-1131_700-alpha1_2527] [autodetect/7377] [CMultimodalPriorUtils.h@236] Unable to bracket left percentile = -11.2898, (a,b) = (-1.79769e+308,-1.79769e+308), (f(a),f(b)) = (7.58798,7.58798) | repeated [9]
Analysis config
{
"bucket_span": "1h",
"influencers": [
"clientip"
],
"detectors": [
{
"over_field_name": "clientip",
"by_field_name": "status",
"function": "rare"
},
{
"over_field_name": "clientip",
"by_field_name": "uri",
"function": "freq_rare"
},
{
"over_field_name": "clientip",
"by_field_name": "status",
"function": "high_count"
},
{
"over_field_name": "clientip",
"by_field_name": "uri",
"function": "high_count"
},
{
"over_field_name": "clientip",
"by_field_name": "method",
"function": "sum",
"field_name": "bytes"
}
]
}Full job log
[2018-07-16T11:32:02,512][INFO ][o.e.x.m.a.TransportPutDatafeedAction] [_bx3G7m] Created datafeed [datafeed-gallery1_20180716-1131_700-alpha1_2527]
[2018-07-16T11:32:02,666][INFO ][o.e.x.m.j.p.a.AutodetectProcessManager] [_bx3G7m] Opening job [gallery1_20180716-1131_700-alpha1_2527]
[2018-07-16T11:32:02,681][INFO ][o.e.c.r.a.AllocationService] [_bx3G7m] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.ml-notifications][0]] ...]).
[2018-07-16T11:32:02,752][INFO ][o.e.x.m.j.p.a.AutodetectProcessManager] [_bx3G7m] [gallery1_20180716-1131_700-alpha1_2527] Loading model snapshot [N/A], job latest_record_timestamp [N/A]
[2018-07-16T11:32:02,938][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [gallery1_20180716-1131_700-alpha1_2527] [autodetect/7377] [CResourceMonitor.cc@66] Setting model memory limit to 1024 MB
[2018-07-16T11:32:03,012][INFO ][o.e.x.m.j.p.a.AutodetectProcessManager] [_bx3G7m] Successfully set job state to [opened] for job [gallery1_20180716-1131_700-alpha1_2527]
[2018-07-16T11:32:03,135][INFO ][o.e.x.m.d.DatafeedJob ] [gallery1_20180716-1131_700-alpha1_2527] Datafeed started (from: 1970-01-01T00:00:00.000Z to: 2018-07-16T10:32:03.037Z) with frequency [600000ms]
[2018-07-16T11:32:03,812][INFO ][o.e.c.m.MetaDataCreateIndexService] [_bx3G7m] [.ml-state] creating index, cause [auto(bulk api)], templates [.ml-state], shards [1]/[1], mappings [doc]
[2018-07-16T11:32:03,812][INFO ][o.e.c.r.a.AllocationService] [_bx3G7m] updating number_of_replicas to [0] for indices [.ml-state]
[2018-07-16T11:32:03,966][INFO ][o.e.c.r.a.AllocationService] [_bx3G7m] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.ml-state][0]] ...]).
[2018-07-16T11:32:08,181][INFO ][o.e.x.m.j.p.DataCountsReporter] [_bx3G7m] [gallery1_20180716-1131_700-alpha1_2527] 10000 records written to autodetect; missingFieldCount=0, invalidDateCount=0, outOfOrderCount=0
[2018-07-16T11:32:11,204][INFO ][o.e.x.m.j.p.DataCountsReporter] [_bx3G7m] [gallery1_20180716-1131_700-alpha1_2527] 20000 records written to autodetect; missingFieldCount=0, invalidDateCount=0, outOfOrderCount=0
[2018-07-16T11:32:14,013][INFO ][o.e.x.m.j.p.DataCountsReporter] [_bx3G7m] [gallery1_20180716-1131_700-alpha1_2527] 30000 records written to autodetect; missingFieldCount=0, invalidDateCount=0, outOfOrderCount=0
[2018-07-16T11:32:18,171][INFO ][o.e.x.m.j.p.DataCountsReporter] [_bx3G7m] [gallery1_20180716-1131_700-alpha1_2527] 40000 records written to autodetect; missingFieldCount=0, invalidDateCount=0, outOfOrderCount=0
[2018-07-16T11:32:19,869][INFO ][o.e.x.m.j.p.DataCountsReporter] [_bx3G7m] [gallery1_20180716-1131_700-alpha1_2527] 50000 records written to autodetect; missingFieldCount=0, invalidDateCount=0, outOfOrderCount=0
[2018-07-16T11:32:22,399][INFO ][o.e.x.m.j.p.DataCountsReporter] [_bx3G7m] [gallery1_20180716-1131_700-alpha1_2527] 60000 records written to autodetect; missingFieldCount=0, invalidDateCount=0, outOfOrderCount=0
[2018-07-16T11:32:24,023][INFO ][o.e.x.m.j.p.DataCountsReporter] [_bx3G7m] [gallery1_20180716-1131_700-alpha1_2527] 70000 records written to autodetect; missingFieldCount=0, invalidDateCount=0, outOfOrderCount=0
[2018-07-16T11:32:26,125][INFO ][o.e.x.m.j.p.DataCountsReporter] [_bx3G7m] [gallery1_20180716-1131_700-alpha1_2527] 80000 records written to autodetect; missingFieldCount=0, invalidDateCount=0, outOfOrderCount=0
[2018-07-16T11:32:28,463][INFO ][o.e.x.m.j.p.DataCountsReporter] [_bx3G7m] [gallery1_20180716-1131_700-alpha1_2527] 90000 records written to autodetect; missingFieldCount=0, invalidDateCount=0, outOfOrderCount=0
[2018-07-16T11:32:30,848][INFO ][o.e.x.m.j.p.DataCountsReporter] [_bx3G7m] [gallery1_20180716-1131_700-alpha1_2527] 100000 records written to autodetect; missingFieldCount=0, invalidDateCount=0, outOfOrderCount=0
[2018-07-16T11:32:43,808][ERROR][o.e.x.m.j.p.l.CppLogMessageHandler] [gallery1_20180716-1131_700-alpha1_2527] [autodetect/7377] [CMultimodalPriorUtils.h@236] Unable to bracket left percentile = -11.2898, (a,b) = (-1.79769e+308,-1.79769e+308), (f(a),f(b)) = (7.61041,7.61041)
[2018-07-16T11:32:48,308][INFO ][o.e.x.m.j.p.DataCountsReporter] [_bx3G7m] [gallery1_20180716-1131_700-alpha1_2527] 200000 records written to autodetect; missingFieldCount=0, invalidDateCount=0, outOfOrderCount=0
[2018-07-16T11:33:09,701][INFO ][o.e.x.m.j.p.DataCountsReporter] [_bx3G7m] [gallery1_20180716-1131_700-alpha1_2527] 300000 records written to autodetect; missingFieldCount=0, invalidDateCount=0, outOfOrderCount=0
[2018-07-16T11:33:35,647][INFO ][o.e.x.m.j.p.DataCountsReporter] [_bx3G7m] [gallery1_20180716-1131_700-alpha1_2527] 400000 records written to autodetect; missingFieldCount=0, invalidDateCount=0, outOfOrderCount=0
[2018-07-16T11:33:58,145][INFO ][o.e.x.m.j.p.DataCountsReporter] [_bx3G7m] [gallery1_20180716-1131_700-alpha1_2527] 500000 records written to autodetect; missingFieldCount=0, invalidDateCount=0, outOfOrderCount=0
[2018-07-16T11:34:02,024][INFO ][o.e.x.m.d.DatafeedJob ] [gallery1_20180716-1131_700-alpha1_2527] Lookback has finished
[2018-07-16T11:34:02,024][INFO ][o.e.x.m.d.DatafeedManager] [no_realtime] attempt to stop datafeed [datafeed-gallery1_20180716-1131_700-alpha1_2527] for job [gallery1_20180716-1131_700-alpha1_2527]
[2018-07-16T11:34:02,024][INFO ][o.e.x.m.d.DatafeedManager] [no_realtime] try lock [20s] to stop datafeed [datafeed-gallery1_20180716-1131_700-alpha1_2527] for job [gallery1_20180716-1131_700-alpha1_2527]...
[2018-07-16T11:34:02,025][INFO ][o.e.x.m.d.DatafeedManager] [no_realtime] stopping datafeed [datafeed-gallery1_20180716-1131_700-alpha1_2527] for job [gallery1_20180716-1131_700-alpha1_2527], acquired [true]...
[2018-07-16T11:34:02,026][INFO ][o.e.x.m.d.DatafeedManager] [no_realtime] datafeed [datafeed-gallery1_20180716-1131_700-alpha1_2527] for job [gallery1_20180716-1131_700-alpha1_2527] has been stopped
[2018-07-16T11:34:02,103][INFO ][o.e.x.m.j.p.a.AutodetectProcessManager] [_bx3G7m] Closing job [gallery1_20180716-1131_700-alpha1_2527], because [close job (api)]
[2018-07-16T11:34:02,104][ERROR][o.e.x.m.j.p.l.CppLogMessageHandler] [gallery1_20180716-1131_700-alpha1_2527] [autodetect/7377] [CMultimodalPriorUtils.h@236] Unable to bracket left percentile = -11.2898, (a,b) = (-1.79769e+308,-1.79769e+308), (f(a),f(b)) = (7.58798,7.58798) | repeated [9]
[2018-07-16T11:34:02,104][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [gallery1_20180716-1131_700-alpha1_2527] [autodetect/7377] [CCmdSkeleton.cc@45] Handled 511910 records
[2018-07-16T11:34:02,107][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [gallery1_20180716-1131_700-alpha1_2527] [autodetect/7377] [CAnomalyJob.cc@1441] Pruning all models
[2018-07-16T11:34:06,055][INFO ][o.e.c.m.MetaDataMappingService] [_bx3G7m] [.ml-anomalies-shared/GvGFj83cQw6ELHcJmiLnlA] update_mapping [doc]
[2018-07-16T11:34:06,287][INFO ][o.e.x.m.j.p.a.NativeAutodetectProcess] [gallery1_20180716-1131_700-alpha1_2527] State output finished
[2018-07-16T11:34:06,287][INFO ][o.e.x.m.j.p.a.o.AutoDetectResultProcessor] [gallery1_20180716-1131_700-alpha1_2527] 3577 buckets parsed from autodetect output
[2018-07-16T11:34:06,680][INFO ][o.e.x.m.j.p.a.AutodetectCommunicator] [gallery1_20180716-1131_700-alpha1_2527] job closed