Skip to content

Error ingesting Office 365 events due to invalid IP address field #18587

Closed

Description

For confirmed bugs, please report:

Some (Exchange audit?) events from Office 365 Management API are causing an ingestion error due to an invalid IP address field (client.ip, maybe others).

May 14 16:51:23 ainfcp1esl00001 filebeat: 2020-05-14T16:51:23.611-0700#011WARN#011[elasticsearch]#011elasticsearch/client.go:384#011Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0x0, ext:63724844335, loc:(*time.Location)(nil)}, Meta:{"_id":"17efaaf6-8bb5-4829-27b1-08d7f6153bf3","pipeline":"filebeat-7.7.0-o365-audit-pipeline"}, Fields:{"agent":{"ephemeral_id":"c7b7e853-c8ba-4749-80e0-b52d98db95ed","hostname":"hostname.com","id":"9ff5gad7-a611-4443-bc73-3d377f156e21","type":"filebeat","version":"7.7.0"},"client":{"address":"[1.2.2.3]:7972","ip":"[1.2.2.3]","port":"7972"},"domain":"domain.com","ecs":{"version":"1.5.0"},"error":{"message":"GoError: cannot override existing key with client.ip"},"event":{"action":"Create","category":"web","code":"ExchangeItem","dataset":"o365.audit","id":"17efaag6-7bb5-4829-27b1-08d7f6153bf3","kind":"event","module":"o365","outcome":"success","provider":"Exchange","type":"info"},"fields":{},"fileset":{"name":"audit"},"host":{"architecture":"x86_64","containerized":false,"hostname":"hostname.com","id":"a962373b0c22485eae53ee6e404cd357","ip":["10.1.2.200"fe80::e75a:ad3:88ff:1ad0"],"mac":["00:50:56:a7:f8:4d"],"name":"hostname.com","os":{"codename":"Core","family":"redhat","kernel":"3.10.0-1127.el7.x86_64","name":"CentOS Linux","platform":"centos","version":"7 (Core)"}},"input":{"type":"o365audit"},"o365audit":{"ClientIP":"[1.2.2.3]:7972","ClientIPAddress":"[1.2.2.3]:7972","ClientInfoString":"Client=MSExchangeRPC","ClientProcessName":"OUTLOOK.EXE","ClientVersion":"16.0.12730.20144"
...
Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse field [client.ip] of type [ip] in document with id '17efaaf6-8bb5-4829-27b1-08d7f6153bf3'. Preview of field's value: '[1.2.2.3]'","caused_by":{"type":"illegal_argument_exception","reason":"'[1.2.2.3]' is not an IP string literal."}}

The important error messages being:

(error.message): GoError: cannot override existing key with client.ip

and

{"type":"mapper_parsing_exception","reason":"failed to parse field [client.ip] of type [ip]"
"caused_by":{"type":"illegal_argument_exception","reason":"'[1.2.2.3]' is not an IP string literal."}}

Caused by the following fields in the original O365 event:

"ClientIP":"[10.1.1.3]:7972",
"ClientIPAddress":"[10.1.1.3]:7972"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions