Open
Description
openedon Sep 20, 2024
Background:
#40854 adds a fallback to fix index 0 conflict for the DynamoDB events. We kept the error level log for the original conflict error even though the fallback could be successful afterwards, so we could capture which events might still send the conflict index. However, many customers monitor Teleport's log and these errors give false positives.
Expected behavior:
- (optional) We should fix the origins that still send conflicting indexes to avoid the fallback, if possible
- If the fallback is successful, the original error must not generate an error log
Current behavior:
The following error is spamming despite followed by a successful overwrite:
2024-09-19T05:43:06Z ERRO [DYNAMODB] Conflict on event session_id and event_index error:[
ERROR REPORT:
Original Error: *trace.AlreadyExistsError ConditionalCheckFailedException: The conditional request failed
{
RespMetadata: {
StatusCode: 400,
RequestID: "3V6F4JBK7AD6ELMLI2QO1F0LK3VV4KQNSO5AEMVJF66Q9ASUAAJG"
},
Message_: "The conditional request failed"
}
Stack Trace:
github.com/gravitational/teleport/lib/events/dynamoevents/dynamoevents.go:1089 github.com/gravitational/teleport/lib/events/dynamoevents.convertError
github.com/gravitational/teleport/lib/events/dynamoevents/dynamoevents.go:455 github.com/gravitational/teleport/lib/events/dynamoevents.(*Log).putAuditEvent
github.com/gravitational/teleport/lib/events/dynamoevents/dynamoevents.go:376 github.com/gravitational/teleport/lib/events/dynamoevents.(*Log).EmitAuditEvent
(zendesk ticket 11706)
Bug details:
- Teleport version: v16
- Recreation steps: DynamoDB events + session.command
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
