Add support for Logstash Dead Letter Queue

[zamazan4ik]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

We want to read DLQ messages from Logstash with Vector.

Attempted Solutions

No solution is available in Vector right now. The only possible solution is to use other software like Logstash.

Proposal

Add support for Logstash DLQ. Probably directly in Logstash source plugin or via yet another plugin.

Possibly useful links:

• https://www.elastic.co/guide/en/logstash/8.4/dead-letter-queues.html
• https://www.elastic.co/guide/en/logstash/8.4/plugins-inputs-dead_letter_queue.html

References

No response

Version

No response

[jszwedko]

Thanks for filing this! Given the differences with receiving data from Logstash via its native protocol vs. reading DLQ messages from files, I think a new source, logstash_dlq would be better.