Use Kubernetes API to scrape logs instead of reading log files from disk

[GreyLilac09]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

1. Collect logs while running Vector as non-root
2. Collect logs while running Vector as STS or Deployment

Attempted Solutions

Alloy employs this strategy to collect logs. We prefer Vector's configuration flexibility and metrics/non-Kube applications, so it would be very useful to have it in Vector.

Proposal

Create a new mode config on Kubernetes logs

This mode causes Vector to read logs via the Kubernetes API instead of reading it from files on disk.

References

No response

Version

No response

[LinPr]

yeah, that will be cool, I just read the source code of alloy, and found the alloy can fetch logs stream from apiserver via client-go, (actually the logs api will forward request to kubelet, and kubelet will do most of the work)

currently vector is more like the promtail, only can tail log files from the mouted kubernetes log dir , this need to access the filesystem in the deployment stage

[titaneric]

I would like to dig into this issue. I have read how Alloy implement it and I could try to port them into Vector.