Request/Response Logging for HTTP/gRPC

[yashrsharma44]

Request/Response Logging in HTTP/gRPC

This issue has been filed to list out the why do we need Request/Response logging for HTTP/gRPC in Thanos, and to track the progress of enabling it.

This issue is a part of #1706 #2694

Why do we need?

Currently, for those who don't want to invest on tracing in Thanos, the logging of Request/Response metadata and request-id in HTTP/gRPC can be very useful for tracking and getting details about them.

We can have the following features -

• request-id for each request made, so that we can track them
• response carrying the same request-id, so we can track them for debugging purposes
• duration taken by each of the request to be handled
• logging of errors for a given request, and selecting the level of logging based on error

How are we looking to implement?

Similar to instrumentation middleware, I am looking to write up middleware(for servers) and tripperwares(for clients) for the logging.
The logging would have some options much like grpc middleware -

• WithDecider - Decider whether we want to log in the given function signature or not. True for logging all the method calls
• WithLevel - Map the error code with the level

Other features

• We can obtain the flag of enabling the logging, it would be switched off by default
• We can also accept if we want to log in a certain log file, instead of stdout

I would be implementing this feature for Thanos, so if any suggestions are there, I would be really happy to discuss it. 🤓