Feature Request: configurable telemetry

[aThorp96]

In a web service which may be making tens of thousands of call to Github, it's desirable to have instrumentation to identify hot code paths, bad tenants, or poor optimizations resulting in extra API calls.

Currently there are three ways to gain metrics on the API requests:

1. Wrapping every API call with a wrapper that emits the telemetry data
   Ugly, a little verbose, easy for new code to accidentally forget to wrap the API call or manually emit the telemetry data, isn't accurate when paging
2. Wrapping API client access with a wrapper which emits the telemetry data every time get the client
   Pretty clean, very easy to subvert by saving the returned client in a variable, isn't accurate when paging
3. Wrapping the entire struct and every necessary struct method
   Very safe from accidentally using the client without metrics, boilerplate hell, isn't accurate when paging

It would be very helpful for the library to provide a mechanism to instrument the API usage with some form of telemetry. Any kind of pluggable middleware, callbacks, or hook would add a lot of utility. If there is a way to do this which is both hard to accidentally workaround and doesn't require a lot of boilerplate, that would be awesome to know about and document!

One very simple or rudimentary option would be to add an optional field Client.OnDo: func() (also settable via Client.WithCallback() or something) which, if set, would be called during Client.bareDo just before caller.Do(). A example usage might be:

func metricEmitter(org: string, repo: string) func() {
    return func() {
        metrics.emit("api-request", tags: map[string]string{"organization": org, "repository": repo})
    }

...
prs, _, err := client
  .WithCallback(metricEmitter(request.Args.Owner, request.Args.Repository))
  .PullRequests
  .List(request.Args.Owner, request.Args.Repository)

[gmlewis]

Very interesting, @aThorp96!
Would you like to work on adding an optional feature like what you have suggested?
If so, I'll assign this issue to you. If not, I'll open this up to our other awesome contributors.

[aThorp96]

I wouldn't mind working on it, but I have not contributed to this project before and would at least want to get the community's thoughts on the design. There's a lot of ways to skin this cat with various tradeoffs 😁

I'd also be interested if anyone has found patterns for instrumenting their API usage besides the three I've thought of

[stevehipwell]

@aThorp96 would you mind expanding on what you can't get using OpenTelemetry on the client transport?

[Not-Dhananjay-Mishra]

Hi, I’d like to work on this. My plan is to add a wrapper that runs before/after client function calls to log request details, track how many requests are made, which function made them, and whether they succeeded or failed.

[gmlewis]

@Not-Dhananjay-Mishra - as mentioned in #3681, we will not move forward with this issue until we hear back from @aThorp96 in regards to the question above.