add tracing support for the embeddings router filter

[codefromthecrypt]

Description:

@Xunzhuo asked me about tracing support for embeddings, and here's info from @mikeldking (phoenix lead)

We do have some ways to evaluate embeddings, notably finding outliers (e.g. toxic content, out of bounds questions) as well as EDA (question space exploration)
But the benefit of logging embeddings does come at a storage cost.
In general if you're doing things like query expansion or agentic RAG, I can see it being useful to see the text getting embedded.

It makes sense to add support for the embeddings endpoint, by recording openai responses, openinference spans, and creating the impl like we did for chat completions.

[mathetake]

Let's aim for v0.4!!!

[codefromthecrypt]

Cool. I found 1 or 2 embedding requests so far that cause hiccups even before tracing occurs. I wont add tracing before 0.4 but will figure out if there is a bug somewhere and report back today.

[codefromthecrypt]

in one case it seems the OpenAI platform in an embedding error sends a response with 1 character less than the Content-Length it sets. I think that's not something we'll likely correct (clients may struggle, but same bug can happen in any SaaS or http service). Unless I find anything else, probably fine on embeddings.

[codefromthecrypt]

here are the two glitches and backfill of all test data without adding tracing yet. This ensures RAG apps that use token embeddings don't crash on the 0.3 release #1087

[codefromthecrypt]

about the content-length issue, it seems typical for extproc to adjust this, so I think the thing I found when upstream openai sends a buggy reply (content-length longer than the content) we can just fix this (by not crashing on EOF as long as we have valid json). Will give a go.

[codefromthecrypt]

the content-length situation isn't specific to embeddings as it is reproducible generally. I'm not sure we can affect it without changing the filter modes, as the crash seems to occur in envoy after response headers and before the body hook is sent back to the extproc. So, I added a new issue and we can forget about it here #1091

[codefromthecrypt]

@Xunzhuo @ion-elgreco @mikeldking I've not forgotten about this. Over the weekend I spend maybe a dozen hours getting things together, but this time wanted to get the spec polished before a PR here. Since recent spec polishing has been a good experience, I don't think this endangers the timeline and also helps avoid drift on our eventual PR. So you can subscribe to Arize-ai/openinference#2162 if you like

[codefromthecrypt]

Arize-ai/openinference#2210 is enough to proceed, so when this is up, I'll continue (have code on the laptop working so it shouldn't be later than next week)