Add e2e test infrastructure and implementations for AI Transport guides#3213
Add e2e test infrastructure and implementations for AI Transport guides#3213GregHolmes wants to merge 2 commits intomainfrom
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Introduce guides/ai-transport/ directory with full runnable code for all 8 AI Transport guides (4 providers x 2 streaming patterns), each with publisher, subscriber, and e2e tests hitting real LLM APIs and Ably. - OpenAI, Anthropic, Vercel AI SDK, and LangGraph providers - Message-per-token and message-per-response streaming patterns - 24 e2e tests total (3 per guide): lifecycle ordering, response reconstruction, and token concatenation verification - Yarn workspaces with shared Vitest config - Exclude guides/ from root Jest config
Move all guide contents into javascript/ subdirectories to prepare for future multi-language support (Java, Python, Swift). Update workspace paths, vitest config, and tsconfig extends accordingly.
GregHolmes
force-pushed
the
AIT-489-Introduce-E2E-testing-for-Guides-code
branch
from
6bb14d3 to
05b88e2
Compare
There was a problem hiding this comment.
Overall looks great, thanks for this! Couple of minor comments
I think there is an outstanding question of how we can get this to work for different languages, in which case I think we might want to consider execing processes for the publisher/subscriber and asserting on the stdout output (or something better?)
Either way, this is a great start for JS, so happy to get this merged, and then iterate from there
| }); | ||
|
|
||
| await publish(pubChannel, 'Reply with exactly: OK'); | ||
| await new Promise((resolve) => setTimeout(resolve, 2000)); |
There was a problem hiding this comment.
Using timeouts like this are prone to flake - instead, we could wrap the channel.subscribe call with a promise that resolves with the first message received. I expect we could benefit from a helper function that can be used more generally in these tests to yield messages from the subscription that match a predicate e.g. something like
async function waitForMessage(
channel: Ably.RealtimeChannel,
predicate: MessagePredicate = () => true,
timeoutMs = 2000
): Promise<Ably.Message>
| let doneTimer: ReturnType<typeof setTimeout> | null = null; | ||
|
|
||
| const resetTimer = () => { | ||
| if (doneTimer) clearTimeout(doneTimer); |
There was a problem hiding this comment.
This is breaking the eslint curly rule: Expected { after 'if' condition.eslintcurly
| switch (message.action) { | ||
| case 'message.create': | ||
| console.log('\n[Response started]', message.serial); | ||
| responses.set(message.serial, message.data as string); |
There was a problem hiding this comment.
This line has a ts error:
Argument of type 'string | undefined' is not assignable to parameter of type 'string'.
Type 'undefined' is not assignable to type 'string'.ts(2345)
and one on the line below
Type 'string | undefined' is not assignable to type 'string | null'.
Type 'undefined' is not assignable to type 'string | null'.ts(2322)
Can we please add a lint and lint:fix script to the top level package json?
2 participants
Introduce guides/ai-transport/ directory with full runnable code for all 8 AI Transport guides (4 providers x 2 streaming patterns), each with publisher, subscriber, and e2e tests hitting real LLM APIs and Ably.