Documentation Updates for RTVI version 1.0.0 and corresponding client-js and client-react libraries. #265

mattieruth · 2025-06-19T00:54:04Z

This PR updates the client-side documentation to match the new PipecatClient and the underlying, simplified RTVI Message protocol. Corresponding code PRs:

This should be merged after the above PRs have been merged and released.

Copilot

Pull Request Overview

This PR updates the client-side documentation to reflect the migration from RTVIClient to the new PipecatClient, updating import names, code examples, and documentation files across the JS and React SDKs. Key changes include:

Renaming of client classes, hooks, and components from RTVI* to Pipecat*.
Updates to code examples and configuration details in multiple transport and API reference documentations.
Removal of outdated documentation pages and streamlining of migration guides.

Reviewed Changes

Copilot reviewed 24 out of 26 changed files in this pull request and generated 1 comment.

File	Description
mint.json	Updated navigation pages to include new client documentation and migration guides.
client/react/*.md	Updated migration guides, introductions, hooks, and components to replace RTVI references with Pipecat references.
client/js/*.md	Revised API references, transport guides, and examples to use the new PipecatClient naming.

Comments suppressed due to low confidence (1)

client/js/api-reference/client-constructor.mdx:68

[nitpick] Consider renaming the instance variable (e.g., to 'pcClient') rather than reusing the class name 'PipecatClient' to improve code clarity and avoid confusion.

const PipecatClient = new PipecatClient({

client/react/hooks.mdx

kompfner · 2025-06-25T14:13:53Z

client/introduction.mdx

@@ -3,6 +3,10 @@ title: "Client SDKs"
 description: "Client libraries for building real-time AI applications with Pipecat"
 ---

+<Warning>
+  The Client SDKs are currently in transition to a new, simpler API design. The js and react libraries have already been deployed with these changes. Their corresponding documentation along with this top-level documentation has been updated to reflect the latest changes, but some specific API references may still be in the old format. For transitioning to the new API, please refer to the [migration guide](/client/migration-guide).


Which API references would still be in the old format?

Oh, do you mean the API references for the other libraries (iOS, Android, React Native)?

no, i think this was the first paragraph i wrote and figured i'd miss something, so betting on failure, but i feel pretty confident in my updates now 😂 -- i'll remove that language. maybe also specifically point out that iOS, Android, ReactNative have not been ported and therefore their documentation has also not been updated.

kompfner · 2025-06-25T14:25:11Z

client/introduction.mdx

    } catch (e) {
      console.error(e.message);
    }

-    // Events
-    rtviClient.on(RTVIEvent.Connected, () => {
+    // Alternative Event Handling Setup


What's this pattern an alternative to? Specifying handlers in the callbacks constructor arg?

looks like i just updated what was here in this case and noticed that it was setting up callbacks two different ways, so added the "alternative" language. not sure what's best here really...

Seems reasonable to show both patterns, as you do here! Maybe a comment tweak would've made it clearer to me, like // Events (alternative to constructor-provided callbacks)

kompfner · 2025-06-25T14:30:35Z

client/introduction.mdx

@@ -111,66 +118,71 @@ All Pipecat client SDKs provide:
    });

    try {
-      await rtviClient.connect();
+      // you can provide your Daily connection credentials directly as


Ah took me a moment to remember that the connect endpoint (shown below) is the way to get the connection credentials. This comment didn't quite get me there quickly—wondering if it could be clearer, like:

Below, we use a REST endpoint to fetch connection credentials for our Daily Transport. Alternatively, you could provide those credentials directly to `connect()`.

kompfner · 2025-06-25T14:32:26Z

client/introduction.mdx

      console.log("[EVENT] User disconnected");
    });
    ```

+    ```react react
+    // Example: Using PipecatClient in a React component


Is this missing?

kompfner · 2025-06-25T14:32:35Z

client/introduction.mdx

+  ```
+
+  ```react react
+  // Example: Using PipecatClient in a React component


Is this missing?

kompfner · 2025-06-25T14:35:59Z

client/introduction.mdx

-  console.error(e.message);
-}
-```
+  await pcClient.connect({url: "https://your-daily-room-url", token: "your-daily-token"});


Like that you found a way to also illustrate the alternative method of connecting 👍

kompfner · 2025-06-25T14:43:18Z

client/introduction.mdx

-            content: "tell me a joke",
-          },
-        ],
+<Tab title="Custom Messaging">


Philosophical question: if we're no longer selling Pipecat Client as a tool for sending "non-custom" bot configuration messages (we're recommending folks do their bot configuration server-side), then can we just call this "messaging" as opposed to "custom messaging"?

i don't think i was considering anything about configuration when i labeled this. I believe i was thinking that RTVI is a bunch of messages, this looks at sending custom ones. But also, that seems nuanced and agree it could just be "messaging" 🤷‍♀️

i think i had also planned on maybe adding more tabs for any other most-common patterns, but only did messaging for now as a replacement for what was here, which was an "actions" example.

kompfner · 2025-06-25T15:21:19Z

client/js/api-reference/client-constructor.mdx

-`@pipecat-ai/daily-transport`) exports multiple transports classes, you can
-specify which to use here.
+<Note>
+For best practice, we recommend you pass in a transport using its factory function as access to the transport is typically unnecessary. For advanced use cases that do require access to the transport, we recommend doing so via the `PipecatClient.transport` property.


Not sure I follow—how is creating a transport via a factory function v a constructor related to whether or not you then hold a reference to the transport?

um... uh... good point. you know, i think i initially introduced factory functions thinking we would pass the function to the constructor for the constructor to call, but ended up realizing you need to provide constructor opts to the transport, so backed into using the factory function but calling it to get the transport when just doing a new myTransport(opts) would work just the same.

The real note here is that the best practice is to construct the transport inline in the client constructor so that you are forced to get the transport (if needed) from the client, and not tempted to just hold on to the one you made and passed in, since client.transport !== rawTransport (remember that client.transport returns a wrapped version of the transport with a safety harness).

i.e.

const pcClient = new PipecatClient({transport: new DailyTransport(dailyOpts)}); // forces me to do (pcClient.transport as DailyClient).preAuth({url});

vs.

const transport = new DailyTransport(dailyOpts); const pcClient = new PipecatClient({transport}); // this works but now i'm more likely to do: transport.preAuth({url});

GIVEN THAT BREATHY RESPONSE, should i get rid of the create() factory methods? you're right, they don't really accomplish anything. it's more about documenting best practice.

should i get rid of the create() factory methods

Maybe! I suppose they don't really seem necessary.

The real note here is that the best practice is to construct the transport inline in the client constructor so that you are forced to get the transport (if needed) from the client, and not tempted to just hold on to the one you made and passed in

👍 . Maybe the action item is to simplify this note to just focus on this recommendation, then? Like:

"As a best practice, we recommend you construct the transport inline in the client constructor, as opposed to holding a reference to it. Access to the transport is typically unnecessary. For advanced use cases that do require access to the transport, we recommend doing so via the PipecatClient.transport property, which provides some additional safeguards."

done and factory functions are now removed from code and docs :)

kompfner · 2025-06-25T15:25:29Z

client/js/api-reference/messages.mdx

@@ -1,45 +1,288 @@
 ---
-title: "Messages"
+title: "Custom Messaging"


See earlier note about the need for "custom"

same response. I'm happy either way, but let me know your thoughts after reading why I used "Custom messaging" here.

No strong feelings about this! Keeping it as "custom messaging" sounds fine to me.

kompfner · 2025-06-25T15:26:56Z

client/js/api-reference/messages.mdx


-Messages are used by a client for passing an instruction to a bot that typically results in a [callback or event](./callbacks).
+Oftentimes clients need to provide configuration data to the server when connecting. This can include things like preferred language, user preferences, initial messages, or any other data that the server needs to know about the client. This must occur before the bot is started and therefore not part of the RTVI standard, but rather a custom implementation. That said, the PipecatClient makes it easy to send this data as part of the `connect()` method, by passing an object with the `requestData` property. Your server endpoint can then handle this data as needed. In the example below, we demonstrate sending an initial prompt and preferred language to the server when connecting.


Nit: I'm noticing that sometimes we say PipecatClient and sometimes PipecatClient.

yeah... that's probably laziness or copilot'ing (or i guess those are one and the same 😂 ) -- What should I try to be consistent towards? i'm guessing PipecatClient?

kompfner · 2025-06-25T15:29:55Z

client/js/api-reference/messages.mdx


-Examples of messages include:
+<CodeGroup>


great example!

thanks ☺️

kompfner · 2025-06-25T15:31:21Z

client/js/api-reference/messages.mdx

+        # Kick off the conversation
+        await task.queue_frames([context_aggregator.user().get_context_frame()])
+
+    @daily_transport.event_handler("on_participant_left")


stepping back: would it be better if this were an rtvi event?

maybe. other than supporting the new behaviors and deprecating the ones going away, though, i didn't do a ton of work on the pipecat side of all this. maybe we do a better sweep of things like this in post?

kompfner · 2025-06-25T15:42:57Z

client/js/api-reference/messages.mdx

+
+```python bot
+# Listen for a RTVIClientMessageFrame and push a STTUpdateSettingsFrame
+class CustomFrameProcessor(FrameProcessor):


I'm sure we talked about this on a call and I don't remember, but: this is the recommended way for a bot to handle a message? Creating a custom frame processor feels like rather a lot of verbose boilerplate for a customer.

It also feels a bit unintuitive to me...a client message feels conceptually like something that is coming from outside the pipeline and controlling it, not like something that automatically originates within it. I would've expected something more like an event handler in the bot's code file.

Oh wait, I also see an example below of using an "on_client_message" event handler. That feels much more natural to me.

When are we recommending using one approach v the other? Why are we only showing the custom processor approach in this example, and showing both approaches in the other example?

Same question re rtvi.send_server_response v pushing a RTVIServerResponseFrame

ah! GREAT question. walking away from that discussion, my understanding was that handlers are definitely the preferred and simpler approach, so long as you do not need to mess with the pipeline. In this example, I'm updating the STT language and thought that action required a FrameProcessor, but I could be wrong.

...

I just checked with mark and he agrees, this sort of action is best done as a frame processor.

so long as you do not need to mess with the pipeline

Not sure I understand—what's the downside to calling task.queue_frame(frame) inside an event handler, if event handlers are otherwise the preferred and simpler approach?

um. uh. @aconchillo ? @markbackman ?

(Also, just realized that using the event handler makes it so you don't have to be so careful about where in your pipeline you put your custom frame processor for it to have the desired effect—in the event handler you'll just shove frames into the top of the pipeline)

kompfner · 2025-06-25T15:51:47Z

client/js/api-reference/messages.mdx

+class CustomFrameProcessor(FrameProcessor):
+    async def process_frame(self, frame: Frame, direction: FrameDirection):
+        await super().process_frame(frame, direction)
+		if isinstance(frame, STTUpdateSettingsFrame):


The presence of this frame tells us that the language has been updated only if this CustomFrameProcessor is downstream from the STT service, right?

Is it worth showing how the CustomFrameProcessor is inserted into the pipeline? (And maybe also showing how the CustomObserver is wired up, for that matter?)

maybe? though, that feels more a documentation on how observers and processors work and therefore the job of the documentation on those, rather than client-side documentation on sending rtvi messages. you have to draw the line somewhere.

kompfner · 2025-06-25T15:55:15Z

client/js/migration-guide.mdx

+});
+```
+
+5. Pipeline Configuration Initialization


doh. yes, i forgot to come back to this.

kompfner · 2025-06-25T17:47:42Z

client/js/transports/transport.mdx

@@ -3,23 +3,27 @@ title: "Transport Overview"
 sidebarTitle: "Overview"
 ---

-Transports are the means by which Pipecat clients communicate with their bot services. They handle both message exchange between client and server and real-time media transport. Pipecat implements the RTVI standard for these communications.
+Transports are the means by which Pipecat clients communicate with their bot services. They handle both message exchange between client and server and real-time media transport.


There was list of additional transport functionality (like device management) somewhere else in these docs. Should it go here too?

kompfner · 2025-06-25T17:50:06Z

client/migration-guide.mdx

+description: "A Guide to migrating from RTVIClient to PipecatClient"
+---
+
+This guide will cover the high-level changes between the old `RTVIClient` and the new `PipecatClient`. For specific code updates, refer to the platform-specific migration guides.


Should we also have a warning somewhere in this guide—like we have in the introduction page—that not all client libraries have yet been updated to the new patterns?

kompfner · 2025-06-25T17:58:00Z

client/rtvi-standard.mdx

+
+#### client-ready
+
+Indicates that the client is ready to receive messages and interact with the server. Typically send after the transport media channels have connected.


nit:

Suggested change

Indicates that the client is ready to receive messages and interact with the server. Typically send after the transport media channels have connected.

Indicates that the client is ready to receive messages and interact with the server. Typically sent after the transport media channels have connected.

kompfner · 2025-06-25T18:00:25Z

client/rtvi-standard.mdx

+
+#### bot-transcription
+
+Transcription of the bot's speech. Note: This protocol currently does not the user transcription format to support real-time timestamping for bot transcriptions. The event is typically sent for each sentence of the bot's response. In the future, this may be updated to include the user with real-time transcription capabilities. Currently, most TTS services do not support (or support well), accurate timing information to make this possible. For now, if you want to attempt real-time transcription to match your bot's speaking, you can try using the `bot-tts-text` message type.


missing a word here? This protocol currently does not the user transcription

was i drunk? pushing an update that's hopefully much better...

was i drunk?

that would've made writing this huge amount of documentation more fun! (i'm trying to give a compliment: very impressive overhaul of the docs)

kompfner · 2025-06-25T18:03:24Z

client/rtvi-standard.mdx

+    The payload of the message, which can be any data structure relevant to the message type.
+</ParamField>
+
+## RTVI Message Types


Wonder if there should be a structured way of denoting who (the bot or client) is expected to send each of the messages. Having them mixed together with no labels could be a bit confusing.

good idea... 🤔

:) emojis :)

🤖 🏄

kompfner · 2025-06-25T18:03:50Z

client/rtvi-standard.mdx

+
+### Client-Server Messaging
+
+#### server-message


apparently 🤦‍♀️

kompfner

Pre-approving, as all comments (except one new question) have been acknowledged. Nice work!

mattieruth · 2025-06-25T21:54:40Z

I have made all the suggested updates and filled in the missing sections!

marcus-daily · 2025-06-26T11:48:27Z

client/js/api-reference/messages.mdx

+                    ),
+                )
+                return
+			else:


Looks like something weird's happened with the indentation here -- I think this is using tabs for the else and spaces for the if

good catch. not sure where those tabs came from 🤔

marcus-daily · 2025-06-26T11:54:04Z

client/js/api-reference/messages.mdx

+## From inside an Observer, call `send_server_message` directly on your rtvi instance
+class CustomObserver(BaseObserver):
+	async def on_push_frame(self, data: FramePushed):
+        if isinstance(frame, STTUpdateSettingsFrame):


Again it looks like the indentation is weird here (tabs vs spaces mixed)

marcus-daily · 2025-06-26T11:58:22Z

client/js/migration-guide.mdx

+**New**
+```javascript
+const client = new PipecatClient({
+  transport: new DailyTransport(),


From the discussions I thought we were going to pass in a factory rather than the transport directly?

This was the topic of an earlier comment discussion, in which @mattieruth explains:

"i think i initially introduced factory functions thinking we would pass the function to the constructor for the constructor to call, but ended up realizing you need to provide constructor opts to the transport"

So she made the call to just ditch the factory pattern since it wasn't adding anything.

exactly what kompfner said i said. I had to back away because you need to be able to pass args to the transport constructor/factory function. If you know of a way around this, i'm all ears. The only thing i came up with was to pass the args as a separate param, but that felt ugly. So i'm just trying to document best practice and update all the examples to show inline construction.

…nd methods pages

…ient messages

mattieruth requested review from markbackman and Copilot June 19, 2025 00:54

This comment was marked as outdated.

Sign in to view

mattieruth force-pushed the v2-client-docs branch 2 times, most recently from 4960756 to 7fee2fb Compare June 19, 2025 14:34

mattieruth changed the title ~~A start to updating docs for v2 client changes~~ Documentation Updates for RTVI version 1.0.0 and corresponding client-js and client-react libraries. Jun 24, 2025

mattieruth marked this pull request as ready for review June 24, 2025 17:50

mattieruth requested review from Copilot, kompfner, aconchillo, jptaylor, filipi87 and marcus-daily June 24, 2025 17:51

Copilot AI reviewed Jun 24, 2025

View reviewed changes

client/react/hooks.mdx Outdated Show resolved Hide resolved

kompfner reviewed Jun 25, 2025

View reviewed changes

mattieruth requested a review from Regaddi June 25, 2025 14:59

kompfner reviewed Jun 25, 2025

View reviewed changes

kompfner approved these changes Jun 25, 2025

View reviewed changes

mattieruth mentioned this pull request Jun 25, 2025

RTVI 1.0.0 Updates 🎉 pipecat-ai/pipecat#1999

Open

marcus-daily reviewed Jun 26, 2025

View reviewed changes

mattieruth added 17 commits June 27, 2025 14:44

A start to updating docs for v2 client changes

e158382

Removed pages no longer relevant and updated the client-constructor a…

f8730f4

…nd methods pages

Documented/updated the RTVI errors page

40b1687

Update messages page to outline all the ways to send/handle server-cl…

b183fd9

…ient messages

Update callbacks page

d7450b4

Update Transport docs to new version

40145e3

update React docs

7b9806a

Add version info to client/bot-ready RTVI message standard

b257d52

Add an section to the RTVI bot/client-ready protocols

b533292

minor edit

805e676

Update various overlooked RTVIClient /old naming references

0f98b6f

copilot nits

d30261b

use underscores in keys

346887f

Updates from PR Review

516bf6d

removed factory function references

28b00dd

Fix tab/spacing

0641a11

Add better documentation of the llm search bits

3c8e116

mattieruth force-pushed the v2-client-docs branch from a914407 to 3c8e116 Compare June 27, 2025 22:29


		Messages are used by a client for passing an instruction to a bot that typically results in a [callback or event](./callbacks).
		Oftentimes clients need to provide configuration data to the server when connecting. This can include things like preferred language, user preferences, initial messages, or any other data that the server needs to know about the client. This must occur before the bot is started and therefore not part of the RTVI standard, but rather a custom implementation. That said, the PipecatClient makes it easy to send this data as part of the `connect()` method, by passing an object with the `requestData` property. Your server endpoint can then handle this data as needed. In the example below, we demonstrate sending an initial prompt and preferred language to the server when connecting.


		#### client-ready

		Indicates that the client is ready to receive messages and interact with the server. Typically send after the transport media channels have connected.


		#### bot-transcription

		Transcription of the bot's speech. Note: This protocol currently does not the user transcription format to support real-time timestamping for bot transcriptions. The event is typically sent for each sentence of the bot's response. In the future, this may be updated to include the user with real-time transcription capabilities. Currently, most TTS services do not support (or support well), accurate timing information to make this possible. For now, if you want to attempt real-time transcription to match your bot's speaking, you can try using the `bot-tts-text` message type.


		Examples of messages include:
		<CodeGroup>

Documentation Updates for RTVI version 1.0.0 and corresponding client-js and client-react libraries. #265

Are you sure you want to change the base?

Documentation Updates for RTVI version 1.0.0 and corresponding client-js and client-react libraries. #265

Conversation

mattieruth commented Jun 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kompfner Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kompfner Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

mattieruth commented Jun 19, 2025 •

edited

Loading

kompfner Jun 25, 2025 •

edited

Loading

kompfner Jun 25, 2025 •

edited

Loading

kompfner Jun 25, 2025 •

edited

Loading

kompfner Jun 25, 2025 •

edited

Loading