This SDK has been developed and tested on a esp32s3 and linux. You don't
need any physical hardware to run this SDK, but it will be very limited
(e.g. you won't hear an audio on Linux).
To use it on hardware purchase any of these microcontrollers. Others may work, but this is what has been developed against.
Clone this repository:
git clone --recursive https://github.com/pipecat-ai/pipecat-esp32.git
Install the ESP-IDF toolchain following these instructions.
After that, just open a terminal and load ESP-IDF tools:
source PATH_TO_ESP_IDF/export.sh
We also need to set a few environment variables before we can build:
export WIFI_SSID=foo
export WIFI_PASSWORD=bar
export PIPECAT_SMALLWEBRTC_URL=URL (e.g. http://192.168.1.10:7860/api/offer)
where WIFI_SSID and WIFI_PASSWORD are just needed to connect the device to
the network. PIPECAT_SMALLWEBRTC_URL is the URL endpoint to connect to your
Pipecat bot.
Go inside the esp32-s3-box-3 directory.
The first thing to do is to set the desired target, for example:
idf.py --preview set-target esp32s3
You can also set linux instead of esp32s3.
Then, just build:
idf.py build
If you built for linux you can run the binary directly:
./build/src.elf
If you built for esp32s3 you can flash your device using the following commands:
idf.py -p /dev/ttyACM0 flash
where /dev/ttyACM0 is the device where your ESP32 is connected. You can run
sudo dmesg to know the device on your system.
On Debian systems, you will want to add your user to the dialout group so you
don't need root access.
idf.py flash
Currently, you can try pipecat-esp32 with one of the Pipecat foundational
examples. For example, from the Pipecat repository you can run:
python examples/foundational/07-interruptible.py --host IP --esp32
where IP is just your machine IP address (e.g. 192.168.1.10). Then, you would
set the environment variable PIPECAT_SMALLWEBRTC_URL as explained above.
