SKRRTN's Twitch OpenAI TTS

🎮 🤖 Real-time interaction with Twitch chat using OpenAI's text-to-speech (TTS) capabilities. 🤖 🎮

Click image to see it in action with OBS!

Overview

This Python script integrates OpenAI's TTS capabilities with Twitch chat. It allows users to ask questions in the Twitch chat, and the bot responds with synthesized speech generated by OpenAI's GPT-3 model. This script is EXTREMELY easy to use and requires almost no effort to get started. It uses almost no computer resources as all processing is done via OpenAI's API and services.

Important

Please note this method DOES COST MONEY to use!! OpenAI's API is NOT FREE!!

After about 3 hours of use with a few hundred questions we racked up $1.30 USD in usage.
(Thats roughly 83,000 characters synthesized w/TTS and 74,000 Tokens for generating response text)

Your mileage my vary depending which model, TTS engine and prompts you use!

Features

• 🤖 Real-time Interaction: Engage with Twitch chat in real-time, processing questions and generating responses almost instantly.
• 🔊 Speech Synthesis: Automatically generate speech responses using OpenAI's powerful GPT-3 model.
• 🎲 Random Question Prompts: Keep chat engagement high by prompting random questions at configurable intervals.
• 🚫 Bad Word Filtering: Maintain a clean chat environment by filtering out inappropriate language.
• ⚙️ Configurability: Easily configure various aspects of the bot, such as Twitch credentials, OpenAI API settings, system options, and limits.

Future Plans

• Ability to change prompt in realtime for admins/mods in chat using !prompt command.
• Better bad word filtering.
• Ability to use OpenAI assistants for better customization of responses.
• Better queue system.
• Better code optimization.
• Get a real job.

Installation

1. Clone this repository:

   git clone https://github.com/skrrtn/Twitch-OpenAI-TTS.git

2. Install the required dependencies from requirements.txt:

   pip install -r requirements.txt

3. Configure the config.json file with your Twitch and OpenAI API credentials, as well as other settings. Detailed explanation of configuration options can be found below.

Configuration

The config.json file contains several sections for configuring different aspects of the bot:

• twitch: Configure Twitch-related settings such as server, port, nickname, token, channel, bad word filtering, and timeout duration.
• openai: Configure OpenAI API settings including API key, system message, model, and TTS model.
• system: Configure system settings such as audio device IDs for question TTS and response TTS.
• limits: Configure various limits and intervals for the bot, including user question interval, character limit, random question interval, and more.

Here's an example of a config.json file:

{
  "twitch": {
    "server": "irc.chat.twitch.tv", # Main Twitch IRC Server
    "port": 6667, # Twitch IRC Port
    "nickname": "your_bot_nickname", # Nickname from the OAuth Token
    "token": "your_oauth_token", # Your Twitch OAuth Token
    "channel": "#your_channel", # What channel to join and monitor
    "bad_word_filter_enabled": true, # Do you want to filter bad words? Be sure to update badwords.txt
    "timeout_seconds": 600 # How long someone should be timed out for using a bad word
  },
  "openai": {
    "api_key": "your_openai_api_key", # Found at platform.openai.com/api-keys
    "system_message": "My name is Jeff", # Give ChatGPT a prompt or personality
    "model": "gpt-3.5-turbo", # Can be any v1/chat/completions model, 3.5 turbo seems to be decent and cheap
    "tts_model": "tts-1", # Can either be tts-1 or tts-1-hd
    "voice": "fable" # Can be alloy, echo, fable, onyx, nova, and shimmer
  },
  "system": {
    "question_tts_device_id": 0, # Which audio device to play questions on
    "response_tts_device_id": 1 # Which audio device to play answers on
  },
  "limits": {
    "user_question_interval": 30, # How long before a user is allowed to ask another question
    "char_limit": 250, # Character limit of the question
    "random_questions_enabled": true, # If no questions are asked, should we play a random question?
    "random_question_idle_time": 60 # If the queue is empty, how long before we play a random question?
  }
}

Make sure to set these values according to your preferences before running the bot.

Usage

Run the script main.py to start the Twitch bot:

python main.py

You can now start asking the bot questions in chat using !q followed by a question.
Example: !q What does blue taste like?

🎉 Thats it!! 🎉 The script will take the questions and process them into TTS for you!

Whats with the text files?

The text files queue.txt, question.txt and response.txt are setup in a way to use them as sources in OBS Studio. This is will allow you to show the questions, answers and amount of questions in queue on stream in real time. When a question is processed, we add the question to question.txt and update queue.txt to show how many questions are in queue. Once a response is returned, we then add that to response.txt. If you set these files to sources in OBS, it will update automatically. After a response is played, it will clear the files and update the queue.txt accordingly.

Good practice for showing the text effeciently, right click the text source and Edit Transform (or CTRL-E).
Set Positional Alignment and Alignment In Bounding Box to Center.
Set Bounding Box Type to Scale To Inner Bounds.

Now, you can make the text box the size you want, and the text will always stay within that box. Play around for better results!

Which Audio Device To Use

In order to play the audio correctly, we need the correct audio device output. You can find the audio output devices on your system by running this code.

python -m sounddevice

This should show you ALL the outputs on your system. It is up to you to choose the correct one and update the config accordingly. I seperated the audio devices for use with OBS, but you can always use the same one for both the question and response.

Obtaining an OpenAI API Key

To use OpenAI's API, you'll need to obtain an API key from the OpenAI platform. Here's how to get started:

1. Sign up or log in to your OpenAI account at OpenAI's website.
2. Navigate to API keys: Once logged in, go to the "API keys" section of your account settings.
3. Generate an API key: Click on the "Generate API Key" button to create a new API key. Make sure to copy the generated key securely.
4. Add the API key to your config.json: Paste the copied API key into the api_key field under the openai section of your config.json file.

Generating an OAuth Token

To connect the bot to your Twitch channel, you'll need to generate an OAuth token. Here's how to do it:

1. Visit the Twitch Chat OAuth Password Generator: Go to Twitch Chat OAuth Password Generator in your web browser. Make sure you are already logged in to your Twitch account.
2. Click "Connect": Once on the website, simply click the "Connect" button. This will authorize the application to access your Twitch account for chat purposes.
3. Copy the Token: After clicking "Connect", you'll be redirected back to the website, and you should see your OAuth token displayed. Copy this token securely.
4. Use the Token in the Config: Paste the copied OAuth token into the token field under the twitch section of your config.json file. Set the nickname field to your Twitch username.

Things To Consider!

To limit the amount of TTS usage it is recommended to tell OpenAI to not send twenty paragraphs worth of data to process. Easiest way to do this is to include it in your system_message like so:

"system_message": "You are a robot named Mark. You love pie. Keep your response around three small sentences."

This method is the easiest way to limit responses and TTS generation.

License

This project is licensed under the MIT License. See the LICENSE file for details.