Real time voice translation utilizing OpenAI's Whisper for transcript and ChatCompletion for translation.
- From a dropdown, select your spoken language and the language you want to translate to
Speech to text transcription & Translation
- Once you begin recording, Whisper and GPT3.5 will attempt to translate each word you speak in real-timeSummarize
- Once you are done recording, stop the recording and you will be able to summarize the transcriptionVoice
- buggy. Once you are done speaking, stopping the recording session will give you an option forVoice
. Which will read the translated text out via the TTS feature
Requirements:
Setup:
- Follow the Backend README for further steps
Run the websocket server:
- run
cd backend
- run
python3 main.py
Requirements:
- Yarn
- Node (v16+)
Setup:
- run
cd frontend
- run
yarn install
Run the WebApp:
- run
yarn start
- App is very much unstable and not consistent.
- Occasionally will skip words or phrases due to race condition
- Timeout issue with Azure GPT3.5 API calls sometimes
INFO - error_code=Timeout error_message='The operation was timeout.' error_param=None error_type=None message='OpenAI API error received' stream_error=False
Error during translate: The operation was timeout. { "error": { "code": "Timeout", "message": "The operation was timeout." } } 408 {'error': {'code': 'Timeout', 'message': 'The operation was timeout.'}} {'Content-Length': '75', 'Content-Type': 'application/json', 'apim-request-id': 'a0cecbef-1a20-406e-850a-138d89755b6c', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'x-content-type-options': 'nosniff', 'x-ms-region': 'South Central US', 'Date': 'Tue, 28 Mar 2023 19:15:28 GMT'}
- Our Design board
- DeepL Translator - Helpful during our testing process to measure accuracy of translated text