Skip to content

PolyglotSiri is based on [ChatGPT-Siri](https://github.com/Yue-Yang/ChatGPT-Siri) and enhances its multilingual and speech capabilities.

License

Notifications You must be signed in to change notification settings

Munntein/PolyglotSiri-Apple-Shortcut

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
 
 
 
 
 
 

Repository files navigation

PolyglotSiri Apple Shortcut

GitHub stars GitHub forks GitHub license

中文介绍

PolyglotSiri is based on ChatGPT-Siri and enhances its multilingual and speech capabilities.

Now you can talk to siri with any languge. And siri responses bilingually(English and Chinese as i set) with a more natural vocie. Now you can treat Siri as a foreign friend and practice language training with it.

In addition to the OpenAI chatGPT API, Three other REST APIs are used:

  1. OpenAI Whisper API: used as an STT service which can handle multiple languages. Native Siri can only understand one language at a time.
  2. Azure Language Cognitive API: used to recognize the language of text for later Azure TTS language configuration.
  3. Azure TTS: using Azure TTS to replace iOS native ones, resulting in more natural-sounding voice.

Shortchut Link

https://www.icloud.com/shortcuts/61bbd941bfce433cb9193d990a8f4004
(the previous shortcut link has an issue, the url suffix after endpoint "/text/analytics/v3.0/languages" of azure language service was lost by mistake)

Configurition and Customization

  1. First you have to apply for openai api and Azure cloud
  2. Fill your api keys in this shortcut.
    • you need two openai keys, one for whisper and the other for completion.
    • Fill in Azure tts and Azure language cognitive service Keys.
  3. Substitute Azure service REST API urls. There are three of them, one for text language cognition, and two for Azure TTS (aiming for bilingual voice support, you could customize more for mutiple language)
  4. Customize Azure TTS params
  5. Specify mulit-language if statement
    • To make azure TTS support multi-languge vocie, we need to use Azure language cognition to recognize text language. And using if statements to make it using the corresponding TTS language engine when speaking.

Usage

  1. Download the shortcut.
  2. Simply tap the shortcut and let the magic happen.
  3. The audio recording will stop after 15 seconds by default. You can stop it either by tapping or waiting for the countdown to end.
  4. Wait for responce.

Issues

This shortcut is somewhat useful for language learning, but it’s not perfect. There are some issues.

  1. Why are two OpenAI keys necessary?

    • In my practice, using the same key to request whisper and completion APIs, within very few rounds, say 2 or 3, the token would reach the completion api limits. Seperating the API keys would solve the problem. However, I'm still not quite clear on the rules, why whisper API requests counts towards the completion token size.
  2. Since there are three APIs to be called, performance may not be particularly smooth - especially if you need a proxy to access those services.

  3. It doesn’t support streaming output like many other text chatgpt projects. This, coupled with previous network issues, results in slower response times, particularly when ChatGPT generates longer replies.

  4. The most critial issue, this shortcut cannot function well in vocie wake-up mode. I tried my best to debug it, while giving up at end. Looking forward someone could solve the issue.

  5. The second imperfect aspect is that the voice input section cannot automatically stop; manual tapping or waiting for the countdown to finish is required instead.

Acknowledge

  1. ChatGPT-Siri

About

PolyglotSiri is based on [ChatGPT-Siri](https://github.com/Yue-Yang/ChatGPT-Siri) and enhances its multilingual and speech capabilities.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published