If you want a custom Alexa-esque voice assistant, but in your browser, you got it! Why would you need a voice assistant in your browser, you may ask? This voice assistant can be accessed from anywhere - your phone, your laptop, just by going to the hosted URL. Link it to your Dialogflow agent and you're all set!
live demo: https://millasml.github.io/in-browser-voice-assistant/
To make the voice assistant listen to you, say "Hey Pico". If you do not want to use the wake-word detection, you can click the "Start Listening" button on the top right.
The other button on the top right reflects the state of the voice assistant (either waiting, listening, thinking, or speaking). Clicking on it turns off the entire voice assistant and it won't be listening for anything. Click it again to turn it on. Each state of the voice assistant is also reflected with multiple pictures of cute dogs.
Right now, the voice assistant can say hi, and tell you a joke if you ask it to! Jokes are acquired form icanhazdadjoke.com
above is the rough architecture of the in browser voice assistant.
above are the state transitions allowed in this application
| DIRECTORY NAME | DESCRIPTION |
|---|---|
| containers | includes all react UI components that make up the front end |
| moods | the logic of this application - includes the logic of the 4 different moods |
| helpers | helper scripts that the moods use. Includes hotword detection and the mainIntentHandler |
| fulfillment | Includes the different fulfillment handlers that handle fulfillments locally. All these will be imported in the mainIntentHandler (found in helper) |
| assets | static items that might be used. Now it only includes a default fulfillment speech wav file. Not useful at the moment |
| utils | utilities |
download everything that is needed with
npm install
to make HTTP requests:
- request
- request-promise
- axios
*it would be good if we could cut the number of libraries we rely on here to 1
to build HTTP requests:
- xml-builder
To support front end stuff:
- d3-ease
- d3-interpolate
- react-move
- @material-ui
- firebase
- moment
To support back end stuff:
- porcupine hot word detector
- web speech API
- google cloud functions
- dialogflow
- blobs
- web audio API
Just like how you would do with any react app!
npm start
your fulfillment handler's return should be one of two things:
- a fulfillment string
- a fulfillment array that consists of [fulfillmentText, name, showRecentSightings, showTickerTape]
- Create the fulfillment handler in the "fulfillment" folder
- export your fulfillment handler function
- import your fulfillment handler in "helpers/intentHandler.js"
- Add your fulfillment handler to the intentMap
- yay!
- allow change in language
- do all initialization stuff in initializing state
- refactor thinking and speaking to one component
- use only request-promise or axios
- handle the CORS thing, stop using the heroku app quick fix
- Create debug mode
- create separate sessions