Expose status of Assist as a sensor#6098
Expose status of Assist as a sensor#6098ujjol1234 wants to merge 9 commits intohome-assistant:mainfrom
Conversation
There was a problem hiding this comment.
Hi @ujjol1234
It seems you haven't yet signed a CLA. Please do so here.
Once you do that we will be able to review and accept this pull request.
Thanks!
|
Please take a look at the requested changes, and use the Ready for review button when you are done, thanks 👍 |
common/src/main/kotlin/io/homeassistant/companion/android/common/sensors/AssistSensorManager.kt
Fixed
Show fixed
Hide fixed
common/src/main/kotlin/io/homeassistant/companion/android/common/sensors/AssistSensorManager.kt
Fixed
Show fixed
Hide fixed
common/src/main/kotlin/io/homeassistant/companion/android/common/sensors/AssistSensorManager.kt
Fixed
Show fixed
Hide fixed
dshokouhi
left a comment
There was a problem hiding this comment.
Thanks for the contribution!
just a couple things to keep the code in sync with the rest of the project.
I wonder are we able to determine if Assist is properly setup to add a state for that? Or maybe even hide it if not configured in hasSensor ?
app/src/main/kotlin/io/homeassistant/companion/android/HomeAssistantApplication.kt
Outdated
Show resolved
Hide resolved
app/src/main/kotlin/io/homeassistant/companion/android/assist/AssistViewModel.kt
Outdated
Show resolved
Hide resolved
|
Did you test the app with this sensor and is it updating the state as expected (idle > listening > speaking > idle)? Did you also test with keyboard input / interrupting the microphone manually / dismissing the sheet while recording/playing? This will also need a documentation PR to be accepted. |
common/src/main/kotlin/io/homeassistant/companion/android/common/sensors/AssistSensorManager.kt
Outdated
Show resolved
Hide resolved
common/src/main/kotlin/io/homeassistant/companion/android/common/assist/AssistViewModelBase.kt
Outdated
Show resolved
Hide resolved
|
@dshokouhi @TimoPtr thank you for the feedback. I have implemented the changes. |
common/src/main/kotlin/io/homeassistant/companion/android/common/sensors/AssistSensorManager.kt
Show resolved
Hide resolved
common/src/main/kotlin/io/homeassistant/companion/android/common/sensors/AssistSensorManager.kt
Outdated
Show resolved
Hide resolved
app/src/main/kotlin/io/homeassistant/companion/android/assist/AssistViewModel.kt
Show resolved
Hide resolved
common/src/main/kotlin/io/homeassistant/companion/android/common/sensors/AssistSensorManager.kt
Outdated
Show resolved
Hide resolved
| type = "sensor", | ||
| name = R.string.sensor_name_assist, | ||
| descriptionId = R.string.sensor_description_assist, | ||
| statelessIcon = "mdi:robot", |
There was a problem hiding this comment.
@jpelgrom @dshokouhi Do you have a preference for the icon?
There was a problem hiding this comment.
mdi:account-voice, to match the core sensor for an Assist satellite which is probably the closest thing?
common/src/main/kotlin/io/homeassistant/companion/android/common/sensors/AssistSensorManager.kt
Outdated
Show resolved
Hide resolved
common/src/main/kotlin/io/homeassistant/companion/android/common/sensors/AssistSensorManager.kt
Show resolved
Hide resolved
app/src/main/kotlin/io/homeassistant/companion/android/HomeAssistantApplication.kt
Outdated
Show resolved
Hide resolved
common/src/main/kotlin/io/homeassistant/companion/android/common/sensors/AssistSensorManager.kt
Show resolved
Hide resolved
common/src/main/kotlin/io/homeassistant/companion/android/common/sensors/AssistSensorManager.kt
Show resolved
Hide resolved
common/src/main/kotlin/io/homeassistant/companion/android/common/sensors/AssistSensorManager.kt
Outdated
Show resolved
Hide resolved
common/src/main/kotlin/io/homeassistant/companion/android/common/sensors/AssistSensorManager.kt
Outdated
Show resolved
Hide resolved
app/src/main/kotlin/io/homeassistant/companion/android/assist/AssistViewModel.kt
Show resolved
Hide resolved
common/src/main/kotlin/io/homeassistant/companion/android/common/sensors/AssistSensorManager.kt
Outdated
Show resolved
Hide resolved
| AssistSensorManager.updateState(app, AssistSensorManager.AssistState.SPEAKING) | ||
| val result = audioUrlPlayer.playAudio(it.toString()) | ||
| AssistSensorManager.updateState(app, AssistSensorManager.AssistState.IDLE) | ||
| result |
There was a problem hiding this comment.
FYI I'm currently changing how the playAudio works to support streaming TTS but it shouldn't be hard to adjust.
There was a problem hiding this comment.
That's cool. I have taken care of it.
TimoPtr
left a comment
There was a problem hiding this comment.
In order to merge this PR we need two things
- merge main and solve the conflict
- a PR on the companion doc repo https://github.com/home-assistant/companion.home-assistant to describe this.
| val state = when (inputMode) { | ||
| AssistInputMode.VOICE_ACTIVE -> AssistSensorManager.AssistState.LISTENING | ||
| AssistInputMode.VOICE_INACTIVE, AssistInputMode.TEXT -> AssistSensorManager.AssistState.IDLE | ||
| AssistInputMode.BLOCKED -> AssistSensorManager.AssistState.CLOSED |
There was a problem hiding this comment.
Blocked -> Closed why one or the other? @jpelgrom do you have some takes here?
There was a problem hiding this comment.
When I added BLOCKED it was for input options being blocked because the server doesn't support Assist (temporarily), not considering a sensor state.
I'm not sure how we ended up with these states. My first thought would be to model it after the Assist satelite states where possible, which is partly done. However, I think there is a difference between 'input blocked because not supported' and 'dialog closed' that would be useful to have, so unsupported and closed?
TimoPtr
left a comment
There was a problem hiding this comment.
Now that I'm looking at it again. Don't we want also this sensor on the watch? (Should be made in another PR)
| get() = commonR.string.sensor_name_assist | ||
| override suspend fun getAvailableSensors(context: Context): List<SensorManager.BasicSensor> { |
There was a problem hiding this comment.
| get() = commonR.string.sensor_name_assist | |
| override suspend fun getAvailableSensors(context: Context): List<SensorManager.BasicSensor> { | |
| get() = commonR.string.sensor_name_assist | |
| override suspend fun getAvailableSensors(context: Context): List<SensorManager.BasicSensor> { |
| attributes = mapOf( | ||
| "options" to AssistState.entries.map(AssistState::value), | ||
| ), | ||
| mdiIcon = "mdi:robot", |
There was a problem hiding this comment.
Use a private const to only set the icon once.
There was a problem hiding this comment.
Typically we use assistSensor.statelessIcon instead
| AssistInputMode.BLOCKED -> AssistSensorManager.AssistState.CLOSED | ||
| AssistInputMode.TEXT_ONLY -> AssistSensorManager.AssistState.IDLE | ||
| } | ||
| AssistSensorManager.updateState(app, state) |
There was a problem hiding this comment.
You probably also want to set the state in the onCleared of the viewModel to CLOSED?
As the sensor is in |
|
@ujjol1234 I did modify the AssistViewModel to support TTS, you are going to need to adjust your PR to it. |
|
Changing this to draft for now. Once you've rebased on top of/merged the latest changes and addressed all comments, you can mark it as ready for review. |
Summary
This PR introduces a Assist Sensor in the companion-app that shows whether Assist on the phone is listening.
Documentation PR for this sensor: home-assistant/companion.home-assistant#1270
Checklist