[Feature] Websocket support

[o0Zz]

Hi,

It seems that Home Assistant is moving away from maintaining the REST API in favor of the WebSocket API, as indicated by discussions on various forums. Due to this shift, the REST API has limited support for certain features, such as retrieving forecast data. Specifically, calling a service like get_forecasts does not work properly via the REST API, as it doesn't support returning data responses (even when using return_response=True).

For example, the following code:

response = client.trigger_service("weather", "get_forecasts", 
                                  entity_id='weather.home', 
                                  type='hourly', 
                                  return_response=True)

results in a 500 error.

Related issues are tracked here:

Issue #106379
Issue #99820
Currently, the only reliable way to call this service and receive a response is through the WebSocket API, which complicates things if this library doesn't natively support WebSocket connections.

Here's a reference on how to interact with the WebSocket API in Python: https://jeroenboumans.medium.com/listening-to-the-home-assistent-websocket-api-with-python-7a074f8c81ea

Would you be open to supporting WebSocket integration in this library? It would make it much easier to interact with Home Assistant services moving forward.

Thanks for your time and consideration!

[GrandMoff100]

We can totally use the web socket API!

We can just add a WebSocketClient that follows a similar protocol to other clients. I would expect most of the existing endpoints to be supported with websockets too but I haven't looked into it much yet. I think it'd be epic to wrap the websocket api. I wish I could do it myself but frankly I'm swamped this semester 😅

I'll work on it here and there on a branch.

[GrandMoff100]

Hey @o0Zz! Now that the semester is over my time has been freed up and I'm working on this now.

[o0Zz]

Hi! Great to see you are back and you survived to this semester ;)! I have started working on a dashboard for e-paper displays: ha-ipaper (still a work in progress). Once you have a working version of the websocket implementation, I will integrate the weather API.

[GrandMoff100]

It turns out you can use both APIs to get the weather!

From my testing it seems that putting ?return_response at the end of the url works but putting return_response": true in this request body doesn't, which is where return_response=True to Client.trigger_service was going. Just making the following modification to Client.trigger_service made Homeassistant return the right weather responses (however I ran into pydantic errors because the trigger service endpoint was returning arbitrary service response data instead of returning changed entity states anymore):

def trigger_service(
    self,
    domain: str,
    service: str,
+   return_response: bool = True,
    **service_data,
) -> Tuple[State, ...]:
    """
    Tells Home Assistant to trigger a service, returns all states changed while in the process of being called.
    :code:`POST /api/services/<domain>/<service>`
    """
    data = self.request(
+       join("services", domain, service) + "?return_response" * return_response,
-       join("services", domain, service),
        method="POST",
        json=service_data,
    )
    return tuple(map(State.from_json, cast(List[Dict[str, Any]], data)))

Referenced testing code:

from homeassistant_api import Client

url = "http://localhost:8123/api/"

with Client(url, token) as client:
    response = client.trigger_service(
        "weather",
        "get_forecasts", 
        entity_id='weather.forecast_home', 
        type='hourly', 
        return_response=True
    )
    print(response)

Output:

Traceback (most recent call last):
 File "/home/nathan/HomeAssistantAPI/main.py", line 50, in <module>
   main()
 File "/home/nathan/HomeAssistantAPI/main.py", line 18, in main
   response = client.trigger_service("weather", "get_forecasts", 
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/home/nathan/HomeAssistantAPI/homeassistant_api/rawclient.py", line 293, in trigger_service
   return tuple(map(State.from_json, cast(List[Dict[str, Any]], data)))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/home/nathan/HomeAssistantAPI/homeassistant_api/models/states.py", line 44, in from_json
   return cls.model_validate(json)
          ^^^^^^^^^^^^^^^^^^^^^^^^
 File "/home/nathan/HomeAssistantAPI/.venv/lib/python3.12/site-packages/pydantic/main.py", line 568, in model_validate
   return cls.__pydantic_validator__.validate_python(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for State
 Input should be a valid dictionary or instance of State [type=model_type, input_value='changed_states', input_type=str]
   For further information visit https://errors.pydantic.dev/2.8/v/model_type

However, the good news is that my WebSocket client works!

from homeassistant_api import WebSocketClient

with WebSocketClient(ws_url, token) as ws_client:
        response = ws_client.trigger_service("weather", "get_forecasts", 
                                entity_id='weather.forecast_home', 
                                type='hourly',
                                return_response=True)
        print(response)

{'id': 1, 'type': 'result', 'success': True, 'result': {'context': {'id': '01JFJQ5FY2A8YSXVM6BK7QPFAZ', 'parent_id': None, 'user_id': 'e85fc7b7b8924dc9b024ce90ad23799e'}, 'response': {'weather.forecast_home': {'forecast': [{'condition': 'snowy', 'datetime': '2024-12-20T19:00:00+00:00', 'wind_bearing': 198.8, 'cloud_coverage': 100.0, 'uv_index': 0.0, 'temperature': 0.2, 'wind_speed': 21.2, 'precipitation': 0.6, 'humidity': 86}, {'condition': 'snowy', 'datetime': '2024-12-20T20:00:00+00:00', 'wind_bearing': 199.6, 'cloud_coverage': 100.0, 'uv_index': 0.0, 'temperature': 0.1, 'wind_speed': 22.0, 'precipitation': 0.5, 'humidity': 88}, {'condition': 'snowy', 'datetime': '2024-12-20T21:00:00+00:00', 'wind_bearing': 206.5, 'cloud_coverage': 100.0, 'uv_index': 0.0, 'temperature': 0.2, 'wind_speed': 24.8, 'precipitation': 0.3, 'humidity': 92}, {'condition': 'snowy-rainy', 'datetime': '2024-12-20T22:00:00+00:00', 'wind_bearing': 213.7, 'cloud_coverage': 100.0, 'uv_index': 0.0, 'temperature': 0.5, 'wind_speed': 23.4, 'precipitation': 0.9, 'humidity': 94}, {'condition': 'snowy-rainy', 'datetime': '2024-12-20T23:00:00+00:00', 'wind_bearing': 211.6, 'cloud_coverage': 100.0, 'uv_index': 0.0, 'temperature': 0.6, 'wind_speed': 23.0, 'precipitation': 0.9, 'humidity': 95}, {'condition': 'snowy-rainy', 'datetime': '2024-12-21T00:00:00+00:00', 'wind_bearing': 217.4, 'cloud_coverage': 100.0, 'uv_index': 0.0, 'temperature': 0.9, 'wind_speed': 22.3, 'precipitation': 1.0, 'humidity': 95}, {'condition': 'snowy-rainy', 'datetime': '2024-12-21T01:00:00+00:00', 'wind_bearing': 234.8, 'cloud_coverage': 100.0, 'uv_index': 0.0, 'temperature': 1.2, 'wind_speed': 19.8, 'precipitation': 0.2, 'humidity': 96}, {'condition': 'cloudy', 'datetime': '2024-12-21T02:00:00+00:00', 'wind_bearing': 249.6, 'cloud_coverage': 90.6, 'uv_index': 0.0, 'temperature': 1.3, 'wind_speed': 16.2, 'precipitation': 0.0, 'humidity': 94}, {'condition': 'partlycloudy', 'datetime': '2024-12-21T03:00:00+00:00', 'wind_bearing': 251.0, 'cloud_coverage': 53.9, 'uv_index': 0.0, 'temperature': 1.3, 'wind_speed': 15.8, 'precipitation': 0.0, 'humidity': 93}, {'condition': 'cloudy', 'datetime': '2024-12-21T04:00:00+00:00', 'wind_bearing': 245.7, 'cloud_coverage': 96.9, 'uv_index': 0.0, 'temperature': 0.7, 'wind_speed': 15.8, 'precipitation': 0.0, 'humidity': 94}, {'condition': 'cloudy', 'datetime': '2024-12-21T05:00:00+00:00', 'wind_bearing': 240.6, 'cloud_coverage': 88.3, 'uv_index': 0.0, 'temperature': 0.3, 'wind_speed': 16.2, 'precipitation': 0.0, 'humidity': 94}, {'condition': 'cloudy', 'datetime': '2024-12-21T06:00:00+00:00', 'wind_bearing': 238.1, 'cloud_coverage': 100.0, 'uv_index': 0.0, 'temperature': 0.5, 'wind_speed': 17.6, 'precipitation': 0.0, 'humidity': 93}, {'condition': 'cloudy', 'datetime': '2024-12-21T07:00:00+00:00', 'wind_bearing': 233.1, 'cloud_coverage': 100.0, 'uv_index': 0.0, 'temperature': 0.5, 'wind_speed': 18.7, 'precipitation': 0.0, 'humidity': 93}, {'condition': 'cloudy', 'datetime': '2024-12-21T08:00:00+00:00', 'wind_bearing': 229.9, 'cloud_coverage': 100.0, 'uv_index': 0.0, 'temperature': 0.6, 'wind_speed': 21.2, 'precipitation': 0.0, 'humidity': 93}, {'condition': 'cloudy', 'datetime': '2024-12-21T09:00:00+00:00', 'wind_bearing': 226.7, 'cloud_coverage': 100.0, 'uv_index': 0.2, 'temperature': 1.2, 'wind_speed': 24.1, 'precipitation': 0.0, 'humidity': 90}, {'condition': 'cloudy', 'datetime': '2024-12-21T10:00:00+00:00', 'wind_bearing': 223.4, 'cloud_coverage': 100.0, 'uv_index': 0.3, 'temperature': 1.6, 'wind_speed': 24.8, 'precipitation': 0.0, 'humidity': 89}, {'condition': 'cloudy', 'datetime': '2024-12-21T11:00:00+00:00', 'wind_bearing': 218.6, 'cloud_coverage': 100.0, 'uv_index': 0.4, 'temperature': 1.9, 'wind_speed': 27.0, 'precipitation': 0.0, 'humidity': 88}, {'condition': 'rainy', 'datetime': '2024-12-21T12:00:00+00:00', 'wind_bearing': 222.3, 'cloud_coverage': 100.0, 'uv_index': 0.4, 'temperature': 2.2, 'wind_speed': 28.8, 'precipitation': 0.4, 'humidity': 81}, {'condition': 'rainy', 'datetime': '2024-12-21T13:00:00+00:00', 'wind_bearing': 224.7, 'cloud_coverage': 100.0, 'uv_index': 0.3, 'temperature': 2.4, 'wind_speed': 31.0, 'precipitation': 0.3, 'humidity': 86}, {'condition': 'rainy', 'datetime': '2024-12-21T14:00:00+00:00', 'wind_bearing': 227.9, 'cloud_coverage': 100.0, 'uv_index': 0.2, 'temperature': 2.7, 'wind_speed': 33.1, 'precipitation': 0.5, 'humidity': 90}, {'condition': 'rainy', 'datetime': '2024-12-21T15:00:00+00:00', 'wind_bearing': 229.0, 'cloud_coverage': 100.0, 'uv_index': 0.0, 'temperature': 3.0, 'wind_speed': 32.0, 'precipitation': 0.2, 'humidity': 90}, {'condition': 'rainy', 'datetime': '2024-12-21T16:00:00+00:00', 'wind_bearing': 229.4, 'cloud_coverage': 100.0, 'uv_index': 0.0, 'temperature': 3.3, 'wind_speed': 31.3, 'precipitation': 1.1, 'humidity': 90}, {'condition': 'rainy', 'datetime': '2024-12-21T17:00:00+00:00', 'wind_bearing': 236.5, 'cloud_coverage': 100.0, 'uv_index': 0.0, 'temperature': 3.3, 'wind_speed': 31.7, 'precipitation': 0.3, 'humidity': 93}, {'condition': 'cloudy', 'datetime': '2024-12-21T18:00:00+00:00', 'wind_bearing': 242.5, 'cloud_coverage': 100.0, 'uv_index': 0.0, 'temperature': 3.6, 'wind_speed': 30.2, 'precipitation': 0.0, 'humidity': 95}]}}}}

There's still a lot more I need to do with the WebSocketClient in terms of replicating the REST API client methods. But something thats exciting now, is that the websocket connection is two-way meaning that we can listen and subscribe to events (and automation triggers and stuff) from homeassistant now, instead of just the one way connection, firing them like the REST API did.

One thing to note is that when this is all said and done I'm planning on having both a Client.trigger_service method and a Client.trigger_service_with_response method to separate the differences in parsing logic between the two of them, and to make the type hinting more clean (as opposed to passing a return_response=True flag.

tldr; We're supporting the Websocket API now which is good, but turns out we only had to switch from putting "return_response":true in the body to putting ?return_response in the query string to get the forecasts to work, contrary to what home-assistant/core#115046 lead me to believe. I'm gonna look into that a bit more later.

I hope to finish the Websocket client, do the trigger_service redesign and ship it in version 5.0.0 by the end of next week or the week after depending on how my Christmas plans work out.

Cheers!

[GrandMoff100]

Of course there's a lot of documentation that I'm going to need to write and rewrite with this version.

There's much more to be said and discussed about the future of the REST API vs the Websocket API from the back and forth I've seen on the ha core repo.

[GrandMoff100]

Looking at the actual file changes of home-assistant/core#115046 it's clear that only the query-string option was implemented.

[GrandMoff100]

Hey @o0Zz I can ship the trigger_service_with_response function of the rest api client right now if you'd prefer to integrate the weather into your project sooner rather than later.

But if you would rather use the websocket api anyway I'll just ship the WS api client and trigger_service_with_response in the same release a little bit later.

Let me know.

[o0Zz]

Hi, I think it's better to focus on WS since the REST api is not maintain by HA team in all cases. Thus it will just post pone the problem. I don't have any deadline so, let focus on the WS that's totally fine to me.