Refreshing issue

[luzik]

Describe the issue

I believe this is same as #281

Entity data is refreshing only at HA restart. Last update and last change can show for example "2 days ago" while landroid is working everyday.

What version of Home Assistant Core has the issue?

2023.4.6

What was the last working version of Home Assistant Core?

No response

What version of the Landroid Cloud integration do you have installed

beta5

What type of installation are you running?

Home Assistant Container

Which make and model is the mower used for this integration?

M700 Plus

Diagnostics information (NOT log entries!)

config_entry-landroid_cloud-c4bd914bb626e24a9c18fb752ec9a13b.json-2.txt

Relevant log entries

Do not know is this related but found this in logs:


Logger: aiohttp.server
Source: /usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py:403 
First occurred: 08:49:06 (5 occurrences) 
Last logged: 08:51:32

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 332, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
  File "aiohttp/_http_parser.pyx", line 551, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: 400, message="Bad status line 'Invalid method encountered'"

Additional information

No response

[MTrab]

281 is fixed in B5

Nothing more in the logfile than that? Not sure that is related to this

[luzik]

One more:

Logger: homeassistant
Source: /usr/src/homeassistant/homeassistant/runner.py:137 
First occurred: 09:16:40 (1 occurrences) 
Last logged: 09:16:40

Error doing job: Future exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/asyncio/selector_events.py", line 949, in _write_ready
    n = self._sock.send(self._buffer)
TimeoutError: [Errno 110] Operation timed out

[MTrab]

Still doesn't look like anything related to this integration

[MTrab]

You should look for entries mentioning either pyworxcloud or landroid_cloud

And do yourself (and me) the favour of looking directly in the logfile rather than in the log menu. The log from the menu sucks, if you ask me.

[arifroni]

I also have a similar problem. here are relevant log that i found in log file

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 539, in async_unload
    result = await component.async_unload_entry(hass, self)
  File "/config/custom_components/landroid_cloud/__init__.py", line 81, in async_unload_entry
    await hass.async_add_executor_job(
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.10/site-packages/pyworxcloud/__init__.py", line 219, in disconnect
    self.mqtt.disconnect()
  File "/usr/local/lib/python3.10/site-packages/pyworxcloud/utils/mqtt.py", line 222, in disconnect
    self.client.unsubscribe(self._topic.pop(topic))
TypeError: 'str' object cannot be interpreted as an integer

[MTrab]

I also have a similar problem. here are relevant log that i found in log file

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 539, in async_unload
    result = await component.async_unload_entry(hass, self)
  File "/config/custom_components/landroid_cloud/__init__.py", line 81, in async_unload_entry
    await hass.async_add_executor_job(
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.10/site-packages/pyworxcloud/__init__.py", line 219, in disconnect
    self.mqtt.disconnect()
  File "/usr/local/lib/python3.10/site-packages/pyworxcloud/utils/mqtt.py", line 222, in disconnect
    self.client.unsubscribe(self._topic.pop(topic))
TypeError: 'str' object cannot be interpreted as an integer

That's not (I guess) the same as this issue.
That error more looks like this: #347

[luzik]

I did not found any bad logs related to this integration. After one day of not updating information about moving, it is now working quite fine (checked two days). I will close this issue and eventualy open in back.

[Phoenix-DH]

Hey, I only want to add that I habe the same impression that at any point the integration seems to stop updating. I was wondering why my newly created automations are not working.

Mower was moving and had 41 percent battery. Same hours after returning home.

To be honest I am not familiar to get the logs as file, but I checked in GUI and found also nothing.

Only thing what I saw is that the last update time of the mower was days ago. I am talking from
state_updated_at

So I decided to restart. All went fine. Until this morning, no updates. When I thwn go to the Landroid app and refresh the start view, there is also an update in the integration.

[luzik]

Yeah, especially after returning to base for charging it stop updating, but please confirm something for me - does your landroid phone app, do update charge percentage (without force refresh) ? My personal feeling is that landroid do not update its information to cloud so frequently as while moving, thus ain't landroid app nor this integration have updated values. @MTrab - Maybe there is some service in API to ask cloud to force refresh landroid status that we could run sequentially ?

[arifroni]

yesterday, when I saw integration didn't update the status for some time (at that time mower finished mowing and returned to base). I looked at the phone app and saw the same status (battery percentage) as the integration. then I refreshed the app and both the app and the integration were updated.

[MTrab]

The service is already there, but should not be used.
I have 2 Landroids running, in 2 different accounts, and both are updating as expected without the need for forcing anything or restarting.
That's why I keep asking for logs.
Perhaps enable debug on the device and send what it generates (ALOT, so no copy/paste, drag the file in the reply)

And yes, when idle i the charger the device does not update as frequent as when mowing

[luzik]

I believe that landroid is sending push notification to the cloud when something changes (battery is fully charged, robot was lifted, opened and while moving - every minute or so). And changing battery % while charging do not trigger such push. Nevertheless landroid app have it's way to refresh robot values - so I believe that we should have it also. My humble opinion is that this is quite good (efficient) push model that save lot of data usage - yet still quite responsible for an mobile app, but for home assistant - init algorithm can be like this (if we will be force refresh data like app can)

Force refresh data when:

• battery is less then 100% and it's state is charging - refresh every one minute
• just normal force refresh every hour

[luzik]

Another idea - if @MTrab do not see any need to force refresh, please consider providing home assistant service for such job, so we could write automation with algorithm like that i wrote.

I believe that main problem here is that when users see that entity status did not refresh for so many hours - he do not know is something is wrong ..just psychological problem with that :-)

[MTrab]

Force refresh data when:

• battery is less then 100% and it's state is charging - refresh every one minute

This is a VERY bad ide, unless you want to be banned!

Again, I do not see this issue on any of the robots I have access to, they all refresh as expected in both my developer install and in my regular production HA.

So without any logs I cannot do much, sorry.

[Phoenix-DH]

Will try to provide them, need to check how.

App Info I did not check to be honest.

[MTrab]

Would be a good idea to do a check in the app if that has more recent data also

[Phoenix-DH]

Force refresh data when:

• battery is less then 100% and it's state is charging - refresh every one minute

This is a VERY bad ide, unless you want to be banned!

Again, I do not see this issue on any of the robots I have access to, they all refresh as expected in both my developer install and in my regular production HA.

So without any logs I cannot do much, sorry.

I have some light in the garden which should change based on the battery level. Does then taht mean that this would not be possible?

App stats I will check with the next run.

[MTrab]

No - as the battery level is updated when the robot is sending updates I don't see why that should not be possible.

[MTrab]

Battery state of the mower located at my kids place. See, it updates just as it should.

[MTrab]

Could even go this far, if you look closely there's 2 drops in the data, that's 2 reboots due to updates of Home Assistant, otherwise it's pretty consistent.

[Phoenix-DH]

Ok thanks!
I have just implemented the integration.
Will have a look at home if i also get this data.

[luzik]

Cool! I am assuming that You created entity from landroid attribute ? My landroid entity was last updated 18 hours ago so no way to have chart like yours

[MTrab]

Correct - a template sensor looking at the battery attribute.
But saying the entity was updated 18 hours ago doesn't mean it's not updating attributes.
The last changed timestamp is from when the STATE changed last, not when an attribute was changed.

So if it's 18 hours since the mower entered the charger and has been home ever since, then that is perfectly normal.

[luzik]

Oh!
I have to check that!
Would You be so kind to provide such a template ?

[MTrab]

It is really just basic templating, but you can find it in this small package: https://github.com/MTrab/HAConfig/blob/master/configurations/packages/horsensvej/landroid.yaml

[MTrab]

Also, remember as I have mentioned a few times, due to the huge amount of attributes, you cannot trust the UI when looking at attributes for the vacuum - you need to do a forced reload of the page (CTRL+F5) to be sure to see the latest attribute data.
This is a Home Assistant issue and not a Landroid issue, as the data is there - just not always up to date in the UI.

[Phoenix-DH]

Sure, but if there will be an updste, automations will react.

Thanks for sll the additional info.
Will let you know.

[MTrab]

Yes, will be fixed in next release - also not related to this issue!

[luzik]

No, haven't received credential from anyone with this exact issue. So every attempt to recreate this have been on my own mower or the ones I have credentials from due to other issues

Malene, I am sorry but with b6 I also have outdated information. Please provide email, I will send my credentials.

[cm86]

So for me it‘s working since your last release…
Thanks so far.
I will have a closer look next week…

[MTrab]

Any updates on this with the latest release?

[CubieMedia]

I tested with b6 and and no more issues with updating. Just installed your new release 3.0.0 and will continue testing this week.

It looks good, awesome work!

[Phoenix-DH]

I have also the latest release installed, v.3.0.0b6
Not sure to install the official latest release with the other mentioned pyworx bug.

Issue is still there as far as I can see it with the battery stats. With the next mowing cycle I will check in detail.

[MTrab]

I have also the latest release installed, v.3.0.0b6 Not sure to install the official latest release with the other mentioned pyworx bug.

If you don't experience the pyworxcloud bug in b6 then you shouldn't in the latest release either.

But if you still have refreshing issues, please share your account via landroid_cloud (at) trab.dk (And please mention issue 351 somewhere in the text so I can keep track of which issue that account is for)

[Phoenix-DH]

Ok, I will try. Thank you for the info.

[Phoenix-DH]

Same issue …
You got mail.
Thank you.

[prof7bit]

2023-05-04 07:06:51.420 ERROR (Thread-5 (_thread_main)) [root] Uncaught thread exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.10/site-packages/paho/mqtt/client.py", line 3591, in _thread_main
    self.loop_forever(retry_first_connection=True)
  File "/usr/local/lib/python3.10/site-packages/paho/mqtt/client.py", line 1756, in loop_forever
    rc = self._loop(timeout)
  File "/usr/local/lib/python3.10/site-packages/paho/mqtt/client.py", line 1185, in _loop
    return self.loop_misc()
  File "/usr/local/lib/python3.10/site-packages/paho/mqtt/client.py", line 1610, in loop_misc
    self._check_keepalive()
  File "/usr/local/lib/python3.10/site-packages/paho/mqtt/client.py", line 2560, in _check_keepalive
    self._do_on_disconnect(rc)
  File "/usr/local/lib/python3.10/site-packages/paho/mqtt/client.py", line 3475, in _do_on_disconnect
    on_disconnect(self, self._userdata, rc)
  File "/usr/local/lib/python3.10/site-packages/pyworxcloud/utils/mqtt.py", line 215, in _on_disconnect
    self.client.reconnect()
  File "/usr/local/lib/python3.10/site-packages/paho/mqtt/client.py", line 1044, in reconnect
    sock = self._create_socket_connection()
  File "/usr/local/lib/python3.10/site-packages/paho/mqtt/client.py", line 3685, in _create_socket_connection
    return socket.create_connection(addr, timeout=self._connect_timeout, source_address=source)
  File "/usr/local/lib/python3.10/socket.py", line 824, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/usr/local/lib/python3.10/socket.py", line 955, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Try again

No updates since this error. No new exceptions or any other log entries with "pyworx" or "landriod" since this error.

I got the exact same traceback yesterday evening, running current release version 3.0.1.

The last update I got yesterday was that it has docked after returning to base due to low battery, since then no updates anymore. This morning it appeared to be still at 9% battery.

Reloading the integration fixed it (this time it reloaded fine, without needing HA restart)

Additional info: I have a timer every 15 minutes to call the refresh service.

My mover entered the dock at 20:18 and updated the entity state properly
My traceback happened at 20:27 (first refresh service call after having docked)

11 hours later I reload the integration and it is updating again, so I assume i did not get rate-limit banned, also I found through experimentation that 4 times/h seems slow enough to not get banned.

[MTrab]

That traceback may be related, but doesn't tell what the error is, other than some MQTT error. Not much help unfortunately.

[prof7bit]

That traceback may be related, but doesn't tell what the error is, other than some MQTT error. Not much help unfortunately.

It failed in getaddrinfo(), so probably some kind of temporary network error. But nonetheless, wouldn't it be possible to somehow catch ALL possible errors (or detect connection loss) and recover gracefully or just restart itself completely?

For some reason the service call (from node red) to refresh that repeated all night long every 15 minutes did not produce any error message in the log, neither in homeassistant nor in node red, although it was in this disconnected broken state. So for some reason your code was not aware that the connection did not work anymore.

[MTrab]

As that error happens in Paho-MQTT there's not much I can do about that.

And the code is aware of being disconnected. It doesnt try to send any data to the mower if disconnected, which I think I write as warnings in the log as well.

Auto-reconnect from a disconnected state haven't been implemented yet. Might come later on, but in my tests it have given some strange errors when trying.

[prof7bit]

And the code is aware of being disconnected.

The state of the entity did not change to "unavailable", it was just frozen at the last known state "docked".

If the entity state would indicate the loss of connection I could (as a temporary workaround until it is properly fixed) try to change my timer automation to trigger integration reload instead of calling the refresh service whenever the state of the mower is "unavailable". But nothing visible to home assistant or node red indicated that disconnected state.

[MTrab]

If this checks out, then I might have made a for the next release.
Doing some testing right now, as I can reproduce the behaviour on @Phoenix-DH 's device

[Phoenix-DH]

Oh yes 🙌🏻 Great!
Thanks for the work.
Inobly recognized some more updates during mowing 😂

[MTrab]

Seems like it works as intended.
Currently a forced refresh will be performed 15 minutes after last dataset was received from the mower.
If the mower provides updates before the 15 minutes has elapsed, the scheduler will reset and start a new 15 minutes period - this to prevent eventually being banned for too many requests.

The 15 minutes MIGHT be adjusted (either lowered or extended) in the future, depending on experiences from next release.

[luzik]

Before version bump I can test it from master. Is it ready to test ?

[MTrab]

No, need to test from the Issue-377 repo as the fix is included in that one.

[luzik]

HACS shows only a master branch. Will wait :-P

[MTrab]

If you want to use other branches, you need to manually downloading the zip and copying files to your HA

[Phoenix-DH]

Great! Thank you for the update, I will try.

@MTrab Do you still need my account?

[MTrab]

No - have already deleted the credential from my box

[Phoenix-DH]

Thanks!

[roofburner]

The battery update issue during docked/charging, seems fixed for me (WR147E.1 - landroid cloud v3.0.2)

[roofburner]

Upgraded Landroid firmware to v3.30. Battery refresh still OK.