Recorder warns sqlite3 was not shutdown cleanly, despite controlled shutdown #73679

adamfeldman · 2022-06-18T14:37:01Z

The problem

Recently, I often but not always get a warning that the Recorder's default sqlite3 DB was not shutdown cleanly. This is despite performing a controlled shutdown via the UI on the Hardware settings page.

Currently running 2022.6.x and HAOS 8.1. This issue started within the past month or so. Hard for me to pinpoint if it's correlated with HAOS 8.x or the Core version.

I have a feeling this issue will need to be moved to the HAOS repo, but the direct problem that is logged is from Recorder.

I have run HAOS in a VM for a few years (OVA image running within KVM on Proxmox). I always shut it down from within the HAOS guest via the UI.

I am sure there are more diagnostics I can help pull, I am not sure where to start.

Thank you teams!

2022-06-18 09:21:03 WARNING (Recorder) [homeassistant.components.recorder.util] The system could not validate that the sqlite3 database at //config/home-assistant_v2.db was shutdown cleanly

What version of Home Assistant Core has the issue?

core-2022.6.6

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Recorder

Link to integration documentation on our website

https://www.home-assistant.io/integrations/recorder/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

The text was updated successfully, but these errors were encountered:

probot-home-assistant · 2022-06-18T14:37:07Z

recorder documentation
recorder source
_{^{(message by IssueLinks)}}

probot-home-assistant · 2022-06-18T14:37:10Z

Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (recorder) you are listed as a code owner for? Thanks!
_{^{(message by CodeOwnersMention)}}

mrcfelici · 2022-06-24T08:31:31Z

I have the same problem, is there any news about it? Many thanks

pvizeli · 2022-06-24T08:50:03Z

I guess the issue is, that core now doing online database migration. We need a way to get the information on the supervisor to prevent shutdown/reboots while core migrates live the database. Maybe it's also an issue with sqlite. Coult be that the python 3.10 upgrade solve that in 2022.7

adamfeldman · 2022-07-27T16:50:30Z

Thanks for the info @pvizeli! The issue remains in the 2022.7.x series and on HAOS 8.2.

In my experience it's very consistent – IIRC anytime I've done a controlled shutdown via the UI, the sqlite warning shows up in the logs.

nvOC-Stubo · 2022-08-10T13:41:59Z

Same issue here with Supervisor 2022.07.0 and HA 2022.8.3.

snowmangh · 2022-08-12T14:20:15Z

I also have this error but usually after shutting down/restarting my Synology NAS. HA 2022.7.6 running inside Docker.

jimmyl88 · 2022-11-02T16:41:46Z

I have the same problem after restarting HA from GUI, have to reboot the VM.

Have recently moved my VM to a new host and get a corrupt db-file.
Then i restored to a 5 days old backup and then it all worked for 2 weeks.

Home Assistant 2022.10.5
Supervisor 2022.10.0
Operating System 9.3

Commifreak · 2023-01-02T09:46:39Z

Same here (docker)
2023-01-02 10:40:07.272 WARNING (Recorder) [homeassistant.components.recorder.util] The system could not validate that the sqlite3 database at //config/home-assistant_v2.db was shutdown cleanly but it always was a clean shutdown. Maybe sometimes the shutdown does not finish ongoing events? During my test I had this:

2023-01-02 10:45:31.925 WARNING (Recorder) [homeassistant.components.recorder.util] The system could not validate that the sqlite3 database at //config/home-assistant_v2.db was shutdown cleanly
2023-01-02 10:45:31.984 WARNING (Recorder) [homeassistant.components.recorder.util] Ended unfinished session (id=13 from 2023-01-02 09:44:37.593355)

bdraco · 2023-02-07T14:10:26Z

If you shutdown via docker you can hit the container shutdown timeout (I think its 10 seconds) which causes an abrupt shutdown as it does not give Home Assistant enough time to properly shutdown

TTLucian · 2023-02-15T21:56:29Z

I use VMWare Workstation and I get this error every time I restart the VM. HASOS is shutting down cleanly.

The system could not validate that the sqlite3 database at //config/home-assistant_v2.db was shutdown cleanly

Ended unfinished session (id=462 from 2023-02-15 21:45:30.649049)

Commifreak · 2023-02-16T04:47:50Z

I also checked my docker env: shutdown timeout: 120sec. HA is done earlier but the message keeps popping up (sometimes, not always).

Luca1996O · 2023-03-07T19:26:50Z

Same issue here. Does anyone has a workaround with the exception of removing that database and let it regenerate it (it keeps reboot and appear after some days)?

bdraco · 2023-04-12T08:01:48Z

We need a full log from the run before the issue appears with debug logging turned on for homeassistant.components.recorder

I also opened #91261 which will cause the recorder to try to shutdown cleanly if something really goes wrong.

issue-triage-workflows · 2023-07-11T08:08:49Z

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

Commifreak · 2023-07-11T09:47:49Z

Let us know if that works for you by adding a comment 👍

Issue still there.

ELind77 · 2023-07-21T16:24:10Z

I am experiencing the same issue running HA on Docker.

I run HAnon unRAID and I take nightly backups of all of the Docker volumes (there is an unRAID plugin that makes common practice). The backup process logs show the HA container was stopped in 10 seconds while I have the timeout set at 120 seconds. Not really sure what's going on here.

I can check the logs from the HA container this weekend. Any advice on what to look for?

I can also try moving to MariaDB but it would be nice to get some insight into this issue.

gcarneiro · 2023-09-11T14:52:55Z

I was having this problem, new installations or restored, after reboot HA did not start. I changed my sd card and problem stop.

masi · 2023-10-30T17:57:01Z

I'm using Home Assistant OS on a Home Assistant Yellow and get this error every time I power on HA. I'm always use the UI to do a clean shut down.

I do this quite often as I'm still playing around.

ginjo · 2024-01-23T06:28:42Z

Also experiencing this issue using docker image homeassistant/home-assistant:2024.1

Deimos761 · 2024-02-07T18:00:08Z

Same problem for me
I noted this issue since core 2024.1.1 and now it goes on 2024.1.6.
Never shuttted down abruptly; daily clean host.shutdown.
Before restarting, daily clean database (1 hour before)
I tried to erase the db and starting a new one but the issue is still there.

I use HAOS in a VMware environment

Logger: homeassistant.components.recorder.util
Source: components/recorder/util.py:332
Integration: Recorder (documentation, issues)
First occurred: 00:59:07 (1 occurrences)
Last logged: 00:59:07

The system could not validate that the sqlite3 database at //config/home-assistant_v2.db was shutdown cleanly

RhavoX · 2024-02-13T22:46:58Z

In HA 2024.2 when node-red issues the host_shutdown service I also get this log.

amarofarinha · 2024-03-01T16:04:24Z

Hi all.
i have the same here. I migrated from homeassistant blue to a proxmox cluster and now i allways have this log entry when reestart.
Everything seems to be working fine but allways 2 messages related.

Proxmox 8.1.4

Core 2024.2.5
Supervisor 2024.02.1
Operating System 12.0
Frontend 20240207.1

i have these 2 entries in log:

Logger: homeassistant.components.recorder.util
Source: components/recorder/util.py:611
Integration: Recorder (documentation, issues)
First occurred: 3:51:35 PM (1 occurrences)
Last logged: 3:51:35 PM

Ended unfinished session (id=312 from 2024-03-01 15:47:50.708938)

Logger: homeassistant.components.recorder.util
Source: components/recorder/util.py:332
Integration: Recorder (documentation, issues)
First occurred: 3:51:35 PM (1 occurrences)
Last logged: 3:51:35 PM

The system could not validate that the sqlite3 database at //config/home-assistant_v2.db was shutdown cleanly

litinoveweedle · 2024-04-20T15:10:21Z

I have the issue on the 2024.04.03 on my supervised installation as well. I realized it during the development of the custom integration. If I restart HA (from the web menu) after it was running from the previous day (means more than ~ ten hours) I will get same error after HA restart:

2024-04-20 16:02:02.369 WARNING (Recorder) [homeassistant.components.recorder.util] The system could not validate that the sqlite3 database at //config/home-assistant_v2.db was shutdown cleanly
2024-04-20 16:02:02.384 WARNING (Recorder) [homeassistant.components.recorder.util] Ended unfinished session (id=565 from 2024-04-19 23:51:45.488175)

The HA shutdown phase seems to be very long (very close to 300s / 5min) and there is nothing in the logs after HA restart is triggered (with normal warning log level). Is there any 5min shutdown timeout? Can it be extended?

But if I restart HA after it was running only for few tens of minutes (up to hour or two), the restart is almost immediate.

I use HA on supervised installation on server hardware, with SQLite being placed on NVMe disk, so bare HW performance shall not be an issue, even with my DB size around 1GB.

@bdraco: I just started HA recorder with debug set, but due to said above to reproduce error I will need to keep it running for some time and therefore the produced log would be huge. I am also not completely comfortable to share all the data publicly. Is there anything we can do about it? Thank you.

bdraco · 2024-04-20T15:52:59Z

It seems like you have something blocking shutdown which is why it gives up and shuts down uncleanly. You might be able enable debug logs for homeassistant.core, and than down a shutdown to reveal what it is waiting for

dlamblin · 2024-05-23T14:50:00Z

I installed HomeAssistant using the WSL instructions in Windows 11's WSL2 Ubuntu 24.04 LTS. The UI at localhost:8123 (homeassistant.local:8123 wasn't working) doesn't have a shutdown option. Under http://localhost:8123/config/system I only see Restart.

So I've been seeing:

dlamblin@whitebox:~$ cat start-home-assistant.sh
sudo -u homeassistant -H -s source /srv/homeassistant/bin/activate-with-hass
dlamblin@whitebox:~$ . start-home-assistant.sh
[sudo] password for dlamblin:
2024-05-23 10:23:13.189 WARNING (Recorder) [homeassistant.components.recorder.util] The system could not validate that the sqlite3 database at //home/homeassistant/.homeassistant/home-assistant_v2.db was shutdown cleanly
2024-05-23 10:23:29.662 ERROR (SyncWorker_5) [aiodhcpwatcher] Cannot watch for dhcp packets without a functional packet filter: libpcap is not available. Cannot compile filter !

And I'm unsure how I'd avoid it. Choosing reset does actually exit the running process above. And it doesn't appear to restart.

activate-with-hass is just the venv activate script with hass added as the last line.

litinoveweedle · 2024-05-29T08:54:23Z

@bdraco thank you for guidance, but I am lost.

If definitely shouldn't take 4 minutes to get there.

You could try turning on full debug logs and see if anything comes up during the restart, if not you'll probably have to instrument the code to find the source of shutdown being blocked. Alternatively, if the UI is still responsive for the 4 minute after shutdown is requested, we could add a service to the profiler to dump all the current running tasks

The GUI stop responding immediately when I execute restart (full restart), so using profile after that is not an option.

I did installed profiles and executed:
SERVICE PROFILER.LOG_THREAD_FRAMES
and
SERVICE PROFILER.LOG_EVENT_LOOP_SCHEDULED
and started
SERVICE PROFILER.SET_ASYNCIO_DEBUG

I am attaching log. For the first two I do not see any obvious issue, number of pending threads seems to be OK to my usage, also the scheduled event loop seems to be fine as well.

When I execute restart, there is almost nothing to be logged in the HA log, except for:

2024-05-29 10:33:07.096 ERROR (MainThread) [homeassistant] Error doing job: Error on transport creation for incoming connection
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 235, in _accept_connection2
    await waiter
  File "/usr/local/lib/python3.12/asyncio/sslproto.py", line 578, in _on_handshake_complete
    raise handshake_exc
ConnectionResetError
2024-05-29 10:33:07.106 ERROR (MainThread) [homeassistant] Error doing job: Error on transport creation for incoming connection
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 235, in _accept_connection2
    await waiter
  File "/usr/local/lib/python3.12/asyncio/sslproto.py", line 578, in _on_handshake_complete
    raise handshake_exc
ConnectionResetError
2024-05-29 10:33:07.110 ERROR (MainThread) [homeassistant] Error doing job: Error on transport creation for incoming connection
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 235, in _accept_connection2
    await waiter
  File "/usr/local/lib/python3.12/asyncio/sslproto.py", line 578, in _on_handshake_complete
    raise handshake_exc
ConnectionResetError
2024-05-29 10:34:12.189 WARNING (MainThread) [asyncio] Executing <Task pending name='Task-1174979' coro=<HomeAssistant.async_stop() running at /usr/src/homeassistant/homeassistant/core.py:1167> created at /usr/local/lib/python3.12/asyncio/tasks.py:420> took 48.419 seconds

this is the end of the log, nothing else is logged! Then HA is restarted after ~4-5 minutes and complains about not properly closed DB.

LOG_EVENT_LOOP_SCHEDULED.log
LOG_THREAD_FRAMES.log
home-assistant.log.1.txt

bdraco · 2024-05-29T09:14:26Z

I added a new service in 2024.6.0 profiler.log_current_tasks that will log the current tasks. This should help tracking down this issue.

https://my.home-assistant.io/redirect/developer_call_service?service=profiler.log_current_tasks

https://next.home-assistant.io/integrations/profiler/#service-profilerlog_current_tasks

litinoveweedle · 2024-05-29T12:17:26Z

I added a new service in 2024.6.0 profiler.log_current_tasks that will log the current tasks. This should help tracking down this issue.

Thank you, I will use it once 2024.6.0 is ready and post results here.

litinoveweedle · 2024-06-10T18:28:05Z

@bdraco

Thank you for update profiler. Using new profiler service Profiler: Log current asyncio tasks I was able to dump pending asyncio tasks.I will keep running HA till tomorrow (so it will goes into state when could not be terminated correctly), dump tasks again and use also log event loop just before restart.

Please disregard my previous post, as my log was not complete list of the asyncio task. I edited my post to delete it. I will try to post here when I will find hopefully blocking tasks tomorrow

bdraco · 2024-06-10T18:35:43Z

One thing to keep in mind is that all tasks are not blocking ones, only tasks created with hass.async_create_task are tracked and will be waited for. If its created with asyncio.create_task or create_eager_task, shutdown won't wait for these tasks.

Its a bit of work to figure out where they were created, but once you have a list of tasks, usually you can use the name of the task to figure out where it was created and than work backwards to figure out if its a tracked (one that will block shutdown) or untracked task.

litinoveweedle · 2024-06-10T18:42:23Z

Thank you for additional information, just where I will see how the task were created?

bdraco · 2024-06-10T18:58:02Z

Its a bit of digging to work it out. Once you have the name/location of the task, you have to search out the code that created the task.

Usually you can checkout core and do some git grep to try to find it.

litinoveweedle · 2024-06-10T19:03:27Z

Ok, no problem I can do that, grep -R is my good fried. I was more thinking if in the profiler you are able to list/dump task caller routine of the task. ;-) Anyway thank you I will post here tomorrow results.

litinoveweedle · 2024-06-11T10:48:19Z

OK, I just did:

SERVICE PROFILER.LOG_THREAD_FRAMES
SERVICE PROFILER.LOG_EVENT_LOOP_SCHEDULED
SERVICE PROFILER.LOG_CURRENT_TASKS
SERVICE PROFILER.SET_ASYNCIO_DEBUG (enable)
restarted HA

The whole log is attached, some findings:

Modbus sensors

# grep -c 'ModbusRegisterSensor.async_update' home-assistant.log.1 
176408

seems too many tasks for modbus sensors updates exists. I would expect, that if task is created to update modbus sensors, timeout shall exists to cancel the task if either timeout is reached or new succeeding task to update same sensor is created. So ModbusRegisterSensor.async_update tasks seems to be growing constantly...

Kodi integration

is seems, that Kodi HA native integration is unable to cleanly handle HA shutdown

2024-06-11 11:20:34.061 ERROR (Recorder) [homeassistant] Error doing job: Task exception was never retrieved (None):   File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 672, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 639, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1980, in _run_once
    handle._run()
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/homeassistant/homeassistant/helpers/event.py", line 1603, in _interval_listener
    self.hass.async_run_hass_job(self._run_job, dt_util.utcnow(), background=True)
  File "/usr/src/homeassistant/homeassistant/core.py", line 939, in async_run_hass_job
    return self._async_add_hass_job(hassjob, *args, background=background)
  File "/usr/src/homeassistant/homeassistant/core.py", line 748, in _async_add_hass_job
    task = create_eager_task(
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 37, in create_eager_task
    return Task(coro, loop=loop, name=name, eager_start=True)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/kodi/media_player.py", line 453, in _async_connect_websocket_if_disconnected
    await self._ping()
  File "/usr/src/homeassistant/homeassistant/components/kodi/media_player.py", line 439, in _ping
    await self._kodi.ping()
  File "/usr/local/lib/python3.12/site-packages/pykodi/kodi.py", line 164, in ping
    response = await self._server.JSONRPC.Ping()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/jsonrpc_websocket/jsonrpc.py", line 45, in send_message
    await self._client.send_str(message.serialize())
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client_ws.py", line 165, in send_str
    await self._writer.send(data, binary=False, compress=compress)
  File "/usr/local/lib/python3.12/site-packages/aiohttp/http_websocket.py", line 729, in send
    await self._send_frame(message, WSMsgType.TEXT, compress)
  File "/usr/local/lib/python3.12/site-packages/aiohttp/http_websocket.py", line 628, in _send_frame
    raise ConnectionResetError("Cannot write to closing transport")
ConnectionResetError: Cannot write to closing transport

Some integration are not able to close at time during the shutdown. Those includes both native HA integration onvif, tradfri, cloud...

2024-06-11 11:20:34.061 ERROR (Recorder) [homeassistant] Error doing job: Task exception was never retrieved (None):   File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 672, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 639, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1980, in _run_once
    handle._run()
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/homeassistant/homeassistant/helpers/event.py", line 1603, in _interval_listener
    self.hass.async_run_hass_job(self._run_job, dt_util.utcnow(), background=True)
  File "/usr/src/homeassistant/homeassistant/core.py", line 939, in async_run_hass_job
    return self._async_add_hass_job(hassjob, *args, background=background)
  File "/usr/src/homeassistant/homeassistant/core.py", line 748, in _async_add_hass_job
    task = create_eager_task(
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 37, in create_eager_task
    return Task(coro, loop=loop, name=name, eager_start=True)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/kodi/media_player.py", line 453, in _async_connect_websocket_if_disconnected
    await self._ping()
  File "/usr/src/homeassistant/homeassistant/components/kodi/media_player.py", line 439, in _ping
    await self._kodi.ping()
  File "/usr/local/lib/python3.12/site-packages/pykodi/kodi.py", line 164, in ping
    response = await self._server.JSONRPC.Ping()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/jsonrpc_websocket/jsonrpc.py", line 45, in send_message
    await self._client.send_str(message.serialize())
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client_ws.py", line 165, in send_str
    await self._writer.send(data, binary=False, compress=compress)
  File "/usr/local/lib/python3.12/site-packages/aiohttp/http_websocket.py", line 729, in send
    await self._send_frame(message, WSMsgType.TEXT, compress)
  File "/usr/local/lib/python3.12/site-packages/aiohttp/http_websocket.py", line 628, in _send_frame
    raise ConnectionResetError("Cannot write to closing transport")
ConnectionResetError: Cannot write to closing transport

Comfoconnect custom integration seems to have some issues during shuttdown as well.

2024-06-11 11:20:34.324 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved (None):   File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 672, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 639, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1980, in _run_once
    handle._run()
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/homeassistant/homeassistant/helpers/event.py", line 1603, in _interval_listener
    self.hass.async_run_hass_job(self._run_job, dt_util.utcnow(), background=True)
  File "/usr/src/homeassistant/homeassistant/core.py", line 939, in async_run_hass_job
    return self._async_add_hass_job(hassjob, *args, background=background)
  File "/usr/src/homeassistant/homeassistant/core.py", line 748, in _async_add_hass_job
    task = create_eager_task(
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 37, in create_eager_task
    return Task(coro, loop=loop, name=name, eager_start=True)
  File "/config/custom_components/comfoconnect/__init__.py", line 159, in send_keepalive
    await bridge.connect(entry.data[CONF_LOCAL_UUID])
  File "/usr/local/lib/python3.12/site-packages/aiocomfoconnect/comfoconnect.py", line 116, in connect
    reconnect_task = self._loop.create_task(_reconnect_loop())
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aiocomfoconnect/comfoconnect.py", line 101, in _reconnect_loop
    await read_task
  File "/usr/local/lib/python3.12/site-packages/aiocomfoconnect/bridge.py", line 113, in _read_messages
    await self._process_message()
  File "/usr/local/lib/python3.12/site-packages/aiocomfoconnect/bridge.py", line 218, in _process_message
    message = await self._read()
              ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiocomfoconnect/bridge.py", line 181, in _read
    msg_len_buf = await self._reader.readexactly(4)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/streams.py", line 752, in readexactly
    await self._wait_for_data('readexactly')
  File "/usr/local/lib/python3.12/asyncio/streams.py", line 531, in _wait_for_data
    raise RuntimeError(
RuntimeError: readexactly() called while another coroutine is already waiting for incoming data

@bdraco where do you please suggest to start?

home-assistant.log.zip

issue-triage-workflows · 2024-09-09T11:08:26Z

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

mathgim · 2024-09-09T12:12:56Z

still pending

xeberth · 2024-10-04T06:57:59Z

Still happening on HA GREEN Platform

Versions: HA OS:13.1 and HA CORE: 2024.10.0

[core-ssh homeassistant]$ ha core info
arch: aarch64
audio_input: null
audio_output: null
backups_exclude_database: false
boot: true
image: ghcr.io/home-assistant/green-homeassistant
machine: green
port: 8123
ssl: false
update_available: false
version: 2024.10.0
version_latest: 2024.10.0
watchdog: true


[core-ssh homeassistant]$ ha os info
board: green
boot: B
boot_slots:
  A:
    state: inactive
    status: good
    version: "12.4"
  B:
    state: booted
    status: good
    version: "13.1"
data_disk: 032G01-0xe3a50c10
update_available: false
version: "13.1"
version_latest: "13.1"
[core-ssh homeassistant]$
```

litinoveweedle · 2024-10-04T10:51:33Z

Same here, problem is present after any restart after more than few hours of HA uptime.

issue-triage-workflows · 2025-01-02T11:07:48Z

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

mathgim · 2025-01-02T19:25:02Z

Still going on for me

litinoveweedle · 2025-01-10T05:02:46Z

@bdraco please can you help us here? It would be really nice to get repetitive process dump during the HA stop - to see what is still running and preventing HA to stop/restart properly. This problem still persists and affect multiple users. I fully understand that it can be caused by 3rd party integration, so I am not asking for fix, but to implement code, which would help to identify component causing this issue. This would significantly help to all community also for the future similar issues. So far we were not able to identify root cause using current profiler implementation. I am fully ready to test any code - I have full control over my HA (Supervised) so I can patch HA code easily. Thank you very much in adance,

Deimos761 · 2025-01-10T08:00:12Z

(...) I fully understand that it can be caused by 3rd party integration (...)

Maybe I am wrong but I am actually not fully convinced that the problem can be caused by 3rd party integration: I did a clean installation of HA (supervised on VMware) and the problem was there almost immediately.

litinoveweedle · 2025-01-10T08:03:00Z

Do you by any chance use native modbus or any other modbus based integration? During my troubleshooting with the profiler I caught many pending scheduler modbus tasks.

pedolsky · 2025-01-10T09:38:37Z

I am actually not fully convinced that the problem can be caused by 3rd party integration

In my case, the problem occurs particularly after an Internet outage.

litinoveweedle · 2025-01-10T09:40:43Z

And do you have any internet based integration?

And in addition n do you suffer the same issue as me, so HA restart takes about 4-5min? (After hitting restart the HA becomes unresponsive immediately and will start only after 4-5 minutes)

Deimos761 · 2025-01-10T10:42:39Z

in my case, not any integration based on internet (at the time of the first error messages of sqlite3).
All working on local network

litinoveweedle · 2025-01-10T10:48:42Z

And the slow restart.as I described? Do you have this issue as well?

Deimos761 · 2025-01-10T10:56:13Z

no: the performance itself seems not to be affected, in my case

pedolsky · 2025-01-10T12:41:52Z

Neither me.

masi · 2025-01-10T14:19:25Z

I am using HA Yellow without any fiddling with modbus (whatever that may be). The restart time seems to be normal. At least it does not take so long that I start to get worried :)

probot-home-assistant bot added the integration: recorder label Jun 18, 2022

bdraco added the database: sqlite3 label Jan 30, 2023

bdraco mentioned this issue Apr 12, 2023

Ensure recorder always attempts clean shutdown if recorder thread raises #91261

Merged

20 tasks

issue-triage-workflows bot added the stale label Jul 11, 2023

issue-triage-workflows bot removed the stale label Jul 11, 2023

issue-triage-workflows bot added the stale label Sep 9, 2024

issue-triage-workflows bot removed the stale label Sep 9, 2024

issue-triage-workflows bot added the stale label Jan 2, 2025

issue-triage-workflows bot removed the stale label Jan 2, 2025

Recorder warns sqlite3 was not shutdown cleanly, despite controlled shutdown #73679

Recorder warns sqlite3 was not shutdown cleanly, despite controlled shutdown #73679

Comments

adamfeldman commented Jun 18, 2022 • edited Loading

The problem

What version of Home Assistant Core has the issue?

What was the last working version of Home Assistant Core?

What type of installation are you running?

Integration causing the issue

Link to integration documentation on our website

Diagnostics information

Example YAML snippet

Anything in the logs that might be useful for us?

Additional information

probot-home-assistant bot commented Jun 18, 2022

probot-home-assistant bot commented Jun 18, 2022

mrcfelici commented Jun 24, 2022

pvizeli commented Jun 24, 2022 • edited Loading

adamfeldman commented Jul 27, 2022

nvOC-Stubo commented Aug 10, 2022

snowmangh commented Aug 12, 2022

jimmyl88 commented Nov 2, 2022

Commifreak commented Jan 2, 2023

bdraco commented Feb 7, 2023

TTLucian commented Feb 15, 2023 • edited Loading

Commifreak commented Feb 16, 2023

Luca1996O commented Mar 7, 2023

bdraco commented Apr 12, 2023

issue-triage-workflows bot commented Jul 11, 2023

Commifreak commented Jul 11, 2023

ELind77 commented Jul 21, 2023

gcarneiro commented Sep 11, 2023

masi commented Oct 30, 2023

ginjo commented Jan 23, 2024

Deimos761 commented Feb 7, 2024

RhavoX commented Feb 13, 2024

amarofarinha commented Mar 1, 2024

litinoveweedle commented Apr 20, 2024

bdraco commented Apr 20, 2024

dlamblin commented May 23, 2024

litinoveweedle commented May 29, 2024

bdraco commented May 29, 2024 • edited Loading

litinoveweedle commented May 29, 2024

litinoveweedle commented Jun 10, 2024 • edited Loading

bdraco commented Jun 10, 2024

litinoveweedle commented Jun 10, 2024

bdraco commented Jun 10, 2024

litinoveweedle commented Jun 10, 2024

litinoveweedle commented Jun 11, 2024

issue-triage-workflows bot commented Sep 9, 2024

mathgim commented Sep 9, 2024

xeberth commented Oct 4, 2024 • edited Loading

litinoveweedle commented Oct 4, 2024

issue-triage-workflows bot commented Jan 2, 2025

mathgim commented Jan 2, 2025

litinoveweedle commented Jan 10, 2025

Deimos761 commented Jan 10, 2025

litinoveweedle commented Jan 10, 2025

pedolsky commented Jan 10, 2025

litinoveweedle commented Jan 10, 2025 • edited Loading

Deimos761 commented Jan 10, 2025

litinoveweedle commented Jan 10, 2025

Deimos761 commented Jan 10, 2025

pedolsky commented Jan 10, 2025

masi commented Jan 10, 2025

adamfeldman commented Jun 18, 2022 •

edited

Loading

pvizeli commented Jun 24, 2022 •

edited

Loading

TTLucian commented Feb 15, 2023 •

edited

Loading

bdraco commented May 29, 2024 •

edited

Loading

litinoveweedle commented Jun 10, 2024 •

edited

Loading

xeberth commented Oct 4, 2024 •

edited

Loading

litinoveweedle commented Jan 10, 2025 •

edited

Loading