Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Watchdog can freeze ODROID-M1 and ODROID-XU4 (maybe others) #2675

Open
agners opened this issue Aug 8, 2023 · 4 comments
Open

Watchdog can freeze ODROID-M1 and ODROID-XU4 (maybe others) #2675

agners opened this issue Aug 8, 2023 · 4 comments
Labels
board/odroid Hardkernel's ODROID Boards bug os pinned Prevents from getting marked as stale

Comments

@agners
Copy link
Member

agners commented Aug 8, 2023

Describe the issue you are experiencing

Since #2628 watchdog is enabled in systemd.

It seems that when stressing the system with stress-ng, it can trigger a watchdog reset by systemd. Typically, this should lead to a reboot. However, it seems that certain embedded boards have bugs in their watchdog implementation or elsewhere, which causes the system to not reboot but just freeze.

This has been observed with ODROID-M1 and ODROID-XU4

What operating system image do you use?

odroid-m1 (Hardkernel ODROID-M1)

What version of Home Assistant Operating System is installed?

11.0.dev20230803

Did you upgrade the Operating System.

Yes

Steps to reproduce the issue

  1. Boot the board with the dev version
  2. Use stress-ng --all 4 to stress the system a lot
  3. Wait for 5-10 minutes until a the board stops responding

A easier way is to use a stable release (e.g. 10.4) which does not have systemd watchdog enabled:

  1. Use cat /dev/watchdog to start the watchdog (this will lead to cat: read error: Invalid argument, but at this point the watchdog has been started because the device file has been opened.
  2. Wait for a freeze to happen

Anything in the Supervisor logs that might be useful for us?

No

Anything in the Host logs that might be useful for us?

Some board warn that the watchdog continues to run after catting the file:


Aug 08 21:03:35 ha-shelf2-om1 kernel: watchdog: watchdog0: watchdog did not stop!


### System information

_No response_

### Additional information

_No response_
@agners agners added bug board/odroid Hardkernel's ODROID Boards os labels Aug 8, 2023
agners added a commit to agners/operating-system that referenced this issue Aug 9, 2023
agners added a commit to agners/operating-system that referenced this issue Aug 9, 2023
In current Linux 6.1 kernel, the watchdog leads to a system freeze
instead of a reboot.
agners added a commit to agners/operating-system that referenced this issue Aug 9, 2023
In current Linux 6.1 kernel, the watchdog leads to a system freeze
instead of a reboot.
agners added a commit that referenced this issue Aug 9, 2023
In current Linux 6.1 kernel, the watchdog leads to a system freeze
instead of a reboot.
agners added a commit that referenced this issue Aug 9, 2023
In current Linux 6.1 kernel, the watchdog leads to a system freeze
instead of a reboot.
@mingzhangqun
Copy link
Contributor

mingzhangqun commented Aug 30, 2023

Hi, everyone. I've tried enable BR2_INIT_BUSYBOX and disable BR2_INIT_SYSTEMD, the watchdog freeze also.
So I guess it has nothing to do with systemd.

watchdog.txt

@mingzhangqun
Copy link
Contributor

mingzhangqun commented Aug 30, 2023

Using the official source code (linux 5.10) the watchdog works well. I'm trying the newest official sdk (linux 6.1).

@mingzhangqun
Copy link
Contributor

I've tried official sdk(linux 6.1), the watchdog works well.
I replaced the kernel partition(dd if=kernel.squash of=/dev/sdx2) with haos, the watchdog works too.
kernel_img.zip

Copy link

There hasn't been any activity on this issue recently. To keep our backlog manageable we have to clean old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant OS version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale label Nov 30, 2023
@agners agners added pinned Prevents from getting marked as stale and removed stale labels Nov 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
board/odroid Hardkernel's ODROID Boards bug os pinned Prevents from getting marked as stale
Projects
None yet
Development

No branches or pull requests

2 participants