Duplicated entities after upgrading ESPHome firmware

[felipecrs]

The problem

Today I triggered a firmware upgrade for all my ESPHome devices from within Home Assistant itself, through their update entities.

The upgrade path was from 2023.3.0 to 2023.5.0.

However, as every device was being upgraded, Home Assistant was starting to duplicate their entities. Example:

Funnily, only the main entities of the devices got duplicated, not every entity of the device.

What version of Home Assistant Core has the issue?

core-2023.5.3

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

ESPHome

Link to integration documentation on our website

https://www.home-assistant.io/integrations/esphome

Diagnostics information

A device which was upgraded and had the entity duplicated:

config_entry-esphome-32631d630ccc67a9048f618b937cf594.json.txt

A device which wasn't yet upgraded:

config_entry-esphome-6790a04f0a12304d9adb6233c95c06f4.json.txt

After upgrading it, which generated the duplicated entity problem:

config_entry-esphome-6790a04f0a12304d9adb6233c95c06f4.json (1).txt

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant]

Hey there @OttoWinter, @jesserockz, mind taking a look at this issue as it has been labeled with an integration (esphome) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of esphome can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign esphome Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

esphome documentation
esphome source
_{^{(message by IssueLinks)}}

[felipecrs]

From my 8 ESPHome devices, this issue happened for 7 of them. Although they had different entity domains (lights, switches and covers), they all had something in common: they only exposed a single main entity.

The only device in which the issue did not happen was my SONOFF 4CH, which exposes not a single main entity but 4 lights entities instead.

I hope this helps figuring out the root cause of the problem, maybe.

[RoboMagus]

This has been hapening to me for a while as well. Not just on HA upgrades, but any HA restart there is a chance that (all) entities of an ESPHome device are re-registered under a different entity_id.

I believe this is a pre existing issue for a while already, but could not find an open issue about it.

This happened to me again this morning for 2 ESPHome power plugs:

All of the original entities are now unavailable and the new ones have entity_id formated like <old_entity_id>_2.

No errors in the logs, just the very "helpful" mention of new entities found:

2023-05-21 03:02:14.967 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new number.esphome entity: number.neo_coolcam_2_power_variance_threshold_2
2023-05-21 03:02:14.971 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_2_voltage_2
2023-05-21 03:02:14.973 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_2_amperage_2
2023-05-21 03:02:14.976 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_2_power_2
2023-05-21 03:02:14.978 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_2_total_daily_energy_2
2023-05-21 03:02:14.979 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_2_apparent_power_2
2023-05-21 03:02:14.981 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_2_power_factor_2
2023-05-21 03:02:14.982 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_2_power_variance_2
2023-05-21 03:02:14.984 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_2_power_variance_riemann_sum_2
2023-05-21 03:02:14.987 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_2_energy_2
2023-05-21 03:02:14.990 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new button.esphome entity: button.neo_coolcam_2_restart_2
2023-05-21 03:02:14.992 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new binary_sensor.esphome entity: binary_sensor.neo_coolcam_2_button_2
2023-05-21 03:02:14.995 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new switch.esphome entity: switch.neo_coolcam_2_power_variance_error_2
2023-05-21 03:02:15.218 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new binary_sensor.esphome entity: binary_sensor.neo_coolcam_3_button_2
2023-05-21 03:02:15.222 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_3_voltage_2
2023-05-21 03:02:15.225 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_3_amperage_2
2023-05-21 03:02:15.227 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_3_power_2
2023-05-21 03:02:15.229 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_3_energy_2
2023-05-21 03:02:15.234 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_3_apparent_power_2
2023-05-21 03:02:15.237 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_3_power_factor_2
2023-05-21 03:02:15.239 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.esphome entity: sensor.neo_coolcam_3_total_daily_energy_2
2023-05-21 03:02:15.245 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new button.esphome entity: button.neo_coolcam_3_restart_2
2023-05-21 03:02:15.761 INFO (MainThread) [aioesphomeapi.reconnect_logic] Successfully connected to neo-coolcam-2 @ 192.168.1.108
2023-05-21 03:02:15.881 INFO (MainThread) [aioesphomeapi.reconnect_logic] Successfully connected to neo-coolcam-3 @ 192.168.1.205

[RoboMagus]

Perhaps related to #89104?

[bdraco]

Do you by chance have two config entires for the same device?

I just fixed a bug in #95678 that would allow you to set up the same device twice

[felipecrs]

Do you by chance have two config entires for the same device?

If you mean the same device multiple times in this page:

No, I'm pretty sure it is not the case and it was not the case before the upgrade as well.

[felipecrs]

But I can try upgrading my devices after the bugfix gets released, and then I can check if the same reappeared or not.

[bdraco]

Can you check .storage/core.entity_registry and compare the unique ids of the duplicates?

[felipecrs]

Hm... I no longer have the environment with the duplicated entities (because it's my "production" home, so I had to fix the entities manually - deleting the outdated ones), so, unfortunately, I can't check this. :(

Next time I upgrade my ESPHome devices, I'll take a backup of HA before and after the upgrade so we can have data to compare in case the issue happens again.

[felipecrs]

I noticed the PR was merged. I'll update all my devices and report back.

[felipecrs]

I updated my 8 devices and the issue did not happen this time.

Maybe the PR fixed the problem (thanks @bdraco!), or maybe I was just lucky.

Either way, I'll close this issue. In case I notice the problem again, I can reopen it.

If someone else faces the same problem, feel free to chime in and I can reopen the issue.