Significantly speed up single use callback dispatchers #117934
Conversation
We have many cases where we only use a dispatcher once. For example, in MQTT we use the dispatchers for discovery and once the first one fires, we disconnect and setup a different one. During startup this can quickly reach 10000+ dispatcher connect and disconnects. Because we wrapped each job with catch_log_exceptions which calls wraps() the cost of connecting up the dispatcher vastly exceeded the cost of firing the dispatcher. This made the dispatcher a poor choice for single use signals. To mitigate this, we no longer wrap callback dispatchers in catch_log_exceptions and catch the exception inline.
We have many cases where we only use a dispatcher once. For example, in MQTT we use the dispatchers for discovery and once the first one fires, we disconnect and setup a different one. During startup this can quickly reach 10000+ dispatcher connect and disconnects. Because we wrapped each job with catch_log_exceptions which calls wraps() the cost of connecting up the dispatcher vastly exceeded the cost of firing the dispatcher. This made the dispatcher a poor choice for single use signals. To mitigate this, we no longer wrap callback dispatchers in catch_log_exceptions and catch the exception inline.
|
looks like we need to fix some stuff in mqtt first. |
At 66 entities per device its quite the speed up #109030 |
|
Another option would be to not use the dispatcher in mqtt since its rare that we have these single use high volume use cases |
|
I think I might be overthinking that. The maint burden is low, and this change is strait-forward so better to not develop something one off for mqtt. |
|
It does also nicely speed up mobile app setup as well |
|
I was hoping to make |
|
I'm not sure what the solution to replace |
|
This looks solid, but will run some more profiles in the morning just to be sure since its too late here |
|
Even if we can make I'm happy with this after sleeping on it. |
Proposed change
We have many cases where we only use a dispatcher once. For example, in MQTT we use
the dispatchers for discovery and once the first one fires, we disconnect and
setup a different one. During startup this can quickly reach 10000+ dispatcher
connect and disconnects. Because we wrapped each job with
catch_log_exceptionswhich calls
wraps()the cost of connecting up the dispatcher vastly exceededthe cost of firing the dispatcher. This made the dispatcher a poor choice for
single use signals. To mitigate this, we no longer wrap callback dispatchers
in
catch_log_exceptionsand catch the exception inline to ensure the behavioris the same but avoids the
wraps()overhead.Type of change
Additional information
Checklist
ruff format homeassistant tests)If user exposed functionality or configuration variables are added/changed:
If the code communicates with devices, web services, or third-party tools:
Updated and included derived files by running:
python3 -m script.hassfest.requirements_all.txt.Updated by running
python3 -m script.gen_requirements_all..coveragerc.To help with the load of incoming pull requests: