Weight sensor average statistics by state durations

[emontnemery]

Proposed change

Weight sensor average statistics by state durations.

This makes the average calculation more accurate when there are successive occurrences of the same state and for sensors which updates states on an irregular basis.

Type of change

• Dependency upgrade
• Bugfix (non-breaking change which fixes an issue)
• New integration (thank you!)
• New feature (which adds functionality to an existing integration)
• Breaking change (fix/feature causing existing functionality to break)
• Code quality improvements to existing code or addition of tests

Additional information

• This PR fixes or closes issue: fixes #
• This PR is related to issue:
• Link to documentation pull request:

Checklist

• The code change is tested and works locally.
• Local tests pass. Your PR cannot be merged unless tests pass
• There is no commented out code in this PR.
• I have followed the development checklist
• The code has been formatted using Black (black --fast homeassistant tests)
• Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

• No score or internal
• 🥈 Silver
• 🥇 Gold
• 🏆 Platinum

To help with the load of incoming pull requests:

• I have reviewed two other open pull requests in this repository.

[dgomes]

This is actually implemented in the filter sensor https://www.home-assistant.io/integrations/filter/

https://github.com/home-assistant/core/blob/dev/homeassistant/components/filter/sensor.py#L573

[emontnemery]

@dgomes Right, but the TimeSMAFilter is not well suited when we already have all the samples on hand, or do you suggest to refactor Filter to allow passing in an iterable with states?

[dgomes]

The filter sensor also uses history, I propose we do some refactoring in order for both to use the same code/algorithm.

[emontnemery]

Right, I missed that it does that in an async_added_to_hass.

Priming of the filters from history is done like this:

            # Replay history through the filter chain
            for state in history_list:
                if state.state not in [STATE_UNKNOWN, STATE_UNAVAILABLE, None]:
                    self._update_filter_sensor_state(state, False)

The complexity of this is O(n^2) for the TimeSMAFilter because it loops over all samples for each call to _update_filter_sensor_state, so we shouldn't do like that for the sensor statistics which will run every hour.

What we could do is perhaps to add a helper which takes as input an iterable of some state abstraction, and only does the actual calculation:

        moving_sum = 0
        start = new_state.timestamp - self._time_window
        prev_state = self.last_leak or self.queue[0]
        for state in self.queue:
            moving_sum += (state.timestamp - start).total_seconds() * prev_state.state
            start = state.timestamp
            prev_state = state
        new_state.state = moving_sum / self._time_window.total_seconds()

[frenck]

I suggest we might need to get this refactoring out of the critical path.
This change is part of the MVP to start collecting stats, which needs to go into beta.

[dgomes]

What we could do is perhaps to add a helper which takes as input an iterable of some state abstraction, and only does the actual calculation:

That's the way forward I believe

[frenck]

That's the way forward I believe

In a follow-up PR for another time.