Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set last_reset for integrated entities in IoTaWatt #65143

Merged
merged 3 commits into from
Feb 2, 2022

Conversation

agners
Copy link
Member

@agners agners commented Jan 28, 2022

Breaking change

Proposed change

For entities which integrate over time (like energy in watt hours) the
iotawattpy library uses the beginning of the year as start date by
default (using relative time specification "y", see 1). Since
PR #56974 state class has been changed from TOTAL_INCREASING to TOTAL.
However, the relative start date of "y" causes the value to reset to
zero at the beginning of the year.

This fixes it by setting last_reset properly, which takes such resets
into account.

While at it, let's set the cycle to one day. This lowers the load on
IoTaWatt (fetching with start date beginning of the day seems to
response faster than beginning of the year).

image

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • The code has been formatted using Black (black --fast homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

  • No score or internal
  • 🥈 Silver
  • 🥇 Gold
  • 🏆 Platinum

To help with the load of incoming pull requests:

@probot-home-assistant
Copy link

Hey there @gtdiehl, @jyavenard, mind taking a look at this pull request as it has been labeled with an integration (iotawatt) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)

For entities which integrate over time (like energy in watt hours) the
iotawattpy library uses the beginning of the year as start date by
default (using relative time specification "y", see [1]). Since
PR home-assistant#56974 state class has been changed from TOTAL_INCREASING to TOTAL.
However, the relative start date of "y" causes the value to reset to
zero at the beginning of the year.

This fixes it by setting last_reset properly, which takes such resets
into account.

While at it, let's set the cycle to one day. This lowers the load on
IoTaWatt (fetching with start date beginning of the day seems to
response faster than beginning of the year).

[1]: https://docs.iotawatt.com/en/master/query.html#relative-time
@agners agners force-pushed the fix-iotawatt-wh-sensor branch from 70ff794 to a635d55 Compare January 28, 2022 17:41
@agners
Copy link
Member Author

agners commented Jan 28, 2022

It seems that the entities with the accumulated suffix (introduced with #55512) are not affected by this problem, which make sense since those query energy values only in time slices.

That said, I am question why we need to have two entities for energy (watt hours), at least today: When using a fixed starting point, there should be no lost data (unless HA is offline more than a period length, which would be 1d in this case). The accumulating sensor seems to be somewhat of a re-implementation of features available with state_class TOTAL and using last_reset. To calculate energy generation/consumption using a single CT with negative values, IoTaWatt has now an on-board integration option which is much more accurate since its operating on 5s interval (see https://docs.iotawatt.com/en/master/integrators.html).

@emontnemery @jyavenard thoughts?

@agners
Copy link
Member Author

agners commented Jan 28, 2022

Btw, it seems to me that accumulated sensors have a slight inaccuracy. To compare, I added both sensors to the energy dashboard (the regular wh and the wh accumulated sensor, besides the utility_meter which splits daytime/nighttime usage is on there too, but that is irrelevant).

image

image

@frenck frenck changed the title Set last_reset for integrated entities Set last_reset for integrated entities in IoTaWatt Jan 28, 2022
Co-authored-by: Franck Nijhof <frenck@frenck.nl>
@jyavenard
Copy link
Contributor

Btw, it seems to me that accumulated sensors have a slight inaccuracy. To compare, I added both sensors to the energy dashboard (the regular wh and the wh accumulated sensor, besides the utility_meter which splits daytime/nighttime usage is on there too, but that is irrelevant).

image

image

Accumulated sensors do no measure exactly the same period as the energy one.

They round to the previous minute while energy the value calculated since the beginning of the year up to the current query time.

It is possible that they accumulate overtime some rounding error as by default it’s using only 3 digits accuracy.

But that should be barely measurable over an hour or a day as used by the energy screen

@agners
Copy link
Member Author

agners commented Jan 31, 2022

They round to the previous minute while energy the value calculated since the beginning of the year up to the current query time.

It is possible that they accumulate overtime some rounding error as by default it’s using only 3 digits accuracy.

I understand the difference, but do you feel there is still a need for that accumulating sensor (now that there are integration sensors on IoTaWatt side)?

But that should be barely measurable over an hour or a day as used by the energy screen

Surprisingly, my screenshot show a difference of 0.3kWh at a daily use of 6kWh, that is about 5%. Not huge, but also somewhat surprising that rounding leads to such a big difference... I did not restart or do anything to the instance at that day (I picked that day because I know I was not at home) 😄

@jyavenard
Copy link
Contributor

They round to the previous minute while energy the value calculated since the beginning of the year up to the current query time.
It is possible that they accumulate overtime some rounding error as by default it’s using only 3 digits accuracy.

I understand the difference, but do you feel there is still a need for that accumulating sensor (now that there are integration sensors on IoTaWatt side)?

For most, I'd say we now longer need the accumulating sensor. However, issue with IotaWatt integration side is that they never reset nor can you reset them.
I recently changed my setup when I added a new 3-phase circuit. As I didn't have enough input on the iotawatt; I used two inputs on what used to be a 3-phase monitoring and use those.

The advantage of the accumulated sensor is that their lifetime is entirely controlled by HA, I can delete their value and they will start accruing at the time I want.
The default Wh sensor will show value from the beginning of the year, regardless of what they measure. For me it would have made the new values nonsensical.

Hence, why I've kept the accumulated sensors, because selfishly, I use and need them.

@agners
Copy link
Member Author

agners commented Feb 2, 2022

However, issue with IotaWatt integration side is that they never reset nor can you reset them.

So far we asked for a yearly total, so they did reset once a year.

The advantage of the accumulated sensor is that their lifetime is entirely controlled by HA, I can delete their value and they will start accruing at the time I want.

Do you use the state/values of the accumulated energy entities directly? If so, what is the use case for that?

The default Wh sensor will show value from the beginning of the year, regardless of what they measure. For me it would have made the new values nonsensical.

I see, but with that change we ask for the daily total, which leads to a reset every day. You will still get a peak at the hour you add a new sensor, but it won't be as bad as before. So would this change then cover your use case?

Copy link
Contributor

@emontnemery emontnemery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@agners agners merged commit c8e6435 into home-assistant:dev Feb 2, 2022
@github-actions github-actions bot locked and limited conversation to collaborators Feb 4, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants