Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

drivers/dht: reversal of #16934 regression #19674

Closed
wants to merge 1 commit into from

Conversation

hugueslarrive
Copy link
Contributor

Contribution description

This PR reintroduces the specific handling for DHT22 that was mistakenly removed by #16934 (drivers/dht: correct interpreting raw values).

The improvement in value handling for DHT11 has been preserved, except for the detection of negative values, which now occurs on the 4th byte for DHT11 instead of the 3rd byte for DHT22.

Testing procedure

With a DHT21 using tests/drivers/dht:

2023-05-27 18:25:35,220 # DHT values - temp: 24.9°C - relative humidity: 78.7%
2023-05-27 18:25:37,245 # DHT values - temp: 25.1°C - relative humidity: 78.3%
2023-05-27 18:25:39,270 # DHT values - temp: 25.2°C - relative humidity: 78.0%
2023-05-27 18:25:41,294 # DHT values - temp: 25.4°C - relative humidity: 77.6%
2023-05-27 18:25:43,319 # DHT values - temp: 25.5°C - relative humidity: 77.3%
2023-05-27 18:25:45,343 # DHT values - temp: 25.6°C - relative humidity: 76.9%
2023-05-27 18:25:47,368 # DHT values - temp: 25.7°C - relative humidity: 76.6%
2023-05-27 18:25:49,392 # DHT values - temp: 25.9°C - relative humidity: 76.3%
2023-05-27 18:25:51,417 # DHT values - temp: 26.0°C - relative humidity: 76.3%
2023-05-27 18:25:53,442 # DHT values - temp: 26.1°C - relative humidity: 75.9%
2023-05-27 18:25:55,467 # DHT values - temp: 26.3°C - relative humidity: 75.6%
2023-05-27 18:25:57,491 # DHT values - temp: 26.4°C - relative humidity: 75.3%
2023-05-27 18:25:59,516 # DHT values - temp: 26.6°C - relative humidity: 75.0%
2023-05-27 18:26:01,541 # DHT values - temp: 26.7°C - relative humidity: 74.6%
2023-05-27 18:26:03,566 # DHT values - temp: 26.8°C - relative humidity: 74.2%
2023-05-27 18:26:05,590 # DHT values - temp: 26.9°C - relative humidity: 73.8%
2023-05-27 18:26:07,615 # DHT values - temp: 27.0°C - relative humidity: 74.0%

I don't have a DHT11 to test with.

Issues/PRs references

Reversal of #16934 regression

@hugueslarrive

This comment was marked as outdated.

@hugueslarrive
Copy link
Contributor Author

2023-05-29 07:41:11,212 # Running test 5 times with 5 distinct sleep times
2023-05-29 07:41:11,213 # Slept for 81 us (expected: 40 us) Offset: 41 us
2023-05-29 07:41:11,213 # Slept for 1032 us (expected: 1000 us) Offset: 32 us
2023-05-29 07:41:12,206 # Slept for 1000032 us (expected: 1000000 us) Offset: 32 us
2023-05-29 07:41:12,238 # Slept for 20032 us (expected: 20000 us) Offset: 32 us
2023-05-29 07:41:12,239 # Slept for 46 us (expected: 1 us) Offset: 45 us
2023-05-29 07:41:12,240 # Slept for 73 us (expected: 40 us) Offset: 33 us
2023-05-29 07:41:12,240 # Slept for 1033 us (expected: 1000 us) Offset: 33 us
2023-05-29 07:41:13,233 # Slept for 1000033 us (expected: 1000000 us) Offset: 33 us
2023-05-29 07:41:13,265 # Slept for 20032 us (expected: 20000 us) Offset: 32 us
2023-05-29 07:41:13,266 # Slept for 45 us (expected: 1 us) Offset: 44 us
2023-05-29 07:41:13,266 # Slept for 72 us (expected: 40 us) Offset: 32 us
2023-05-29 07:41:13,267 # Slept for 1032 us (expected: 1000 us) Offset: 32 us
2023-05-29 07:41:14,260 # Slept for 1000032 us (expected: 1000000 us) Offset: 32 us
2023-05-29 07:41:14,293 # Slept for 20033 us (expected: 20000 us) Offset: 33 us
2023-05-29 07:41:14,293 # Slept for 45 us (expected: 1 us) Offset: 44 us
2023-05-29 07:41:14,294 # Slept for 72 us (expected: 40 us) Offset: 32 us
2023-05-29 07:41:14,294 # Slept for 1032 us (expected: 1000 us) Offset: 32 us
2023-05-29 07:41:15,287 # Slept for 1000032 us (expected: 1000000 us) Offset: 32 us
2023-05-29 07:41:15,320 # Slept for 20032 us (expected: 20000 us) Offset: 32 us
2023-05-29 07:41:15,320 # Slept for 45 us (expected: 1 us) Offset: 44 us
2023-05-29 07:41:15,321 # Slept for 73 us (expected: 40 us) Offset: 33 us
2023-05-29 07:41:15,321 # Slept for 1033 us (expected: 1000 us) Offset: 33 us
2023-05-29 07:41:16,314 # Slept for 1000033 us (expected: 1000000 us) Offset: 33 us
2023-05-29 07:41:16,352 # Slept for 20032 us (expected: 20000 us) Offset: 32 us
2023-05-29 07:41:16,352 # Slept for 45 us (expected: 1 us) Offset: 44 us
2023-05-29 07:41:16,353 # Test ran for 5137492 us

Okay, if it sleeps for 45 µs in the _wait_for_level() function, there is little chance that it will work!

@hugueslarrive
Copy link
Contributor Author

On nucleo-f303k8:

2023-05-29 08:01:11,041 # Slept for 57 us (expected: 40 us) Offset: 17 us
2023-05-29 08:01:11,047 # Slept for 1017 us (expected: 1000 us) Offset: 17 us
2023-05-29 08:01:12,052 # Slept for 1000016 us (expected: 1000000 us) Offset: 16 us
2023-05-29 08:01:12,077 # Slept for 20016 us (expected: 20000 us) Offset: 16 us
2023-05-29 08:01:12,081 # Slept for 26 us (expected: 1 us) Offset: 25 us

Strange...

@hugueslarrive
Copy link
Contributor Author

On esp8266, replacing xtimer_usleep() by xtimer_spin(xtimer_ticks_from_usec()):

2023-05-29 08:16:58,254 # Slept for 47 us (expected: 40 us) Offset: 7 us
2023-05-29 08:16:58,255 # Slept for 1007 us (expected: 1000 us) Offset: 7 us
2023-05-29 08:16:59,247 # Slept for 1000007 us (expected: 1000000 us) Offset: 7 us
2023-05-29 08:16:59,285 # Slept for 20006 us (expected: 20000 us) Offset: 6 us
2023-05-29 08:16:59,286 # Slept for 7 us (expected: 1 us) Offset: 6 us

Much better! Anyway, who can expect to watch a signal while sleeping...

@hugueslarrive
Copy link
Contributor Author

2023-05-29 08:35:41,301 # Initializing DHT sensor...	[OK]
2023-05-29 08:35:41,302 # 
2023-05-29 08:35:41,328 # DHT values - temp: 25.1°C - relative humidity: 73.2%
2023-05-29 08:35:43,353 # DHT values - temp: 25.1°C - relative humidity: 73.2%
2023-05-29 08:35:45,378 # DHT values - temp: 25.1°C - relative humidity: 73.2%
2023-05-29 08:35:47,403 # DHT values - temp: 25.1°C - relative humidity: 73.2%
2023-05-29 08:35:49,428 # DHT values - temp: 25.1°C - relative humidity: 73.2%
2023-05-29 08:35:51,453 # DHT values - temp: 25.1°C - relative humidity: 73.2%
2023-05-29 08:35:53,478 # DHT values - temp: 25.1°C - relative humidity: 73.1%
2023-05-29 08:35:55,503 # DHT values - temp: 25.1°C - relative humidity: 73.1%
2023-05-29 08:35:57,528 # DHT values - temp: 25.1°C - relative humidity: 73.2%

Great! It makes me want to try again on the nano where it has never worked with riot's driver...

@maribu
Copy link
Member

maribu commented May 29, 2023

It is also possible to use a GPIO IRQ for decoding, e.g. see #18591 That might allow sleeping while still being reliable enough.

(And with the spurious IRQs in periph_timer many drivers now fixed except for a few rare platforms, it should now actually work reliable.)

@maribu maribu added Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) Process: needs backport Integration Process: The PR is required to be backported to a release or feature branch labels May 29, 2023
Copy link
Member

@maribu maribu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this. I only have a few nitpicks inline.

@maribu
Copy link
Member

maribu commented May 30, 2023

Thanks for reporting and fixing the issue. Please squash :)

@maribu maribu added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label May 30, 2023
@maribu
Copy link
Member

maribu commented May 30, 2023

I don't have a DHT11 to test with.

I just tested with an DHT11 connected to an AVR: To works fine :)

@riot-ci
Copy link

riot-ci commented May 30, 2023

Murdock results

✔️ PASSED

5fa999e drivers/dht: reversal of #16934 regression

Success Failures Total Runtime
6933 0 6933 12m:51s

Artifacts

This PR reintroduces the specific handling for DHT22 that was mistakenly removed by RIOT-OS#16934 (drivers/dht: correct interpreting raw values).

The improvement in value handling for DHT11 has been preserved, except for the detection of negative values, which now occurs on the 4th byte for DHT11 instead of the 3rd byte for DHT22.

Moreover :
- improvement in reliability on ESP8266
- make it works on avr

Co-authored-by: Marian Buschsieweke <maribu@users.noreply.github.com>
@hugueslarrive
Copy link
Contributor Author

I still have a doubt regarding the handling of negative temperatures on the new DHT11. I couldn't find a datasheet with the data format of the new DHT11, so I looked here https://github.com/adafruit/DHT-sensor-library/blob/master/DHT.cpp and here https://github.com/Seeed-Studio/Grove_Temperature_And_Humidity_Sensor/blob/master/DHT.cpp.

@hugueslarrive
Copy link
Contributor Author

Outdated by #19718

bors bot added a commit that referenced this pull request Jun 20, 2023
19718: drivers/dht: busy wait reimplementation r=maribu a=hugueslarrive

### Contribution description

In PR #19674, I also provided quick and dirty fixes to restore functionality on esp8266 and enable operation on AVR. While reviewing PR #18591, it became apparent to me that this driver needed a refresh, particularly its migration to ztimer.

The cause of the malfunction on esp8266 was that since the default switch to ztimer as the backend for xtimer, XTIMER_BACKOFF was no longer taken into account. Therefore, the correction I provided in PR #19674 simply made explicit what was previously done implicitly with xtimer and now needs to be done explicitly with ztimer (spinning instead of sleeping).

Moreover, it was unnecessarily complex to measure the pulse duration in a busy-wait implementation, which required 2 calls to ztimer_now() and  32-bit operations expensive on 8-bit architecture. Instead, it is sufficient to read the state of the bus at the threshold moment.

Finally, in practice, it is possible to reduce the read interval (down to less than 0.5s for DHT22) by "harassing" the DHT with start signals until it responds.

This re-implementation brings the following improvements:
- Many backports from `@maribu's` IRQ based implementation (#18591):
  - Use of ztimer
  - Use of errno.h
  - Use of a dht_data structure to pass arguments, to facilitate integration
  - Adaptation of the bit parsing technique to parse bits into the data array
- Reintroduction of DHT11/DHT22 differentiation.
- Separation of `dht_read()` steps into functions for better readability and the ability to share certain functions among different implementations
- Sensor presence check in `dht_init()`
- ~~Automatic adjustment to a minimum data hold time~~
- Default input mode changed to open drain (a pull-up resistor should be placed close to the output if necessary but not close to the input)
- AVR support without platform-specific handling by avoiding  ztimer_spin() and using the overflow of an 8-bit variable as a pre-timeout to minimize time-consuming ztimer_now() calls

Regarding the changes in the start signal sequence and the removal of the `_reset()` function:
![nano_dht_read_2](https://github.com/RIOT-OS/RIOT/assets/67432403/95966813-2b5f-4a0f-a388-8ac630526ab2)

~~In the previous implementation, there was an unnecessary spike at the beginning of the signal sequence, corresponding to START_HIGH_TIME. This spike has been removed in the re-implementation, as it is unnecessary. Instead, the MCU now simply pulls the signal low for START_LOW_TIME and then releases the bus, which is sufficient for initiating communication with the DHT sensor.~~ Actually, it is necessary to raise the bus level; otherwise, the _wait_for_level() called immediately after to check the response of the DHT may read the port before the signal level is raised, resulting in a false positive.

Additionally, the previous implementation had an issue where the MCU switched back to output mode and went high immediately after reading the 40 bits of data. However, the DHT sensor was still transmitting 2 or 3 additional bytes of '0' at that point, causing a conflict. This issue has been resolved in the re-implementation:
![nano_dht_read_optimized](https://github.com/RIOT-OS/RIOT/assets/67432403/ff124839-5ec5-4df3-bab7-5348d8160a25)

~~Regarding the optimization for AVR, I have performed measurements of `_wait_for_level()` until timeout (85 loops):~~
~~- on esp8266-esp-12x: 264 µs, which is 3.11 µs per loop~~
~~- on nucleo-f303k8: 319 µs, which is 3.75 µs per loop~~
~~- on arduino-nano: 3608 µs, which is 42.45 µs per loop~~
~~Duration measurements on the Arduino Nano:~~


Co-authored-by: Hugues Larrive <hlarrive@pm.me>
bors bot added a commit that referenced this pull request Jun 20, 2023
19718: drivers/dht: busy wait reimplementation r=benpicco a=hugueslarrive

### Contribution description

In PR #19674, I also provided quick and dirty fixes to restore functionality on esp8266 and enable operation on AVR. While reviewing PR #18591, it became apparent to me that this driver needed a refresh, particularly its migration to ztimer.

The cause of the malfunction on esp8266 was that since the default switch to ztimer as the backend for xtimer, XTIMER_BACKOFF was no longer taken into account. Therefore, the correction I provided in PR #19674 simply made explicit what was previously done implicitly with xtimer and now needs to be done explicitly with ztimer (spinning instead of sleeping).

Moreover, it was unnecessarily complex to measure the pulse duration in a busy-wait implementation, which required 2 calls to ztimer_now() and  32-bit operations expensive on 8-bit architecture. Instead, it is sufficient to read the state of the bus at the threshold moment.

Finally, in practice, it is possible to reduce the read interval (down to less than 0.5s for DHT22) by "harassing" the DHT with start signals until it responds.

This re-implementation brings the following improvements:
- Many backports from `@maribu's` IRQ based implementation (#18591):
  - Use of ztimer
  - Use of errno.h
  - Use of a dht_data structure to pass arguments, to facilitate integration
  - Adaptation of the bit parsing technique to parse bits into the data array
- Reintroduction of DHT11/DHT22 differentiation.
- Separation of `dht_read()` steps into functions for better readability and the ability to share certain functions among different implementations
- Sensor presence check in `dht_init()`
- ~~Automatic adjustment to a minimum data hold time~~
- Default input mode changed to open drain (a pull-up resistor should be placed close to the output if necessary but not close to the input)
- AVR support without platform-specific handling by avoiding  ztimer_spin() and using the overflow of an 8-bit variable as a pre-timeout to minimize time-consuming ztimer_now() calls

Regarding the changes in the start signal sequence and the removal of the `_reset()` function:
![nano_dht_read_2](https://github.com/RIOT-OS/RIOT/assets/67432403/95966813-2b5f-4a0f-a388-8ac630526ab2)

~~In the previous implementation, there was an unnecessary spike at the beginning of the signal sequence, corresponding to START_HIGH_TIME. This spike has been removed in the re-implementation, as it is unnecessary. Instead, the MCU now simply pulls the signal low for START_LOW_TIME and then releases the bus, which is sufficient for initiating communication with the DHT sensor.~~ Actually, it is necessary to raise the bus level; otherwise, the _wait_for_level() called immediately after to check the response of the DHT may read the port before the signal level is raised, resulting in a false positive.

Additionally, the previous implementation had an issue where the MCU switched back to output mode and went high immediately after reading the 40 bits of data. However, the DHT sensor was still transmitting 2 or 3 additional bytes of '0' at that point, causing a conflict. This issue has been resolved in the re-implementation:
![nano_dht_read_optimized](https://github.com/RIOT-OS/RIOT/assets/67432403/ff124839-5ec5-4df3-bab7-5348d8160a25)

~~Regarding the optimization for AVR, I have performed measurements of `_wait_for_level()` until timeout (85 loops):~~
~~- on esp8266-esp-12x: 264 µs, which is 3.11 µs per loop~~
~~- on nucleo-f303k8: 319 µs, which is 3.75 µs per loop~~
~~- on arduino-nano: 3608 µs, which is 42.45 µs per loop~~
~~Duration measurements on the Arduino Nano:~~


19737: dist/tools/openocd: start debug-server in background and wait r=benpicco a=fabian18



19746: buildsystem: Always expose CPU_RAM_BASE & SIZE flags r=benpicco a=Teufelchen1

### Contribution description

Hello 🐧 

This moves the definition of `CPU_RAM_BASE/SIZE` from being only available in certain situation to be always available.
Reason for change is to simplify common code in the cpu folder.

In cooperation with `@benpicco` 

### Testing procedure

Passing CI


### Issues/PRs references

First usage will be in the PMP driver. Although there is more code in RIOT that could be refactored to use these defines instead of hacks / hardcoded values.

Co-authored-by: Hugues Larrive <hlarrive@pm.me>
Co-authored-by: Fabian Hüßler <fabian.huessler@ml-pa.com>
Co-authored-by: Teufelchen1 <bennet.blischke@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: drivers Area: Device drivers CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Process: needs backport Integration Process: The PR is required to be backported to a release or feature branch Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants