Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Miscellaneous camera controls, IMX582/IMX586 on-sensor HDR #972

Merged
merged 14 commits into from
Sep 23, 2024

Conversation

alex-luxonis
Copy link
Contributor

@alex-luxonis alex-luxonis commented Feb 28, 2024

New misc controls, these should be set on ColorCamera with .initialControl, and only available for IMX378 / IMX477 / IMX577 at THE_1080_P resolution:

.initialControl.setMisc("downsampling-mode", "binning")  # default: "scaling"

-> max FPS for THE_1080_P also back to 118 when binning is enabled (only up to 60 possible with sensor scaling).

For IMX582(LCM48)/IMX586 it's possible to set the binning mode for resolutions of 12MP or lower (implemented with quad-bayer binning)

.initialControl.setMisc("binning-mode", "sum")  # default: "avg" (2x2 pixel average)

-> this results in a brighter image at the same exposure (but slightly more noise possible)


HDR for IMX582 (up to 10fps currently) and IMX586, enable by initialControl:

.initialControl.setMisc("hdr-exposure-ratio", 4)  # HDR enabled when set `> 1`, options: 2, 4, 8

then can adjust also at runtime:

.setMisc("hdr-exposure-ratio", 4)  # valid options: 1, 2, 4, 8 
.setMisc("hdr-local-tone-weight", 0.75)  # default 0.75, normalized range 0.0 - 1.0

3 HDR exposures are configured on sensor as:

  • long --- the one received from AE or manual
  • middle = long / hdr-exposure-ratio
  • short = long / ( hdr-exposure-ratio * hdr-exposure-ratio )

The HDR effect is more pronounced when local tone weight is higher.


Option to bypass 3A for having manual exposure/ISO take effect faster:

.initialControl.setMisc("manual-exposure-handling", "fast")  # default: "default"
  • implemented now for all sensors.
  • higher manual ISO range for IMX378, up to 3200 (and up to 35000 with .initialControl.setMisc("manual-exposure-handling", "fast"))

  • initial support for new Sony 4K Starvis sensors: IMX678 and IMX715
  • enable HCG (high conversion gain) by default for supported sensors (IMX462 1080p and IMX678), can be set with initialControl or at runtime:
.setMisc("high-conversion-gain", 0)  # 1 to enable (default on supported sensors)`

Option to set the main camera to drive auto-exposure and auto-white-balance in multi-camera configurations, applicable for sensors/modules of the same type. E.g. CAM_A being the main camera:

cam['cama'].initialControl.setMisc("3a-follow", dai.CameraBoardSocket.CAM_A)
cam['camb'].initialControl.setMisc("3a-follow", dai.CameraBoardSocket.CAM_A)
cam['camd'].initialControl.setMisc("3a-follow", dai.CameraBoardSocket.CAM_A)

.initialControl.setMisc("downsampling-mode", "binning")  # default: "scaling"
.initialControl.setMisc("binning-mode", "sum")  # default: "avg"

Max FPS for IMX378 1080p back to 118 (from 60) with binning enabled
…misc control:

.initialControl.setMisc("manual-exposure-handling", "fast")
.initialControl.setMisc("hdr-exposure-ratio", 4) # enables HDR when set `> 1`, current options: 2, 4, 8
.initialControl.setMisc("hdr-local-tone-weight", 75) # default 75, range 0..100
…bled by default. Values: 0, 1, can also change during streaming
- better config for 10fps HDR
- runtime configurable hdr-exposure-ratio (1,2,4,8), hdr-local-tone-weight (0..1, float normalized)
moratom pushed a commit that referenced this pull request Apr 3, 2024
* Update install_dependencies.sh

Fixed installing on ARM and raspbian

Tested on:
    - RPI - armv7l - Raspbian 10 (buster)
    - nvidia jetson orin nano - aarch64 - ("Ubuntu 20.04.6 LTS")
    - RPI 64 bit OS (aarch64 - luxonis/based on Bookworm - 12)
    - x86 host ("Ubuntu 21.04")

* Update install_dependencies.sh

Added ubuntu 23.10 support
```
cam["cama"].initialControl.setMisc("3a-follow", dai.CameraBoardSocket.CAM_A)
cam["camb"].initialControl.setMisc("3a-follow", dai.CameraBoardSocket.CAM_B)
```

generic fast manual exposure (for all sensors)
@alex-luxonis alex-luxonis requested a review from moratom August 26, 2024 12:08
@alex-luxonis alex-luxonis merged commit 9b2725f into develop Sep 23, 2024
28 of 29 checks passed
@alex-luxonis alex-luxonis deleted the camera_controls_misc branch September 23, 2024 12:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants