OpenNI2 Stream setProperty(7) - mirroring not implemented?

[starmaid]

Required Info
Camera Model	D435 specifically but maybe applies to more
Firmware Version	5.15.1
Operating System & Version	Win10
Kernel Version (Linux Only)	NA
Platform	PC
SDK Version	2.54.2
Language	NA
Segment	OpenNI2 Wrapper

Issue Description

I'm trying to use the realsense OpenNI sdk for 3D scanning with other software that support generic OpenNI sensors. I have compiled the Realsense sdk with OpenNI support, and copied the new dlls to wherever they are needed, but I'm running into issues. For example:

Skanect throws this:

INFO: OpenNI2: Device "043422070239" present.
INFO: OpenNI2: Device info: [PS1080] [7463] [1537].
INFO: OpenNI2: Number of devices: 1
INFO: No support for softkinetic, skipping.
INFO: OpenNI2: Opening: 043422070239
INFO: OpenNI2: Number of devices: 1
WARNING: Could not switch to VGA depth mode.
ERROR: OpenNI2: Couldn't prepare depth stream: 	Stream setProperty(7) failed

And ReconstructMe throws this:

2023-11-06T02:03:54 ; Found 1 OpenNI compatible sensors
2023-11-06T02:03:54 ; PS1080, PrimeSense, 043422070239
2023-11-06T02:03:54 ; Error: ..\..\..\..\source\robvis\lib_rgbdsensor\src\openni2_sensor.cpp(398): Throw in function void __cdecl LibRGBDSensor::openni2_sensor::open(class boost::shared_ptr<class google::protobuf::Message>)
Dynamic exception type: class boost::exception_detail::clone_impl<class LibFundament::robvis_error>
std::exception::what: Check '_data->depth.setMirroringEnabled(cfg->depth_stream().mirror_enabled()) == openni::STATUS_OK' failed [3 != 0] Failed to setMirroringEnabled 	Stream setProperty(7) failed

So I took a look at what properties the Rs2Stream supports, and it looks like setProperty() only has cases for:

	ONI_STREAM_PROPERTY_VIDEO_MODE                    = 3, // OniVideoMode*
       ...
	ONI_STREAM_PROPERTY_AUTO_WHITE_BALANCE     = 100, // OniBool
	ONI_STREAM_PROPERTY_AUTO_EXPOSURE               = 101, // OniBool
	ONI_STREAM_PROPERTY_EXPOSURE                         = 102, // int
	ONI_STREAM_PROPERTY_GAIN                                  = 103, // int

Is there a reason stream mirroring (type 7) and others aren't implemented? Is this some hardware limitation? Could it be done in the driver? I'm not an OpenNI expert, but if this makes realsense 2 cameras work with more software, I'd dive into this.

Or, to address the real issue I'm having, does anyone know a free 3D scanning software (not Dot3D free trial) that doesn't crash when these features aren't supported?

[MartyG-RealSense]

Hi @starmaid The RealSense OpenNI2 wrapper has not been updated for 5 years (in November 2018) at the time of writing this. So whilst it will be compatible with the D435 camera model, its features are limited to the ones listed below.

The OpenNI2 wrapper is open-source, so RealSense users are welcome to modify it to add new features. I cannot confirm whether it would be possible to add support for stream mirroring though.

In regard to free 3D scanning tools, the RealSense SDK's OpenCV-based open source rs-kinfu tool may be worth investigating. It progressively builds up a point cloud image through frame fusion and then the scan can be exported as a ply format file for import into 3D modelling tools for conversion into a solid-mesh object.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/opencv/kinfu

D435 is also supported by the commercial 3D scanning software RecFusion for Windows that has a free trial version.

https://www.recfusion.net/products/

[starmaid]

Thanks Marty for the quick response! Guess I didn't look at the commit dates to see when it was last touched.

So, I put in a super-simple case in Rs2StreamProps.cpp to see if it would stop the OpenNI scanner programs from crashing:

case ONI_STREAM_PROPERTY_MIRRORING:
{
	return ONI_STATUS_OK;
	break;
}

And it worked? The programs didn't seem to care that the stream wasn't flipped, just that they got the OK code back. I was able to complete scans with Skanect and ReconstructMe with no issues.

Should I make a PR with just this? Is it worth going deeper?

[MartyG-RealSense]

You are very welcome. It's excellent news that you achieved a solution. Thanks very much for sharing the details!

RealSense users are welcome to submit PRs to provide updates to wrappers so that they can be considered by the RealSense team for merging into the official SDK.

[MartyG-RealSense]

Hi @starmaid As you have achieved a solution, do you require further assistance with this case please? Thanks!

[starmaid]

I'm doing some more research on how to either implement it, or a more graceful way to not implement it. I will likely come back with a PR at some point but this is just a side project.

[MartyG-RealSense]

Thanks very much for the update!