Depth color JET range only shows dark/light blue

[itsderek23]

The color JET range in test.py is only showing dark/light blue.

The range needs to be changed for the JET display to show yellow, red, and green.

/cc @Luxonis-Brandon

[chris-piekarski]

I assume COLORMAP_HOT has a similar issue. I get pretty much just solid red for everything.

[Luxonis-Brandon]

Thanks. So I'm guessing there must be some range setting to calling this conversion that should help. Looking now.

[Luxonis-Brandon]

OK, yes, it's just a range thing. So changing this line here to 4 * 65535 instaed of 65535 makes a fairly appropriate range for the BW1097 (the Pi Compute Module Edition, which has 9cm camera spacing).

Note that I've done some bending/etc. to my board (accidentally) so the calibration isn't as good as it could be. But you can still see this is a bit more of an appropriate range for the 1097. This probably is a terrible actual solution... but it gives a quantification of about how far off the current range is.

• https://photos.app.goo.gl/4DKXoYhs3fVKbCQ16
• https://photos.app.goo.gl/uuwubN95LKsRzZG27

Thoughts?

[chris-piekarski]

Interesting, thanks @Luxonis-Brandon . I had looked at that line a few times but convinced myself it was correct given the uint16 input. If I change that line to 2,3 or 4 * 65535 I do get more colorized depth info. However, it quickly drops to 2fps (but reports it as 46 fps) and fills the console with "data queue full errors". I'll play with that line to see if I can find a quicker way to convert the frame.

[Luxonis-Brandon]

Got it. So we recently added a functionality to make this not happen. The reason it happens is the host gets overwhelmed and can't keep up with the frame output from the Myriad X.

So to solve that we implemented the capability to specify on the host side what the Myriad X should output in terms of FPS. I don't know if this is in Master yet, however. Let me look.

[Luxonis-Brandon]

Not clear to me yet @chris-piekarski . So here's the code I'm looking at:
https://github.com/luxonis/depthai-python-extras/blob/1703b69a736e4932d0dd2872648a4828f41cb3cd/test.py

# Make sure to put 'left' always first. Workaround for an issue to be investigated
configs = {
    'streams': [
        'meta_d2h',
        {'name': 'left', "max_fps": 5.0},
        {'name': 'right', "max_fps": 5.0},
        {'name': 'depth_sipp', "max_fps": 10.0},
        'metaout',
        {'name': 'previewout', "max_fps": 10.0},
        # 'depth_color_h',
    ],

[Luxonis-Brandon]

And here's the commit:
1703b69

Need to run now but will look more into this soon.

[Luxonis-Brandon]

Ah, yes, @GergelySzabolcs pointed out that this is in PR #38 #38

(Sorry I'm still a Git n00b, working to get better!)

[chris-piekarski]

Nice. I will try that PR now.

[itsderek23]

@chris-piekarski - test.py is hanging for me with the metaout/previewout streams, so I wouldn't be surprised if you run into issues. We're working those and will likely loop back here.

[chris-piekarski]

could only get the previewout stream to work for me on that PR @itsderek23. The depth stream options never show up and throw a != null assertion exception when I force close it. Same issue if I go back to the commit Brandon called out. I'll just wait until you say its stable :-)

[itsderek23]

Ah sorry Chris ... we'll loop back as we get #38 sorted out.

…

On Tue, Mar 3, 2020 at 3:07 PM Christopher Piekarski < ***@***.***> wrote: could only get the previewout stream to work for me on that PR @itsderek23 <https://github.com/itsderek23>. The depth stream options never show up and throw a != null assertion exception when I force close it. Same issue if I go back to the commit Brandon called out. I'll just wait until you say its stable :-) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#39>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAB5SCR7WEQK2E5WJVJ3JLRFV5Q7ANCNFSM4LAL527A> .

[chris-piekarski]

So the depth stream not working turned out to be a loose flex cable on the right camera @itsderek23. I have had it slightly pop up a few times now. Oh and the 4 * 6553 change is working good now at 10 fps on PR 41. Thanks!

[itsderek23]

So the depth stream not working turned out to be a loose flex cable on the right camera @itsderek23.

Ah ... I ran into this one with a bad connection as well. Remind me again what model you are using @chris-piekarski?

PS - Added an issue to create a troubleshooting entry for this.

[Luxonis-Brandon]

Yes, thanks @chris-piekarski - that's the main hardware issue we're seeing so far. We're wondering if we need to stake the camera connection down on future units. What happened to us is we would accidentally grab the board by that location, which would push the flex down and pop out the connector. I caught myself doing it once.

And @itsderek23 I happen to know, @chris-piekarski has the BW1097 (RPi Compute Module Edition).

[itsderek23]

And @itsderek23 I happen to know, @chris-piekarski has the BW1097 (RPi Compute Module Edition).

Great. Noted in luxonis/depthai-docs-website#29 (comment).

[Luxonis-Brandon]

So as an update on this, using JET on disparity (instead of actual depth data) produces an better visualization.

So this can now be done using python3 test.py -s depth_color_h, which then uses the host to colorize the disparity.

The why of doing it this was is if the Myriad X colorizes the depth (it can, easily!) the data is sent as RGB so 3 bytes (24 bits) per pixel, so 2.7MB/frame or 82.9MB/s for 30FPS - which is QUITE a lot for say a Pi to visualize and is, more importantly, too much data for USB2 to handle (typically USB2 maxes out at ~30MB/s).

So with this depth_color_h option, disparity is sent directly, which is just a single byte, so 0.9MB/frame or 27MB/s for 30FPS, which is a LOT more manageable to say a Raspberry Pi to deal with.

Since the Pi still does struggle even w/ 27MB/s over USB, we added the capability to limit the framerate, which helps to prevent the Pi from getting overwhelmed. Using 10FPS usually helps:

python3 test.py -s depth_color_h,10

And see more details here: https://docs.luxonis.com/faq/#how-do-i-display-multiple-streams