OpenGLDriver::updateDescriptors crash

[dcteris]

Describe the bug
We first create a rendertarget view, and use mouse to pick something by calling View::pick, and works great under 1.54.5
After update to 1.56.0, the render works well, the program crash when we click mouse for picking.
We build release mode, it deos not crash, but we can not pick anything.

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Logs
Editor1.exe!utils::abort() Line 30 C++
Editor1.exe!utils::panic(const char * func, const char * file, int line, const char * assertion) Line 35 C++
Editor1.exe!filament::backend::GLDescriptorSet::validate(filament::backend::HandleAllocator<32,96,136> & allocator, filament::backend::Handlefilament::backend::HwDescriptorSetLayout pipelineLayout) Line 360 C++

Editor1.exe!filament::backend::OpenGLDriver::updateDescriptors::__l2::<lambda_1>::operator()(unsigned __int64 set) Line 3936 C++
Editor1.exe!utils::bitset<unsigned char,1,void>::forEachSetBit<filament::backend::OpenGLDriver::updateDescriptors'::2'::<lambda_1>>(filament::backend::OpenGLDriver::updateDescriptors::__l2::<lambda_1> exec) Line 95 C++
Editor1.exe!filament::backend::OpenGLDriver::updateDescriptors(utils::bitset<unsigned char,1,void> invalidDescriptorSets) Line 3924 C++
Editor1.exe!filament::backend::OpenGLDriver::draw2(unsigned int indexOffset, unsigned int indexCount, unsigned int instanceCount) Line 3963 C++
Editor1.exe!filament::backend::invoke<filament::backend::OpenGLDriver,void __cdecl(unsigned int,unsigned int,unsigned int),filament::backend::OpenGLDriver &,unsigned int,unsigned int,unsigned int>(void(filament::backend::OpenGLDriver::)(unsigned int, unsigned int, unsigned int) m, filament::backend::OpenGLDriver & d, unsigned int && <args_0>, unsigned int && <args_1>, unsigned int && <args_2>) Line 94 C++
Editor1.exe!filament::backend::trampoline<void (__cdecl filament::backend::OpenGLDriver::)(unsigned int,unsigned int,unsigned int),filament::backend::OpenGLDriver &,std::tuple<unsigned int,unsigned int,unsigned int>,0,1,2>(void(filament::backend::OpenGLDriver::)(unsigned int, unsigned int, unsigned int) && m, filament::backend::OpenGLDriver & d, std::tuple<unsigned int,unsigned int,unsigned int> && t, std::integer_sequence<unsigned __int64,0,1,2> __formal) Line 99 C++
Editor1.exe!filament::backend::apply<void (__cdecl filament::backend::OpenGLDriver::)(unsigned int,unsigned int,unsigned int),filament::backend::OpenGLDriver &,std::tuple<unsigned int,unsigned int,unsigned int>>(void(filament::backend::OpenGLDriver::)(unsigned int, unsigned int, unsigned int) && m, filament::backend::OpenGLDriver & d, std::tuple<unsigned int,unsigned int,unsigned int> && t) Line 104 C++
Editor1.exe!filament::backend::CommandType<void (__cdecl filament::backend::Driver::)(unsigned int,unsigned int,unsigned int)>::Command<&filament::backend::Driver::draw2>::execute<void (__cdecl filament::backend::OpenGLDriver::)(unsigned int,unsigned int,unsigned int),filament::backend::OpenGLDriver &>(void(filament::backend::OpenGLDriver::)(unsigned int, unsigned int, unsigned int) && method, filament::backend::OpenGLDriver & driver, filament::backend::CommandBase * base, __int64 * next) Line 144 C++
Editor1.exe!filament::backend::ConcreteDispatcherfilament::backend::OpenGLDriver::draw2(filament::backend::Driver & driver, filament::backend::CommandBase * base, __int64 * next) Line 525 C++
Editor1.exe!filament::backend::CommandBase::execute(filament::backend::Driver & driver) Line 79 C++
Editor1.exe!filament::backend::CommandStream::execute::__l2::<lambda_1>::operator()() Line 99 C++
[External Code]
Editor1.exe!filament::backend::Driver::execute(const std::function<void __cdecl(void)> & fn) Line 218 C++
Editor1.exe!filament::backend::CommandStream::execute(void * buffer) Line 95 C++
Editor1.exe!filament::FEngine::execute() Line 1271 C++
Editor1.exe!filament::Engine::execute() Line 374 C++

Desktop (please complete the following information):

• OS: Windows
• GPU:AMD 6700xt
• Backend: OpenGL

Smartphone (please complete the following information):

• Device: [e.g. Pixel 2]
• OS: [e.g. Android Pie 9.0]

Additional context
Add any other context about the problem here.

[show50726]

Do you enable transparent picking?

[dcteris]

Thanks, by turn off transparent picking, it didn't crash any more, and can pick something.
By the way, transparent picking is enabled by default.
So, transparent picking is not completed yet ?

[show50726]

I thought it was disabled by default...it seems I accidentally modified the default value.
Let me open a PR to disable it by default and investigate why it crashes...

[show50726]

@dcteris Could you share more details about the setup/settings of your scene?

[dcteris]

This is win32 app with imgui, code from filagui and filamentapp, not use SDL.

Create a rendertarget
m_colorTx = filament::Texture::Builder()
.width(m_size.Width).height(m_size.Height).levels(1)
.usage(filament::Texture::Usage::COLOR_ATTACHMENT | filament::Texture::Usage::SAMPLEABLE)
.format(filament::Texture::InternalFormat::RGBA8).build(*engine);
m_depthTx = filament::Texture::Builder()
.width(m_size.Width).height(m_size.Height).levels(1)
.usage(filament::Texture::Usage::DEPTH_ATTACHMENT)
.format(filament::Texture::InternalFormat::DEPTH32F).build(*engine);
m_renderTarget = filament::RenderTarget::Builder()
.texture(filament::RenderTarget::AttachmentPoint::COLOR, m_colorTx)
.texture(filament::RenderTarget::AttachmentPoint::DEPTH, m_depthTx)
.build(*engine);

m_view->setRenderTarget(m_renderTarget);

Then use imgui draw this rendertarget
drawList->AddImage(_TextureID(m_colorTx), canvasPos, canvasPos + ImVec2((float)txSize.Width, (float)txSize.Height), uvMin, uvMax);

I found even if there is only a skybox, when I hit mouse, i got an assert too (transparent picking)

[show50726]

Do you enable transparent picking?

[dcteris]

Thanks, by turn off transparent picking, it didn't crash any more, and can pick something.
By the way, transparent picking is enabled by default.
So, transparent picking is not completed yet ?

[show50726]

I thought it was disabled by default...it seems I accidentally modified the default value.
Let me open a PR to disable it by default and investigate why it crashes...

[show50726]

@dcteris Could you share more details about the setup/settings of your scene?

[dcteris]

This is win32 app with imgui, code from filagui and filamentapp, not use SDL.

Create a rendertarget
m_colorTx = filament::Texture::Builder()
.width(m_size.Width).height(m_size.Height).levels(1)
.usage(filament::Texture::Usage::COLOR_ATTACHMENT | filament::Texture::Usage::SAMPLEABLE)
.format(filament::Texture::InternalFormat::RGBA8).build(*engine);
m_depthTx = filament::Texture::Builder()
.width(m_size.Width).height(m_size.Height).levels(1)
.usage(filament::Texture::Usage::DEPTH_ATTACHMENT)
.format(filament::Texture::InternalFormat::DEPTH32F).build(*engine);
m_renderTarget = filament::RenderTarget::Builder()
.texture(filament::RenderTarget::AttachmentPoint::COLOR, m_colorTx)
.texture(filament::RenderTarget::AttachmentPoint::DEPTH, m_depthTx)
.build(*engine);

m_view->setRenderTarget(m_renderTarget);

Then use imgui draw this rendertarget
drawList->AddImage(_TextureID(m_colorTx), canvasPos, canvasPos + ImVec2((float)txSize.Width, (float)txSize.Height), uvMin, uvMax);

I found even if there is only a skybox, when I hit mouse, i got an assert too (transparent picking)