CommandBufferGL::drawElements Crash

[kiranb47]

• axmol version: Latest dev branch
• devices test on:
• developing environments
  • NDK version: r19c
  • Xcode version: 12.4
  • Visual Studio:
    • VS version: 2019 (16.11), 2022 (17.4)
    • MSVC version: 1929, 1934
    • Windows SDK version: 10.0.22621.0
  • cmake version:
    Steps to Reproduce:

We have migrated our Match Animal game to Axmol from Cocos2dx V3 latest yesterday. Getting this crash now. How to fix this issue?

backtrace:
#00 pc 0x00000000005ffa58 /data/app/games.spearmint.matchanimal-ayOwq5oTYpeKYIRIPMWVUQ==/lib/arm64/libMatchAnimal.so (ax::Renderer::TriangleCommandBufferManager::createBuffer()+1681)
#1 pc 0x00000000005fdbc0 /data/app/games.spearmint.matchanimal-ayOwq5oTYpeKYIRIPMWVUQ==/lib/arm64/libMatchAnimal.so (ax::Renderer::init()+964)
#2 pc 0x000000000063b060 /data/app/games.spearmint.matchanimal-ayOwq5oTYpeKYIRIPMWVUQ==/lib/arm64/libMatchAnimal.so (ax::Director::setOpenGLView(ax::GLView*)+394)
#3 pc 0x00000000005640cc /data/app/games.spearmint.matchanimal-ayOwq5oTYpeKYIRIPMWVUQ==/lib/arm64/libMatchAnimal.so (Java_org_axmol_lib_AxmolRenderer_nativeInit+99)
#4 pc 0x00000000001c5ff0 /data/app/games.spearmint.matchanimal-ayOwq5oTYpeKYIRIPMWVUQ==/oat/arm64/base.odex

[rh101]

How to fix this issue?

At risk of stating the obvious, have you actually attempted to use a debugger or other methods of tracking down why it crashes? A breakpoint in the right place may be the only thing you need to find out the issue. You've provided no source code, no context to the crash, whether it's on a real device or emulator, or what the app was doing at the time.

It's something in your own code causing this issue, and unlikely given to be something in the game engine, and given that you ported it from what I can only assume is a working version of the app, then something like this must be trivial for you to fix.

[kiranb47]

Hey @rh101 The issue is not reproducible in any of our test devices. Crash logs are from Google Play live users. 3.6% of our GPLAY users have the issue.

[kiranb47]

Another similar crash:

[split_config.armeabi_v7a.apk!libMatchAnimal.so] CommandBufferGL.cpp - ax::backend::CommandBufferGL::drawArrays(ax::backend::PrimitiveType, unsigned int, unsigned int, bool)
SIGSEGV
filter_list

date_range
Last 28 days
arrow_drop_down
Affected users
9
Last 28 days
Events
20
Last 28 days
Last occurred
3 hours ago
Last updated
Today, 10:30 AM
Affected users
Apr 27, 2023 - May 25, 2023
Affected users
Events
By app version
10205 (1.2.5)
By Android version
Android 11 (SDK 30)
Android 12 (SDK 31)
By device
samsung a10s
samsung a02
Redmi ice
TCL Cruze_Lite
By issue visibility
help_outline
Foreground
Stack trace
help_outline
1
Sample attributes:
samsung a10s (Galaxy A10s)

Android 11 (SDK 30)

Version: 10205 (1.2.5)Occurred: 3 hours ago
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 10195 >>> games.spearmint.matchanimal <<<

backtrace:
  #00  pc 0x00000000000701ae  /vendor/lib/egl/libGLESv2_mtk.so
  #01  pc 0x0000000000089ca1  /vendor/lib/egl/libGLESv2_mtk.so
  #02  pc 0x00000000000888d9  /vendor/lib/egl/libGLESv2_mtk.so (glDrawArrays+2592)
  #03  pc 0x000000000002d549  /vendor/lib/egl/libGLES_meow.so (MEOW::meow_call_ddk_gl_2_glDrawArrays(unsigned int, int, int)+20)
  #04  pc 0x00000000004255d7  /data/app/~~ZtkX28x-dzXdBYditxORXg==/games.spearmint.matchanimal-6vE2kmMTB0DCfes9iuUpeg==/split_config.armeabi_v7a.apk!libMatchAnimal.so (ax::backend::CommandBufferGL::drawArrays(ax::backend::PrimitiveType, unsigned int, unsigned int, bool)+222)
  #05  pc 0x000000000040feef  /data/app/~~ZtkX28x-dzXdBYditxORXg==/games.spearmint.matchanimal-6vE2kmMTB0DCfes9iuUpeg==/split_config.armeabi_v7a.apk!libMatchAnimal.so (ax::Renderer::drawCustomCommand(ax::RenderCommand*)+758)
  #06  pc 0x000000000040ff3d  /data/app/~~ZtkX28x-dzXdBYditxORXg==/games.spearmint.matchanimal-6vE2kmMTB0DCfes9iuUpeg==/split_config.armeabi_v7a.apk!libMatchAnimal.so (ax::Renderer::doVisitRenderQueue(std::__ndk1::vector<ax::RenderCommand*, std::__ndk1::allocator<ax::RenderCommand*>> const&)+398)
  #07  pc 0x000000000040f7d5  /data/app/~~ZtkX28x-dzXdBYditxORXg==/games.spearmint.matchanimal-6vE2kmMTB0DCfes9iuUpeg==/split_config.armeabi_v7a.apk!libMatchAnimal.so (ax::Renderer::visitRenderQueue(ax::RenderQueue&)+386)
  #08  pc 0x000000000040f539  /data/app/~~ZtkX28x-dzXdBYditxORXg==/games.spearmint.matchanimal-6vE2kmMTB0DCfes9iuUpeg==/split_config.armeabi_v7a.apk!libMatchAnimal.so (ax::Renderer::processGroupCommand(ax::GroupCommand*)+292)
  #09  pc 0x000000000040f8e3  /data/app/~~ZtkX28x-dzXdBYditxORXg==/games.spearmint.matchanimal-6vE2kmMTB0DCfes9iuUpeg==/split_config.armeabi_v7a.apk!libMatchAnimal.so (ax::Renderer::processRenderCommand(ax::RenderCommand*)+342)
  #10  pc 0x000000000040ff3d  /data/app/~~ZtkX28x-dzXdBYditxORXg==/games.spearmint.matchanimal-6vE2kmMTB0DCfes9iuUpeg==/split_config.armeabi_v7a.apk!libMatchAnimal.so (ax::Renderer::doVisitRenderQueue(std::__ndk1::vector<ax::RenderCommand*, std::__ndk1::allocator<ax::RenderCommand*>> const&)+398)
  #11  pc 0x000000000040f7d5  /data/app/~~ZtkX28x-dzXdBYditxORXg==/games.spearmint.matchanimal-6vE2kmMTB0DCfes9iuUpeg==/split_config.armeabi_v7a.apk!libMatchAnimal.so (ax::Renderer::visitRenderQueue(ax::RenderQueue&)+386)
  #12  pc 0x000000000040ff8b  /data/app/~~ZtkX28x-dzXdBYditxORXg==/games.spearmint.matchanimal-6vE2kmMTB0DCfes9iuUpeg==/split_config.armeabi_v7a.apk!libMatchAnimal.so (ax::Renderer::render()+415)
  #13  pc 0x00000000003d64b1  /data/app/~~ZtkX28x-dzXdBYditxORXg==/games.spearmint.matchanimal-6vE2kmMTB0DCfes9iuUpeg==/split_config.armeabi_v7a.apk!libMatchAnimal.so (ax::Scene::render(ax::Renderer*, ax::Mat4 const&, ax::Mat4 const*)+225)
  #14  pc 0x000000000043953d  /data/app/~~ZtkX28x-dzXdBYditxORXg==/games.spearmint.matchanimal-6vE2kmMTB0DCfes9iuUpeg==/split_config.armeabi_v7a.apk!libMatchAnimal.so (ax::Director::drawScene()+295)
  #15  pc 0x000000000043af85  /data/app/~~ZtkX28x-dzXdBYditxORXg==/games.spearmint.matchanimal-6vE2kmMTB0DCfes9iuUpeg==/split_config.armeabi_v7a.apk!libMatchAnimal.so (ax::Director::mainLoop()+1485)
  #16  pc 0x00000000000ad5d7  /data/app/~~ZtkX28x-dzXdBYditxORXg==/games.spearmint.matchanimal-6vE2kmMTB0DCfes9iuUpeg==/oat/arm/base.odex (art_jni_trampolin

[rh101]

Crash logs are from Google Play live users. 3.6% of our GPLAY users have the issue.

Is it limited to specific devices or Android versions, and if so, what devices and Android versions are having this issue?

I notice the armeabi_v7a in that dump. Is the crash limited to 32bit devices running armeabi_v7a?

Also, please put code tags around the on the crash dumps so they're formatted and easier to read. Just add 3 backtick ` symbols before and after the crash dumps.

[kiranb47]

@rh101 Issue also in arm64_v8 devices.

[split_config.arm64_v8a.apk!libMatchAnimal.so] CommandBufferGL.cpp - ax::backend::CommandBufferGL::drawElements(ax::backend::PrimitiveType, ax::backend::IndexFormat, unsigned long, unsigned long, bool)

backtrace:
  #00  pc 0x0000000000088b78  /apex/com.android.runtime/lib64/bionic/libc.so (__memcpy+232)
  #01  pc 0x00000000001eb86c  /vendor/lib64/egl/libGLESv2_adreno.so (!!!0000!6642a709aefb598e3c7ff817eaf381!03dd3ba!+3524)
  #02  pc 0x0000000000360b08  /vendor/lib64/egl/libGLESv2_adreno.so (!!!0000!928ddd272828b5ecd302b519177d41!03dd3ba!+904)
  #03  pc 0x0000000000332b54  /vendor/lib64/egl/libGLESv2_adreno.so (!!!0000!4c38aea95be9faf0b3861d1af73f50!03dd3ba!+2916)
  #04  pc 0x0000000000320078  /vendor/lib64/egl/libGLESv2_adreno.so (!!!0000!591ab8a9d75351b2e63b06236cc5c1!03dd3ba!+16)
  #05  pc 0x0000000000133bfc  /vendor/lib64/egl/libGLESv2_adreno.so (!!!0000!6b200851123c7898055fe62ff9f71f!03dd3ba!+1876)
  #06  pc 0x000000000012be84  /vendor/lib64/egl/libGLESv2_adreno.so (!!!0000!77df12deb6a622478efa8fb9929034!03dd3ba!+1004)
  #07  pc 0x000000000061f8a8  /data/app/~~jlm2IhawQGELdTRWJwWbqw==/games.spearmint.matchanimal-oUvgtVnie2xt30tW0BFwmg==/split_config.arm64_v8a.apk!libMatchAnimal.so (ax::backend::CommandBufferGL::drawElements(ax::backend::PrimitiveType, ax::backend::IndexFormat, unsigned long, unsigned long, bool)+242)
  #08  pc 0x00000000005fee60  /data/app/~~jlm2IhawQGELdTRWJwWbqw==/games.spearmint.matchanimal-oUvgtVnie2xt30tW0BFwmg==/split_config.arm64_v8a.apk!libMatchAnimal.so (ax::Renderer::drawCustomCommand(ax::RenderCommand*)+752)
  #09  pc 0x00000000005fef04  /data/app/~~jlm2IhawQGELdTRWJwWbqw==/games.spearmint.matchanimal-oUvgtVnie2xt30tW0BFwmg==/split_config.arm64_v8a.apk!libMatchAnimal.so (ax::Renderer::doVisitRenderQueue(std::__ndk1::vector<ax::RenderCommand*, std::__ndk1::allocator<ax::RenderCommand*>> const&)+398)
  #10  pc 0x00000000005fe4b0  /data/app/~~jlm2IhawQGELdTRWJwWbqw==/games.spearmint.matchanimal-oUvgtVnie2xt30tW0BFwmg==/split_config.arm64_v8a.apk!libMatchAnimal.so (ax::Renderer::visitRenderQueue(ax::RenderQueue&)+386)
  #11  pc 0x00000000005fe27c  /data/app/~~jlm2IhawQGELdTRWJwWbqw==/games.spearmint.matchanimal-oUvgtVnie2xt30tW0BFwmg==/split_config.arm64_v8a.apk!libMatchAnimal.so (ax::Renderer::processGroupCommand(ax::GroupCommand*)+292)
  #12  pc 0x00000000005fe644  /data/app/~~jlm2IhawQGELdTRWJwWbqw==/games.spearmint.matchanimal-oUvgtVnie2xt30tW0BFwmg==/split_config.arm64_v8a.apk!libMatchAnimal.so (ax::Renderer::processRenderCommand(ax::RenderCommand*)+342)
  #13  pc 0x00000000005fef04  /data/app/~~jlm2IhawQGELdTRWJwWbqw==/games.spearmint.matchanimal-oUvgtVnie2xt30tW0BFwmg==/split_config.arm64_v8a.apk!libMatchAnimal.so (ax::Renderer::doVisitRenderQueue(std::__ndk1::vector<ax::RenderCommand*, std::__ndk1::allocator<ax::RenderCommand*>> const&)+398)
  #14  pc 0x00000000005fe4b0  /data/app/~~jlm2IhawQGELdTRWJwWbqw==/games.spearmint.matchanimal-oUvgtVnie2xt30tW0BFwmg==/split_config.arm64_v8a.apk!libMatchAnimal.so (ax::Renderer::visitRenderQueue(ax::RenderQueue&)+386)
  #15  pc 0x00000000005fef74  /data/app/~~jlm2IhawQGELdTRWJwWbqw==/games.spearmint.matchanimal-oUvgtVnie2xt30tW0BFwmg==/split_config.arm64_v8a.apk!libMatchAnimal.so (ax::Renderer::render()+415)
  #16  pc 0x00000000005b17b8  /data/app/~~jlm2IhawQGELdTRWJwWbqw==/games.spearmint.matchanimal-oUvgtVnie2xt30tW0BFwmg==/split_config.arm64_v8a.apk!libMatchAnimal.so (ax::Scene::render(ax::Renderer*, ax::Mat4 const&, ax::Mat4 const*)+225)
  #17  pc 0x000000000063a760  /data/app/~~jlm2IhawQGELdTRWJwWbqw==/games.spearmint.matchanimal-oUvgtVnie2xt30tW0BFwmg==/split_config.arm64_v8a.apk!libMatchAnimal.so (ax::Director::drawScene()+295)
  #18  pc 0x000000000063ca68  /data/app/~~jlm2IhawQGELdTRWJwWbqw==/games.spearmint.matchanimal-oUvgtVnie2xt30tW0BFwmg==/split_config.arm64_v8a.apk!libMatchAnimal.so (ax::Director::mainLoop()+1485)
  #19  pc 0x00000000000ada90  /data/app/~~jlm2IhawQGELdTRWJwWbqw==/games.spearmint.matchanimal-oUvgtVnie2xt30tW0BFwmg==/oat/arm64/base.odex (art_jni_trampoline+144)
  #20  pc 0x0000000002018c7c  /memfd:jit-cache (org.axmol.lib.AxmolRenderer.onDrawFrame+220)
  #21  pc 0x000000000069f968  /system/framework/arm64/boot-framework.oat (android.opengl.GLSurfaceView$GLThread.guardedRun+3992)
  #22  pc 0x00000000006a01b0  /system/framework/arm64/boot-framework.oat (android.opengl.GLSurfaceView$GLThread.run+224)
  #23  pc 0x0000000000134564  /apex/com.android.art/lib64/libart.so (art_quick_invoke_stub+548)
  #24  pc 0x0000000000198e94  /apex/com.android.art/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+204)
  #25  pc 0x0000000000532198  /apex/com.android.art/lib64/libart.so (art::(anonymous namespace)::InvokeWithArgArray(art::ScopedObjectAccessAlreadyRunnable const&, art::ArtMethod*, art::(anonymous namespace)::ArgArray*, art::JValue*, char const*)+104)
  #26  pc 0x0000000000533398  /apex/com.android.art/lib64/libart.so (art::JValue art::InvokeVirtualOrInterfaceWithJValues<art::ArtMethod*>(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, art::ArtMethod*, jvalue const*)+440)
  #27  pc 0x00000000005808b8  /apex/com.android.art/lib64/libart.so (art::Thread::CreateCallback(void*)+1272)
  #28  pc 0x00000000000f40c4  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+64)
  #29  pc 0x000000000008ed10  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64)

[rh101]

@kiranb47 Crashes that affect a small subset of users may have very specific triggers, and since you are the one with access to the source code, you're in the best position to narrow down the possible causes. Do you collect any form of diagnostic information from your application? Can you tell what a user is doing in the app just prior to the crash?

[solan-solan]

According to provided logs, all your issues are related to gl buffers. The falt occurred inside gldrawelements, gldrawarrays and while buffer creating. For example, axmol supports int as base type for index array as opposed to cocos. I would check how your buffers creating in your code at first, since the issues looks logical. You created them manually?

[kiranb47]

Hey @rh101 Firebase is not reporting this issue, so not able to identify pre crash events logs. Hey @solan-solan I have no technical knowledge about OpenGL. We have not applied any changes to axmol source code for this game.

[kiranb47]

@rh101 @solan-solan Issue is related to UI ScrollView

[rh101]

It's great that you have managed to narrow down the issue. Can you show the snippet of code related to how you use the ui::ScrollView? Include how you set it up/initialize it, and any other related code. If you could create a small project with just the ui::ScrollView usage then that would help.

[crazyhappygame]

FYI. I migrated my app from cocos2d-x 3.17 to latest axmol (4e664e6) and see in Google Play similar crashes in drawElements and drawArrays

ax::backend::CommandBufferGL::drawElements(ax::backend::PrimitiveType, ax::backend::IndexFormat, unsigned long, unsigned long, bool)

and

ax::backend::CommandBufferGL::drawArrays(ax::backend::PrimitiveType, unsigned long, unsigned long, b

In my game I use ui::ScrollView as well. I used many (100-1000) Sprite and Scale9Sprite.
@kiranb47 how did you narrow problem to ui::ScrollView ?

App with cocos2d-x 3.17 has ~0 crashes after upgrading I see significant increase. I can not reproduce issue on local devices ...

Please check below screenshot how much crashes increased in Google Play after update:

[DelinWorks]

is your project open source? I can debug and potentially fix this for you if so. otherwise why not compile a small project that showcases the issue like @rh101 said?

[crazyhappygame]

@DelinWorks Thank you for an offer.

The problem is that I can not reproduce problem locally ... I see crashes only in Google Play. I am so sorry my project is not open source.
Based on above callstacks it looks like we crash somewhere here

void CommandBufferGL::drawElements(PrimitiveType primitiveType,
                                   IndexFormat indexType,
                                   std::size_t count,
                                   std::size_t offset,
                                   bool wireframe)
{
    prepareDrawing();
#ifndef AX_USE_GLES  // glPolygonMode is only supported in Desktop OpenGL
    if (wireframe) glPolygonMode(GL_FRONT_AND_BACK, GL_LINE);
#else
    if (wireframe) primitiveType = PrimitiveType::LINE;
#endif
    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, _indexBuffer->getHandler());
    glDrawElements(UtilsGL::toGLPrimitiveType(primitiveType), count, UtilsGL::toGLIndexType(indexType),
                   (GLvoid*)offset);

That could mean that _indexBuffer is incorrect or parameters to glDrawElements are incorrect.

Unfortunately I am not familiar with GL. Do you think that it would be possible having above callstack analyze existing code and narrow problem through code inspection?
@DelinWorks

[DelinWorks]

You were able to produce it on your device or is that a google analysis ghost crash