Getting OutOfMemoryError: pthread_create on Android 4, 6, 7

[Sdghasemi]

First of all thanks for the great library.
As I mentioned I'm getting below error on all Android APIs I'm using except 21 and 22 (Android 5):

Fatal Exception: java.lang.OutOfMemoryError: pthread_create (1040KB stack) failed: Try again
       at java.lang.Thread.nativeCreate(Thread.java)
       at java.lang.Thread.start(Thread.java:1063)
       at org.java_websocket.client.WebSocketClient.run(WebSocketClient.java:280)
       at java.lang.Thread.run(Thread.java:818)

Also there are about 620 identical threads with below stack along with the crash on Crashlytics:

pool-33-thread-8
       at java.lang.Object.wait(Object.java)
       at java.lang.Thread.parkFor$(Thread.java:1220)
       at sun.misc.Unsafe.park(Unsafe.java:299)
       at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
       at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2013)
       at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:410)
       at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1036)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1098)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
       at java.lang.Thread.run(Thread.java:818)

I was hoping if someone can help me with the issue, thanks in advance.

[marci4]

Hello @Sdghasemi,

please use this issue template to help me understand the issue at hand.

Are you using wss?

Normally we are not spawning a lot of new threads, especially on the client side....
I would love to get a example application to help me understand the issue more!

Greetings
marci4

[Sdghasemi]

I've read the issue template, I'll provide you as much as info I can so we can spot where the problem is coming from.
The problem is obvious as you know. There's a function somewhere which creates new threads without using them.
Because the web socket instances are not reusable, I managed to create a retry procedure aside from the library which will retry to connect after timer timed out (delays will increase exponentially based on previous retry counts) until the socket opens. BTW this process stops when application is not focused or when there is no network connectivity and resumes when both of those are on board.
Here is the code for the retry procedure:

    /**
     * Thread executor for web socket retry handler
     */
    private ScheduledFuture<?> mScheduledFuture;

    /**
     * Stores previous retry timer as a power of 2 (2^mRetryTimer) to set next retry timer based on
     */
    private byte mRetryTimer = 0;


    @Override
    public void onConnectionEstablished() {     // websocket onOpen
        mIsNetworkConnected = Config.isOnline();
        mIsConnected = true;
        cancelRetryTimer(true);
        if (isViewAvailable()) {
            if (mIsNetworkConnected) mViewWeakReference.get().onConnectionEstablished();
            else mViewWeakReference.get().onNetworkConnectionEstablished();
        }
    }
    @Override
    public void onConnectionClosed(MessagingRequest request, int code, String reason, boolean remote) {     // websocket onClose
        if (request != mRequest) return;
        mIsConnected = false;
        if (isViewAvailable()) {
            if (mIsNetworkConnected) mViewWeakReference.get().onConnectionClosed();
            else mViewWeakReference.get().onNetworkConnectionLost();
        }
        if (code == NORMAL &&
                (reason != null && (reason.equals(CLOSE_MESSAGE_TOKEN_REFRESH) ||
                        reason.equals(CLOSE_MESSAGE_LOST_FOCUS)))) return;
        if (!Config.isTokenValid()) {
            mRequest.refreshAuthorizationToken();
            return;
        }

        if (mScheduledFuture == null) {
            establishConnection(true);      // try to reconnect
        }
    }
    private void setupRetryTimer() {
        long retryDelay = (long) (Math.pow(2, mRetryTimer));
        if (mRetryTimer < 3) mRetryTimer++;
        mScheduledFuture = Executors.newSingleThreadScheduledExecutor().schedule(new Runnable() {
            @Override
            public void run() {
                try {
                    if (mRequest != null) {
                        if (mRequest.isOpen()) {
                            cancelRetryTimer(false);
                        } else {
                            mRequest.close();
                            establishConnection(true);
                        }
                    }
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        }, retryDelay, TimeUnit.SECONDS);
        Logger.log(WEB_SOCKET, "Retrying in " + retryDelay + " sec.");
    }
    private void cancelRetryTimer(boolean interrupt) {
        Logger.log(WEB_SOCKET, "Canceling retry timer...");
        if (mScheduledFuture != null) {
            mScheduledFuture.cancel(interrupt);
            mScheduledFuture = null;
        }
        resetRetryTimer();
    }
    private void resetRetryTimer() {
        mRetryTimer = 0;
    }
    @Override
    public void establishConnection(boolean forceReconnect) {
        try {
            if (!(mIsAppVisible = MainApplication.isAppVisible())) {
                cancelRetryTimer(true);
                return;
            }
            if ((mRequest != null && !mRequest.isInConnectionPhase()) || forceReconnect) {
                mRequest = new MessagingRequest(this, mRequest);
                setupRetryTimer();
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

As you see, the instances disposed properly in cancelRetryTimer(). Is there somewhere in the library which creates threads without disposal?

[marci4]

Hello @Sdghasemi,

you should not read it but use it instead!
This helps me to understand the environment and everything I need to reproduce this issue.

For example when you started this issue, you did not mention that you are somehow reconnecting the websocket, Without such information I would just have closed this issue with "cannot-reproduce".

For me it is not clear what you are doing so such a template is important!!
Especially on android a simple demonstration app would also help me to debug and reproduce it.
The more details it contains the easier it is for me to reproduce and fix it (if there even exists a problem).

Greetings
marci4

[Sdghasemi]

Since this I'm using web socket as a part of a huge codebase, I'm afraid I can not make a simple sample representing the problem to reproduce (I myself couldn't reproduce it either, but there are a number of crashes tell me this is happening). I'm trying to find out where this crashes might get produced, or maybe there is no way the library could cause it!
Knowing that I use the issue template as you said if there's any help:

Expected Behavior

There's a function somewhere creating pool threads repeatedly which shouldn't without proper disposal.

Current Behavior

The threads memory runs out and a crash happens with this stack trace:

Fatal Exception: java.lang.OutOfMemoryError: pthread_create (1040KB stack) failed: Try again
       at java.lang.Thread.nativeCreate(Thread.java)
       at java.lang.Thread.start(Thread.java:1063)
       at org.java_websocket.client.WebSocketClient.run(WebSocketClient.java:280)
       at java.lang.Thread.run(Thread.java:818)

Possible Solution

Find out where is coming from, maybe a result of creating instances of WebSocketClient to reconnect or something else.

Steps to Reproduce (for bugs)

A link to the problem: http://crashes.to/s/0915cc05563

Debug log (for bugs)

There is no particular debug log, since the bug is not related to the web socket transactions itself.

Context

I'm afraid I have no idea, If I had I could reproduce it somehow.

Your Environment

I have provided everything relevant to the issue in this comment and previous ones.

Thanks for your help.

[marci4]

Hello @Sdghasemi,

thank you again for the details especially for the link.

Am I understanding the site correctly that there are 476 threads running?
(Latest Session: Fatal Exception ... below that there is a button for me with "Show all 475 threads")

Going through this list it looks like you are spawning a lot of pool-threads.
Maybe you can reduce the number there?
I recently added some names for threads but all the threads would have the default ("Thread-x") or something with "WebSocket".

Also just for your information, I will be on holiday for a week so don't expect any answers in this time ;)

Hope this still helps you!

Greetings
marci4

[Sdghasemi]

Hello @marci4,
After days of struggle in project and dependent libraries since you mentioned your pattern for thread names I happily managed to find where the pool threads were coming from.
It was from a separate advertising library called Adad which has an awful code structure in showing and downloading ads. Using the older version fixed the problem.
Thank you very much for helping me finding out the issue.