publishing large messages at high frequency causes out of memory on android

[christiankerl]

I try to publish image messages (320x240 with 2 bytes per pixel) with a frequency of 30Hz from my android device. Unfortunately the buffer copying during message serialization somehow floods the memory and the GC is not able to catch up. Therefore, I see a lot of the following log messages:

GC_FOR_ALLOC freed 1235K, 4% free 156597K/161848K, paused 40ms, total 40ms
10-07 10:01:16.241 17345-17418/my.android_package D/dalvikvm﹕ GC_FOR_ALLOC freed 1406K, 4% free 159421K/164708K, paused 45ms, total 45ms
10-07 10:01:17.142 17345-17418/my.android_package D/dalvikvm﹕ GC_FOR_ALLOC freed 2936K, 4% free 162244K/167568K, paused 36ms, total 36ms
10-07 10:01:18.223 17345-17451/my.android_package D/dalvikvm﹕ GC_FOR_ALLOC freed 4251K, 4% free 164916K/170472K, paused 42ms, total 43ms
10-07 10:01:19.294 17345-17418/my.android_package D/dalvikvm﹕ GC_FOR_ALLOC freed 3325K, 3% free 167894K/173072K, paused 52ms, total 52ms

until an OutOfMemory exception occurs:

10-07 10:01:26.341 17345-17418/my.android_package E/AndroidRuntime﹕ FATAL EXCEPTION: pool-1-thread-12
java.lang.OutOfMemoryError
at org.jboss.netty.buffer.HeapChannelBuffer.(HeapChannelBuffer.java:42)
at org.jboss.netty.buffer.LittleEndianHeapChannelBuffer.(LittleEndianHeapChannelBuffer.java:34)
at org.jboss.netty.buffer.ChannelBuffers.buffer(ChannelBuffers.java:139)
at org.jboss.netty.buffer.HeapChannelBufferFactory.getBuffer(HeapChannelBufferFactory.java:69)
at org.jboss.netty.buffer.DynamicChannelBuffer.ensureWritableBytes(DynamicChannelBuffer.java:86)
at org.jboss.netty.buffer.DynamicChannelBuffer.writeBytes(DynamicChannelBuffer.java:241)
at org.ros.internal.message.field.ChannelBufferField.serialize(ChannelBufferField.java:67)
at org.ros.internal.message.DefaultMessageSerializer.serialize(DefaultMessageSerializer.java:32)
at org.ros.internal.message.DefaultMessageSerializer.serialize(DefaultMessageSerializer.java:26)
at o rg.ros.internal.transport.queue.OutgoingMessageQueue$Writer.loop(OutgoingMessageQueue.java:63)
at org.ros.concurrent.CancellableLoop.run(CancellableLoop.java:56)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1080)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:573)
at java.lang.Thread.run(Thread.java:841)

If I limit the publishing rate to 15 Hz I still see a lot of GC log messages, but the amount of allocated memory stays constant. At this rate there seems to be a balance between object allocation/deallocation.

Is there a way to reuse existing buffers and avoid costly allocations?

[stonier]

Could probably be optimised better, but we don't have anyone actively working on the internals of this part right now so putting this on the 'todo' milestone.

[MallikarjunTirlapur]

Hi christiankerl,

Have you got the solution for your problem.
If yes please let me know the solution.

[christiankerl]

Unfortunately not...

[gitunit]

is there an example how to publish raw images with rosjava? im getting similar error as on top post. maybe i missed the Hz setting? where would i set this? (the compressed image publisher sends with ~15Hz without crashing)

[ParthaEth]

I have similar problem. I can do 640X480 @ 30 HZ raw image but have problems as soon as I go beyond. This only happens only if there is a subscriber! Can I ask ROS to drop messages?

[jubeira]

Hi @ParthaEth ,
If images arrive faster than you can process them, one workaround could be adding a latch or lock to the message listener. That should ensure that you actually finish executing a callback before the next messages arrive. I've tried that and so far it worked fine.

[arpit-korechi]

Hi @ParthaEth ,

If images arrive faster than you can process them, one workaround could be adding a latch or lock to the message listener. That should ensure that you actually finish executing a callback before the next messages arrive. I've tried that and so far it worked fine.

Could you provide some reference code to that could help the optimizing the problem?