startStream issues

[JinYi-Tsinghua]

Package version
5.1.2

Environment

• OS: Android

Describe the bug

.listen has no data unless encoder: AudioEncoder.pcm16bits is used, yet no error is triggered. All other AudioEncoders fail silently, despite working just fine when start() is used instead of startStream().

Furthermore, even if pcm16bits is used with startStream(), there is no documented way to play it back.

Furthermore, when pcm16bits is used with start(), it produces a file that cannot be played back via AudioPlayer(), unlike all other AudioEncoder formats.

If I am wrong and startStream is not unusable (either with pcm16bits or other), then an example file needs to be written to show how to utilize startStream to produce an output that can be played back with AudioPlayer().

[llfbandit]

Thanks for your report.
A bit of documentation has been added to get notified of async errors from onStateChanged() stream.

About PCM concerns, if you don't know this format, just don't use it, this is raw audio. Most players are unable to play it.

[JinYi-Tsinghua]

PCM might be my fault then. That's not the encoder I want to use anyway.

Also I've already enabled onError with no output from it.

Regarding AAC, here is if I try recording a few seconds of aacEld:

D/BufferPoolAccessor2.0(12572): evictor expired: 1, evicted: 0
D/CCodec (12572): allocate(c2.android.aac.encoder)
I/CCodec (12572): setting up 'default' as default (vendor) store
I/CCodec (12572): Created component [c2.android.aac.encoder]
D/CCodecConfig(12572): read media type: audio/mp4a-latm
D/ReflectedParamUpdater(12572): extent() != 1 for single value type: algo.buffers.max-count.values
D/ReflectedParamUpdater(12572): extent() != 1 for single value type: output.subscribed-indices.values
D/ReflectedParamUpdater(12572): extent() != 1 for single value type: input.buffers.allocator-ids.values
D/ReflectedParamUpdater(12572): extent() != 1 for single value type: output.buffers.allocator-ids.values
D/ReflectedParamUpdater(12572): extent() != 1 for single value type: algo.buffers.allocator-ids.values
D/ReflectedParamUpdater(12572): extent() != 1 for single value type: output.buffers.pool-ids.values
D/ReflectedParamUpdater(12572): extent() != 1 for single value type: algo.buffers.pool-ids.values
I/CCodecConfig(12572): query failed after returning 9 values (BAD_INDEX)
D/CCodecConfig(12572): c2 config diff is Dict {
D/CCodecConfig(12572): c2::u32 coded.bitrate.value = 64000
D/CCodecConfig(12572): c2::u32 coded.pl.level = 0
D/CCodecConfig(12572): c2::u32 coded.pl.profile = 8192
D/CCodecConfig(12572): c2::u32 coding.aac-sbr-mode.value = 3
D/CCodecConfig(12572): c2::u32 input.buffers.max-size.value = 2048
D/CCodecConfig(12572): c2::u32 input.delay.value = 0
D/CCodecConfig(12572): string input.media-type.value = "audio/raw"
D/CCodecConfig(12572): string output.media-type.value = "audio/mp4a-latm"
D/CCodecConfig(12572): c2::u32 raw.channel-count.value = 1
D/CCodecConfig(12572): c2::u32 raw.sample-rate.value = 44100
D/CCodecConfig(12572): }
I/MediaCodec(12572): MediaCodec will operate in async mode
D/MediaCodec(12572): shapeMediaFormat: deltas(0): AMessage(what = 0x00000000) = {
D/MediaCodec(12572): }
D/MediaCodec(12572): flushMediametrics
D/CCodec (12572): [c2.android.aac.encoder] buffers are bound to CCodec for this session
D/CCodecConfig(12572): no c2 equivalents for flags
D/CCodecConfig(12572): no c2 equivalents for encoder
D/CCodecConfig(12572): c2 config diff is c2::u32 coded.bitrate.value = 128000
D/CCodecConfig(12572): c2::u32 coded.pl.profile = 8202
D/CCodecConfig(12572): c2::u32 raw.channel-count.value = 2
W/Codec2Client(12572): query -- param skipped: index = 1107298332.
D/CCodec (12572): encoding statistics level = 0
D/CCodec (12572): setup formats input: AMessage(what = 0x00000000) = {
D/CCodec (12572): int32_t sample-rate = 44100
D/CCodec (12572): int32_t channel-count = 2
D/CCodec (12572): int32_t max-input-size = 4096
D/CCodec (12572): string mime = "audio/raw"
D/CCodec (12572): int32_t android._config-pcm-encoding = 2
D/CCodec (12572): }
D/CCodec (12572): setup formats output: AMessage(what = 0x00000000) = {
D/CCodec (12572): int32_t sample-rate = 44100
D/CCodec (12572): int32_t bitrate = 128000
D/CCodec (12572): int32_t channel-count = 2
D/CCodec (12572): int32_t level = 0
D/CCodec (12572): int32_t max-bitrate = 128000
D/CCodec (12572): string mime = "audio/mp4a-latm"
D/CCodec (12572): int32_t profile = 39
D/CCodec (12572): }
I/CCodecConfig(12572): query failed after returning 9 values (BAD_INDEX)
D/CCodecConfig(12572): c2 config diff is c2::u32 input.buffers.max-size.value = 4096
D/MediaCodec(12572): keep callback message for reclaim
W/Codec2Client(12572): query -- param skipped: index = 1342179345.
W/Codec2Client(12572): query -- param skipped: index = 2415921170.
W/Codec2Client(12572): query -- param skipped: index = 3254792206.
W/Codec2Client(12572): query -- param skipped: index = 3254792194.
D/CCodecBufferChannel(12572): [c2.android.aac.encoder#966] Created input block pool with allocatorID 16 => poolID 20 - OK (0)
I/CCodecBufferChannel(12572): [c2.android.aac.encoder#966] Created output block pool with allocatorID 16 => poolID 33 - OK
D/CCodecBufferChannel(12572): [c2.android.aac.encoder#966] Configured output block pool ids 33 => OK
D/MediaCodecEncoder(12572): Output format set: {max-bitrate=128000, sample-rate=44100, level=0, mime=audio/mp4a-latm, profile=39, channel-count=2, bitrate=128000, csd-0=java.nio.HeapByteBuffer[pos=0 lim=4 cap=4]}
W/DartWorker(12572): type=1400 audit(0.0:9106): avc: granted { execmem } for scontext=u:r:untrusted_app:s0:c198,c256,c512,c768 tcontext=u:r:untrusted_app:s0:c198,c256,c512,c768 tclass=process app=com.test.app
D/BufferPoolAccessor2.0(12572): bufferpool2 0xb400d0b72df69a28 : 0(0 size) total buffers - 0(0 size) used buffers - 152/157 (recycle/alloc) - 2/304 (fetch/transfer)
D/BufferPoolAccessor2.0(12572): evictor expired: 1, evicted: 1
D/BufferPoolAccessor2.0(12572): bufferpool2 0xb400d0b72df70a28 : 5(20480 size) total buffers - 4(16384 size) used buffers - 260/265 (recycle/alloc) - 2/521 (fetch/transfer)
D/BufferPoolAccessor2.0(12572): bufferpool2 0xb400d0b72df70a28 : 5(20480 size) total buffers - 4(16384 size) used buffers - 532/537 (recycle/alloc) - 2/1065 (fetch/transfer)
D/CCodecBufferChannel(12572): [c2.android.aac.encoder#966] MediaCodec discarded an unknown buffer
D/CCodecBufferChannel(12572): [c2.android.aac.encoder#966] MediaCodec discarded an unknown buffer
D/CCodecBufferChannel(12572): [c2.android.aac.encoder#966] MediaCodec discarded an unknown buffer

As I said, during this time, there is no callbacks from .listener, so no data from listener (unless I use PCM). The only callback is onDone.

[JinYi-Tsinghua]

....yeah so this issue shouldn't be closed. It is unresolved. startStream is broken on Android except for PCM.