Composition Player Multiple Issues when seeking in PIP and grid mode

[pawaom]

Version

Media3 main branch

More version details

I am trying the new 1.7.0/1.8.0(there was some problem in the version numbers) code

When we seek the video It works fine in sequential media, but crashes in 2/2 grid and pip mode

Devices that reproduce the issue

Pixel 9 pro xl upsidedowncupcake emulator
samsung galaxy tab A7 lite sm-t225 android 14

Devices that do not reproduce the issue

N/A

Reproducible in the demo app?

Yes

Reproduction steps

Start the composition demo app

1. select sequential option select media play then seek it forward/ backward works fine
2. select Grid option select media play then seek it forward/ backward crashes
3. select PIP option select media play then seek it forward/ backward crashes

I had reported a similar issue for the older composition player Since the Sequential video is working I assume it must have been resolved
I know composition player is still in active development , so I tried to report the problems early.
#1489

Screen_recording_20250517_143617.1.mp4

Expected result

media should play successfully in both grid and pip mode after seeking forward or backward

Actual result

Play back crashes here is the bug report

2025-05-17 14:40:42.925 23336-23336 CompositionPlayer       androidx.media3.demo.composition     E  playerFailed [eventTime=26.62, mediaPos=18.61, window=0, errorCode=ERROR_CODE_VIDEO_FRAME_PROCESSING_FAILED (Ask Gemini)
                                                                                                    androidx.media3.common.PlaybackException: Error processing video frames
                                                                                                        at androidx.media3.transformer.CompositionPlayer.maybeUpdatePlaybackError(CompositionPlayer.java:1098)
                                                                                                        at androidx.media3.transformer.CompositionPlayer.lambda$onError$2$androidx-media3-transformer-CompositionPlayer(CompositionPlayer.java:488)
                                                                                                        at androidx.media3.transformer.CompositionPlayer$$ExternalSyntheticLambda0.run(D8$$SyntheticClass:0)
                                                                                                        at android.os.Handler.handleCallback(Handler.java:958)
                                                                                                        at android.os.Handler.dispatchMessage(Handler.java:99)
                                                                                                        at android.os.Looper.loopOnce(Looper.java:230)
                                                                                                        at android.os.Looper.loop(Looper.java:319)
                                                                                                        at android.app.ActivityThread.main(ActivityThread.java:8919)
                                                                                                        at java.lang.reflect.Method.invoke(Native Method)
                                                                                                        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:578)
                                                                                                        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1103)
                                                                                                    Caused by: androidx.media3.common.VideoFrameProcessingException: java.lang.IllegalStateException
                                                                                                        at androidx.media3.effect.VideoFrameProcessingTaskExecutor.handleException(VideoFrameProcessingTaskExecutor.java:306)
                                                                                                        at androidx.media3.effect.VideoFrameProcessingTaskExecutor.lambda$wrapTaskAndSubmitToExecutorService$3$androidx-media3-effect-VideoFrameProcessingTaskExecutor(VideoFrameProcessingTaskExecutor.java:292)
                                                                                                        at androidx.media3.effect.VideoFrameProcessingTaskExecutor$$ExternalSyntheticLambda0.run(D8$$SyntheticClass:0)
                                                                                                        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:487)
                                                                                                        at java.util.concurrent.FutureTask.run(FutureTask.java:264)
                                                                                                        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:307)
                                                                                                        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
                                                                                                        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
                                                                                                        at java.lang.Thread.run(Thread.java:1012)
                                                                                                    Caused by: java.lang.IllegalStateException
                                                                                                        at androidx.media3.common.util.Assertions.checkState(Assertions.java:85)
                                                                                                        at androidx.media3.effect.DefaultVideoCompositor.queueInputTexture(DefaultVideoCompositor.java:193)
                                                                                                        at androidx.media3.effect.MultipleInputVideoGraph.queuePreProcessingOutputToCompositor(MultipleInputVideoGraph.java:454)
                                                                                                        at androidx.media3.effect.MultipleInputVideoGraph.lambda$registerInput$0$androidx-media3-effect-MultipleInputVideoGraph(MultipleInputVideoGraph.java:294)
                                                                                                        at androidx.media3.effect.MultipleInputVideoGraph$$ExternalSyntheticLambda3.onTextureRendered(D8$$SyntheticClass:0)
                                                                                                        at androidx.media3.effect.FinalShaderProgramWrapper.renderFrameToOutputTexture(FinalShaderProgramWrapper.java:517)
                                                                                                        at androidx.media3.effect.FinalShaderProgramWrapper.renderFrame(FinalShaderProgramWrapper.java:464)
                                                                                                        at androidx.media3.effect.FinalShaderProgramWrapper.queueInputFrame(FinalShaderProgramWrapper.java:264)
                                                                                                        at androidx.media3.effect.FrameConsumptionManager.lambda$queueInputFrame$1$androidx-media3-effect-FrameConsumptionManager(FrameConsumptionManager.java:93)
                                                                                                        at androidx.media3.effect.FrameConsumptionManager$$ExternalSyntheticLambda1.run(D8$$SyntheticClass:0)
                                                                                                        at androidx.media3.effect.VideoFrameProcessingTaskExecutor.lambda$wrapTaskAndSubmitToExecutorService$3$androidx-media3-effect-VideoFrameProcessingTaskExecutor(VideoFrameProcessingTaskExecutor.java:290)
                                                                                                        at androidx.media3.effect.VideoFrameProcessingTaskExecutor$$ExternalSyntheticLambda0.run(D8$$SyntheticClass:0) 
                                                                                                        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:487) 
                                                                                                        at java.util.concurrent.FutureTask.run(FutureTask.java:264) 
                                                                                                        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:307) 
                                                                                                        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
                                                                                                        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644) 
                                                                                                        at java.lang.Thread.run(Thread.java:1012) 
                                                                                                  ]
2025-05-17 14:40:42.954 23336-23336 CompPreviewVM           androidx.media3.demo.composition     E  Preview error (Ask Gemini)
                                                                                                    androidx.media3.common.PlaybackException: Error processing video frames
                                                                                                        at androidx.media3.transformer.CompositionPlayer.maybeUpdatePlaybackError(CompositionPlayer.java:1098)
                                                                                                        at androidx.media3.transformer.CompositionPlayer.lambda$onError$2$androidx-media3-transformer-CompositionPlayer(CompositionPlayer.java:488)
                                                                                                        at androidx.media3.transformer.CompositionPlayer$$ExternalSyntheticLambda0.run(D8$$SyntheticClass:0)
                                                                                                        at android.os.Handler.handleCallback(Handler.java:958)
                                                                                                        at android.os.Handler.dispatchMessage(Handler.java:99)
                                                                                                        at android.os.Looper.loopOnce(Looper.java:230)
                                                                                                        at android.os.Looper.loop(Looper.java:319)
                                                                                                        at android.app.ActivityThread.main(ActivityThread.java:8919)
                                                                                                        at java.lang.reflect.Method.invoke(Native Method)
                                                                                                        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:578)
                                                                                                        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1103)
                                                                                                    Caused by: androidx.media3.common.VideoFrameProcessingException: java.lang.IllegalStateException
                                                                                                        at androidx.media3.effect.VideoFrameProcessingTaskExecutor.handleException(VideoFrameProcessingTaskExecutor.java:306)
                                                                                                        at androidx.media3.effect.VideoFrameProcessingTaskExecutor.lambda$wrapTaskAndSubmitToExecutorService$3$androidx-media3-effect-VideoFrameProcessingTaskExecutor(VideoFrameProcessingTaskExecutor.java:292)
                                                                                                        at androidx.media3.effect.VideoFrameProcessingTaskExecutor$$ExternalSyntheticLambda0.run(D8$$SyntheticClass:0)
                                                                                                        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:487)
                                                                                                        at java.util.concurrent.FutureTask.run(FutureTask.java:264)
                                                                                                        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:307)
                                                                                                        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
                                                                                                        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
                                                                                                        at java.lang.Thread.run(Thread.java:1012)
                                                                                                    Caused by: java.lang.IllegalStateException
                                                                                                        at androidx.media3.common.util.Assertions.checkState(Assertions.java:85)
                                                                                                        at androidx.media3.effect.DefaultVideoCompositor.queueInputTexture(DefaultVideoCompositor.java:193)
                                                                                                        at androidx.media3.effect.MultipleInputVideoGraph.queuePreProcessingOutputToCompositor(MultipleInputVideoGraph.java:454)
                                                                                                        at androidx.media3.effect.MultipleInputVideoGraph.lambda$registerInput$0$androidx-media3-effect-MultipleInputVideoGraph(MultipleInputVideoGraph.java:294)
                                                                                                        at androidx.media3.effect.MultipleInputVideoGraph$$ExternalSyntheticLambda3.onTextureRendered(D8$$SyntheticClass:0)
                                                                                                        at androidx.media3.effect.FinalShaderProgramWrapper.renderFrameToOutputTexture(FinalShaderProgramWrapper.java:517)
                                                                                                        at androidx.media3.effect.FinalShaderProgramWrapper.renderFrame(FinalShaderProgramWrapper.java:464)
                                                                                                        at androidx.media3.effect.FinalShaderProgramWrapper.queueInputFrame(FinalShaderProgramWrapper.java:264)
                                                                                                        at androidx.media3.effect.FrameConsumptionManager.lambda$queueInputFrame$1$androidx-media3-effect-FrameConsumptionManager(FrameConsumptionManager.java:93)
                                                                                                        at androidx.media3.effect.FrameConsumptionManager$$ExternalSyntheticLambda1.run(D8$$SyntheticClass:0)
                                                                                                        at androidx.media3.effect.VideoFrameProcessingTaskExecutor.lambda$wrapTaskAndSubmitToExecutorService$3$androidx-media3-effect-VideoFrameProcessingTaskExecutor(VideoFrameProcessingTaskExecutor.java:290)
                                                                                                        at androidx.media3.effect.VideoFrameProcessingTaskExecutor$$ExternalSyntheticLambda0.run(D8$$SyntheticClass:0) 
                                                                                                        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:487) 
                                                                                                        at java.util.concurrent.FutureTask.run(FutureTask.java:264) 
                                                                                                        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:307) 
                                                                                                        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
                                                                                                        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644) 
                                                                                                        at java.lang.Thread.run(Thread.java:1012) 

Media

Media that is used is the one provided by Android exoplayer media3 composition demo app

Bug Report

• You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.

[pawaom]

I have few more question,

1. the demo app has one audio overlay. can we have multiple overlay
2. Can we remove an PIP video overlay after some time, example the background video is 25 seconds long, overlay pip video is 10 seconds long, starts from 5th second of the background video plays this 15th second and then we remove it.
3. There is no option to set the duration of the overlay settings , like we want to set the setRotationDegrees for certain time, or set scale for certain time only

override fun getOverlaySettings(inputId: Int, presentationTimeUs: Long): OverlaySettings {
            return if (inputId == 0) {
              val cycleRadians = 2 * Math.PI * (presentationTimeUs / cycleTimeUs)
              StaticOverlaySettings.Builder()
                .setScale(0.35f, 0.35f)
                .setOverlayFrameAnchor(0f, 1f) // Top-middle of overlay
                .setBackgroundFrameAnchor(sin(cycleRadians).toFloat() * 0.5f, -0.2f)
                .setRotationDegrees(cos(cycleRadians).toFloat() * -10f)

                .build()
            }

4. if anchor video is smaller than the overlay video, the video stops at the last frame of the anchor video while olverlay keeps playing can we show a black screen instead(have both options to show last frame or black screen)

5. better documentation

[pawaom]

another Problem I noticed in PIP mode on Rotation , if we rotate back the video is not fully visible this video explains it. It seems not to happen in the sequential or grid mode, I could not reproduce it in those modes however I will suggest to test for them as well.
I tested this problem on
samsung galaxy tab A7 lite sm-t225 android 14

But this problem of , pip video part not visible , was not seen in Pixel 9 pro xl upsidedowncupcake emulator

XRecorder_20250518_01.mp4

[pawaom]

And Another problem the video keeps playing after the total duration has been reached, exceeding the total duration

The below video explains it

It was reproduced in both Grid and PIP mode,
on both samsung galaxy tab A7 lite sm-t225 android 14 and Pixel 9 pro xl upsidedowncupcake emulator

This video helps to show both the problems regarding total duration and the above pip video being visible on the Pixel 9 pro xl upsidedowncupcake emulator(this is expected behavior) read this along with the above comment.

Screen_recording_20250518_110514.1.1.1.mp4

[droid-girl]

Hi @pawaom,
To answer your first comment:

• Seeking is currently not supported for multiple video sequences like PiP and Grid view. We do support seeking for one video sequence and 2 sequences where one of them is an audio file

Please keep in mind that CompositionPlayer is still in development and not all features are supported yet.

[droid-girl]

I have few more question,

1. the demo app has one audio overlay. can we have multiple overlay

Yes, you should be able to have multiple audio sequences if needed.

3. Can we remove an PIP video overlay after some time, example the background video is 25 seconds long, overlay pip video is 10 seconds long, starts from 5th second of the background video plays this 15th second and then we remove it.

This feature is still in development. It will be supported in the later release

5. There is no option to set the duration of the overlay settings , like we want to set the setRotationDegrees for certain time, or set scale for certain time only> >

This will be supported as part of your previous question

4. if anchor video is smaller than the overlay video, the video stops at the last frame of the anchor video while olverlay keeps playing can we show a black screen instead(have both options to show last frame or black screen)

This is a known limitation and it will be removed in the upcoming releases.

[droid-girl]

Thank you for reporting bugs in the composition demo. We are aware of several limitations and will gradually add improvements and fix problems as we also improve and add new APIs to CompositionPlayer