Flutter : FirebaseMessaging "onBackgroundMessage" with FFmpegKit

[ujwalparker]

Description
In an app (Flutter) where we have configured FirebaseMessaging (https://pub.dev/packages/firebase_messaging/) with the onBackgroundMessage handler the FFmpegKit plugin is not usable as there seems to be some conflict due to which the EventChannel which is created by this plugin is rendered null

Expected behavior
Methods like FFmpegKit.executeAsync or FFprobeKit.getMediaInformationAsync or others execute with a result.

Current behavior
The methods FFmpegKit.executeAsync or FFprobeKit.getMediaInformationAsync or others fail with the following sample error

W/ffmpeg-kit-flutter(32468): ResultHandler can not send event {FFmpegKitExecuteCallbackEvent={createTime=1635353254392, startTime=1635353254414, sessionId=1, type=1, command= -i /data/user/0/com.arre.voice/app_flutter/effects/drumroll.wav -i /data/user/0/com.arre.voice/app_flutter/effects/cheer.wav -i /data/user/0/com.arre.voice/app_flutter/effects/boo.wav -filter_complex amix=inputs=3:duration=longest -y /storage/emulated/0/Android/data/com.arre.voice/files/[#6d178].m4a}} on a null event sink.

To Reproduce
Just add FirebaseMessaging to the app and implement onBackgroundMessage handler.

Screenshots

Logs
successAsync gets called for any method but eventSink is render null thus the error message printed by the plugin
Log.w(LIBRARY_NAME, String.format("ResultHandler can not send event %s on a null event sink.", object));

Environment

• Platform: Flutter Android
• Architecture: arm64, and others too
• Version v4.5.0
• Source branch v4.5.0

Other
FirebaseMessaging has 3 handlers as below ( firebase_messaging: ^10.0.7)

• FirebaseMessaging.onMessage
• FirebaseMessaging.onMessageOpenedApp
• FirebaseMessaging.onBackgroundMessage

When you just do not set the "FirebaseMessaging.onBackgroundMessage" handler the plugin works fine.

[tanersener]

Can you provide more details about which ffmpeg-kit methods you're calling?

And, does calling another method like await FFmpegKitConfig.getFFmpegVersion() before execute methods makes a difference?

[ujwalparker]

@tanersener

Tried with following methods, most of them which generate a future session

• FFmpegKitConfig.getFFmpegVersion()
• FFmpegKit.executeAsync()
• FFprobeKit.getMediaInformationAsync()

Yes FFmpegKitConfig.getFFmpegVersion() works. with or without FCM background and prints the following
v4.5-dev-2008-g90da43557f

Put together a sample app for this use case with 3 branches

https://github.com/ujwalparker/audio-mixer/tree/ffmpeg-kit/issues-209-audio-mixer

• Includes the sample code. All methods implemented work expected.

https://github.com/ujwalparker/audio-mixer/tree/ffmpeg-kit/issues-209-fcm-added

• On Adding FCM only FFmpegKitConfig.getFFmpegVersion() works and the others do not. On the UI in sample app only the first button works and displays the response from the function call. Others fail with the null sink error

https://github.com/ujwalparker/audio-mixer/tree/ffmpeg-kit/issues-209-fcm-background-disabled

• By just disabling FirebaseMessaging.onBackgroundMessage - the functions all work

Only thing not executed is below lines of code

Future<void> _firebaseMessagingBackgroundHandler(RemoteMessage message) async { await Firebase.initializeApp(); print('Handling a background message ${message.messageId}'); }

You can have a look at the change sheet for the above 3 branches with changes made. (issues-209-fcm-added and issues-209-fcm-background-disabled)

[tanersener]

FFmpegKitFlutterPlugin supports both V1 and V2 Android embeddings. There are also logs printed during init and register methods. As far as I see, when FirebaseMessaging.onBackgroundMessage is enabled a second FFmpegKitFlutterPlugin is created. But that second instance is not properly registered; EventSink which is necessary to send back events is null. That's why you have this error.

I suggest asking this behaviour to Firebase devs. There are similar errors reported for other plugins when they are registered through FirebaseMessaging.onBackgroundMessage.

[ujwalparker]

@tanersener I do not think this has any interference with Firebase. I spent some time to understand if there is any conflict in the method channel naming etc. but could not find any.

I left the onBackgroundMessage handler active and tried to invoke FFprobeKit.getMediaInformationAsync using the "ffmpeg_kit_flutter_min" package this time (does not have relevance to changing the package - I was just trying to reduce the APK size - unrelated)

1. On app launch - I see that Firebase Messaging background service has initialized
   
2. On invoking FFprobeKit.getMediaInformationAsync, the FFmpegKitFlutterPlugin is invoked. The following logs also get printed
   I/flutter (25239): Loading ffmpeg-kit-flutter. D/ffmpeg-kit-flutter(25239): FFmpegKitFlutterPlugin started listening to events on io.flutter.plugin.common.EventChannel$IncomingStreamRequestHandler$EventSinkImplementation@f7b8302. I/flutter (25239): Loaded ffmpeg-kit-flutter-android-min-arm64-v8a-4.5.0.

but the execution fails

Why I say its unrelated - because the onListen() is called on the FFmpegKitFlutterPlugin and the log clearly prints the eventSink id. which mean that the successAsync was invoked by asyncMediaInformationSessionExecute and AsyncWriteToPipeTask also invoked.

I could only see one FFmpegKitFlutterPlugin instance created - and that was after the call was made to FFprobeKit.getMediaInformationAsync on a user interaction of a button.

Attached logs, 1st is after launch of app where Firebase plugin is initialized. 2nd on invoking getMediaInformationAsync

1. FlutterFirebaseMessagingBackgroundService.txt
2. FFprobeKit.getMediaInformationAsync.txt

Thoughts?

[tanersener]

Why don't you use the version we have on issue-209 branch, it has more detailed logs?

[ujwalparker]

Ok will use that and report back.

[ifananomali]

@tanersener @ujwalparker I got this error too when using FFmpeg with Firebase Messaging 'background handler'. Hope to see a solution for this.

[ujwalparker]

@tanersener those logs helped. And yes your initial comment about two instances of FFmpegKitFlutterPlugin objects created is right - but this is expected behavior IMO (but why? more on that towards the end)

Firebase Messaging background service triggers the second object creation and that is where the bug manifests.

The issue is in the way eventSink is used across and since it is reinitialized when the new FFmpegKitFlutterPlugin object gets created, When the FFmpegKitConfig methods are called they are sending the result back on a eventSink which is now null.

I have thought of a crude low impact fix which solves the issue by way of using a static eventSink - to see if this fixes the issue and it does fix - this is crude because this uses a singleton initializer just to use a static eventSink. Checkout the commit @ https://github.com/ujwalparker/ffmpeg-kit/commit/8cc4bd0608551b44142aa25d7173cb7fa7b40fe0

This is not a fix Firebase will accept or review IMO - although I find it weird on why implementation of the background callback triggers an initialization :P. Found an issue which was closed for lack of support on their repo.

I have not raised this as a PR as I would not call this as an efficient fix as I do not fully understand the architecture of this plugin. It might possibly be an easier fix from your POV. Let me know if with this you can see a possible solution which is not as crude as the commit.

Here is a gist with the above fix working with _firebaseMessagingBackgroundHandler enabled
https://gist.github.com/ujwalparker/5afb7b29611119fa90ed3cd38764f3df

cc. @ifananomali

[tanersener]

flutter plugin of ffmpeg-kit registers itself in the same way as the official flutter.dev plugins. None of those plugins utilise static channels, event sinks or plugins. I want to keep it in that way, if possible.

I made a minor change in the issue-209 branch which might help you in your case. Try that please. It's the best I can do at the moment.

[ujwalparker]

@tanersener the change on that branch also works !!. You can take this into the main branch as it will solve this issue for all who want to use this plugin and have FCM integrated (which should be like 90% developers :P)