what is the best way for playing and changing a audio in runtime

[amirc901]

Hi, what is the best way to play audio in run time for example when a user selects an audio file in the UI how could I communicate with the audioservice and tell it to play the audio? I saw you are using MediaItem for this but it has not a property for the filePath or fileUrl

[ryanheise]

The MediaItem id is an arbitrary unique id of your choice, and you can use the URL or file path as this id if you wish.

Then you could use:

AudioService.playFromMediaId(filePath);

[yringler]

This is something that I also worked with. I'm passing in a URL as the ID, but that doesn't tell the service what the title, artist, album etc are. The only way to do that is with a queue, so it would seem that to play a single file with all the right metadata you'd set a queue with a single item and then play (or playFromId)

[hacker1024]

You can also use the IsolateNameServer API to send through complex objects.

My app needs to be able to play stations, playlists, and albums, so I have a MusicProvider interface that has loading and queueing functions and I send the implementation objects through to it.

[nuc134r]

I want to share how I managed to get it working.

I don't use playFromMediaId but instead I'm heavily using customAction to pass data into service. I'm passing a list of my own audio track objects by serializing them into list of UInt8List with protobuf library. You can read about here. Then I'm using custom actions to change current track, reorder items in current playlist and toggle shuffle/repeat mode. As I have complex objects to store metadata (urls, album covers, lyrics) I couldn't use MediaItem for that. But of course I use MediaItem object to update notification. So for me it's just an utility object. I don't have a requirement to support queue on Android level yet but it will be easy to implement as I have everything I need stored in the Service.

It gets interesting when user closes Activity (or Android system kills it) and then app is restarted. You need to connect to service and restore current state. It is easy to pick current track name and position, maybe even current track can be picked from it's id. But how would I get entire playlist with all the URLs and metadata? Shuffle and repeat settings? Since customAction doesn't support returning data (you cannot return data from Java-to-Flutter channel request or so) it seems like a pity.

So I implemented service triggered custom actions or how they call it here bi-directional custom actions. It was quite easy to implement and now when I need state I only send a customAction to service and it triggers customAction in response with everything I need. That's the only way I came up with. Basically that's the only thing that I miss in the library. Also I need to pass current buffered position but it's even easier to support as even PlaybackStateCompat has a field for that.

I think I'll PR it soon. Thank you Ryan for this library. You da real MVP.

[ryanheise]

I think the IsolateNameServer API could be used in the implementation of customAction and also bidirectional custom actions, and should be simpler since it doesn't require any dependencies. But are there any benefits of the protobuf approach that would outweigh the simplicity of IsolateNameServer?

[hacker1024]

I don't know about protobuf, but the IsolateNameServer API doesn't work on Flutter for Web, and I hope we can support that at some point.

[ryanheise]

That's a good point. Although I wonder if this is still unsupported. I know that was the case in the past, but in principle I see no reason why isolates couldn't be supported in Flutter for Web using Web Workers, and I would have expected some work to happen in this direction.

[nuc134r]

But are there any benefits of the protobuf approach that would outweigh the simplicity of IsolateNameServer?

I have nothing to say as I dont know much about isolates communication and IsolateNameServer API in particular. I just shared some approach that worked for me.

What happens is:

• A list of objects is converted into proto objects
• Proto objects are serialized into list of Uint8List
• That list is sent through standard message codec of platform channel into Java
• Java internally deserializes it into Java types
• Then it is sent into platform channel again which serializes it back into Dart types
• On the Dart side list is deserialized into proto objects
• Protos are converted into my objects

Simple as that. :)

Actually I'm new to Android, Java and Flutter so my approach is probably naive and not very performant. Might research better ways later but for now it completely suits my needs. If Ryan is going to implement bi-directional custom actions into plugin I'm going to instantly switch to them as it probably will work better in every possible way.

[dustin-graham]

@ryanheise I also have a question about how this is supposed to work right now. The example code shows a couple MediaItems hardcoded into a list for a media queue. However, this is hardly a typical use case. Do you have some sample code on how to select a media item and play that dynamically with this library?

This thread has mentioned a few things but no sample code has been provided yet. I would appreciate some official guidance on this use case.

[ryanheise]

@dustin-graham Apologies for not replying sooner. The intention was to work with thin clients where your background task is the one that has access to your database of media items, and contains the logic for playing them, while your Flutter UI acts as a thin client. So AudioService.playFromMediaId is intended for the purpose you describe.

One question is that it may depend on where you're selecting the item from. If from the queue, the way it is intended to work is that the background task broadcasts the queue, the UI can show it, if the user selects an item, you can pass its id back to the background task via AudioService.playFromMediaId.

Aside from the queue, you will be able to implement more custom use cases via the plugin's extension mechanism: custom actions (and now also custom events). With these, if there is a method on AudioService that you wish existed, you can add a custom one, and if there is a stream you wish existed, you can add one.

There is also a plan to one day fully support the type of catalogue that Android Auto supports. The way it would work is that the background audio task contains the catalogue, and can broadcast it to interested clients including Android Auto, and also your Flutter UI. Your Flutter UI can then select a media item from that catalogue, and then pass it into AudioService.playFromMediaId. This isn't ready for prime time yet, so if you have a more specialised use case that's not a queue, I recommend using custom actions and events described previously. (Note that custom events were just added, and are only available in the git version).

[ryanheise]

I've also just added another convenience method (to git): AudioService.playMediaItem. This allows you to pass in all the metadata for a media item, not just the URL. While convenient, note that this is not a standard protocol, so you'll still need to implement onPlayFromMediaId if you want to respond to other remote clients on Android that follow the standard protocol.