Audio over AOA Issues

[khburdette]

Bug Report

With the new router service logic, media apps are connecting over AOA (USB) and unable to play media.

When starting the AOA protocol, the head unit will send a product ID (see AOA spec) which can be used to determine if the head unit supports audio or not. This information should be used in determining whether or not to connect a media app to a router service using AOA.

In our testing some devices don't support media over AOA. When connected over USB the device will still sent the audio via A2DP. Samsung devices were among the devices that observed this behavior.

Because of the inconsistency with devices, we decided not to support media over AOA on SYNC head units. I propose we provide a hot fix to not allow media apps to connect to a router service that's connected via AOA until we can better determine if both the head unit and device support media over AOA before connecting.

Worth noting that with the 4.8 library, we are able to specify transport(s) to be used. Older libraries using the router service will still connect via AOA when any router service connects via AOA. Another possible hot fix is to require apps using the 4.8 library to specify their transport and only allowing apps with 4.8+ to connect to a router service over AOA. Again, another temporary solution until we can determine if both the head unit and device support media over AOA and then connecting.

Reproduction Steps

1. Install app that supports AOA (Hello SDL)
2. Install media app using the router service
3. Connect phone via USB and select the AOA app when prompted on device

Expected Behavior

Media apps will only connect over AOA if both the device and head unit support it

Observed Behavior

Media app connects over AOA and is unable to play media

OS & Version Information

• Android Version: 9
• SDL Android Version: 4.8
• Testing Against: SYNC Gen 3

[joeygrover]

This is a rather complicated issue, so I apologize for the novel below.

First and foremost, the module implementing Core should deny registration to a MEDIA app if the media component isn't ready or available.

However, with the Gen3 system it seems that is not the case. While it is a much more intended action, a MEDIA app could have used the pre-multiplexed AOA connection and still registered with the module and hit the same issue.

Moving onto the library side, the router service is merely a pipe for SDL messages to be sent to and from SDL apps. It has the bare minimum knowledge of the apps that are binding to it by design. This means it has no knowledge of which apps have certain requirements as in this case a method for audio arbitration. It will send the connected intent and allow usage to any type of app , it will be up to the app (or client side of the binding relationship) to decide if the current situation is acceptable to use. This can be seen where apps have the ability to require a high bandwidth transport.

Neither of the two presented solutions would be acceptable. There is no specific requirement that an IVI system has to support audio over AOA in order to use AOA for SDL. An IVI system could use AOA for SDL and send audio through the AOA, bluetooth A2DP, or even an aux cord. While the latter is not likely, the second is a very real scenario. Not allowing MEDIA apps to connect over AOA entirely would block this type of integration. Which would make the solution unacceptable to any adopter/SDLC member that has implemented this type of connection.

Furthermore, one issue with SDL is discoverability. If we prevent apps from connecting in any scenario there is less of chance that app will be discovered. Providing a message prompt to the user to connect an audio source in this instance would be a better end goal in my opinion.

Based on this information I think we have to create a better solution. One potential work around is giving apps the ability to select if they requireAudioConnection into their multiplexing transport config. The library itself can check the Android AudioManager for connected devices and ensure that at least one of the supported audio transports are connected. However, finding the correct timing for this might be a little tricky as we don't have any requirement, for example, saying A2DP has to be connected before SPP. If we are early in the process we may miss a potential acceptable scenario which is also a concern.

Alternatively to AudioManager, the Android MediaRouter API is the more direct path to find where audio is being router, but it was introduced in api level 16 (Android 4.1) so it wouldn't cover all Android devices. According to Android's distribution dashboard (which oddly hasn't been updated since October 26, 2018) this would cover 99.5% of Android phones though.

I would recommend taking options to the SDLC steering committee next week and explaining the situation. Then give the members a chance to investigate and review this issue, and vote on a solution the following meeting.

1. Do Nothing. Core module should reject MEDIA app registration if no audio arbitration method is available.
2. Tell Media apps to remove USB as an acceptable connection from MultiplexTransportConfig
3. Library does not allow Media Apps to use USB at all from client side of router service binding relationship
4. Increase the amount of data the router service collects to prevent MEDIA apps from connecting over USB
5. Add additional flag, requireAudioConnection, to MultiplexTransportConfig and library checks AudioManager if an acceptable audio output device is present (would need SDLC to define which devices are acceptable See Andorid's AudioDeviceInfo)
6. Same as 5. but with MediaRouter
7. Similar to 5, add additional option to config to send Show message to inform developer to connect audio source vs just closing their service if no acceptable audio device is present.
8. Same as 7 but with MediaRouter

[khburdette]

I think options 4-8 are all good options for a more permanent solution and can be discussed in the SDLC steering committee. I also think we need to come up with a solution for media apps using the router service on library versions <4.8. Can we brainstorm a hotfix solution for this?

[theresalech]

As discussed during the 2019-05-07 Steering Committee call, members will review this issue and possible solutions provided by the Project Maintainer. Members should comment with additional solutions if applicable and plan to vote on a solution during the 2019-05-14 Steering Committee meeting. Members should also advise if they are using AOA and if they rely on other sources of audio routing outside of AOA, and take this into consideration when determining the best solution. We will plan to include a fix for this issue in the 4.9 Release, as agreed to during the 2019-05-07 Steering Committee call.

[ashwink11]

As suggested in solution 5, I agree that library should ensure that at least one of the supported audio output device is connected. However, instead of using flag requireAudioConnection, can we use AppHMIType::MEDIA?

In case we keep this flag, what would be the behavior if requireAudioConnection is not set, but App is an MEDIA App and still tries to route Audio through IVI?

In addition to these solutions mentioned in 4-8, I think, we should also consider solution 1 as well. Core module should reject MEDIA app registration if no audio arbitration method is available.
This would be preventive step to make sure that IVI does not go in undefined state, if App does not select requireAudioConnection , but registers on IVI as an MEDIA App and try to route Audio.

[khburdette]

An example of how AOA is currently behaving:
Install (from the play store): App Catalog and Sygic Car Navigation
Install Hello SDL with 4.8.1 library version

Scenario 1:

• Plug in device
• Select Hello SDL on the AOA popup:
  • observed behavior: App Catalog and Hello SDL show up on the head unit

Scenario 2:

• Plug in device
• Select Sygic on the AOA popup:
  • observed behavior: Sygic shows up, Hello SDL has foreground service but never shows up
  • when you unplug the device, Hello SDL never removed it's foreground notification, so this is putting Hello SDL into a bad state

This might weigh into our possible AOA hotfix, because I don't think this is ideal ...

[joeygrover]

I hesitate to force a developers integration, so simply saying that MEDIA apps can't connect over AOA when audio isn't present is a hard rule and prevents some discoverability. Instead, we could say the requireAudioConnection will be set to true by default for MEDIA apps, but can be overridden if set to false by the developer.

I'm 50/50 on performing 1 along with any from 4-8. It creates two solutions to the same problem, but 1 is more straightforward. I'd leave it to the SDLC to decide.

As stated before though, a MEDIA app could put up an alert of information on screen that tells the user to connect a media pipe of somesort.

@khburdette I don't think that has anything to do with the specific issue called out. I would get logs to see why that's happening first as it would appear to be a separate problem. What happens when you change the Hello SDL hmi type to NONMEDIA? Same behavior? I would open another issue to further discuss what you're seeing.

[khburdette]

Yes, this also happens with NONMEDIA. I agree this needs to be another issue. I'll have to grab some logs

[theresalech]

An SDLC member reached out via email to confirm that they are comfortable with option 1, and 4-8, as they have assumed a Bluetooth A2DP connection was a prerequisite for SDL apps to register.

[khburdette]

I just tested with a Samsung Galaxy S9 running Android 8.0 and it doesn't support audio over AOA. Instead it sends the audio to A2DP.

My Pixel 3 XL running Android 9 does send the audio over AOA.

@theresalech, even if A2DP is a prerequisite, it needs to be very clear that different devices function differently, so having a media app connected via AOA will have different behavior depending on the device. I'm not sure how an OEM can figure out where the device is sending the audio to be able to actually play it.

[joeygrover]

The IVI system will have to support the Audio over AOA though for audio to get routed by the phone through AOA though right? So the IVI system would know it supports AOA audio and would follow the a priority (if AOA audio, use that. Else if A2DP use that). Or it only supports one and uses that one. The app developer doesn't specify where the audio is being streamed as it is handled by the OS (AFAIK).

I for sure don't believe A2DP should be a prerequisite, but I agree it does bring up a need for clarification around audio arbitration prioritization.

[khburdette]

I dug of some of our findings when we were looking to introduce audio over AOA for SYNC3:

Phone paired via bluetooth and connected via AOA:

• Samsung devices route audio through A2DP
• Pixel devices route audio through AOA

Phone NOT paired via bluetooth and connected via AOA:

• Samsung devices route audio through AOA
• Pixel devices route audio through AOA

So the issue is that where the audio is being streamed is handled by the OS, but different handsets have different behavior.

If the head unit only supports AOAv1, the devices do route the audio through A2DP, which seems to be the only thing consistent here.

[khburdette]

Can we rev the router service version and not allow older router services to connect via AOA? This way only AOA apps would be required to upgrade.

This would also mean that when other apps finally upgrade they can decide to use bluetooth only or benefit from one of the more permanent solutions listed above.

[joeygrover]

I'm not following your comment. You want to remove the AOA connecting ability for the current version of the router service and only allow it for new versions of the router service? If that's what you mean, that won't work. All apps would still have to upgrade to have the functionality removed from that version of the router service, as well as that's a breaking change. And even still, if an AOA app is installed with the newer router service that allows the AOA connection, all the other apps will still bind to it and we would be back in this situation.

[KainGC]

Hi @joeygrover,

Can you please summarize the latest idea and conversation regarding this issue here in the ticket?
I know that some detail was included in the latest meeting minutes but anyone looking just at this ticket does not know the current status and it could cause some confusion.