[Withdrawn] SDL 0324 - Function to control the priority of audio and video when switching between apps

[jordynmackool]

Hello SDL community,

The review of "SDL 0324 - Function to control the priority of audio and video when switching between apps" began on November 18, 2020, and continues through January 12, 2021. The proposal is available here:

https://github.com/smartdevicelink/sdl_evolution/blob/master/proposals/0324-Function-to-control-the-priority-of-audio-and-video-when-switching-between-apps.md

Reviews are an important part of the SDL evolution process. All reviews should be sent to the associated Github issue at:

#1095

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of SDL. When writing your review, here are some questions you might want to answer in your review:

• Is the problem being addressed significant enough to warrant a change to SDL?
• Does this proposal fit well with the feel and direction of SDL?
• If you have used competitors with a similar feature, how do you feel that this proposal compares to those?
• How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
  Please state explicitly whether you believe that the proposal should be accepted into SDL.

More information about the SDL evolution process is available at

https://github.com/smartdevicelink/sdl_evolution/blob/master/process.md

Thank you,
Jordyn Mackool

Program Manager - Livio
Jordyn@livio.io

[joeljfischer]

1. The motivation of this proposal is not at all clear. Uncommon terminology like "app status transition rule" is used and not defined. Please define your terms and provide examples. Despite reading this proposal I don't know what problem it's solving.

I have additional points, but this proposal is, in my opinion, un-reviewable until the above is fixed.

[Jack-Byrne]

1. @joeljfischer I agree terminology should be updated. My understanding is that this proposal wants to customize the rules that are documented in this proposal: https://github.com/smartdevicelink/sdl_evolution/blob/master/proposals/0150-video-streaming-state.md

Also I agree the motivation needs to be updated. Motivation should explain what problem this proposal solves.

When the SDL app is launched, SDL Core loads two configuration files, the default configuration and the customized one by the OEM. Then, SDL Core selects one of the status transition rules with each situation.

I don't agree with creating two separate configuration files for this feature. Any SDL Core customizations should be added to the existing smartDeviceLink.ini configuration file.

3. Examples of the configurations need to be added to the proposal. Examples should include key/values for configurations.

4. The state management of SDL core is already very complex and testing all of the current state transitions takes a lot of time. I believe adding this feature would make maintaining SDL Core's states management near impossible because of all of the possible state configurations that would need to be tested.

5. Do any of the example tables show cases where it might be useful to customize the hmiLevel, streaming, or audible states? It seems these tables only show existing behavior for switching states.

[Akihiro-Miyazaki]

@joeljfischer -san,
1.
Thank you for your comment.
Regarding to "app status transition rule", we will add the example, and will modify the Motivation like below.

For this reason, we define the rules for the app status transition such as "only one app can be FULL at a time", we propose to add a function in which OEMs can set the priority control of audio and video according to the rules of status transition.

Currently, the app behavior of when app switches is implemented in SDL Core, so it does not work as OEM intended.
In this proposal, by OEM is defining the configuration file, OEMs can control the apps behavior by themselves according to the rules of the app status transition when an app switches other app.

If our reply above is not answered to the intention of your comment, could you please explicitly show us the details.

@JackLivio -san,
1.
We will confirm the proposal documentation of SDL-0150, then we will reply on this matter.

We would like to reply No.3-5 comments, after they are in focus on each other in this matter.

Thank you for your advice.
Since the policy table has two files(preloaded_pt and policy table), we think the policy table is matched than smartDeviceLink.ini.
therefore, we will modify the word from configuration file to the policy table.

Thank you.

[jordynmackool]

The Steering Committee voted to keep this proposal in review on 2020-11-24 to allow for conversation to continue on the review issue.

[joeljfischer]

1.

For this reason, we define the rules for the app status transition such as "only one app can be FULL at a time", we propose to add a function in which OEMs can set the priority control of audio and video according to the rules of status transition.

I'm sorry, but I'm still struggling to understand this sentence. Do you mean something like, "we propose to add the ability to configure the rules SDL Core uses to change connected apps' video and audio HMI statuses?"

[Akihiro-Miyazaki]

@joeljfischer -san,
I am very sorry for my poor english.

If an app is launched during other app was launched, the rules such as which app is audible or which app is streamable and etc. are already implemented by SDL Core.

To be able to modify the rules by the OEMs themselves, I would like to move the rules to policy table.

[joeljfischer]

1. Thanks for the response, I think I understand what you are attempting to do with this proposal. I believe that the motivation needs to be made clearer in order to help others understand the problem that you are attempting to solve.

[Jack-Byrne]

2. It is possible to have configuration without using policies. This is done via the existing smartDeviceLink.ini config file. I do not recommend using policies for configuring the rules in this feature.

[jordynmackool]

To ensure all open items are addressed and make the comments easier to follow, please respond to the open items in one comment moving forward. To provide clarity of the current status of this proposal review, please see a summary of the items below:

Author to respond to the following open items:

2. It is possible to have configuration without using policies. This is done via the existing smartDeviceLink.ini config file. I do not recommend using policies for configuring the rules in this feature.

3. Examples of the configurations need to be added to the proposal. Examples should include key/values for configurations.

4. The state management of SDL core is already very complex and testing all of the current state transitions takes a lot of time. I believe adding this feature would make maintaining SDL Core's states management near impossible because of all of the possible state configurations that would need to be tested.

5. Do any of the example tables show cases where it might be useful to customize the hmiLevel, streaming, or audible states? It seems these tables only show existing behavior for switching states.

Agreed upon revisions:

1. Author to update the Motivation to explain what problem this proposal is solving.

[jordynmackool]

The Steering Committee voted to keep this proposal in review on 2020-12-01 to allow for the author to respond to the remaining open items, summarized in this comment.

[Akihiro-Miyazaki]

@JackLivio -san,
2.
Thank you for your comment.
We think there is no problem to store to the smartDeviceLink.ini file itself.
Just in case, we would like to confirm about using the smartDeviceLink.ini file.
If the OEMs want to modify the data for customization, do the OEMs modify the data in the smartDeviceLink.ini file directly?

Also, we consider that it needs the two tables.
One of them is for default data.
And the other is for customization by OEMs.
In this case, could you teach us where are better to store the tables?

The table example which we are considering is below.

In this proposal, since the SDL Core will work according to the table, I think it will reduce the processing load of SDL Core.

In this proposal, since it is possible for OEMs to modify audio and video output by themselves when an app is switched other app, OEMs can customize according to native specifications.

[Jack-Byrne]

2. Yes the smartDeviceLink.ini file is read by SDL Core each time the application starts. Any SDL Core integrator can make updates to the smartDeviceLink.ini to better match the needs of their system.

3. Ok thank you.

i) How should this table be represented in the smartDeviceLink.ini file?

ii) I think the selected audio and selected screen columns are redundant. These columns should not be needed since AUDIBLE and FULL states note which app should be heard/seen.

iii) The table does not include the impact of OnEventChanged notifications from the HMI.

i) I am not sure how it will reduce the processing load of SDL Core? What do you imagine the implementation of the sdl core state manager would like like to accomplish this?

ii) My concern is with the number of testable cases that will be needed by allowing customization of these options.

25 app type combinations * 4 App1 HMI level options * 3 App1 Audible options * 4 App2 HMI level options * 3 App2 Audible options = 3600 possible configurations for this feature. There are probably more combinations since OnEventChanged notifications are not included in the table.

Testing that this feature's behavior is true its configuration will be difficult. IMO allowing the customization of these fields will reduce the reliability of SDL's OnHMIStatus message.

I still would like to see a specific use case on why customizing these fields would be beneficial. What about the current behavior is not working for the authors integration?

[Akihiro-Miyazaki]

@JackLivio -san,
3i
We are considering quantifying these parameters and storing into the smartDeviceLink.ini.

3ii
I think so.
I will remove the selected audio and selected screen columns from the list above.

3iii
I am very sorry but I can not understand the relationship with OnEventChanged.
Could you please explain us?

4i
First of all, I had misunderstanding.
I thought the processing load.
I am very sorry.

4ii
We do not know if this test pattern is a lot compared others.
However, we think it should to address test pattern if it is beneficial.
We think you have the same feeling.

We think by clarifying, the behavior becomes easy to visible.

5
We think by being able to customize the behavior of audible and streamable by OEMs themselves, it is possible to get closer to the Native specifications.

[Jack-Byrne]

3i) Thank you, but I would like to see the .ini entries included in the proposal. See how ini updates were included in this proposal

3ii) 👍

3iii) This proposal has a chart that includes documented behavior for OnEventStatus

Picture of chart for reference:

OnEventChanged is a message sent by the HMI to let Core know when a native functionality of the HMI overrides what an SDL App might be doing. For example taking a phone call, switching to cd/radio, started embedded navigation will affect an App's HMI level, streamable, and audible statuses. How these events are handled should be mentioned in the proposal. Does the author wish to customize these events?

5. Could you please provide a specific example of how it would align better? For example, do you want two media apps to be audible at the same time? Two nav apps to be streamable at the same time?

Maybe there is an easier way to accomplish a solution instead of completely changing Core's hmi state logic with a high resolution configuration. Also, is the issue only with the rules for audible & streamable?

[jordynmackool]

The Steering Committee voted to keep this proposal in review on 2020-12-08 to allow for conversation to continue on the review issue.

[Akihiro-Miyazaki]

@JackLivio -san,
3i)
Thank you for your information.
We will consider like below.

;Describe about control the priority of audio and video when switching between apps.
;The structure is like below.
;
; (appHMIType of app1 and app2) = (HMI Level, audible status and streamable status of app1),(HMI Level, audible status and streamable status of app2)
;
;The following are description of each content.
;
;Indicate appHMIType of app1
; Media
; Navigation
; ProjectionTrue = Projection(isMedia = True)
; ProjectionFalse = Projection(isMedia = False)
; Other = App except above three apps(Media, Navigation, Projection)
;
;Indicate appHMIType of app2
; Media
; Navigation
; ProjectionTrue = Projection(isMedia = True)
; ProjectionFalse = Projection(isMedia = False)
; Other = App except above three apps(Media, Navigation, Projection)
;
;Indicate HMI LEVEL status of app1
; 0 = BACKGROUND
; 1 = FULL
; 2 = LIMITED
; 3 = NONE
;
;Indicate Audible status of app1
; 0 = NOT AUDIBLE
; 1 = AUDIBLE
;
;Indicate Streamable status of app1
; 0 = NOT STREAMABLE
; 1 = STREAMABLE
;
;Indicate HMI LEVEL status of app2
; 0 = BACKGROUND
; 1 = FULL
; 2 = LIMITED
; 3 = NONE
;
;Indicate Audible status of app2
; 0 = NOT AUDIBLE
; 1 = AUDIBLE
;
;Indicate Streamable status of app2
; 0 = NOT STREAMABLE
; 1 = STREAMABLE
MediaMedia = 0, 0, 0, 1, 1, 0
MediaNavigation = 2, 1, 0, 1, 1, 1
MediaProjectionTrue = 0, 0, 0, 1, 1, 0
MediaProjectionFalse = 2, 1, 0, 1, 0, 0
MediaOther = 2, 1, 0, 1, 0, 0
NavigationMedia = 2, 1, 1, 1, 1, 0
NavigationNavigation = 0, 0, 0, 1, 1, 1
NavigationProjectionTrue = 0, 0, 0, 1, 1, 1
NavigationProjectionFalse = 0, 1, 0, 1, 0, 1
NavigationOther = 2, 1, 1, 1, 0, 0
ProjectionTrueMedia = 2, 0, 1, 1, 1, 0
ProjectionTrueNavigation = 0, 0, 0, 1, 1, 1
ProjectionTrueProjectionTrue = 0, 0, 0, 1, 1, 1
ProjectionTrueProjectionFalse = 0, 0, 0, 1, 0, 1
ProjectionTrueOther = 2, 1, 1, 1, 0, 0
ProjectionFalseMedia = 2, 0, 1, 1, 1, 0
ProjectionFalseNavigation = 0, 0, 0, 1, 1, 1
ProjectionFalseProjectionTrue = 0, 0, 0, 1, 1, 1
ProjectionFalseProjectionFalse = 0, 0, 0, 1, 0, 1
ProjectionFalseOther = 2, 0, 1, 1, 0, 0
OtherMedia = 0, 0, 0, 1, 1, 0
OtherNavigation = 0, 0, 0, 1, 1, 1
OtherProjectionTrue = 0, 0, 0, 1, 1, 1
OtherProjectionFalse = 0, 0, 0, 1, 0, 1
OtherOther = 0, 0, 0, 1, 0, 0

If you have idea better than ours, could you please teach us?

3iii)
Thank you for your explanation.
We understood.
Do you mention that we should include how the events are controlled(handled) in this proposal.
Since this proposal describes which app's audible and streamable are selected in the SDL system, we assume the events from HU are out of the scope.

Also, I think that the control (handle) between SDL and HU is supported on the HU side.
We're proposing a specification inside SDL, so we're assuming it's out of scope.

[Akihiro-Miyazaki]

@JackLivio -san,
5.
I'm very sorry for this comment, but there is no specific example.
However, we believe that OEMs have specifications that require audible and streamable output in each case.
Therefore, it would be useful if the OEM could customize it.

[jordynmackool]

Please respond to all open items in one comment moving forward. To provide clarity of the current status of this proposal review, please see a summary of the items below:

Open Items:

3i.[Nexty] Thank you, but I would like to see the .ini entries included in the proposal. See how ini updates were included in this proposal.

3iii. [Nexty] Thank you for your explanation. We understood.
Do you mention that we should include how the events are controlled(handled) in this proposal.
Since this proposal describes which app's audible and streamable are selected in the SDL system, we assume the events from HU are out of the scope. Also, I think that the control (handle) between SDL and HU is supported on the HU side.
We're proposing a specification inside SDL, so we're assuming it's out of scope.

5. [Nexty] I'm very sorry for this comment, but there is no specific example. However, we believe that OEMs have specifications that require audible and streamable output in each case. Therefore, it would be useful if the OEM could customize it.

Agreed upon revisions:

1. Author to update the Motivation to explain what problem this proposal is solving.

3. Examples of the configurations need to be added to the proposal. Examples should include key/values for configurations.
(See 3 in this comment.)

ii. Remove the selected audio and selected screen columns from the list above.

2. See 2 in this comment - Add configurations to the ini file instead of policies.

Resolved, no action need:

4, 4i, 4ii

[Jack-Byrne]

i) Thank you for this. Will consider review after discussion from number 5.

iii) I would request that a note be added in the proposal that OnEventChange request hmistatus outcomes will use not be altered by these the customizations.

5. The desired outcome for the proposal is that multiple apps can have a streamable and audible state at the same time?

I don't think it is necessary to add detailed customization for all app types, hmi levels, and streaming states.

I think the desired outcome could be resolved by a single configuration.

smartDeviceLink.ini:

; When true, an app that gains an audible/streamable state will continue to stay 
; audible/streamable until the app is exited to hmi level none. 
; Multiple apps will be able to obtain audible/streamable states at the same time.
;
; When false, the default rules are used for managing audible/streamable states.

; persistStreamingState = false
persistStreamingState = true

[jordynmackool]

The Steering Committee voted to keep this proposal in review on 2020-12-15 to allow for the conversation to continue on the open items, summarized in this comment.

[Akihiro-Miyazaki]

@JackLivio -san,

3i.
Thank you for your suggestion.
However, although I described in No.5, I think your suggestion maybe not match our proposal.
If you can, it is help us that you teach us the advices based on our consideration.

3iii.
Thank you for your adveice.
We understood.
We will modify to add the comment to the comment below to the Potential downsides.

In this proposal, it describes which app's audible and streamable are selected in the SDL system.
Therefore, this proposal does not define audible and streamable including the events from HU such as OnEventChange request hmistatus outcomes.
Also, the events from HU will use not be altered by these the customizations.
The HMI should define as the system.

Does this comment match your requirement?

Thank you for your suggestion.
However, for example.
Precondition :

• persistStreamingState = true
• navigation app1 : Audible and Streamable
• navigation app2 : Audible and Streamable
  (1) The navigation app1 is worked on Full and audible and streamable.
  (2) The navigation app2 is launched on audible and streamble.

Our proposal: the navigation app1 moves to Background and not audible and not streamable.
Your suggestion: the navigation app1 moves to Background and audible and streamable.
Is it correct?
Our proposal looks like usual.
However, by adding the customized table, the system can behave through your suggestion.
In this way, this proposal aims to allow you to customize the table to the specifications for OEMs.
Therefore, I think your suggestion does not match ours.

[Jack-Byrne]

@Akihiro-Miyazaki

3i. I am sorry but I think the example table provided for the .ini config is not very readable and would be error prone. I am still trying to understand why all of these items need customizations. Continued in number 5.

3ii. 👍

5. My mistake I misunderstood the previous comment for what customizations were envisioned.

If you are proposing these changes without a reason for how customizing these options would be helpful, I will not be able to endorse this proposal. I do not see a reason for making large changes to the SDL Core code to implement a customizable table that is envisioned to keep the same behavior.

[Akihiro-Miyazaki]

@JackLivio -san,
3i.
We understand the .ini config is not very readable and would be error prone.
Thus, we need your help that you teach us the advices based on our consideration.

3ii.
I assume it is for reply of 3iii.
Is it correct?

Currently, as SDL, which screen is prioritized and which audio is prioritized when an app switches other app is implemented in SDL Core, but I assume that it does not define explicitly.
There is no problem written in .ini or policy table, but it is a merit that it is easy to understand which screen and audio are selected as SDL by the priority order listing, and I assume that it is also a merit that it can be customized for each OEM.
Since there are also screen and audio on the HU side, I assume it is necessary to implement on HU (HMI) which screen and audio will be output as the system in the end.

[Jack-Byrne]

@Akihiro-Miyazaki

3i). Blocked by discussion number 5.

3iii). Yes sorry I meant 3iii. 👍 .

5. I believe the behavior for which apps should be visible, streamable, and audible are well defined. Tables in this proposal describe all cases.

I am sorry but I do not see the merit in customizing app switching behavior.

Since there are also screen and audio on the HU side, I assume it is necessary to implement on HU (HMI) which screen and audio will be output as the system in the end.

We already have HMI RPC's that tell Core what context the app is being shown in by the HMI:

Show an app -> OnAppActivated
Hide an app -> OnAppDeactivated
Exit an app -> OnExitApplication

[Akihiro-Miyazaki]

@JackLivio -san,
5.
Thank you for teaching reference.
We will confirm this proposal and reply to you.
By the way, we have a question about proposal and specification documentation.
We recognize the smartdevicelink.com as the specification documentation.
Is it correct?
If correct, please teach us how to find out SDL 0150 proposal?
In the case of app services, the SDL 0167 proposal is described as the reference. How about the SDL 0150 proposal?

We already have HMI RPC's that tell Core what context the app is being shown in by the HMI:

We know these HMI RPC.
However, These are RPC for notification to SDL Core.
Thus, by receiving these RPC, SDL Core can confirm the behavior of each app, but SDL Core can not know the behavior which screen and audio will be output as the system.
Therefore, since HU(HMI) controls which screen and audio will be output, we showed the comment above.

[Jack-Byrne]

5. I am sorry that the documentation is fragmented for this feature, we can add the tables in the 150 proposal to the developer portal.

[jordynmackool]

To provide clarity of the current status of this proposal review, please see a summary of the items below:

Open Items:

3i. Blocked by 5.

5. See comment here.

Agreed upon revisions:

1. Author to update the Motivation to explain what problem this proposal is solving.

2. See 2 in this comment - Add configurations to the ini file instead of policies.

3. Examples of the configurations need to be added to the proposal. Examples should include key/values for configurations.
(See 3 in this comment.)

ii. Remove the selected audio and selected screen columns from the list above.

3iii. Add a note to the Potential downside:
In this proposal, it describes which app's audible and streamable are selected in the SDL system.
Therefore, this proposal does not define audible and streamable including the events from HU such as OnEventChange request hmistatus outcomes. Also, the events from HU will use not be altered by these the customizations. The HMI should define as the system.

Resolved, no action need:

4, 4i, 4ii

[jordynmackool]

As the author has requested additional time to respond to open item number 5 in this comment, which is blocking the other remaining open item (3i). The project maintainer suggests deferring this proposal to allow other review issues to be brought into review.

[jordynmackool]

The Steering Committee voted to defer this proposal on 2021-01-12 to allow the author additional time to respond to the open items summarized in this comment. The proposal will remain deferred until the author has acknowledged (via email to sdlc@smartdevicelink.com) that they are ready to respond.

[jordynmackool]

This proposal has been withdrawn per the author's request via email.