Feature/0296 possibility to update video streaming capabilities (#223)

* init commit * add Videosteaming Switching Example Notification to OnSystemCapabilityUpdated index * Add `appID` to OnAppCapabilityUpdated notification * process review comments * Apply suggestions from code review Co-authored-by: Shobhit Adlakha <[email protected]> * add OnAppCapabilityUpdated to config.json * Apply suggestions from code review Co-authored-by: Shobhit Adlakha <[email protected]> * update OnAppCapabilityUpdated.png Co-authored-by: Dubovyk <[email protected]> Co-authored-by: Shobhit Adlakha <[email protected]> Co-authored-by: Dmitriy Boltovskiy <[email protected]>
smartdevicelink · Feb 12, 2021 · fdb7116 · fdb7116
1 parent de5c401
commit fdb7116
Show file tree

Hide file tree

Showing 9 changed files with 239 additions and 0 deletions.
diff --git a/config.json b/config.json
@@ -133,6 +133,9 @@
             },
             {
               "name": "OnAppPropertiesChange"
+            },
+            {
+              "name": "OnAppCapabilityUpdated"
             }
           ]
         },

diff --git a/docs/BasicCommunication/OnAppCapabilityUpdated/assets/OnAppCapabilityUpdated.png b/docs/BasicCommunication/OnAppCapabilityUpdated/assets/OnAppCapabilityUpdated.png
diff --git a/docs/BasicCommunication/OnAppCapabilityUpdated/assets/OnAppCapabilityUpdated.txt b/docs/BasicCommunication/OnAppCapabilityUpdated/assets/OnAppCapabilityUpdated.txt
@@ -0,0 +1,26 @@
+participant App
+participant SDL
+participant HMI
+
+
+seq Obtaining VideoStreamingCapabilities during HMI initialization  
+
+SDL->HMI: UI.GetCapabilities
+HMI-->SDL:<align:center> UI.GetCapabilities\n(systemCapabilities=videoStreamingCapability,\n[additionalVideoStreamingCapabilities])
+end 
+
+note over App, HMI: RAI
+
+seq App receives VideoStreamingCapabilities supported by the system
+
+App->SDL:<align:center> GetSystemCapability\n(systemCapabilityType=VIDEO_STREAMING,\nsubscribe=true)
+SDL-->App:<align:center> GetSystemCapability\n(systemCapability=videoStreamingCapability,\n[additionalVideoStreamingCapabilities])
+
+end
+
+seq App provides supported VideoStreamingCapabilities
+
+App->>SDL:<align:center> OnAppCapabilityUpdated\n(appCapability=videoStreamingCapability,\n[additionalVideoStreamingCapabilities])
+SDL->>HMI:<align:center> BC.OnAppCapabilityUpdated\n(appCapability=videoStreamingCapability,\n[additionalVideoStreamingCapabilities])
+
+end
diff --git a/docs/BasicCommunication/OnAppCapabilityUpdated/index.md b/docs/BasicCommunication/OnAppCapabilityUpdated/index.md
@@ -0,0 +1,121 @@
+## OnAppCapabilityUpdated
+
+Type
+: Notification
+
+Sender
+: SDL
+
+Purpose
+: Inform the HMI that a specific application capability has changed
+
+### Notification
+
+SDL sends OnAppCapabilityUpdated notification to HMI to provide <abbr title="Video Streaming Capabilities">VSC</abbr> supported by application
+
+!!! may
+
+HMI may switch video streaming resolution according to values provided by application in OnAppCapabilityUpdated notification. (_see also [OnSystemCapabilityUpdated](../OnSystemCapabilityUpdated)_)
+
+If HMI doesn't receive OnAppCapabilityUpdated notification or receives OnAppCapabilityUpdated without `additionalVideoStreamingCapabilities`, HMI should not send OnSystemCapabilityUpdated notification and the user should not be able to change the resolution.
+!!!
+
+!!! note
+
+In case older application does not support sending `OnAppCapabilityUpdated` notification, HMI must treat this application as one that does not support dynamic resolution switching and allow only default screen mode for it.
+
+!!!
+
+#### Parameters
+
+|Name|Type|Mandatory|Additional|
+|:---|:---|:--------|:---------|
+|appCapability|[Common.AppCapability](../../common/structs/#appcapability)|true||
+|appID|Integer|true||
+
+### Sequence Diagrams
+
+|||
+Negotiation of Video Streaming Capabilities between HMI and Mobile Application
+![OnAppCapabilityUpdated](./assets/OnAppCapabilityUpdated.png)
+|||
+
+### JSON Message Examples
+
+#### Example Notification
+
+```json
+{
+   "jsonrpc":"2.0",
+   "method":"BasicCommunication.OnAppCapabilityUpdated",
+   "params":{
+      "appID":65544,
+      "appCapability":{
+         "appCapabilityType":"VIDEO_STREAMING",
+         "videoStreamingCapability":{
+            "hapticSpatialDataSupported":false,
+            "scale":2.5,
+            "additionalVideoStreamingCapabilities":[
+               {
+                  "scale":1,
+                  "hapticSpatialDataSupported":true,
+                  "preferredResolution":{
+                     "resolutionWidth":800,
+                     "resolutionHeight":350
+                  }
+               },
+               {
+                  "hapticSpatialDataSupported":false,
+                  "preferredResolution":{
+                     "resolutionWidth":320,
+                     "resolutionHeight":240
+                  }
+               },
+               {
+                  "hapticSpatialDataSupported":true,
+                  "preferredResolution":{
+                     "resolutionWidth":400,
+                     "resolutionHeight":480
+                  }
+               },
+               {
+                  "hapticSpatialDataSupported":true,
+                  "preferredResolution":{
+                     "resolutionWidth":640,
+                     "resolutionHeight":480
+                  }
+               },
+               {
+                  "hapticSpatialDataSupported":true,
+                  "preferredResolution":{
+                     "resolutionWidth":800,
+                     "resolutionHeight":240
+                  }
+               },
+               {
+                  "scale":2.5,
+                  "hapticSpatialDataSupported":false
+               },
+               {
+                  "scale":5,
+                  "hapticSpatialDataSupported":true
+               }
+            ],
+            "preferredResolution":{
+               "resolutionWidth":800,
+               "resolutionHeight":350
+            },
+            "diagonalScreenSize":10,
+            "maxBitrate":10000,
+            "pixelPerInch":150,
+            "supportedFormats":[
+               {
+                  "protocol":"RAW",
+                  "codec":"H264"
+               }
+            ]
+         }
+      }
+   }
+}
+```
diff --git a/.../OnSystemCapabilityUpdated/assets/OnSystemCapabilityUpdated_VIDEO_STREAMING.png b/.../OnSystemCapabilityUpdated/assets/OnSystemCapabilityUpdated_VIDEO_STREAMING.png
diff --git a/...munication/OnSystemCapabilityUpdated/assets/OnSystemCapabilityUpdated_VIDEO_STREAMING.txt b/...munication/OnSystemCapabilityUpdated/assets/OnSystemCapabilityUpdated_VIDEO_STREAMING.txt
@@ -0,0 +1,36 @@
+participant App
+participant SDL
+participant HMI
+
+==<align:center>Pre-conditions: 1.Streaming is in process\n2.App is subscribed to capabilities updates ==
+
+seq HMI notifies application of new Video Streaming Capabilities
+
+note over HMI:<align:center> User changes\nstreaming resolution
+
+HMI->HMI: Stop displaying video
+HMI->>SDL:<align:center> BC.OnSystemCapabilityUpdated\n(appID, videoStreamingCapability=VideoStreamingCapability)
+SDL->>App:<align:center> OnSystemCapabilityUpdated\n(videoStreamingCapability=VideoStreamingCapability)
+App->App: CallBack
+end
+
+seq Application stops streaming
+App->SDL: EndService (VIDEO)
+SDL-->App: EndServiceACK (VIDEO)
+SDL->HMI: Navi.StopStream
+HMI-->SDL: Navi.StopStream (SUCCESS)
+SDL->>HMI: Navi.OnVideoDataStreaming (false)
+end
+
+seq Application starts streaming after updating streaming content window according to the new VSCs
+App->SDL: StartService (VIDEO, new_streaming_params)
+SDL->HMI: Navi.SetVideoConfig (new_streaming_params)
+HMI-->SDL: Navi.SetVideoConfig (SUCCESS)
+SDL-->App: StartServiceACK (VIDEO, new_streaming_params)
+SDL->HMI: Navi.StartStream
+HMI-->SDL: Navi.StartStream (SUCCESS)
+App->SDL:<align:center> StreamingData ...\n(new_streaming_params)
+SDL->HMI: <align:center> StreamingData ...\n(new_streaming_params)
+SDL->>HMI: Navi.OnVideoDataStreaming (true)
+HMI->HMI: Display new video stream
+end
diff --git a/docs/BasicCommunication/OnSystemCapabilityUpdated/index.md b/docs/BasicCommunication/OnSystemCapabilityUpdated/index.md
@@ -18,13 +18,24 @@ Purpose
     * `menuName` and `templateTitle` should be included, if they are visible on HMI.
     * `mainField1-4`, `statusBar` and `mediaTrack` should not be included if OEM implemented the `WEB_VIEW` template without these text fields.
     * `availabletemplates`, `buttonCapabilities` and `imageTypeSupported` should reflect the general capabilities of the window/system.
+5. HMI sends OnSystemCapabilityUpdated(VIDEO_STREAMING) notification to notify a subscribed mobile application that video streaming resolution has switched.
 
 !!! NOTE
 
 Template `WEB_VIEW` is only allowed for the WebEngine applications with `AppHMIType`=`WEB_VIEW`.
 
 !!!
 
+#### Recommendations for how various screen modes have to be handled
+
+If the change is a Picture-in-Picture-type change, scale the current video stream to the size you desire without going through this flow. Touches should never be passed to the app. Either a system menu/buttons should be displayed when selected, or the selection should bring the user immediately back to the full-screen app.
+
+If the change is a split-screen type change (and cannot be handled by scaling the original video), then go through this flow. If the app does not support your custom split-resolution, either don't allow the user to put the app in split screen, or scale the video stream to a size that fits the window and use "black bars" to fill the rest of the window. Touches may be passed to the app in certain limited cases.
+
+ * The window must have a width of at least 3" and a height of at least 3".
+
+ * The touches must be offset for the developer so that 0,0 continues to be in the top-left corner of the app window. If these requirements are not met, either a system menu / buttons should be displayed when selected, or the selection should bring the user immediately back to the full-screen app.
+
 #### Parameters
 
 |Name|Type|Mandatory|Additional|
@@ -44,6 +55,11 @@ OnSystemCapabilityUpdated(APP_SERVICES, REMOVED)
 ![OnSystemCapabilityUpdated_REMOVED](./assets/OnSystemCapabilityUpdated_REMOVED.png)
 |||
 
+|||
+OnSystemCapabilityUpdated(VIDEO_STREAMING) Streaming Resolution Switching
+![OnSystemCapabilityUpdated_VIDEO_STREAMING](./assets/OnSystemCapabilityUpdated_VIDEO_STREAMING.png)
+|||
+
 ### JSON Message Examples
 
 #### Example Notification
@@ -98,3 +114,26 @@ OnSystemCapabilityUpdated(APP_SERVICES, REMOVED)
   }
 }
 ```
+
+#### Videosteaming Switching Example Notification
+
+```json
+{
+   "jsonrpc":"2.0",
+   "method":"BasicCommunication.OnSystemCapabilityUpdated",
+   "params":{
+      "appID":489377304,
+      "systemCapability":{
+         "systemCapabilityType":"VIDEO_STREAMING",
+         "videoStreamingCapability":{
+            "preferredResolution":{
+               "resolutionWidth":800,
+               "resolutionHeight":380
+            },
+            "maxBitrate":20000,
+            "scale":1
+         }
+      }
+   }
+}
+```
diff --git a/docs/Common/Enums/index.md b/docs/Common/Enums/index.md
@@ -1373,3 +1373,9 @@
 |ENABLE_INPUT_KEY_MASK|0||
 |DISABLE_INPUT_KEY_MASK|1||
 |USER_CHOICE_INPUT_KEY_MASK|2||
+
+### AppCapabilityType
+
+|Name|Value|Description|
+|:---|:----|:----------|
+|VIDEO_STREAMING|0||
diff --git a/docs/Common/Structs/index.md b/docs/Common/Structs/index.md
@@ -51,6 +51,7 @@
 |pixelPerInch|Float|false|minvalue: 0|PPI is the diagonal resolution in pixels divided by the diagonal screen size in inches.|
 |scale|Float|false|minvalue: 1<br>maxvalue: 10|The scaling factor the app should use to change the size of the projecting view.|
 |preferredFPS|integer|false|minvalue: 0<br>maxvalue: 2147483647|The preferred frame rate per second of the head unit.|
+|additionalVideoStreamingCapabilities|Common.VideoStreamingCapability|false|array: true<br>minsize: 1<br>maxsize: 100||
 
 ### DynamicUpdateCapabilities
 
@@ -1253,3 +1254,10 @@ There are no defined parameters for this struct
 |:---|:---|:--------|:---------|:----------|
 |keyboardLayout|[Common.KeyboardLayout](../enums/#keyboardlayout)|true|||
 |numConfigurableKeys|Integer|true|minvalue: 0<br>maxvalue: 10|Number of keys available for special characters, App can customize as per their needs.|
+
+### AppCapability
+
+|Name|Type|Mandatory|Additional|Description|
+|:---|:---|:--------|:---------|:----------|
+|appCapabilityType|[Common.AppCapabilityType](../enums/#appcapabilitytype)|true||Used as a descriptor of what data to expect in this struct. The corresponding param to this enum should be included and the only other param included|
+|videoStreamingCapability|Common.VideoStreamingCapability|false||Describes supported capabilities for video streaming|
-Original file line number
+Diff line change
@@ Expand Up / @@ -133,6 +133,9 @@ @@
                 },
                 {
                   "name": "OnAppPropertiesChange"
+                },
+                {
+                  "name": "OnAppCapabilityUpdated"
                 }
               ]
             },
@@ Expand Down @@