[SDL 0173] - Read Generic Network Signal Data Implementation #2977

LuxoftAKutsan · 2019-07-29T15:18:53Z

This PR is ready for review.

Risk

This PR makes major API changes.

Testing Plan

Manual testing is done with smartdevicelink/sdl_hmi#210
Automated testing is done with smartdevicelink/sdl_atf_test_scripts#2217
Testing reports attached : RGNSD_test_reports.zip

Summary

Implementation of SDL evolution issue: smartdevicelink/sdl_evolution#768

Enhancements

Implemented generating of Policy enums from RPC spec xml. This is done to avoid hight coupling of generated API schema to application manager and policies. Also was done the additional generation of schemas to validate enum string according to RPC spec.
VehicleInfo commands and command factory refactored to get struct with the list of dependencies in constructor.

Bug Fixes

Known issues

GetPolicyConfigurationData is not able to provide access to ConsumerFriendlyMessages:messages section of the policy table, GetUserFriendlyMessage RPC should be used for it. Quick fox is available Fix messages conversion to json #2985. But this fix has drawbacks describe in Fix messages conversion to json #2985 (comment). After merging feature PR minor but should be created and an appropriate solution should be discussed.

CLA

I have signed the CLA

LuxoftAKutsan · 2019-07-29T15:40:05Z

@atiwari9 please approve feature implementation

atiwari9 · 2019-07-29T15:42:04Z

Ford has reviewed and Approved the changes. Ready for Livio review

src/appMain/sdl_preloaded_pt.json

...nents/application_manager/include/application_manager/policies/custom_vehicle_data_povider.h

...ger/rpc_plugins/sdl_rpc_plugin/src/commands/hmi/sdl_get_policy_configuration_data_request.cc

...c_plugins/vehicle_info_plugin/include/vehicle_info_plugin/custom_vehicle_data_manager_impl.h

src/components/rpc_base/include/rpc_base/rpc_base_json_inl.h

tools/InterfaceGenerator/generator/parsers/JSONRPC.py

tools/InterfaceGenerator/generator/parsers/RPCBase.py

tools/InterfaceGenerator/generator/generators/PolicyTypes.py

jacobkeeler · 2019-08-23T20:13:24Z

A few error flows that I found while testing

any mode
- Connect app1
- Send SubscribeVehicleData with any allowed vehicle data type: SUCCESS response is received
- Disconnect app1, then reconnect again
- Send SubscribeVehicleData with any allowed vehicle data type: expected SUCCESS, got DATA_NOT_AVAILABLE
PROPRIETARY mode
- Add custom vehicle data to preloaded policy table
- Start Core
- Connect an app with custom vehicle data permissions, then restart Core while still connected. Repeat this process several times (shouldn't need more than 10)
- Check contents of policy table, vehicle_data_item_definition and
  vehicle_date_item_parameters are empty

This issue seems to be tied to the following message in the Core logs, and as implied by the multiple restarts it only happens sometimes:

ERROR [23 Aug 2019 15:55:14,244][0x7f8ca7fff700][Policy] /home/jacobkeeler/sdl_core/src/components/policy/policy_regular/src/sql_pt_representation.cc:2609 SQLPTRepresentation::InsertVehicleDataItem: Vehicle data item is not initialized.
TRACE [23 Aug 2019 15:55:14,244][0x7f8ca7fff700][Policy] /home/jacobkeeler/sdl_core/src/components/policy/policy_regular/src/sql_pt_representation.cc:1754 SQLPTRepresentation::SaveVehicleDataItems: Exit
TRACE [23 Aug 2019 15:55:14,244][0x7f8ca7fff700][Policy] /home/jacobkeeler/sdl_core/src/components/policy/policy_regular/src/sql_pt_representation.cc:1728 SQLPTRepresentation::SaveVehicleData: Exit

Edit: revised flow of issue 2

.../application_manager/rpc_plugins/vehicle_info_plugin/src/custom_vehicle_data_manager_impl.cc

LuxoftAKutsan · 2019-08-24T09:59:05Z

@jacobkeeler
Unable to reproduce the issues you specified.
Tried manually and also with the attached script.
Could you please attach logs for analysis?

All other comments addressed

jacobkeeler · 2019-08-26T16:14:34Z

Issue 1: SmartDeviceLinkCore.log

Issue 2: SmartDeviceLinkCore.log

tools/InterfaceGenerator/generator/generators/PolicyTypes.py

LuxoftAKutsan · 2019-08-27T08:53:32Z

Issue 1 is an HMI issue fixed in smartdevicelink/sdl_hmi@cb240f4

Issue 2: It is known SDL issue: #2282
With fix available: #2501
Should we include this fix in this feature?

jacobkeeler · 2019-08-27T14:29:24Z

@LuxoftAKutsan
Looks like both of the links you provided are related to issue 1.

Issue 2 is a policy table corruption issue, which is completely unrelated to the HMI logic and #2282.

Considering it is an existing issue, I will review #2501 separately, rather than including it in this feature

LuxoftAKutsan · 2019-08-27T14:35:00Z

@jacobkeeler

Issue 2 :
Should I include already an existing fix for #2282: #2501 to develop into feature branch?

jacobkeeler · 2019-08-27T14:54:31Z

#2282 is not related to issue 2, as mentioned.

#2501 should remain targeted against develop

jacobkeeler · 2019-08-28T14:33:29Z

@LuxoftAKutsan Are you still investigating the second issue?

PROPRIETARY mode

Add custom vehicle data to preloaded policy table

Start Core

Connect an app with custom vehicle data permissions, then restart Core while still connected. Repeat this process several times (shouldn't need more than 10)

Check contents of policy table, vehicle_data_item_definition and
vehicle_date_item_parameters are empty

This issue seems to be tied to the following message in the Core logs, and as implied by the multiple restarts it only happens sometimes:
ERROR [23 Aug 2019 15:55:14,244][0x7f8ca7fff700][Policy] /home/jacobkeeler/sdl_core/src/components/policy/policy_regular/src/sql_pt_representation.cc:2609 SQLPTRepresentation::InsertVehicleDataItem: Vehicle data item is not initialized.
TRACE [23 Aug 2019 15:55:14,244][0x7f8ca7fff700][Policy] /home/jacobkeeler/sdl_core/src/components/policy/policy_regular/src/sql_pt_representation.cc:1754 SQLPTRepresentation::SaveVehicleDataItems: Exit
TRACE [23 Aug 2019 15:55:14,244][0x7f8ca7fff700][Policy] /home/jacobkeeler/sdl_core/src/components/policy/policy_regular/src/sql_pt_representation.cc:1728 SQLPTRepresentation::SaveVehicleData: Exit

alexkutsan · 2019-08-28T15:24:09Z

@jacobkeeler we reproduced the issue, found the root cause.
The fix is ready, regression check is in progress.

alexkutsan · 2019-08-28T15:41:47Z

@jacobkeeler please see the fix for Issue 2 :
6564f61

Right now we are in progress of rebasing feature branch on the latest develop and resolve conflicts.
Checking that feature tests are not affected by new commits in develop.
The feature does not introduce regression in delivered features.

Looks like all comments addressed. Please let me know if you have additional questions.
And are you OK with forcing push to feature branch (rebased to develop with commit squashing) after your approval?

jacobkeeler · 2019-08-28T18:56:07Z

@LuxoftAKutsan I'm fine with squashing and merging, I'd prefer to avoid force pushing before that though.

jacobkeeler · 2019-08-29T17:10:43Z

@LuxoftAKutsan please fix merge conflicts

alexkutsan · 2019-08-29T18:54:16Z

@jacobkeeler Additional merge conflicts were found after delivery of RPC message protection to develop branch.

Conflicts are rather significant and should be resolved carefully to avoid regression in both features.
RPC message protection and RGNSD tests are in regression area, so should be checked.

Conflicts resolving will be ready for tomorrow.

Policy manager HMI level enum reorder

Set default version of custom vehicle data mapping

SQL storage implementation for VehicleDataItems Fix validation of double values in policy table Change isDouble to isNumeric during validation double values "0" should be validated as correct double value Add validation vehicle data during PTU Send in PT snapshot only version of custom VDI - During snapshot generation remove vehicle data items section - Add validation for Policy table depended on PT type Fix wrong check for vehicle data snapshot Allow empty vehicle data in PTU Change max value of string for URL from 255 to INT_MAX

Vehicle info params added to hmi commands

Allow unknown params for Subscribe VD

Fixes : #2975

GetPolicyCOnfigurationData Implementation

Fixes : #2961 fix sending OnPermissionChanged after PTU with removing functional group. OnPermissionChange notification is now sent after applying PTU Replace ModuleConfig with Whole PT

Fixes #2962 Add additional check for functional group content before OnPermissionChangeNotification. * rename comparing functions to make more clear their return value sense * change variable names * extra check for null values in HasNewGroups() New unit test to cover changes within functional group

Change max value url for external policy fixes for external flow * since unti validation for the database * empty vehicle data validation

Introduces validation of vehicle_data_item name and key validation: * they should not contain spaces; * they should not be empty or consist only spaces; * they should not contain invalid chars like '!@#$%^&*'.

* Avoid converting to Json non object values Json::write adds additional `"` to string type as json value * Fix unit tests

to Subscribe, Unsubscribe vehicle data

Extract SMember from CObjectSchemaItem Add methods to ISchemaItem (Using composite pattern) - GetMemberSchemaItem - AddMemberSchemaItem Add implementation of VehicleDataItemSchema class Add appropriate unit tests Add creation of vehicle_data items schemes on policy event

alexkutsan · 2019-08-30T13:01:51Z

@jacobkeeler feature branch rebased on latest develop

LuxoftAKutsan changed the title ~~Feature/read generic network signal data~~ [SDL 0173] - Read Generic Network Signal Data Implementation Jul 29, 2019

mkorniichuk force-pushed the feature/read_generic_network_signal_data branch from 60b281d to 3ade2bc Compare August 1, 2019 11:39

LuxoftAKutsan mentioned this pull request Aug 8, 2019

Fix messages conversion to json #2985

Closed

1 task

jacobkeeler requested changes Aug 8, 2019

View reviewed changes

jacobkeeler requested changes Aug 23, 2019

View reviewed changes

.../application_manager/rpc_plugins/vehicle_info_plugin/src/custom_vehicle_data_manager_impl.cc Outdated Show resolved Hide resolved

.../application_manager/rpc_plugins/vehicle_info_plugin/src/custom_vehicle_data_manager_impl.cc Outdated Show resolved Hide resolved

jacobkeeler reviewed Aug 26, 2019

View reviewed changes

tools/InterfaceGenerator/generator/generators/PolicyTypes.py Outdated Show resolved Hide resolved

jacobkeeler approved these changes Aug 28, 2019

View reviewed changes

LitvinenkoIra force-pushed the feature/read_generic_network_signal_data branch from 6564f61 to 13465f4 Compare August 29, 2019 17:21

LuxoftAKutsan added 7 commits August 30, 2019 11:54

Generate policy enums from MIBILE_API

7ca13ca

Policy manager HMI level enum reorder

Add RPC vehicle data and endpoint_properties to sdl_preloaded_pt

cf6bfe3

Set default version of custom vehicle data mapping

Make Policy Handler forward declared to avoid high coupling

e8d62de

CustomVehicleDataManager implementation

e3c7460

Add vehicle_info_command_params for vehicle info plugin

da51509

Vehicle info params added to hmi commands

Process CustomVehicleData in GetVehicleData request

918452c

LuxoftAKutsan and others added 21 commits August 30, 2019 14:14

Fix for message convertion to SO and back to message

6971204

Allow unknown params for Subscribe VD

Fix resumption of subscriptions

d36ffab

Fixes : #2975

Add GetPolicyConfigurationData

0f77029

GetPolicyCOnfigurationData Implementation

Fix OnPermissionChangeNotif data corruption

7e5719c

Fixes : #2961 fix sending OnPermissionChanged after PTU with removing functional group. OnPermissionChange notification is now sent after applying PTU Replace ModuleConfig with Whole PT

Add CustomVehicleData functionality for POLICY_EXTERNAL flow

94b100f

Change max value url for external policy fixes for external flow * since unti validation for the database * empty vehicle data validation

Convert data_type to VehicleDataType enum value

a4584d0

Validation of VehicleDataItem name and key

a147200

Introduces validation of vehicle_data_item name and key validation: * they should not contain spaces; * they should not be empty or consist only spaces; * they should not contain invalid chars like '!@#$%^&*'.

Fix clean up of DB iv vehicle_data.schema_items are empty in PT

4734d3b

Avoid converting to Json non object values (#2982)

027599c

* Avoid converting to Json non object values Json::write adds additional `"` to string type as json value * Fix unit tests

Fix spelling and remove some redundant code

23d8d83

Parameter enum is removed from enums in order to be generated from API

cfc4be8

Implementation of new field in mobile API for response

ef82b5d

to Subscribe, Unsubscribe vehicle data

Update code generator

3714de7

Add engineOilLife to HMI_API

12c6559

Remove deprecated RPC GetUrls due to major version update

071ef6f

Fix generated enums to keep Google coding style

d49a130

Remove CollectRPCSpecVDItems method

1de81a0

Check group content in a separate method

d258471

DeleteVehicleDataItems without transaction

ce477b8

mkorniichuk force-pushed the feature/read_generic_network_signal_data branch from 13465f4 to ce477b8 Compare August 30, 2019 12:54

jacobkeeler approved these changes Aug 30, 2019

View reviewed changes

jacobkeeler merged commit 0186b8c into develop Aug 30, 2019

This was referenced Aug 30, 2019

[SDL 0173] Read Generic Network Signal Data #2840

Closed

engineOilLife parameter is missed for OnVehicleData notification in HMI_API.xml #2992

Closed

SDL sends empty SubscribeVehicleData request to HMI in case of resumption #2975

Closed

jacobkeeler mentioned this pull request Sep 24, 2019

SDL does not send OnPermissionChange after PTU with updated parameters list in assigned group for app #2962

Closed

AByzhynar deleted the feature/read_generic_network_signal_data branch February 27, 2020 08:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SDL 0173] - Read Generic Network Signal Data Implementation #2977

[SDL 0173] - Read Generic Network Signal Data Implementation #2977

LuxoftAKutsan commented Jul 29, 2019 •

edited

Loading

LuxoftAKutsan commented Jul 29, 2019

atiwari9 commented Jul 29, 2019

jacobkeeler commented Aug 23, 2019 •

edited

Loading

LuxoftAKutsan commented Aug 24, 2019

jacobkeeler commented Aug 26, 2019

LuxoftAKutsan commented Aug 27, 2019 •

edited

Loading

jacobkeeler commented Aug 27, 2019 •

edited

Loading

LuxoftAKutsan commented Aug 27, 2019 •

edited

Loading

jacobkeeler commented Aug 27, 2019 •

edited

Loading

jacobkeeler commented Aug 28, 2019

alexkutsan commented Aug 28, 2019

alexkutsan commented Aug 28, 2019 •

edited

Loading

jacobkeeler commented Aug 28, 2019

jacobkeeler commented Aug 29, 2019

alexkutsan commented Aug 29, 2019 •

edited

Loading

alexkutsan commented Aug 30, 2019

[SDL 0173] - Read Generic Network Signal Data Implementation #2977

[SDL 0173] - Read Generic Network Signal Data Implementation #2977

Conversation

LuxoftAKutsan commented Jul 29, 2019 • edited Loading

Risk

Testing Plan

Summary

Enhancements

Bug Fixes

Known issues

CLA

LuxoftAKutsan commented Jul 29, 2019

atiwari9 commented Jul 29, 2019

jacobkeeler commented Aug 23, 2019 • edited Loading

LuxoftAKutsan commented Aug 24, 2019

jacobkeeler commented Aug 26, 2019

LuxoftAKutsan commented Aug 27, 2019 • edited Loading

jacobkeeler commented Aug 27, 2019 • edited Loading

LuxoftAKutsan commented Aug 27, 2019 • edited Loading

jacobkeeler commented Aug 27, 2019 • edited Loading

jacobkeeler commented Aug 28, 2019

alexkutsan commented Aug 28, 2019

alexkutsan commented Aug 28, 2019 • edited Loading

jacobkeeler commented Aug 28, 2019

jacobkeeler commented Aug 29, 2019

alexkutsan commented Aug 29, 2019 • edited Loading

alexkutsan commented Aug 30, 2019

LuxoftAKutsan commented Jul 29, 2019 •

edited

Loading

jacobkeeler commented Aug 23, 2019 •

edited

Loading

LuxoftAKutsan commented Aug 27, 2019 •

edited

Loading

jacobkeeler commented Aug 27, 2019 •

edited

Loading

LuxoftAKutsan commented Aug 27, 2019 •

edited

Loading

jacobkeeler commented Aug 27, 2019 •

edited

Loading

alexkutsan commented Aug 28, 2019 •

edited

Loading

alexkutsan commented Aug 29, 2019 •

edited

Loading