Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch DataModel::Provider interface to default enabled (except on linux where it stays as check) #36042

Merged
merged 152 commits into from
Oct 31, 2024

Conversation

andy31415
Copy link
Contributor

@andy31415 andy31415 commented Oct 11, 2024

This will switch from using ember-compatibility-functions.cpp to using the codegen datamodel provider.

PR should show FLASH and RAM sideffect as well as if any compilation issues may occur.

Changes

  • enable data model interface usage by default (except linux where the state remains "check" and runs both in parallel)
  • Fix build issues and bugs (some split in other PRs). Specifically I have found:
    • k32w0 is very close to full flash, so enabled progress-level logs and above only (no detail)
    • fixed bug in caching being maintained across dynamic endpoint enable/disable (if the same endpoint id is re-used, the same path may reference freed clusters)

Size notes

@andy31415 andy31415 requested a review from a team as a code owner October 11, 2024 15:05
Copy link

semanticdiff-com bot commented Oct 11, 2024

Review changes with  SemanticDiff

@andy31415
Copy link
Contributor Author

chip-k32w0x-light-example does not fit in flash anymore ... that is not good at all.

Looking through the rest of the failures and will have separate PRs to fix them.

@andy31415 andy31415 marked this pull request as draft October 11, 2024 16:37
@andy31415
Copy link
Contributor Author

Converted to draft: we need to figure out all build failures and even then figure out how to manage flash increases.

… with dynamic dispatcher. Dynamic dispatcher seems to only care about commands so implemented all the rest as empty, which is not ideal
@github-actions github-actions bot added controller icd Intermittently Connected Devices labels Oct 11, 2024
@project-chip project-chip deleted a comment from github-actions bot Oct 11, 2024
@project-chip project-chip deleted a comment from github-actions bot Oct 11, 2024
@project-chip project-chip deleted a comment from github-actions bot Oct 11, 2024
@project-chip project-chip deleted a comment from github-actions bot Oct 11, 2024
…ogress logging and up (no detail/automation)
@project-chip project-chip deleted a comment from github-actions bot Oct 28, 2024
@project-chip project-chip deleted a comment from github-actions bot Oct 28, 2024
@project-chip project-chip deleted a comment from github-actions bot Oct 28, 2024
@project-chip project-chip deleted a comment from github-actions bot Oct 28, 2024
@project-chip project-chip deleted a comment from github-actions bot Oct 28, 2024
@project-chip project-chip deleted a comment from github-actions bot Oct 29, 2024
Copy link

github-actions bot commented Oct 29, 2024

PR #36042: Size comparison from 3d80093 to 5e230fa

Increases above 0.2%:

platform target config section 3d80093 5e230fa change % change
bl702 lighting-app bl702+eth FLASH 647346 649564 2218 0.3
RAM 25233 25321 88 0.3
bl702+wifi FLASH 824910 827144 2234 0.3
RAM 13965 14053 88 0.6
bl706+mfd+rpc+littlefs FLASH 1053644 1056804 3160 0.3
RAM 23821 23901 80 0.3
bl702l lighting-app bl702l+mfd+littlefs RAM 16468 16556 88 0.5
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 836396 838644 2248 0.3
pump-app LP_EM_CC1354P10_6 FLASH 768388 770444 2056 0.3
pump-controller-app LP_EM_CC1354P10_6 FLASH 752648 754664 2016 0.3
cyw30739 light CYW30739B2-P5-EVK-01 FLASH 678365 680305 1940 0.3
CYW30739B2-P5-EVK-02 FLASH 698217 700157 1940 0.3
CYW30739B2-P5-EVK-03 FLASH 698217 700157 1940 0.3
CYW930739M2EVB-02 FLASH 655153 657093 1940 0.3
light-switch CYW30739B2-P5-EVK-01 FLASH 614813 616577 1764 0.3
CYW30739B2-P5-EVK-02 FLASH 634449 636205 1756 0.3
CYW30739B2-P5-EVK-03 FLASH 634449 636205 1756 0.3
lock CYW30739B2-P5-EVK-01 FLASH 634173 635881 1708 0.3
CYW30739B2-P5-EVK-02 FLASH 653889 655597 1708 0.3
CYW30739B2-P5-EVK-03 FLASH 653889 655597 1708 0.3
thermostat CYW30739B2-P5-EVK-01 FLASH 609429 611253 1824 0.3
CYW30739B2-P5-EVK-02 FLASH 629281 631113 1832 0.3
CYW30739B2-P5-EVK-03 FLASH 629281 631113 1832 0.3
efr32 lock-app BRD4187C FLASH 926100 928524 2424 0.3
nrfconnect all-clusters-app nrf7002dk_nrf5340_cpuapp FLASH 885884 889176 3292 0.4
lighting-app nrf52840dk_nrf52840+rpc FLASH 883684 886320 2636 0.3
nrf52840dongle_nrf52840 FLASH 823976 826076 2100 0.3
nrf5340dk_nrf5340_cpuapp FLASH 780240 782340 2100 0.3
lock-app nrf5340dk_nrf5340_cpuapp FLASH 732352 734236 1884 0.3
pump-app nrf52840dk_nrf52840 FLASH 759568 761512 1944 0.3
pump-controller-app nrf52840dk_nrf52840 FLASH 744768 746652 1884 0.3
nxp contact k32w0+release FLASH 582280 584192 1912 0.3
mcxw71+release FLASH 596768 598728 1960 0.3
light k32w1+release FLASH 682976 685096 2120 0.3
lock mcxw71+release FLASH 746568 748584 2016 0.3
qpg lighting-app qpg6105+debug FLASH 660376 662408 2032 0.3
lock-app qpg6105+debug FLASH 618460 620380 1920 0.3
stm32 light STM32WB5MM-DK FLASH 481672 483776 2104 0.4
telink bridge-app tlsr9258a FLASH 680868 683210 2342 0.3
contact-sensor-app tlsr9528a_retention FLASH 620466 622578 2112 0.3
light-switch-app-ota-shell-factory-data tlsr9528a FLASH 708404 710598 2194 0.3
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 625310 627802 2492 0.4
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 811320 814362 3042 0.4
tizen all-clusters-app arm unknown 4912 4932 20 0.4
FLASH 1722268 1732856 10588 0.6
RAM 90140 90608 468 0.5
chip-tool-ubsan arm unknown 10792 10820 28 0.3
FLASH 18157678 18223782 66104 0.4
RAM 7915516 7961952 46436 0.6
Full report (79 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
platform target config section 3d80093 5e230fa change % change
bl602 lighting-app bl602+mfd+littlefs+rpc FLASH 1349924 1352490 2566 0.2
RAM 104120 104080 -40 -0.0
bl702 lighting-app bl702+eth FLASH 647346 649564 2218 0.3
RAM 25233 25321 88 0.3
bl702+wifi FLASH 824910 827144 2234 0.3
RAM 13965 14053 88 0.6
bl706+mfd+rpc+littlefs FLASH 1053644 1056804 3160 0.3
RAM 23821 23901 80 0.3
bl702l lighting-app bl702l+mfd+littlefs FLASH 974428 976714 2286 0.2
RAM 16468 16556 88 0.5
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 836396 838644 2248 0.3
RAM 123676 123628 -48 -0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 821824 823848 2024 0.2
RAM 125556 125516 -40 -0.0
pump-app LP_EM_CC1354P10_6 FLASH 768388 770444 2056 0.3
RAM 114048 113984 -64 -0.1
pump-controller-app LP_EM_CC1354P10_6 FLASH 752648 754664 2016 0.3
RAM 114240 114192 -48 -0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 626450 627994 1544 0.2
RAM 206132 205780 -352 -0.2
lock CC3235SF_LAUNCHXL FLASH 666426 667730 1304 0.2
RAM 206284 205932 -352 -0.2
cyw30739 light CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 678365 680305 1940 0.3
RAM 78692 78684 -8 -0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 698217 700157 1940 0.3
RAM 81324 81324 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 698217 700157 1940 0.3
RAM 81324 81324 0 0.0
CYW930739M2EVB-02 unknown 2040 2040 0 0.0
FLASH 655153 657093 1940 0.3
RAM 73760 73752 -8 -0.0
light-switch CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 614813 616577 1764 0.3
RAM 71644 71644 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 634449 636205 1756 0.3
RAM 74196 74188 -8 -0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 634449 636205 1756 0.3
RAM 74196 74188 -8 -0.0
lock CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 634173 635881 1708 0.3
RAM 74692 74692 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 653889 655597 1708 0.3
RAM 77244 77236 -8 -0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 653889 655597 1708 0.3
RAM 77244 77236 -8 -0.0
thermostat CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 609429 611253 1824 0.3
RAM 68780 68772 -8 -0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 629281 631113 1832 0.3
RAM 71412 71412 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 629281 631113 1832 0.3
RAM 71412 71412 0 0.0
efr32 lock-app BRD4187C FLASH 926100 928524 2424 0.3
RAM 159920 159916 -4 -0.0
BRD4338a FLASH 741208 742464 1256 0.2
RAM 232580 232572 -8 -0.0
window-app BRD4187C FLASH 1018008 1019224 1216 0.1
RAM 128264 128260 -4 -0.0
esp32 all-clusters-app c3devkit DRAM 95256 95328 72 0.1
FLASH 1538710 1541790 3080 0.2
IRAM 82538 82542 4 0.0
m5stack DRAM 116192 116256 64 0.1
FLASH 1548950 1550542 1592 0.1
IRAM 117039 117039 0 0.0
linux air-purifier-app debug unknown 4688 4688 0 0.0
FLASH 2736743 2736743 0 0.0
RAM 129616 129616 0 0.0
all-clusters-app debug unknown 5528 5528 0 0.0
FLASH 6039048 6039048 0 0.0
RAM 523872 523872 0 0.0
all-clusters-minimal-app debug unknown 5424 5424 0 0.0
FLASH 5371642 5371642 0 0.0
RAM 242448 242448 0 0.0
bridge-app debug unknown 5408 5408 0 0.0
FLASH 4705204 4705204 0 0.0
RAM 218400 218400 0 0.0
chip-tool debug unknown 5960 5960 0 0.0
FLASH 12979062 12979064 2 0.0
RAM 584466 584466 0 0.0
chip-tool-ipv6only arm64 unknown 21416 21416 0 0.0
FLASH 11114184 11114184 0 0.0
RAM 635400 635400 0 0.0
fabric-admin debug unknown 5792 5792 0 0.0
FLASH 11308853 11308853 0 0.0
RAM 584922 584922 0 0.0
fabric-bridge-app debug unknown 4632 4632 0 0.0
FLASH 4531634 4531634 0 0.0
RAM 205368 205368 0 0.0
fabric-sync debug unknown 4840 4840 0 0.0
FLASH 5244117 5244117 0 0.0
RAM 465432 465432 0 0.0
lighting-app debug+rpc+ui unknown 6056 6056 0 0.0
FLASH 5646289 5646289 0 0.0
RAM 228520 228520 0 0.0
lock-app debug unknown 5344 5344 0 0.0
FLASH 4754080 4754080 0 0.0
RAM 204520 204520 0 0.0
ota-provider-app debug unknown 4720 4720 0 0.0
FLASH 4380820 4380820 0 0.0
RAM 198240 198240 0 0.0
ota-requestor-app debug unknown 4656 4656 0 0.0
FLASH 4519860 4519860 0 0.0
RAM 202808 202808 0 0.0
shell debug unknown 4216 4216 0 0.0
FLASH 3063773 3063773 0 0.0
RAM 160336 160336 0 0.0
thermostat-no-ble arm64 unknown 9456 9456 0 0.0
FLASH 4123816 4123816 0 0.0
RAM 242936 242936 0 0.0
tv-app debug unknown 5624 5624 0 0.0
FLASH 5980517 5980517 0 0.0
RAM 596304 596304 0 0.0
tv-casting-app debug unknown 5208 5208 0 0.0
FLASH 11187469 11187469 0 0.0
RAM 694552 694552 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 914348 916444 2096 0.2
RAM 143357 143281 -76 -0.1
nrf7002dk_nrf5340_cpuapp FLASH 885884 889176 3292 0.4
RAM 141496 141468 -28 -0.0
all-clusters-minimal-app nrf52840dk_nrf52840 FLASH 847916 850016 2100 0.2
RAM 142265 142189 -76 -0.1
light-switch-app nrf52840dk_nrf52840 FLASH 791740 793672 1932 0.2
RAM 132331 132259 -72 -0.1
nrf7002dk_nrf5340_cpuapp FLASH 936144 938028 1884 0.2
RAM 130904 130832 -72 -0.1
lighting-app nrf52840dk_nrf52840+rpc FLASH 883684 886320 2636 0.3
RAM 146749 146677 -72 -0.0
nrf52840dongle_nrf52840 FLASH 823976 826076 2100 0.3
RAM 154611 154539 -72 -0.0
nrf5340dk_nrf5340_cpuapp FLASH 780240 782340 2100 0.3
RAM 145759 145687 -72 -0.0
nrf7002dk_nrf5340_cpuapp FLASH 936144 938028 1884 0.2
RAM 130904 130832 -72 -0.1
lock-app nrf52840dk_nrf52840 FLASH 806800 808688 1888 0.2
RAM 132886 132814 -72 -0.1
nrf5340dk_nrf5340_cpuapp FLASH 732352 734236 1884 0.3
RAM 133016 132944 -72 -0.1
nrf7002dk_nrf5340_cpuapp FLASH 952140 953976 1836 0.2
RAM 131459 131387 -72 -0.1
pump-app nrf52840dk_nrf52840 FLASH 759568 761512 1944 0.3
RAM 131677 131605 -72 -0.1
pump-controller-app nrf52840dk_nrf52840 FLASH 744768 746652 1884 0.3
RAM 131436 131364 -72 -0.1
nxp contact k32w0+release FLASH 582280 584192 1912 0.3
RAM 71092 71044 -48 -0.1
mcxw71+release FLASH 596768 598728 1960 0.3
RAM 63184 63144 -40 -0.1
light k32w0+release FLASH 618884 611284 -7600 -1.2
RAM 70556 70436 -120 -0.2
k32w1+release FLASH 682976 685096 2120 0.3
RAM 48816 48776 -40 -0.1
lock mcxw71+release FLASH 746568 748584 2016 0.3
RAM 67340 67300 -40 -0.1
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1643572 1645244 1672 0.1
RAM 212408 212056 -352 -0.2
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1550524 1552180 1656 0.1
RAM 209208 208856 -352 -0.2
light cy8ckit_062s2_43012 FLASH 1466516 1468172 1656 0.1
RAM 201200 200840 -360 -0.2
lock cy8ckit_062s2_43012 FLASH 1463956 1465292 1336 0.1
RAM 225560 225200 -360 -0.2
qpg lighting-app qpg6105+debug FLASH 660376 662408 2032 0.3
RAM 105420 105380 -40 -0.0
lock-app qpg6105+debug FLASH 618460 620380 1920 0.3
RAM 99880 99832 -48 -0.0
stm32 light STM32WB5MM-DK FLASH 481672 483776 2104 0.4
RAM 144844 144844 0 0.0
telink bridge-app tlsr9258a FLASH 680868 683210 2342 0.3
RAM 91304 91232 -72 -0.1
contact-sensor-app tlsr9528a_retention FLASH 620466 622578 2112 0.3
RAM 50600 50464 -136 -0.3
light-switch-app-ota-shell-factory-data tlsr9528a FLASH 708404 710598 2194 0.3
RAM 73940 73804 -136 -0.2
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 625310 627802 2492 0.4
RAM 144468 144404 -64 -0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 811320 814362 3042 0.4
RAM 99100 99036 -64 -0.1
tizen all-clusters-app arm unknown 4912 4932 20 0.4
FLASH 1722268 1732856 10588 0.6
RAM 90140 90608 468 0.5
chip-tool-ubsan arm unknown 10792 10820 28 0.3
FLASH 18157678 18223782 66104 0.4
RAM 7915516 7961952 46436 0.6

@mergify mergify bot merged commit daf2bce into project-chip:master Oct 31, 2024
67 checks passed
yyzhong-g pushed a commit to yyzhong-g/connectedhomeip that referenced this pull request Dec 12, 2024
…linux where it stays as check) (project-chip#36042)

* Switch data model provider usage to default to enabled

* Decouple pump configuration and control from compatibility functions. Still 100% ember tied.

* Fix up some private/public dependencies: instance header should always be a private dependency

* Allow java (or generally dynamic server) controller builds to compile with dynamic dispatcher. Dynamic dispatcher seems to only care about commands so implemented all the rest as empty, which is not ideal

* Restyled by clang-format

* Restyled by gn

* Better dynamic dispatch implementation, removed extra TODO functions

* Restyled by whitespace

* Update dep a bit more

* Fix linter

* Fix linter again

* Update NXP to support various logging targets and for K32W0 enable progress logging and up (no detail/automation)

* Fix nxp compile (ordering issue)

* Update logic a bit to use log level for low power

* Update builds to consider low power already removing logs

* Make microwave oven cluster compilable

* Restyle

* Add codegen data model to darwin xproject: given that we add ember, we need these to access the ember bits as there is no app or data model to provide these

* Add missing override for darwin dispatch when codegen data model uses lower level methods for ember access

* Fix linter

* Add ability to track structural changes in ember for the purpose of codegen data model caching

* Fix typo: uint is unsigned

* Restyle

* Fix android to also provide a generation version

* Fix another uint/unsigned typo ... oops

* Add assertion of stack lock in darwin override of emberAfWriteAttribute

* Restyled by isort

* Remove formatting changes from python files

* Pick up restyle changes too

* Add comment on attribute laregest

* Use TLV-based encoding for ember data buffer.

This saves 2K of flash on some test devices, being much more
efficient in code size compared to using datamodel code.

* Restyled by clang-format

* Restyled by shfmt

* Undo unrelated change

* Add some casts to make android compiler happy

* Update darwin build project as well with the new files

* Updates based on code review

* Added unit tests for min/max int64 values

* Rename PascalString to PascalStringType

* Fix rename

* Restyle

* Add helper methods inside odd sized integers to localize code logic

* Restyled by clang-format

* Fix up negative ranges

* Fixed ranges

* Fix signed max bug and update unit tests

* Make android happy

* Typo fix

* Switch up unit tests

* Update a nullable check to make use of ValueToNullValue

* Add namespace prefix

* Update src/app/codegen-data-model-provider/EmberDataBuffer.cpp

Co-authored-by: Boris Zbarsky <[email protected]>

* Update src/app/codegen-data-model-provider/EmberDataBuffer.h

Co-authored-by: Boris Zbarsky <[email protected]>

* Correct comments:signed, not unsigned

* Use constructors for the buffer info

* Rename things to EmberAttributeDataBuffer

* Undo submodule updates

* Restyled by clang-format

* Use EmberAttributeDataBuffer for codegen provider _Read

* Fix comments

* Restyled by clang-format

* Update DynamicDispatcher comment

* Rename file

* Use a pointer for the subject descriptor.

This seems to save about 88 bytes of flash on a test NRF board.

* Restyle

* Fix include

* Fix include

* Also fix PW rpc

* This saves 136 bytes...

* More changes to save slightly more flash for code

* Restyle

* Fix typo

* Fix includes

* make more detailed logging optional in the codegen data model and enable it only on known large platforms

* Pull back the event path validity mixin, start with a datamodel implementation

* Fix compile logic after I moved things away

* Add one more check

* Restyle

* Fix typo

* Fix includes

* Restyle

* Move decodable lists bits as a non-template class to save flash

* Update src/app/InteractionModelEngine.cpp

Co-authored-by: Boris Zbarsky <[email protected]>

* Rename method

* Restyle

* More renames

* Restyle

* Fix some renames

* Restyle

* A few more renames

* Restyle

* Use chip::app::IsSignedAttributeType

* Fix up put as well as naming for null value and comment

* Fix up nullable tests

* Test that you cannot decode a null value for non-nullable double and single

* Allow NAN for non-nullable floating points

* Add test case for non nullable bool

* Restyle

* Add a header for efr32

* Update src/app/codegen-data-model-provider/EmberAttributeDataBuffer.cpp

Co-authored-by: Boris Zbarsky <[email protected]>

* Update src/app/codegen-data-model-provider/EmberAttributeDataBuffer.cpp

Co-authored-by: Boris Zbarsky <[email protected]>

* Remove extra comment

* Replace switch with if

* Comment fix

* Another try to make efr32 build of tests happy

* Move includes around, to try to work around issues within efr32 compiles...

* more updates, this time local efr32 compiles

* Remove lookup tables from ember attribute data buffer

---------

Co-authored-by: Andrei Litvin <[email protected]>
Co-authored-by: Restyled.io <[email protected]>
Co-authored-by: Boris Zbarsky <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

5 participants