Skip to content

Commit

Permalink
Create a codegen that converts idl back into .matter formats (project…
Browse files Browse the repository at this point in the history
…-chip#29867)

* Start creating a IDL codegen so we can self-test parsed output

* Start with listing clusters

* Enum listing

* A lot more things supported

* Attribute rendering

* Support for string and octet string sizes

* Timed command support

* Restyle

* Add descriptions to clusters

* Attempt to fix up alignment of things

* Alignment looks slightly better

* Better command separation

* Align comments

* Align and output descriptions including clusters

* More work regarding loop structures

* Apply hex formatting to bitmaps. output now seems identical except one whitespace change

* Identical output for now

* Support API maturity. Notice that doccomments are lost on maturity :(

* Fix doxygen parsing for api maturity at the cluster level

* Restyle

* Support endpoints, although that is not 1:1 as hex encoding and ordering for events is lost

* Restyle

* Add todo note that default value does not string escaping

* Default rendering and add to files

* More updates on file dependencies

* Unit test IDL generator

* Add the IDL unit test as a standard unit test

* Update for python compatibility

* Fix unit testing of builds when GSDK root is defined

* Added a readme file

* Restyle

* Make xml parser use the idl codegen

* Restyle

* look to fix misspell warnings

* Undo repo update

* Fix linter errors

---------

Co-authored-by: Andrei Litvin <[email protected]>
  • Loading branch information
2 people authored and HunsupJung committed Oct 23, 2023
1 parent 9693c71 commit 422cc39
Show file tree
Hide file tree
Showing 12 changed files with 536 additions and 7 deletions.
1 change: 1 addition & 0 deletions scripts/build/test.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ def build_actual_output(root: str, out: str, args: List[str]) -> List[str]:
'IMX_SDK_ROOT': 'IMX_SDK_ROOT',
'TI_SYSCONFIG_ROOT': 'TEST_TI_SYSCONFIG_ROOT',
'JAVA_PATH': 'TEST_JAVA_PATH',
'GSDK_ROOT': 'TEST_GSDK_ROOT',
})

retval = subprocess.run([
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
cd "{root}"

# Generating efr32-brd4161a-light-rpc-no-version
gn gen --check --fail-on-unused-args --export-compile-commands --root={root}/examples/lighting-app/silabs '--args=silabs_board="BRD4161A" is_debug=false import("//with_pw_rpc.gni")' {out}/efr32-brd4161a-light-rpc-no-version
gn gen --check --fail-on-unused-args --export-compile-commands --root={root}/examples/lighting-app/silabs '--args=silabs_board="BRD4161A" is_debug=false import("//with_pw_rpc.gni") efr32_sdk_root="TEST_GSDK_ROOT" openthread_root="TEST_GSDK_ROOT/util/third_party/openthread"' {out}/efr32-brd4161a-light-rpc-no-version

# Building efr32-brd4161a-light-rpc-no-version
ninja -C {out}/efr32-brd4161a-light-rpc-no-version
1 change: 1 addition & 0 deletions scripts/py_matter_idl/BUILD.gn
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ pw_python_package("matter_idl") {
"matter_idl/test_backwards_compatibility.py",
"matter_idl/test_matter_idl_parser.py",
"matter_idl/test_generators.py",
"matter_idl/test_idl_generator.py",
"matter_idl/test_xml_parser.py",
]

Expand Down
2 changes: 2 additions & 0 deletions scripts/py_matter_idl/files.gni
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ matter_idl_generator_templates = [
"${chip_root}/scripts/py_matter_idl/matter_idl/generators/cpp/application/PluginApplicationCallbacksHeader.jinja",
"${chip_root}/scripts/py_matter_idl/matter_idl/generators/cpp/tlvmeta/TLVMetaData_cpp.jinja",
"${chip_root}/scripts/py_matter_idl/matter_idl/generators/cpp/tlvmeta/TLVMetaData_h.jinja",
"${chip_root}/scripts/py_matter_idl/matter_idl/generators/idl/MatterIdl.jinja",
"${chip_root}/scripts/py_matter_idl/matter_idl/generators/java/CHIPCallbackTypes.jinja",
"${chip_root}/scripts/py_matter_idl/matter_idl/generators/java/ChipClustersCpp.jinja",
"${chip_root}/scripts/py_matter_idl/matter_idl/generators/java/ChipClustersRead.jinja",
Expand All @@ -27,6 +28,7 @@ matter_idl_generator_sources = [
"${chip_root}/scripts/py_matter_idl/matter_idl/generators/cpp/application/__init__.py",
"${chip_root}/scripts/py_matter_idl/matter_idl/generators/cpp/tlvmeta/__init__.py",
"${chip_root}/scripts/py_matter_idl/matter_idl/generators/filters.py",
"${chip_root}/scripts/py_matter_idl/matter_idl/generators/idl/__init__.py",
"${chip_root}/scripts/py_matter_idl/matter_idl/generators/java/__init__.py",
"${chip_root}/scripts/py_matter_idl/matter_idl/generators/registry.py",
"${chip_root}/scripts/py_matter_idl/matter_idl/generators/types.py",
Expand Down
150 changes: 150 additions & 0 deletions scripts/py_matter_idl/matter_idl/generators/idl/MatterIdl.jinja
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
{% macro render_field(field) -%}{#
Macro for the output of a single field entry such as:
int16u identifyTime = 0;
optional int16u transitionTime = 3;
optional nullable int16u transitionTime = 2;
optional ExtensionFieldSet extensionFieldSets[] = 5;
#}

{%- if field.qualities %}{{field.qualities | idltxt}} {% endif -%}
{{field.data_type.name}}
{%- if field.data_type.max_length -%} <{{field.data_type.max_length}}> {%- endif -%}
{##} {{field.name}}
{%- if field.is_list %}[]{% endif -%}
{##} = {{field.code}};
{%- endmacro -%}

{% macro render_struct(s) -%}{#
Macro for the output of a complete struct
#}
{%- if s.tag %}{{s.tag | idltxt}} {% endif -%}
{% if s.qualities %}{{s.qualities | idltxt}} {% endif -%}
struct {{s.name}} {##}
{%- if s.code is not none %}= {{s.code}} {% endif -%}
{
{% for field in s.fields %}
{{render_field(field)}}
{% endfor %}
}
{%- endmacro -%}


// This IDL was auto-generated from a parsed data structure

{% for cluster in idl.clusters %}
{% if cluster.description %}/** {{cluster.description}} */
{% endif %}
{{cluster.api_maturity | idltxt}}{{cluster.side | idltxt}} cluster {{cluster.name}} = {{cluster.code}} {
{%- for enum in cluster.enums %}

enum {{enum.name}} : {{ enum.base_type}} {
{% for entry in enum.entries %}
{{entry.name}} = {{entry.code}};
{% endfor %}
}
{% endfor %}

{%- for bitmap in cluster.bitmaps %}

bitmap {{bitmap.name}} : {{ bitmap.base_type}} {
{% for entry in bitmap.entries %}
{{entry.name}} = 0x{{"%X" | format(entry.code)}};
{% endfor %}
}
{% endfor %}

{%- for s in cluster.structs | rejectattr("tag") %}
{% if loop.first %}

{% endif %}
{{render_struct(s)}}
{% if not loop.last %}

{% endif %}
{% endfor %}

{%- for e in cluster.events %}
{% if loop.first %}

{% endif %}
{##} {##}{% if e.qualities %}{{e.qualities | idltxt}} {% endif -%}
{{e.priority | idltxt}} event {{e | event_access}}{{e.name}} = {{e.code}} {
{% for field in e.fields %}
{{render_field(field)}}
{% endfor %}
}
{% if not loop.last %}

{% endif %}
{% endfor %}

{%- for a in cluster.attributes %}
{% if loop.first %}

{% endif %}
{{a.qualities | idltxt}}attribute {{a | attribute_access}}{{render_field(a.definition)}}
{% endfor %}

{%- for s in cluster.structs | selectattr("tag") %}

{{render_struct(s)}}
{% endfor %}

{%- for c in cluster.commands %}
{% if loop.first %}

{% endif %}
{% if c.description %}
/** {{c.description}} */
{% endif %}
{{c.qualities | idltxt}}command {{c | command_access}}{{c.name}}(
{%- if c.input_param %}{{c.input_param}}{% endif -%}
): {{c.output_param}} = {{c.code}};
{% endfor %}
}

{% endfor %}

{%- if idl.endpoints %}
{%- for endpoint in idl.endpoints %}
endpoint {{endpoint.number}} {
{% for t in endpoint.device_types %}
device type {{t.name}} = {{t.code}}, version {{t.version}};
{% endfor%}

{%-for b in endpoint.client_bindings %}
{% if loop.first %}

{% endif %}
binding cluster {{b}};
{% endfor %}

{%-for c in endpoint.server_clusters %}

server cluster {{c.name}} {
{% for e in c.events_emitted %}
emits event {{e}};
{% if loop.last %}

{% endif %}
{% endfor %}
{% for a in c.attributes %}
{{"%-8s" | format(a.storage|idltxt) }} attribute {{a.name}}
{%- if a.default is not none %} default = {{a.default|render_default}} {%- endif %};
{% endfor %}
{% for cmd in c.commands %}
{% if loop.first %}

{% endif %}
handle command {{cmd.name}};
{% endfor %}
}
{% endfor %}

}
{% if not loop.last %}

{% endif %}
{% endfor %}
{% endif %}
32 changes: 32 additions & 0 deletions scripts/py_matter_idl/matter_idl/generators/idl/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
## Generator description

Generates a structured `Idl` data type into a human-readable text format
(`.matter` file).

It is useful for tools that ingest non-idl data but convert into idl data (e.g.
`zapxml` or CSA data model XML data.)

### Usage

A no-op usage can be:

```
./scripts/codegen.py -g idl --output-dir out/idlgen examples/all-clusters-app/all-clusters-common/all-clusters-app.matter
```

which would re-generate the entire `all-clusters-app.matter` into
`out/idlgen/idl.matter`

This generation is useful for testing/validating that both parsing and
generation works. Actual usage of this generator would be inside XML tools.

### Within XML parsing

A XML parser will use this code generator to output a human readable view of the
parsed data:

```
./scripts/py_matter_idl/matter_idl/xml_parser.py \
./src/app/zap-templates/zcl/data-model/chip/onoff-cluster.xml \
./src/app/zap-templates/zcl/data-model/chip/global-attributes.xm
```
Loading

0 comments on commit 422cc39

Please sign in to comment.