Add targets that generate test coverage information. #20578

andy31415 · 2022-07-11T21:30:37Z

Problem

Need to visually see test coverage information.

Change overview

Added all-clusters and chip-tool coverage targets (for future use for integration/yaml tests)
Added tests with coverage information, including html result output.

Testing

Unit test of build scripts
ran linux-x64-tests-coverage as a target, saw HTML output of coverage.

github-actions · 2022-07-11T21:53:49Z

PR #20578: Size comparison from 0d703bb to bd428b9

Increases (1 build for cc13x2_26x2)

platform	target	config	section	`0d703bb`	`bd428b9`	change	% change
cc13x2_26x2	shell	LP_CC2652R7	(read only)	658734	658742	8	0.0
			.text	573500	573508	8	0.0

Decreases (1 build for cc13x2_26x2)

platform	target	config	section	`0d703bb`	`bd428b9`	change	% change
cc13x2_26x2	shell	LP_CC2652R7	(read/write)	187992	187984	-8	-0.0

Full report (30 builds for cc13x2_26x2, cyw30739, efr32, esp32, k32w, linux, mbed, nrfconnect, p6, telink)

platform	target	config	section	`0d703bb`	`bd428b9`	change	% change
cc13x2_26x2	all-clusters-app	LP_CC2652R7	(read only)	666243	666243	0	0.0
			(read/write)	184980	184980	0	0.0
			.bss	74116	74116	0	0.0
			.data	3356	3356	0	0.0
			.rodata	88139	88139	0	0.0
			.text	577788	577788	0	0.0
	all-clusters-minimal-app	LP_CC2652R7	(read only)	632051	632051	0	0.0
			(read/write)	157684	157684	0	0.0
			.bss	73412	73412	0	0.0
			.data	3356	3356	0	0.0
			.rodata	77379	77379	0	0.0
			.text	554348	554348	0	0.0
	lock-ftd	LP_CC2652R7	(read only)	669151	669151	0	0.0
			(read/write)	172216	172216	0	0.0
			.bss	71148	71148	0	0.0
			.data	3280	3280	0	0.0
			.rodata	76279	76279	0	0.0
			.text	592392	592392	0	0.0
	lock-mtd	LP_CC2652R7	(read only)	618559	618559	0	0.0
			(read/write)	144264	144264	0	0.0
			.bss	66868	66868	0	0.0
			.data	3280	3280	0	0.0
			.rodata	76159	76159	0	0.0
			.text	541912	541912	0	0.0
	pump-app	LP_CC2652R7	(read only)	678175	678175	0	0.0
			(read/write)	164040	164040	0	0.0
			.bss	71228	71228	0	0.0
			.data	3280	3280	0	0.0
			.rodata	88503	88503	0	0.0
			.text	589188	589188	0	0.0
	pump-controller-app	LP_CC2652R7	(read only)	664007	664007	0	0.0
			(read/write)	178328	178328	0	0.0
			.bss	71348	71348	0	0.0
			.data	3276	3276	0	0.0
			.rodata	84367	84367	0	0.0
			.text	579160	579160	0	0.0
	shell	LP_CC2652R7	(read only)	658734	658742	8	0.0
			(read/write)	187992	187984	-8	-0.0
			.bss	76420	76420	0	0.0
			.data	3360	3360	0	0.0
			.rodata	84918	84918	0	0.0
			.text	573500	573508	8	0.0
cyw30739	light	cyw930739m2evb_01	(read/write)	579622	579622	0	0.0
			.app_xip_area	458376	458376	0	0.0
			.bss	64184	64184	0	0.0
			.data	716	716	0	0.0
			.rodata	0	0	0	0.0
			.text	112	112	0	0.0
	lock	cyw930739m2evb_01	(read/write)	585542	585542	0	0.0
			.app_xip_area	459568	459568	0	0.0
			.bss	68912	68912	0	0.0
			.data	720	720	0	0.0
			.rodata	0	0	0	0.0
			.text	112	112	0	0.0
	ota-requestor-no-progress-logging	cyw930739m2evb_01	(read/write)	582774	582774	0	0.0
			.app_xip_area	462376	462376	0	0.0
			.bss	63392	63392	0	0.0
			.data	660	660	0	0.0
			.rodata	0	0	0	0.0
			.text	112	112	0	0.0
efr32	lighting-app	BRD4161A	(read/write)	1081436	1081436	0	0.0
			.bss	132996	132996	0	0.0
			.data	2048	2048	0	0.0
			.text	946372	946372	0	0.0
		BRD4161A+rpc	(read/write)	1135764	1135764	0	0.0
			.bss	149676	149676	0	0.0
			.data	2260	2260	0	0.0
			.text	983808	983808	0	0.0
		BRD4161A+rs911x	(read/write)	947924	947924	0	0.0
			.bss	140768	140768	0	0.0
			.data	2048	2048	0	0.0
			.text	805088	805088	0	0.0
	lock-app	BRD4161A+wf200	(read/write)	1128816	1128816	0	0.0
			.bss	144184	144184	0	0.0
			.data	2060	2060	0	0.0
			.text	982548	982548	0	0.0
	window-app	BRD4161A	(read/write)	`1075244`	`1075244`	0	0.0
			.bss	134468	134468	0	0.0
			.data	2076	2076	0	0.0
			.text	938676	938676	0	0.0
esp32	all-clusters-app	c3devkit	(read only)	`1020100`	`1020100`	0	0.0
			(read/write)	1485642	1485642	0	0.0
			.dram0.bss	70080	70080	0	0.0
			.dram0.data	14600	14600	0	0.0
			.flash.rodata	215528	215528	0	0.0
			.flash.text	`1020100`	`1020100`	0	0.0
			.iram0.text	62902	62902	0	0.0
		m5stack	(read only)	1073987	1073987	0	0.0
			(read/write)	487712	487712	0	0.0
			.dram0.bss	75600	75600	0	0.0
			.dram0.data	34144	34144	0	0.0
			.flash.rodata	245972	245972	0	0.0
			.flash.text	1068603	1068603	0	0.0
			.iram0.text	123267	123267	0	0.0
k32w	light	k32w061+release	(read/write)	658832	658832	0	0.0
			.bss	69516	69516	0	0.0
			.data	1992	1992	0	0.0
			.text	581524	581524	0	0.0
	lock	k32w061+release	(read/write)	685652	685652	0	0.0
			.bss	69980	69980	0	0.0
			.data	2004	2004	0	0.0
			.text	607868	607868	0	0.0
linux	chip-tool-no-interactive-ipv6only	arm64	(read only)	10031716	10031716	0	0.0
			(read/write)	684529	684529	0	0.0
			.bss	42609	42609	0	0.0
			.data	1152	1152	0	0.0
			.data.rel.ro	623432	623432	0	0.0
			.dynamic	528	528	0	0.0
			.got	13520	13520	0	0.0
			.init	24	24	0	0.0
			.init_array	192	192	0	0.0
			.rodata	478260	478260	0	0.0
			.text	`7992788`	`7992788`	0	0.0
	thermostat-no-ble	arm64	(read only)	`2595316`	`2595316`	0	0.0
			(read/write)	158289	158289	0	0.0
			.bss	65249	65249	0	0.0
			.data	1704	1704	0	0.0
			.data.rel.ro	83240	83240	0	0.0
			.dynamic	528	528	0	0.0
			.got	5072	5072	0	0.0
			.init	24	24	0	0.0
			.init_array	400	400	0	0.0
			.rodata	165476	165476	0	0.0
			.text	`2190064`	`2190064`	0	0.0
mbed	lock-app	CY8CPROTO_062_4343W+release	(read only)	6224	6224	0	0.0
			(read/write)	`2448112`	`2448112`	0	0.0
			.bss	213940	213940	0	0.0
			.data	5872	5872	0	0.0
			.text	`1410756`	`1410756`	0	0.0
nrfconnect	all-clusters-app	nrf52840dk_nrf52840	(read/write)	`1175119`	`1175119`	0	0.0
			bss	142900	142900	0	0.0
			rodata	141880	141880	0	0.0
			text	811448	811448	0	0.0
	all-clusters-minimal-app	nrf52840dk_nrf52840	(read/write)	1155315	1155315	0	0.0
			bss	142136	142136	0	0.0
			rodata	133412	133412	0	0.0
			text	800892	800892	0	0.0
p6	all-clusters-app	default	(read/write)	2565392	2565392	0	0.0
			.bss	149120	149120	0	0.0
			.data	2776	2776	0	0.0
			.text	`1523656`	`1523656`	0	0.0
	all-clusters-minimal-app	default	(read/write)	2510440	2510440	0	0.0
			.bss	148400	148400	0	0.0
			.data	2776	2776	0	0.0
			.text	1468704	1468704	0	0.0
	light-app	default	(read/write)	2440736	2440736	0	0.0
			.bss	140456	140456	0	0.0
			.data	2592	2592	0	0.0
			.text	1399000	1399000	0	0.0
	lock-app	default	(read/write)	2467992	2467992	0	0.0
			.bss	140304	140304	0	0.0
			.data	2600	2600	0	0.0
			.text	`1426256`	`1426256`	0	0.0
telink	light-switch-app	tlsr9518adk80d	(read/write)	797268	797268	0	0.0
			bss	70576	70576	0	0.0
			noinit	40416	40416	0	0.0
			text	565668	565668	0	0.0
	lighting-app	tlsr9518adk80d	(read/write)	817100	817100	0	0.0
			bss	71420	71420	0	0.0
			noinit	40416	40416	0	0.0
			text	581994	581994	0	0.0

scripts/build/builders/gn.py

Co-authored-by: Terence Hampson <[email protected]>

github-actions · 2022-07-12T15:36:46Z

PR #20578: Size comparison from 0d703bb to 88d3953

Increases (2 builds for nrfconnect)

platform	target	config	section	`0d703bb`	`88d3953`	change
nrfconnect	all-clusters-app	nrf52840dk_nrf52840	(read/write)	`1175119`	`1175175`	56
			rodata	141880	141888	8
			text	811448	811492	44
	all-clusters-minimal-app	nrf52840dk_nrf52840	(read/write)	1155315	1155367	52
			rodata	133412	133416	4
			text	800892	800936	44

Full report (2 builds for nrfconnect)

platform	target	config	section	`0d703bb`	`88d3953`	change
nrfconnect	all-clusters-app	nrf52840dk_nrf52840	(read/write)	`1175119`	`1175175`	56
			bss	142900	142900	0
			rodata	141880	141888	8
			text	811448	811492	44
	all-clusters-minimal-app	nrf52840dk_nrf52840	(read/write)	1155315	1155367	52
			bss	142136	142136	0
			rodata	133412	133416	4
			text	800892	800936	44

* add support for coverage enabled apps in build_examples.py * Switch to gcc builds * Coverage generation support * Fix typo * Update ordering for builds * Restyle * Update scripts/build/builders/gn.py Co-authored-by: Terence Hampson <[email protected]> Co-authored-by: Terence Hampson <[email protected]>

* add support for coverage enabled apps in build_examples.py * Switch to gcc builds * Coverage generation support * Fix typo * Update ordering for builds * Restyle * Update scripts/build/builders/gn.py Co-authored-by: Terence Hampson <[email protected]> Co-authored-by: Terence Hampson <[email protected]> Co-authored-by: Andrei Litvin <[email protected]> Co-authored-by: Terence Hampson <[email protected]>

) * add support for coverage enabled apps in build_examples.py * Switch to gcc builds * Coverage generation support * Fix typo * Update ordering for builds * Restyle * Update scripts/build/builders/gn.py Co-authored-by: Terence Hampson <[email protected]> Co-authored-by: Terence Hampson <[email protected]>

andy31415 added 6 commits July 11, 2022 16:24

add support for coverage enabled apps in build_examples.py

33c9df0

Switch to gcc builds

ff4fcf0

Coverage generation support

8bec052

Fix typo

d1b7960

Update ordering for builds

16adc39

Restyle

bd428b9

boring-cyborg bot added the scripts label Jul 11, 2022

pullapprove bot requested review from anush-apple, arkq, Byungjoo-Lee, bzbarsky-apple, carol-apple, chrisdecenzo, chshu, chulspro, Damian-Nordic, dhrishi, electrocucaracha, emargolis, erjiaqing, franck-apple, gjc13, harsha-rajendran and hawk248 July 11, 2022 21:30

pullapprove bot requested review from msandstedt, woody-apple, rgoliver, xylophone21, robszewczyk, yufengwangca, saurabhst, selissia, tecimovic, tehampson, turon, vijs, vivien-apple and wbschiller July 11, 2022 21:30

pullapprove bot added the review - pending label Jul 11, 2022

restyled-io bot mentioned this pull request Jul 11, 2022

Restyle Add targets that generate test coverage information. #20579

Closed

turon approved these changes Jul 11, 2022

View reviewed changes

jmeg-sfy approved these changes Jul 12, 2022

View reviewed changes

tehampson approved these changes Jul 12, 2022

View reviewed changes

scripts/build/builders/gn.py Outdated Show resolved Hide resolved

Update scripts/build/builders/gn.py

88d3953

Co-authored-by: Terence Hampson <[email protected]>

andy31415 added the fast track label Jul 13, 2022

andy31415 merged commit a05cc3e into project-chip:master Jul 13, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add targets that generate test coverage information. #20578

Add targets that generate test coverage information. #20578

andy31415 commented Jul 11, 2022

github-actions bot commented Jul 11, 2022 •

edited

Loading

github-actions bot commented Jul 12, 2022

Add targets that generate test coverage information. #20578

Add targets that generate test coverage information. #20578

Conversation

andy31415 commented Jul 11, 2022

Problem

Change overview

Testing

github-actions bot commented Jul 11, 2022 • edited Loading

github-actions bot commented Jul 12, 2022

github-actions bot commented Jul 11, 2022 •

edited

Loading