Increase unauthenticated session pool, fixes #10493

[kghost]

Problem

#10493

Change overview

Increase the pool size of unauthenticated session

Testing

Verified by unit-tests.

[github-actions]

PR #10652: Size comparison from 4c7f664 to 8d7b98c

5 builds (for p6, qpg, telink)

platform	target	config	section	4c7f664	8d7b98c	change	% change
p6	lock-app	default	.bss	68216	68600	384	0.6
			.data	2416	2416	0	0.0
			.heap	962712	962328	-384	-0.0
			.text	1126624	1126624	0	0.0
qpg	lighting-app	qpg6100+debug	.bss	53552	53936	384	0.7
			.data	996	996	0	0.0
			.text	486616	486624	8	0.0
	lock-app	qpg6100+debug	.bss	52488	52872	384	0.7
			.data	952	952	0	0.0
			.text	462684	462692	8	0.0
	persistent-storage-app	qpg6100+debug	.bss	17778	17778	0	0.0
			.data	280	280	0	0.0
			.text	102704	102704	0	0.0
telink	lighting-app	tlsr9518adk80d	bss	71000	71384	384	0.5
			noinit	33216	33216	0	0.0
			text	458304	458304	0	0.0

Increases above 1.0% from 4c7f664 to 8d7b98c:

platform	target	config	section	4c7f664	8d7b98c	change	% change
linux	ota-provider-app	debug	.bss	37472	37952	480	1.3
	shell	debug	.bss	16072	16552	480	3.0
	lighting-app	debug+rpc	.bss	42232	42712	480	1.1

17 builds (for efr32, k32w, linux, mbed)

platform	target	config	section	4c7f664	8d7b98c	change	% change
efr32	lighting-app	BRD4161A	.bss	118036	118420	384	0.3
			.data	1800	1800	0	0.0
			.text	782944	782944	0	0.0
	lock-app	BRD4161A	.bss	115892	116276	384	0.3
			.data	1760	1760	0	0.0
			.text	762088	762088	0	0.0
	window-app	BRD4161A	.bss	116212	116596	384	0.3
			.data	1764	1764	0	0.0
			.text	763012	763012	0	0.0
	lighting-app	BRD4161A+rpc	.bss	131364	131748	384	0.3
			.data	1852	1852	0	0.0
			.text	762700	762700	0	0.0
k32w	lock-app	k32w061+debug	.bss	69052	69436	384	0.6
			.data	1864	1864	0	0.0
			.text	515520	515536	16	0.0
	shell	k32w061+debug	.bss	55080	55464	384	0.7
			.data	672	672	0	0.0
			.text	357332	357332	0	0.0
	lighting-app	k32w061+se05x+release	.bss	78576	78960	384	0.5
			.data	1900	1900	0	0.0
			.text	614304	614304	0	0.0
linux	all-clusters-app	debug	.bss	52176	52656	480	0.9
			.data	978	978	0	0.0
			.data.rel.ro	58688	58688	0	0.0
			.dynamic	592	592	0	0.0
			.got	4072	4072	0	0.0
			.init	27	27	0	0.0
			.init_array	512	512	0	0.0
			.rodata	136373	136373	0	0.0
			.text	1345186	1345186	0	0.0
	chip-tool	debug	.bss	17552	17552	0	0.0
			.data	1584	1584	0	0.0
			.data.rel.ro	86192	86192	0	0.0
			.dynamic	592	592	0	0.0
			.got	4344	4344	0	0.0
			.init	27	27	0	0.0
			.init_array	416	416	0	0.0
			.rodata	178892	178892	0	0.0
			.text	2859941	2859941	0	0.0
	ota-provider-app	debug	.bss	37472	37952	480	1.3
			.data	752	752	0	0.0
			.data.rel.ro	23176	23176	0	0.0
			.dynamic	592	592	0	0.0
			.got	4008	4008	0	0.0
			.init	27	27	0	0.0
			.init_array	440	440	0	0.0
			.rodata	110056	110056	0	0.0
			.text	1010594	1010594	0	0.0
	ota-requestor-app	debug	.bss	205728	206208	480	0.2
			.data	752	752	0	0.0
			.data.rel.ro	24488	24488	0	0.0
			.dynamic	592	592	0	0.0
			.got	4136	4136	0	0.0
			.init	27	27	0	0.0
			.init_array	512	512	0	0.0
			.rodata	128744	128744	0	0.0
			.text	1130130	1130130	0	0.0
	shell	debug	.bss	16072	16552	480	3.0
			.data	242	242	0	0.0
			.data.rel.ro	35120	35120	0	0.0
			.dynamic	592	592	0	0.0
			.got	3496	3496	0	0.0
			.init	27	27	0	0.0
			.init_array	336	336	0	0.0
			.rodata	71855	71855	0	0.0
			.text	569858	569858	0	0.0
	tv-app	debug	.bss	216400	216880	480	0.2
			.data	2032	2032	0	0.0
			.data.rel.ro	55872	55872	0	0.0
			.dynamic	592	592	0	0.0
			.got	4400	4400	0	0.0
			.init	27	27	0	0.0
			.init_array	608	608	0	0.0
			.rodata	152264	152264	0	0.0
			.text	1464706	1464706	0	0.0
	bridge-app	debug+rpc	.bss	52912	53392	480	0.9
			.data	976	976	0	0.0
			.data.rel.ro	25784	25784	0	0.0
			.dynamic	592	592	0	0.0
			.got	3944	3944	0	0.0
			.init	27	27	0	0.0
			.init_array	400	400	0	0.0
			.rodata	110644	110644	0	0.0
			.text	1052629	1052629	0	0.0
	lighting-app	debug+rpc	.bss	42232	42712	480	1.1
			.data	1106	1106	0	0.0
			.data.rel.ro	52496	52496	0	0.0
			.dynamic	608	608	0	0.0
			.got	4104	4104	0	0.0
			.init	27	27	0	0.0
			.init_array	528	528	0	0.0
			.rodata	127857	127857	0	0.0
			.text	1253906	1253906	0	0.0
mbed	lighting-app	CY8CPROTO_062_4343W+release	.bss	172164	172548	384	0.2
			.data	5464	5464	0	0.0
			.heap	858816	858432	-384	-0.0
			.text	1219944	1219944	0	0.0
	lock-app	CY8CPROTO_062_4343W+release	.bss	171084	171468	384	0.2
			.data	5432	5432	0	0.0
			.heap	859928	859544	-384	-0.0
			.text	1197840	1197840	0	0.0

12 builds (for esp32, nrfconnect)

platform	target	config	section	4c7f664	8d7b98c	change	% change
esp32	all-clusters-app	c3devkit	.dram0.bss	60296	60680	384	0.6
			.dram0.data	16192	16192	0	0.0
			.flash.rodata	199008	199008	0	0.0
			.flash.text	872638	872634	-4	-0.0
			.iram0.text	57330	57330	0	0.0
		m5stack	.dram0.bss	62792	63176	384	0.6
			.dram0.data	32084	32084	0	0.0
			.flash.rodata	207432	207432	0	0.0
			.flash.text	902739	902855	116	0.0
			.iram0.text	125115	125115	0	0.0
nrfconnect	lighting-app	nrf52840dk_nrf52840	bss	112332	112716	384	0.3
			rodata	97776	97776	0	0.0
			text	577652	577656	4	0.0
	lock-app	nrf52840dk_nrf52840	bss	111348	111732	384	0.3
			rodata	94112	94112	0	0.0
			text	559152	559156	4	0.0
	pigweed-app	nrf52840dk_nrf52840	bss	51772	51772	0	0.0
			rodata	45772	45772	0	0.0
			text	339392	339392	0	0.0
	pump-app	nrf52840dk_nrf52840	bss	111416	111800	384	0.3
			rodata	95092	95092	0	0.0
			text	562308	562312	4	0.0
	pump-controller-app	nrf52840dk_nrf52840	bss	111356	111740	384	0.3
			rodata	94172	94172	0	0.0
			text	558944	558948	4	0.0
	shell	nrf52840dk_nrf52840	bss	107316	107700	384	0.4
			rodata	71640	71640	0	0.0
			text	519032	519032	0	0.0
	lighting-app	nrf52840dk_nrf52840+rpc	bss	108572	108956	384	0.4
			rodata	88560	88560	0	0.0
			text	550856	550860	4	0.0
		nrf5340dk_nrf5340_cpuapp	bss	113704	114088	384	0.3
			rodata	93016	93016	0	0.0
			text	507116	507120	4	0.0
	lock-app	nrf5340dk_nrf5340_cpuapp	bss	112720	113104	384	0.3
			rodata	89372	89372	0	0.0
			text	488612	488616	4	0.0
	shell	nrf5340dk_nrf5340_cpuapp	bss	108300	108684	384	0.4
			rodata	66284	66284	0	0.0
			text	439632	439632	0	0.0

[github-actions]

Size increase report for "gn_qpg-example-build" from 4c7f664

File	Section	File	VM
chip-qpg6100-lighting-example.out	.bss	0	384
chip-qpg6100-lighting-example.out	.text	8	8
chip-qpg6100-lighting-example.out	.heap	0	-384

Full report output

BLOAT REPORT

Files found only in the build output:
    report.csv

Comparing ./master_artifact/chip-qpg6100-lighting-example.out.map and ./pull_artifact/chip-qpg6100-lighting-example.out.map:

BLOAT EXECUTION FAILED WITH CODE 1:
bloaty: unknown file type for file './pull_artifact/chip-qpg6100-lighting-example.out.map'

Comparing ./master_artifact/chip-qpg6100-lighting-example.out and ./pull_artifact/chip-qpg6100-lighting-example.out:

sections,vmsize,filesize
.bss,384,0
.debug_str,0,62
.debug_abbrev,0,10
.text,8,8
.strtab,0,5
.debug_info,0,4
.shstrtab,0,-1
[Unmapped],0,-8
.heap,-384,0

[github-actions]

Size increase report for "esp32-example-build" from 4c7f664

File	Section	File	VM
chip-all-clusters-app.elf	.dram0.bss	0	384
chip-all-clusters-app.elf	.flash.text	-4	-4

Full report output

BLOAT REPORT

Files found only in the build output:
    report.csv

Comparing ./master_artifact/chip-all-clusters-app.elf and ./pull_artifact/chip-all-clusters-app.elf:

sections,vmsize,filesize
.dram0.bss,384,0
.debug_ranges,0,64
.debug_str,0,61
.debug_line,0,34
.debug_abbrev,0,10
.strtab,0,5
[Unmapped],0,4
.riscv.attributes,0,1
.shstrtab,0,-1
.flash.text,-4,-4
.debug_loc,0,-7
.debug_info,0,-11

[github-actions]

Size increase report for "nrfconnect-example-build" from 4c7f664

File	Section	File	VM
chip-lock.elf	bss	0	384
chip-lock.elf	text	4	4
chip-lock.elf	device_handles	-4	-4
chip-shell.elf	bss	0	384

Full report output

BLOAT REPORT

Files found only in the build output:
    report.csv

Comparing ./master_artifact/chip-lock.elf and ./pull_artifact/chip-lock.elf:

sections,vmsize,filesize
bss,384,0
.debug_str,0,61
.strtab,0,5
text,4,4
.shstrtab,0,3
.debug_info,0,2
.debug_line,0,2
.debug_loc,0,-1
device_handles,-4,-4

Comparing ./master_artifact/chip-shell.elf and ./pull_artifact/chip-shell.elf:

sections,vmsize,filesize
bss,384,0
.debug_str,0,61
.strtab,0,5
.debug_loc,0,3
.debug_info,0,2
.shstrtab,0,-1
.debug_line,0,-2

[andy31415]

Should unauthenticated sessions not expire in time? #10493 seems to complain that if running several commisioning (4 of them?) eventually we run out of resources.

I am wondering if this PR just increases the time it takes for things to fail. Is this pool increase supposed to be a final solution or a temporary change until we figure out how to expire unauthenticated connections? In my mind, we should just LRU unauthenticated connections and have the pool quite small.

[bzbarsky-apple]

I was going to ask the same question @andy31415 asked: this seems like it's just working around the fact that the TE is trying to create 4-5 authenticated connections one after another, not actually fixing things to work when we have had more unauthenticated connection attempts in our lifetime than the pool size.

[msandstedt]

Should unauthenticated sessions not expire in time? #10493 seems to complain that if running several commisioning (4 of them?) eventually we run out of resources.

I am wondering if this PR just increases the time it takes for things to fail. Is this pool increase supposed to be a final solution or a temporary change until we figure out how to expire unauthenticated connections? In my mind, we should just LRU unauthenticated connections and have the pool quite small.

The rule we have is in fact LRU eviction. However, the session manager has no way to forcibly evict when reference counts are non-zero. To do so, the session manager would need to understand (or be able to learn) what resources are associated with its sessions to call on-failed type methods on all associated things.

It is my view that the inability to do this is ultimately the problem.

Since we effectively have a multi-singleton architecture with the single session manager instance and single exchange manager instance, it would probably make most sense for the exchange manager to expose a method to forcibly cancel all exchanges associated with a session. I do not know how difficult this would be to implement.

[kghost]

It is LRU controlled, the session should be released when PASE/CASE session is completed. I'll check why the session is not being freed.

[msandstedt]

It is LRU controlled, the session should be released when PASE/CASE session is completed. I'll check why the session is not being freed.

The nuance to my point was that it may still be too expensive to wait for PASE/CASE session timeout or completion. If so., the session manager would need to have some mechanism for forcible eviction.

However, if even happy-path cleanup is not occurring, then yes, that would be a more immediate problem.