Improve loops performance

[Nerivec]

• Use iterators to avoid nested looping whenever possible.
• Extract cached deleted devices (runtime only) to separate object to avoid lots of unnecessary loop checks.
• Avoid using Object.values with looping on top.

On a database with 100 records (with matching in mid for "find" tests):

┌─────────┬───────────────────────────┬─────────────┬────────────────────┬──────────┬──────────┐
│ (index) │ Task Name                 │ ops/sec     │ Average Time (ns)  │ Margin   │ Samples  │
├─────────┼───────────────────────────┼─────────────┼────────────────────┼──────────┼──────────┤
│ 0       │ 'Old Find by nwk'         │ '46,133'    │ 21676.031102459012 │ '±0.22%' │ 230670   │
│ 1       │ 'New Find by nwk'         │ '149,131'   │ 6705.480378715173  │ '±0.19%' │ 745659   │
│ 2       │ 'Old Find deleted by nwk' │ '42,740'    │ 23396.819942899692 │ '±0.31%' │ 213705   │
│ 3       │ 'New Find deleted by nwk' │ '7,754,479' │ 128.95771808723978 │ '±0.18%' │ 38772399 │
└─────────┴───────────────────────────┴─────────────┴────────────────────┴──────────┴──────────┘
┌─────────┬──────────────────────────────────────┬──────────┬────────────────────┬──────────┬─────────┐
│ (index) │ Task Name                            │ ops/sec  │ Average Time (ns)  │ Margin   │ Samples │
├─────────┼──────────────────────────────────────┼──────────┼────────────────────┼──────────┼─────────┤
│ 0       │ 'Old Device.all looped over'         │ '42,299' │ 23640.73759176489  │ '±0.23%' │ 211500  │
│ 1       │ 'New Device.all looped over'         │ '47,153' │ 21207.48536824284  │ '±0.23%' │ 235766  │
│ 2       │ 'New Device.allIterator looped over' │ '94,555' │ 10575.785707822934 │ '±0.25%' │ 472779  │
└─────────┴──────────────────────────────────────┴──────────┴────────────────────┴──────────┴─────────┘
┌─────────┬────────────────────────────────────────────────┬──────────┬────────────────────┬──────────┬─────────┐
│ (index) │ Task Name                                      │ ops/sec  │ Average Time (ns)  │ Margin   │ Samples │
├─────────┼────────────────────────────────────────────────┼──────────┼────────────────────┼──────────┼─────────┤
│ 0       │ 'Old controller.getDevices() looped over'      │ '37,690' │ 26531.59745629989  │ '±0.29%' │ 188455  │
│ 1       │ 'New controller.getDevices() looped over'      │ '81,494' │ 12270.803126836478 │ '±0.69%' │ 407472  │
│ 2       │ 'Old Z2M zigbee.devices() looped over'         │ '35,652' │ 28048.826201476015 │ '±0.25%' │ 178261  │
│ 3       │ 'New Z2M zigbee.devices() looped over'         │ '58,774' │ 17014.241600040517 │ '±0.42%' │ 293872  │
│ 4       │ 'New Z2M zigbee.devicesIterator() looped over' │ '55,280' │ 18089.687078094114 │ '±0.24%' │ 276401  │
└─────────┴────────────────────────────────────────────────┴──────────┴────────────────────┴──────────┴─────────┘

Z2M code for last 2 perf tests

    devices(includeCoordinator = true): Device[] {
        const devices: Device[] = [];

        for (const device of this.herdsman.getDevicesIterator((d) => (includeCoordinator || d.type !== 'Coordinator'))) {
            devices.push(this.resolveDevice(device.ieeeAddr));
        }

        return devices;
    }

    * devicesIterator(includeCoordinator = true): Generator<Device> {
        for (const device of this.herdsman.getDevicesIterator((d) => (includeCoordinator || d.type !== 'Coordinator'))) {
            yield this.resolveDevice(device.ieeeAddr);
        }
    }

Also:

• Reset runtime lookups on controller stop, per fix: Clear the loaded device database entries when the controller is stopped #1131
• Add a couple of utils for Foundation to avoid repetition.
• Cleanup Database open/write.
• Fix casing in Database class.
• Cleanup temp folder at end of Controller tests (avoids folder getting huge on local test machines).
• Cleanup console calls that slipped through the cracks.

TODO:

• Update Z2M to use iterator functions whenever possible. Use herdsman improved looping. zigbee2mqtt#23541
• Remove herdsman functions that definitely won't have a use anymore (after above).

[Koenkk]

Thanks!