High CPU usage in Energy Organizer, iOS 14+

[thedavidharris]

Bug Report

We're seeing a lot of reports in the Energy organizer for high sustained CPU usage, and this also appears to be affecting our background battery usage metrics. 99% of these reports seem to be from iOS 14.

Sample of some of the stack traces have some slim details:

There's a good number of others but most follow that same pattern. All of these are logged due to high CPU usage, with messages along the lines of "90 seconds cpu time over 94 seconds (96% cpu average), exceeding limit of 50% cpu over 180 seconds."

We haven't been able to locally reproduce at the moment.

Reproduction Steps

TBD

OS & Version Information

• iOS Version: 14.0+ (1-2% or less reports on iOS 13.7, none on anything less)
• SDL iOS Version: Both 6.6 and 6.7

[ghost]

I spend a couple of hours to reproduce the issue and nothing ... So I dig in the code to see what's happening. I focused on 6.7.0/SmartDeviceLink/SDLIAPDataSession.m#L336 as this seems to be the line causing the trouble. My suspicion is the IO thread is in potential risk for an infinity loop!

The Apple documentation for NSRunLoop runmode:beforeDate: confirms the suspicion. Here's the quote

Return Value

YES if the run loop ran and processed an input source or if the specified timeout value was reached; otherwise, NO if the run loop could not be started.

Discussion

If no input sources or timers are attached to the run loop, this method exits immediately and returns NO; otherwise, it returns after either the first input source is processed or limitDate is reached. [...]

After further investigation the only reason for the run loop to immediately return is when both streams (input & output) are removed from the run loop. The reason could be either accidentally by SDL based code or by internal iOS code due to some error. All that without destroying/cancelling the IO thread in the data session instance. My hypothesis is the latter case. iOS removed the NSStream objects from the run loop for whatever reason. I will need to continue testing to find evidence for the hypothesis though.

For sure we need to investigate and find the root cause of the issue. Regardless adding robustness code to [SDLIAPDataSession sdl_accessoryEventLoop] to eliminate the risk of the infinity loop makes sense. Currently the code ignores the result code. My suggestion is to read the result and investigate the status of both streams, the transport instance and the connected accessories. At least an error message should be logged, possibly cancelling the loop if the run loop repeats returning NO. Additional logs about the streams or connected accessories helps investigate the root cause.

@joeljfischer this issue is affecting a couple of Ford apps and I also know of concerns to 3rd party apps. Anything that helps resolving this high priority issue is highly appreciated.

[ghost]

I have received some more information on how to reproduce the issue. Other testers report they have >12 apps and the phone is connected via bluetooth and can reproduce on SYNC3 MY20 which supports iAP via BT. Assuming a USB disconnect could cause some race conditions with so many apps.

Precondition:

• have as many SDL enabled applications installed as possible (>12)
• Head unit must support iAP via Bluetooth
• Connect the iPhone wirelessly to Instruments
• Start instruments energy log (log battery consumption) or activity monitor (log cpu load; probably more interesting)

Action:

1. Turn on
2. Connect iPhone to the head unit over USB
3. Wait until no more apps appear in the Apps area.
4. Disconnect iPhone from USB
5. Observe Instruments
   (6. Shut down head unit)

• Repeat until observed behavior is seen

Expected behavior:

• Most apps should reapper in the Apps area (except navigation apps)
• After apps appeared the phones load should be low
• Optional last step disconnects all apps again but CPU load should remain low

Observed behavior:

• At some point After USB disconnect or BT disconnect one or many apps start to show 100% cpu load.