Crashs since Xcode 14.3 compilation CIContext#render.

[Goule]

Describe the bug

Hello,

Since I compiled my app with Xcode 14.3 I've noticed some crashes:

It's hard to put code here since there are a lot of contexts with the app. But if someone has had the same issue since the Xcode 14.3 compilation maybe it can help.

To Reproduce

1. It appear for some user when the stream start or randomly
2. We have some simple video effect with images
3. We are pushing sample buffer from the camera

Expected behavior

No crash

Version

• HaishinKit 1.3.0 (we are not up to date so maybe it's fixed in the 1.4.5 ?)

Smartphone info.

• iOS 16.4, 16.3 and 16.1
• Xcode 14.3

Additional context

No response

Screenshots

No response

Relevant log output

No response

[shogo4405]

I have heard that Xcode 14.3 has bugs. Even if you update to 14.5, I think crashes will still occur.
Can I see “Données” tab? I want to know model names and free RAM size. By the way, are you using the patch for #986?

[Goule]

I've downgraded my Xcode version and all seems to be ok. It's probably a bug with Xcode 14.3...

Yes we are using the #986 patch !

Here are some data :

[shogo4405]

There may be a possibility that the detailed behavior of DispatchQueue has changed for situations with low memory capacity. Since all of them have low memory capacity, I'm concerned about the possibility of memory leaks occurring in Xcode 14.3.

I will investigate. thank you.

[Goule]

Thanks @shogo4405 !

[shogo4405]

I'm having difficulty reproducing the issue. I believe that the application crashes due to limited available memory, so I recommend that you limit memory usage on the application side to prevent this.
If it's a sample app, it uses around 110MB of memory. We haven't confirmed any memory leaks on the framework side.

Please let me know if there are other people who have the same issue.

[heestand-xyz]

I'm experiencing a similar issue. Crashes here and here.
The devices were low memory when the EXC_BAD_ACCESS happened.
Xcode 14.3 on iOS 16.1.1 & 16.2.0 with HaishinKit 1.4.2

[Goule]

Xcode 14.3.1 is out, hope it will fix it !

[Goule]

The issue still here with XCode 14.3.1 :(

[floriangbh]

Hello,

I have the same issue, but hard to reproduce, there is no tool to overhead the memory of the device...

Did you manage to reproduce this ?

[leo150]

Just a guess to figure out the source, maybe try to use no optimization for target's Build Settings - Swift Compiler - Optimization Level? And test it on small selection of user base.

[leo150]

Recalling a similar issue that happened on iOS 15 (?). There was a bug when user connects headphones, this would make the encoding to freeze for a few seconds, while appendSampleBuffer keeps executing until there's no memory. So the workaround was to wait until videoIO finishes encoding:

if self.limit.value > someNumber {
    return
}

self.limit.mutate { $0 += 1 }

mixer.videoIO.lockQueue.async {
    self.mixer.videoIO.encodeSampleBuffer(sampleBuffer)
    
    self.limit.mutate { $0 -= 1 }
}

[floriangbh]

Thanks @leo150 for the research lead, I'll try. I admit that I was waiting to see under Xcode 15 if this fixes anything..

[leo150]

Does anyone have luck with XCode 15? I can't compile with 14.2 anymore because of the recent changes.

[floriangbh]

Also waiting for feedbacks with Xcode 15 😅

[shogo4405]

Oops! The app being killed due to OOM (Out of Memory) when it exceeds device limits is an iOS specification. I believe it's up to the application to control memory usage and ensure it doesn't exceed the limits. Is everyone pausing xcode updates for now?

[floriangbh]

The weird things is that OOM appear since Xcode 14.3 compilation...
I haven't tested yet with 15.0, maybe in november with progressive release...

[Goule]

Hello,

I've just tested on XCode 15.0.1.
Same issue with HaishinKit v1.6.1, crash appear always with low memory.
I don't know how to investigate on it..

@shogo4405 do you think it's leak on our side? Or in the library?

HaishinKit and Sonoma aren't compatible with XCode 14.2 so I can't use it to build :(

Technical details : I'm using self.rtmpStream.appendSampleBuffer(frame) to push frame from custom AVCaptureVideoDataOutputSampleBufferDelegate

@heestand-xyz did you fix the issue on your side ?

[floriangbh]

In my case the crash appear only in release mode, with GCC_OPTIMIZATION_LEVEL = s and SWIFT_OPTIMIZATION_LEVEL = "-O", only when the available memory is low. Really quickly after the controller with the NetStream and preview appear.

I tried to figure out if there is a memory leak with the profiler, but all seem to be stable even in release mode.

The crash appears here :

I'm also using appendSampleBuffer instead of camera attached.

[shogo4405]

@Goule Goule In that usage, if you follow the next steps, you can save memory usage and reduce the chances of crashing due to memory shortage:

On the app side, set the following value:

• Set the value to AVCaptureVideoDataOutput.alwaysDiscardsLateVideoFrames = true.

For HaishinKit, you need to make the following adjustments:

• Remove the section mixer.audioIO.lockQueue of mixer.videoIO.lockQueue in HaishinKit.
  • Link
  • When processing falls behind, data accumulates in these lockQueues, causing memory pressure.
• Perform stream.appendSampleBuffer on the same queue as set with AVCaptureVideoDataOutput.setSampleBufferDelegate.

[shogo4405]

@floriangbh Is "attachScreen" for iOS or macOS?

[Goule]

Thanks for your help @shogo4405 !

I will try with your recommendation.

Before that I've tested with rtmpStream.attachCamera, and still have the same issue. So it's no coming from the extra code around the appendSampleBuffer I think...

I'll try to reproduce it with the sample project !

[shogo4405]

It is difficult to fix the issue of crashing due to memory capacity limits. I believe that what was previously undetectable as OOM can now be detected as crashes through Firebase. I think it's better to come up with solutions to avoid reaching memory limits rather than trying to fix it afterwards.

I'm adding code to the example that deliberately leads to memory shortage. It uses MemoryUsage.toEmpty() to use up memory without causing an OOM (Out of Memory) situation to that extent.
https://github.com/shogo4405/HaishinKit.swift/blob/main/Examples/iOS/MemoryUsage.swift#L15-L18

[Goule]

I'm not sure this is an OOM :

• the application uses only 130/150Mb
• an OOM is easily detectable with Xcode, after the application closes a message appears with "...has been killed by the operating system because it is using too much memory."

It look more like a memory corruption with a bad access to unaloccated object or something else.

I can reproduce it with Xcode, the memory when the app crash seem to be ok, with 750MB free :

I will try to reduce the scope of my app with only simple screen to isolate the crash.

[shogo4405]

I can reproduce it.

Prerequisites

• GCC_OPTIMIZATION_LEVEL = s
• WIFT_OPTIMIZATION_LEVEL = "-O"
• Prepare two MTHKViews. And stream.attachCamera, stream.attachAudio

Reproduce

1. mthkView1.attachStream(stream)
2. mthkView1.attachStream(nil)
3. mthkView2.attachStream(stream)
   1. Crash CIContext#render method.

How to fix it.

https://github.com/shogo4405/HaishinKit.swift/blob/main/Sources/Media/MTHKView.swift#L71

currentStream.mixer.videoIO.context = CIContext(mtlDevice: device!)

currentStream.mixer.videoIO.lockQueue.async {
  self.currentStream.mixer.videoIO.context = CIContext(mtlDevice: device!)
}