Application Crashing with Multimodal Models

[Vinayak006]

Description:

I’m encountering an application crash while trying to run a multimodal model. The application crashes when attempting to use the following models:

• Primary Model: MobileVLM-1.7B-Q4_K.gguf
• Multimodal Project Model: MobileVLM-1.7B-mmproj-f16.gguf

Steps to Reproduce:

1. Load the primary model (MobileVLM-1.7B-Q4_K.gguf).
2. Load the multimodal project model (MobileVLM-1.7B-mmproj-f16.gguf).
3. Run the prompt.
4. Observe the crash.

Expected Behavior:

The application should run without crashing and successfully process the multimodal models.

Actual Behavior:

The application crashes during the execution phase when using the specified models.

Error Messages / Logs:

I/flutter (30087): [llama.cpp] Loading multimodal model...
F/libc    (30087): Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xdeadc05d in tid 30689 (1.ui), pid 30087 (.fllama_example)
Process name is com.example.fllama_example, not key_process
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
Build fingerprint: 'Realme/RMX1851/RMX1851:11/RKQ1.201217.002/1645426024988:user/release-keys'
Revision: '0'
ABI: 'arm64'
Timestamp: 2024-08-28 15:27:45+0530
pid: 30087, tid: 30689, name: 1.ui  >>> com.example.fllama_example <<<
uid: 12566
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xdeadc05d
    x0  0000007ea60f5ca8  x1  0000007ea6c96c80  x2  00000000deadc04d  x3  0000007ea6c96c80
    x4  0000000000335400  x5  0000000000335400  x6  0000000000000000  x7  0000000000000000
    x8  6ebfdbf6d2949538  x9  0000000000000000  x10 0000007ea6c96c89  x11 0000000000335400
    x12 0000000000000004  x13 00000000000002da  x14 0000007ea5d3aa08  x15 0000000000000001
    x16 0000007ea60b14c0  x17 0000007ea5fced0c  x18 0000007e99ae4000  x19 0000007ea6c96b10
    x20 0000007ea60f5ca8  x21 0000000000000001  x22 0000007ea6c96c80  x23 0000000000000001
    x24 0000007ea5d3c000  x25 000000000000001c  x26 0000007ea09d4f00  x27 00000000000002d9
    x28 0000000000001200  x29 0000007ea5d3a9d0
    lr  0000007ea5e67f18  sp  0000007ea5d3a9a0  pc  0000007ea5fced48  pst 0000000080000000
backtrace:
      #00 pc 000000000028ed48  /data/app/~~4RleT6zBs3xvct1NPgI11A==/com.example.fllama_example-_oVD-W7mbTlVC-tEIe-Iuw==/base.apk!libfllama.so (offset 0xf27000) (BuildId: ec5a262cd245778f6f481ce33a091b78ce7badb7)
      #01 pc 0000000000127f14  /data/app/~~4RleT6zBs3xvct1NPgI11A==/com.example.fllama_example-_oVD-W7mbTlVC-tEIe-Iuw==/base.apk!libfllama.so (offset 0xf27000) (BuildId: ec5a262cd245778f6f481ce33a091b78ce7badb7)
      #02 pc 0000000000127058  /data/app/~~4RleT6zBs3xvct1NPgI11A==/com.example.fllama_example-_oVD-W7mbTlVC-tEIe-Iuw==/base.apk!libfllama.so (offset 0xf27000) (clip_model_load+12872) (BuildId: ec5a262cd245778f6f481ce33a091b78ce7badb7)
      #03 pc 0000000000118964  /data/app/~~4RleT6zBs3xvct1NPgI11A==/com.example.fllama_example-_oVD-W7mbTlVC-tEIe-Iuw==/base.apk!libfllama.so (offset 0xf27000) (fllama_inference_sync+2468) (BuildId: ec5a262cd245778f6f481ce33a091b78ce7badb7)
      #04 pc 0000000000112794  /data/app/~~4RleT6zBs3xvct1NPgI11A==/com.example.fllama_example-_oVD-W7mbTlVC-tEIe-Iuw==/base.apk!libfllama.so (offset 0xf27000) (BuildId: ec5a262cd245778f6f481ce33a091b78ce7badb7)
      #05 pc 0000000000111aec  /data/app/~~4RleT6zBs3xvct1NPgI11A==/com.example.fllama_example-_oVD-W7mbTlVC-tEIe-Iuw==/base.apk!libfllama.so (offset 0xf27000) (InferenceQueue::process_inference()+892) (BuildId: ec5a262cd245778f6f481ce33a091b78ce7badb7)
      #06 pc 00000000001125a8  /data/app/~~4RleT6zBs3xvct1NPgI11A==/com.example.fllama_example-_oVD-W7mbTlVC-tEIe-Iuw==/base.apk!libfllama.so (offset 0xf27000) (BuildId: ec5a262cd245778f6f481ce33a091b78ce7badb7)
      #07 pc 00000000000da278  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+64) (BuildId: e81bf516b888e895d4e757da439c8117)
      #08 pc 000000000007a448  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64) (BuildId: e81bf516b888e895d4e757da439c8117)
Lost connection to device.

Environment:

• Device: Android
• Device version: Android 11