[Barcode / QRCode] Crash NewStringUTF with libart.so

[NasH5169]

Hi all,

The native library libart.so used by ML is crashing. As far as I know, there are 2 ways to crash:

1. The camera has something wrong with utf8 char in its params (only Galaxy S5 for now)
2. The barcode or qrcode contains non utf8 char

Both are the used of NewStringUTF without check to string before?

• Android device: Galaxy S5 klte
• Android OS version: Android 7.1.2
• Google Play Services version: 12.6.85
• Firebase/Play Services SDK version: 16.0.0

Please find below the 2 crash stacks. Both are generated with Vision API but it is excatly the same with ML sample application and I think this repository will be more followed than old vision API.

As a reminder, the link to the original issue from googlesamples/android-vision repo:

googlesamples/android-vision#221

The crash due to invalid char in the qrcode

06-19 09:10:17.459 30848-31438/ A/art: art/runtime/java_vm_ext.cc:410] JNI DETECTED ERROR IN APPLICATION: input is not valid Modified UTF-8: illegal start byte 0xae
    art/runtime/java_vm_ext.cc:410]     string: 'Test Code with �'
    art/runtime/java_vm_ext.cc:410]     in call to NewStringUTF
06-19 09:10:17.460 30848-31438/ A/art: art/runtime/java_vm_ext.cc:410]     from com.google.android.gms.vision.barcode.internal.NativeBarcode[] com.google.android.gms.vision.barcode.internal.NativeBarcodeDetector.recognizeNative(int, int, byte[], com.google.android.gms.vision.barcode.internal.NativeBarcodeDetector$NativeOptions)
    art/runtime/java_vm_ext.cc:410] "Thread-1397" prio=5 tid=25 Runnable
    art/runtime/java_vm_ext.cc:410]   | group="main" sCount=0 dsCount=0 obj=0x12d0a0a0 self=0xab8afee0
    art/runtime/java_vm_ext.cc:410]   | sysTid=31438 nice=0 cgrp=default sched=0/0 handle=0xd6914930
    art/runtime/java_vm_ext.cc:410]   | state=R schedstat=( 6929244074 23452034 473 ) utm=687 stm=5 core=0 HZ=100
    art/runtime/java_vm_ext.cc:410]   | stack=0xd6812000-0xd6814000 stackSize=1038KB
    art/runtime/java_vm_ext.cc:410]   | held mutexes= "mutator lock"(shared held)
    art/runtime/java_vm_ext.cc:410]   native: #00 pc 0035cecd  /system/lib/libart.so (_ZN3art15DumpNativeStackERNSt3__113basic_ostreamIcNS0_11char_traitsIcEEEEiPKcPNS_9ArtMethodEPv+116)
    art/runtime/java_vm_ext.cc:410]   native: firebase/quickstart-android#1 pc 0033d9a3  /system/lib/libart.so (_ZNK3art6Thread4DumpERNSt3__113basic_ostreamIcNS1_11char_traitsIcEEEE+138)
    art/runtime/java_vm_ext.cc:410]   native: firebase/quickstart-android#2 pc 0024f6a1  /system/lib/libart.so (_ZN3art9JavaVMExt8JniAbortEPKcS2_+760)
    art/runtime/java_vm_ext.cc:410]   native: firebase/quickstart-android#3 pc 0024fd3f  /system/lib/libart.so (_ZN3art9JavaVMExt9JniAbortVEPKcS2_St9__va_list+54)
    art/runtime/java_vm_ext.cc:410]   native: firebase/quickstart-android#4 pc 000fc1b3  /system/lib/libart.so (_ZN3art11ScopedCheck6AbortFEPKcz+30)
    art/runtime/java_vm_ext.cc:410]   native: firebase/quickstart-android#5 pc 00101c71  /system/lib/libart.so (_ZN3art11ScopedCheck5CheckERNS_18ScopedObjectAccessEbPKcPNS_12JniValueTypeE.constprop.95+8096)
    art/runtime/java_vm_ext.cc:410]   native: firebase/quickstart-android#6 pc 001088e7  /system/lib/libart.so (_ZN3art8CheckJNI12NewStringUTFEP7_JNIEnvPKc+374)

The crash due to camera getParameters()

06-19 09:05:06.588 11732-11732/ A/art: art/runtime/java_vm_ext.cc:470] JNI DETECTED ERROR IN APPLICATION: input is not valid Modified UTF-8: illegal continuation byte 0x17
    art/runtime/java_vm_ext.cc:470]     string: '��Ŷ��Ŷ��Ŷ��Ŷ=��Ŷ��Ŷ;��Ŷ��Ŷ��Ŷ=;ae-bracket-hdr=Off;ae-bracket-hdr-values=Off,AE-Bracket;anti-shake=0;antibanding=50hz;antibanding-values=off,60hz,50hz,auto;auto-exposure-lock=false;auto-exposure-lock-supported=true;auto-whitebalance-lock=false;auto-whitebalance-lock-supported=true;brightness-step=1;camera-mode=0;contrast=5;contrast-step=1;denoise=denoise-on;denoise-values=denoise-off,denoise-on;dis=disable;dis-values=enable,disable;dual_mode=0;dualrecording-hint=0;effect=none;effect-values=none,mono,negative,solarize,sepia,posterize,whiteboard,blackboard,aqua,emboss,sketch,neon;exif_exptime=0;exif_iso=0;exposure-compensation=0;exposure-compensation-step=0.5;face-detection=off;face-detection-values=off,on;face-recognition=off;face-recognition-values=off,on;fast-fps-mode=0;firmware-mode=none;flash-mode=off;flash-mode-values=off,auto,on,torch;flip-mode-values=off,flip-v,flip-h,flip-vh;fnumber-value-denominator=100;fnumber-value-numerator=220;focal-length=4.80
    art/runtime/java_vm_ext.cc:470] ;intelligent-mode=0;iso=auto;iso-values=auto,ISO_HJR,100,200,400,800,1600;jpeg-quality=96;jpeg-thumbnail-height=288;jpeg-thumbnail-quality=85;jpeg-thumbnail-size-values=512x288,480x288,256x154,432x288,512x384,352x288,320x240,176x144,0x0;jpeg-thumbnail-width=512;lensshade=enable;lensshade-values=enable,disable;llv_mode=0;luma-adaptation=3;max-brightness=6;max-contrast=10;max-exposure-compensation=4;max-num-detected-faces-hw=10;max-num-detected-faces-sw=10;max-num-focus-areas=1;max-num-metering-areas=10;max-saturation=10;max-sce-factor=100;max-sharpness=36;max-zoom=63;maxaperture-value-denominator=100;maxaperture-value-numerator=228;mce=enable;mce-values=enable,disable;metering=center;metering-areas=(0,0,0,0,0);metering-values=matrix,center,spot;min-brightness=0;min-contrast=0;min-exposure-compensation=-4;min-saturation=0;min-sce-factor=-100;min-sharpness=0;num-snaps-per-shutter=1;picture-format=jpeg;picture-format-values=jpeg,bayer-qcom-10gbrg,bayer-qcom-10grbg,bayer-qcom-10rgg
    art/runtime/java_vm_ext.cc:470]     input: '0x01 0x18 0xc5 0xb6 0x17 0x18 0xc5 0xb6 0xe1 <0x17> 0xc5 0xb6 0xe5 0x17 0xc5 0xb6 0x3d 0xe1 0x17 0xc5 0xb6 0xe5 0x17 0xc5 0xb6 0x3b 0x17 0x18 0xc5 0xb6 0xe1 0x17 0xc5 0xb6 0xe5 0x17 0xc5 0xb6 0x3d 0x3b 0x61 0x65 0x2d 0x62 0x72 0x61 0x63 0x6b 0x65 0x74 0x2d 0x68 0x64 0x72 0x3d 0x4f 0x66 0x66 0x3b 0x61 0x65 0x2d 0x62 0x72 0x61 0x63 0x6b 0x65 0x74 0x2d 0x68 0x64 0x72 0x2d 0x76 0x61 0x6c 0x75 0x65 0x73 0x3d 0x4f 0x66 0x66 0x2c 0x41 0x45 0x2d 0x42 0x72 0x61 0x63 0x6b 0x65 0x74 0x3b 0x61 0x6e 0x74 0x69 0x2d 0x73 0x68 0x61 0x6b 0x65 0x3d 0x30 0x3b 0x61 0x6e 0x74 0x69 0x62 0x61 0x6e 0x64 0x69 0x6e 0x67 0x3d 0x35 0x30 0x68 0x7a 0x3b 0x61 0x6e 0x74 0x69 0x62 0x61 0x6e 0x64 0x69 0x6e 0x67 0x2d 0x76 0x61 0x6c 0x75 0x65 0x73 0x3d 0x6f 0x66 0x66 0x2c 0x36 0x30 0x68 0x7a 0x2c 0x35 0x30 0x68 0x7a 0x2c 0x61 0x75 0x74 0x6f 0x3b 0x61 0x75 0x74 0x6f 0x2d 0x65 0x78 0x70 0x6f 0x73 0x75 0x72 0x65 0x2d 0x6c 0x6f 0x63 0x6b 0x3d 0x66 0x61 0x6c 0x73 0x65 0x3b 0x61 0x75 0x74 0x6f 0x2d 0x65 0x
    art/runtime/java_vm_ext.cc:470]     in call to NewStringUTF
    art/runtime/java_vm_ext.cc:470]     from java.lang.String android.hardware.Camera.native_getParameters()

[isaisachen]

Thanks for reporting. Firebase / Google will take a look.

Sounds like firebase/quickstart-android#2 has a consistent way to repro? Do you know if there's a consistent way to repro issue firebase/quickstart-android#1? Thanks!

[NasH5169]

Hi,

Great. Thanks for taking this issue in account.

For the first crash, it is complicated because the Galaxy S5 is on LineageOS 14.1 (Android 7.1.2).

This is the only device with which I have this issue.

[Kinchkun]

Hi,

we have the same issue. The issue arise when reading non-utf8 encoded datamatrix. In our case we are trying to read Latin1 resp. ISO 8859-1 encoded strings from datamatrix.

I think this is also the same issue: googlesamples/android-vision#151

As suggested in the linked github issue, it would be sufficient to provide access to the raw byte array from the datamatrix, so that the application can choose the encoding itself.

[samtstern]

Hi all. We have been able to reproduce this bug internally on some Samsung devices and are working on a fix, thanks for your patience.

[rkshnair]

@samtstern : Any update or release regarding this issue? I can find the same issue in both MLKit and Android Mobile Vision

[oastolen]

We are also experiencing this issue and find that it is a significant problem for us. Any update would be welcome.

[panos-stavrianos]

We are also have the same issue and it's of vital importance. Until this fixes, is there any workaround?

[xianfeng-zhu]

Is this issue solved?

[david99999]

Hi everyone, can somebody that has access to the native code at least add a try/catch statement or something like that to the code that is trying to convert the bytes to a String representation?
I had the expectation that firebase ML could fix this bug because it has been several times asked for a solution here here here and here, but it seems that Firebase ML is only a copy-paste from the GMS Play services

Here you can see that its code is calling the same GMS classes

JNI DETECTED ERROR IN APPLICATION: input is not valid Modified UTF-... in call to NewStringUTF from com.google.android.gms.vision.barcode.internal.NativeBarcode[]...

Attached is a more detailed log if somebody wants to dig deeper

stack_trace.txt

[kobronson]

Same issue here, occurring on all devices when scanning barcodes with special chars. Any chances to fix that issue? We started considering using other barcode library.

Example barcode with special chars:

[david99999]

Hi @samtstern it has been almost a year since you said this https://github.com/firebase/quickstart-android/issues/542#issuecomment-415075156 can you please let us know if at least you guys can add the option to the ML Kit for setting the reader to only return the detected byte[] and let us do the parsing/encoding?
Thanks, hope you can finally help us here

[ShuGc]

Alguien ya encontró una solución para esto?

[GarryKelly]

Hi,

I came across this when trying to use ML kit to scan some barcodes. Its a really brilliant scanning library, but this issue stops it being used for some complex applications

I dont know if its an issue with converting between encoding or if it cant decode some 8 bit characters
Ive created a test barcode the barcode is 256 bytes values 0x00 to 0xff When I scan with the latest ML
com.google.firebase:firebase-ml-vision-barcode-model:16.1.1
it returns a byte array of 142 characters.... I tried it with smaller barcodes and its the same issue, always around certain character codes....

Hope this helps, Garry

[sheepmaster]

@GarryKelly, it doesn't surprise me if the returned string contains fewer characters – the library is trying to interpret the characters as UTF-8, and characters with the highest-order bit set (which are not valid ASCII characters) are interpreted as part of a multi-byte sequence. Some of those characters runs might form a valid Unicode codepoint, but others don't, and the invalid sequences are likely ignored.

AFAICT, the crash issue originally reported has been fixed since some time late 2019; the library doesn't call NewStringUTF() anymore to construct Java strings.