Barcode scanning fails with "Unknown encoding" for ISO-8859-1 encoded data matrix

[dspoeri]

The official German medication plan data matrix ("BMP", Bundeseinheitlicher Medikationsplan) expects data to be encoded with ISO-8859-1. If the data contains a German umlaut, Google Vision barcode scanning fails with an "Unknown encoding" error.

Scanning the following data matrix reproduces the bug:

This bug sadly renders Google Vision barcode scanning useless for the mentioned use case.

Two suggested solutions:

• accept other encodings than ASCII and UTF-8
• provide access to the raw data through a byte array

[GarryKelly]

Just saw this and it is similar to an issue reported last year . I commented on that here #44 (comment) Unfortunately meant the library just didnt work for our use cases.... Its a shame as its an excellent library otherwise....

I agree with the suggested solutions... it would be wonderful for the library to either support the ISO-8859-1 characterset as an option. Or else to provide access to the scanned data as a byte array without going through any character set conversions... Both options would allow reading of all barcodes

I noticed there was some new version com.google.firebase:firebase-ml-vision-barcode-model:16.1.2 released later in 2020 but havent had time to see if these provided that access...

[ivan200]

At least com.google.mlkit:barcode-scanning:16.1.0
contains barcode.rawBytes

Returns raw bytes as it was encoded in the barcode.
Returns null if the raw bytes can not be determined.

so I think you can make
return String(barcode.rawBytes, StandardCharsets.ISO_8859_1)

[dspoeri]

At least com.google.mlkit:barcode-scanning:16.1.0
contains barcode.rawBytes

Returns raw bytes as it was encoded in the barcode.
Returns null if the raw bytes can not be determined.

so I think you can make
return String(barcode.rawBytes, StandardCharsets.ISO_8859_1)

It doesn't help:
rawBytes returns an array with 16 bytes representing the string Unknown encoding.

[cs-googler]

Hi, we are working on a fix internally.

[pke]

so @cs-googler how is the internal fix going? How about letting the user specify the encoding via BarcodeScannerOptions?

[mattemyoo]

@cs-googler

Hello. We are also trying to scan barcodes that include letters with umlauts.

When we are scanning a barcode that contains "ä", it becomes a "d". Even in the rawBytes array, we are getting the value 100, which corresponds to ASCII character "d".

@pke Did you get any new information about this?

[mattemyoo]

So, I did some research, and it seems like the int type that you are using is only covering the first 128 characters, and then the value starts from 0 again.

For example, the character "µ" has the actual ASCII number 181, but in the rawBytes, I am getting the value 53 (ASCII char "5").

Another example is the inverted exclamation mark (¡ with ASCII number 161). The actual rawByte I am receiving is 33 (161 - 128 = 33)