64-bit integers lose precision

[MegaByte]

Environment details

• OS: GCF
• Node.js version: 8.15.0/10.15.3

Steps to reproduce

1. Have data using 64-bit integers with values > 2^53 stored using a non-JS client (e.g. including 5248794116772919364 and larger values)
2. Read the data.
3. Use an orderBy keyed on that field with a startAfter value > 2^53 (e.g. 5248794116772919364)

Actual results:

1. The reported value is rounded. The document is returned with 5248794116772919000.
2. The cursor value is rounded. The document including the 5248794116772919364 value is returned (with 5248794116772919000 as above).

Expected results:

1. The value should not be rounded.
2. The document with the 5248794116772919364 value should not be returned.

In addition, it is not possible to use an alternate type to avoid the numeric rounding. Using a string in startAfter results in no data returned at all. Using a BigInt (Node 10 only) gives the error:

Unexpected value type

See also protocolbuffers/protobuf#3666

[schmidt-sebastian]

@MegaByte Thanks for reporting. This is currently working as intended. We rely on the Number semantics of JavaScript. We may evaluate adding support for BigInt once we phase out Node 8 support. Support for BigInt is however not currently on our roadmap, so don't hold your breath (PRs welcome though!)

[cubuspl42]

@schmidt-sebastian
How is it possible, that the official SDK for accessing Firestore is loosing information, and that's an intended behaviour?

It's not only about precision. Currently, it's absolutely impossible to even learn that the fields stored in database is actually an Integer, or write an Integer to a database.

I believe that it's an officially supported to access Firestore database from multiple platforms. For example, when I'm developing a mobile app in Flutter (Dart) and want to provide some backend logic for it, isn't it an supported, or even encouraged, solution to use Google Cloud Functions with Javascript SDK?

When my application stores an Integer field in the database, how am I supposed to update it / export it? It's beyond inconvenient. It makes a simple task impossible.

At very least, you could box all Integers with a helper Integer class, a bit like it works with Timestamps. It wouldn't really solve the precision issue (9 bits of precision would be lost), but it would allow to read/write an officially supported data type using an official SDK. That's a start.

[schmidt-sebastian]

We are working on dropping Node 8 support across all GCloud SDKs. Once Firestore has been migrated, we can add support for Node 10's BigInt.

Note that we are offering Server SDKs in 7 different languages and are using idiomatic number types on all platforms. While not ideal, if you need help now, you may be better off using a language environment that supports full integer precision by default.

[MegaByte]

The other problem is that the official admin interface also uses this library, which makes it literally impossible to manage that data properly if you don't have another way to access it, and it produces the inaccurate data silently.

[cubuspl42]

@MegaByte You mean the web console? You are absolutely right, it's also unacceptable that the official web interface isn't able to show what type the field really is or not able to produce a variable of certain type. Actually, I made an issue about it a few days ago:

https://issuetracker.google.com/issues/152898936

[schmidt-sebastian]

This has been merged, but won't be released until we push our next major version.

[cubuspl42]

That's great news, thank you! When this is released, could you please ping the team responsible for the Web Console, so they can use it to display/edit the Int64 data type?

[schmidt-sebastian]

We haven't yet committed to fixing this in the Console. While I agree that losing precision is a major problem here as well, the solution is not as straightforward as BigInt support is still not widely available in the browsers we target.

[schmidt-sebastian]

This is part of the 4.0.0 release, which will go out momentarily. You may enable this using the useBigInt flag in our settings.

[andreemic]

Is there a way to use this in "firebase-admin"