Bad value in untransformObject undefined

[zleman1593]

Make sure these boxes are checked before submitting your issue -- thanks for reporting issues back to Parse Server!

• You've met the prerequisites.
• You're running the latest version of Parse Server.
• You've searched through existing issues. Chances are that your issue has been reported or resolved before.

Environment Setup

Local

OSX 10.11.3
MongoDB on ObjectRocket v 3.0.9

Steps to reproduce

Clone this bug reproduction repo, npm install & npm start

https://github.com/zleman1593/parse-server-transform-bug

I migrated a copy of the database to object rocket and I am hosting it from there. This repo uses that db.

If you want to connect to a local db, here is the mongo dump you can use mongorestore with.

https://www.dropbox.com/s/gsfqvwp0lsien90/MongoDump.zip?dl=0
If you want a quick look at the problematic data, I exported it from parse (JSON)(I did not use this for the migration) : https://goo.gl/61X5Jv

Make this query:

curl -X GET \ -H "X-Parse-Application-Id: XXXXXXXXX" \ -H "X-Parse-Master-Key: XXXXXXXXXXX" \ -G \ --data-urlencode "where={\"clientId\":\"QArG4hBu8z\"}" \ http://localhost:1338/parse/classes/ProviderClient

Use this application ID: 3fko3frQqNK20Lx59EbfeP2X3u5Iy3eTstMsIze5 Use this master key: DFYYoY6iWXzgbSDxfZqJHrNaOfHZrpCHlH1vmBau
You can use other keys, just don't forget to switch them out in index.js

Query Response

{"code":1,"message":"Internal server error."}

Logs/Trace

error: Uncaught internal server error. bad value in untransformObject undefined

The error is throw here: https://github.com/ParsePlatform/parse-server/blob/f0ebb7bae6631493eb0fce608c4948ad37636b7c/src/transform.js in the untransformObject function when it comes across a function in the mongoObject when reading the lastMessageTimetoken BSON field which is a 'NumberLong'.

Issue #665 is likely related. @gfosco

[flovilmart]

Did you use the exported json data from parse.com to import in MongoDB?

[zleman1593]

No. I used the parse migration feature. I just provided the JSON above so you could see the structure of the collection.

[flovilmart]

I'm not sure what's going on here, the JSON your provided has numbers on the lastMessageTimetoken.
I've checked with creating a NumberLong from the mongoCLI, and it appears as just a number when read from the JS driver.

[zleman1593]

When exporting the JSON from Parse it formats it to be just a number. However after migrating to another mongoDB using Parse migration the lastMessageTimetoken is visible as NumberLong()

[flovilmart]

uhm.. that's very very odd

[zleman1593]

Did you run the reproduction project and see the error?

[zleman1593]

[flovilmart]

I inserted a document in my DB with a NumberLong:

db.test.Test.insert({'key': NumberLong(1234553243423423)})
WriteResult({ "nInserted" : 1 })
> db.test.Test.find()
{ "_id" : ObjectId("5702bf2759a7f8cecf200a56"), "key" : NumberLong("1234553243423423") }

and then run with the mongo driver:

> db.collection('test.Test').find().toArray(console.log)
undefined
> null [ { _id: 5702bf2759a7f8cecf200a56, key: 1234553243423423 } ]

[zleman1593]

From node REPL (v 5.*) with mongo driver (npm install mongodb) connected to the database provided in the original post I ran the following command
MongoClient.connect(url, function(err, db) { console.log("Connected correctly to server");db.collection('ProviderClient').find({}).toArray(console.log) });

and got:

{ _id: '02P1Y3vpHv', locations: [ 'QgvB76eweL' ], providerId: '88crvxRz79', isArchived: 0, referringProviders: [ 'kV865l1h2D' ], _updated_at: Tue Dec 29 2015 08:31:07 GMT-0800 (PST), clientId: 'k5UrERc5NI', _created_at: Tue Dec 22 2015 11:01:31 GMT-0800 (PST), practiceId: 'aQww4PQ9DU', lastMessageTimetoken: Long { _bsontype: 'Long', low_: 1947518118, high_: 3379319 } },

[flovilmart]

What MongoDB version are you using?

[flovilmart]

Noticed you use node 5.+, I was testing on 4.3

[zleman1593]

3.0.9 hosted on Object Rocket

I can switch to 4.3 and see if any difference.

[zleman1593]

I got the same result using node v4.3.2 (npm v2.14.12)

[zleman1593]

If I switch from using:
db.collection('ProviderClient').find({}).toArray(console.log)

to
db.collection('ProviderClient').find({}).forEach(function (item){console.log(JSON.stringify(item))}) });
I get:

{"_id":"02P1Y3vpHv","locations":["QgvB76eweL"],"providerId":"88crvxRz79","isArchived":0,"referringProviders":["kV865l1h2D"],"_updated_at":"2015-12-29T16:31:07.113Z","clientId":"k5UrERc5NI","_created_at":"2015-12-22T19:01:31.665Z","practiceId":"aQww4PQ9DU","lastMessageTimetoken":"14514066535269542"}

[flovilmart]

Interesting, does the item have a toJSON function or something that would help us sanitize the data at large?

[flovilmart]

nodejs is not capable of representing 64 bits integers, so it wraps them into strings. Not sure this is what you'd expect on the client.

[flovilmart]

@zleman1593 quick question like that, does your number longs are actually 64bit ints?

[zleman1593]

The lastMessageTimetoken field it is set in Parse by
providerClient.set("lastMessageTimetoken", parseFloat(timetoken));
where timetoken is a string. Floats are 32 bit.

[flovilmart]

and it comes from Parse.com? @gfosco can you investigate that?

If they are floats, I suggest you run an mongodb op. to update your LongNumber to regular numbers as you won't loose precision.

[zleman1593]

Yes. All the data were created while still on parse.com

[sdpgames]

hi guys
had the same problem, looks like the date type is full of conversion functions (toInt, etc) that Parse doesn't want to convert. All our dates had the same bug as zleman1593, same scenario too after a migration. Update: though the user on which the crash was happening was new (created after migration)

We use Ubuntu self hosted + parse-server + mongodb (ubuntu also) + rocksdb (self hosted)

As a temporary fix we have done this in untransformObject ( transform.js )

case 'symbol':
throw 'bad value in untransformObject ' + className + ' ' + JSON.stringify(mongoObject);
case 'function':
return mongoObject;

It works so far but probably opens security breach or even crashes later on, so please provide a fix;

Thanks;
Laurent