Empty datetime parsing issue

[Levatius]

Version: v1.0.4

Observed problem:
We suspect it is somehow possible to insert an empty string ("") as a datetime but have been unable to replicate it.
The reason we think this is due to what we see here:

The _email_address_hash_last_checked predicate is of type datetime and is indexed by hour; the schema alter that sets this up occurs before any data is added to the graph.

When trying to query this particular field as above, we get the error:

: rpc error: code = Unknown desc = parsing time "" as "2006": cannot parse "" as "2006"

Attempting to replicate the issue, mutating an empty string onto such a predicate gives us the error:

parsing time "" as "2006": cannot parse "" as "2006"

Then question then becomes, how did an empty string enter the graph as a datetime if it is not possible to mutate it in?

The only way I can see this happening is if the schema definition for the predicate is at any point lost allowing an empty string to be inserted before the definition gets restored*.

*At the start of every processing task, we check whether any definitions are missing and add them to the schema if so.

[pawanrawal]

This can also happen if the predicate initially had default type and an empty value was inserted. Then if you try and do a schema mutation, we skip errors while indexing which we should only do with predicates which have lang tag (for full-text search). Other errors should be returned to the user.

This could be related to predicate move somehow. Do you verify the exact schema for a predicate or only whether the key for the predicate is returned before doing a schema mutation?

[Levatius]

This can also happen if the predicate initially had default type and an empty value was inserted.

Indeed, we think this does not happen initially however because we encountered the error far through the test meaning that the predicate in question had already accepted many non-empty datetimes as properly indexed datetimes.

Do you verify the exact schema for a predicate or only whether the key for the predicate is returned before doing a schema mutation?

Only whether the key exists in the schema:

• If it does, we do not attempt to re-add the key.
• If it does not, we then make a schema alteration to add it with its appropriate index.

It might be worth noting that this particular predicate being empty is even more strange since it is not imported from the original dataset but generated through python's datetime.now() function (using it to record the last time we modified the node) hence we would never expect it to generate an empty value.

[gitlw]

@Levatius Is it ok for you to share your tests so that we can try to reproduce this error?

[srfrog]

Steps to reproduce:

1. Set some correct datetime data and a few empty strings. No schema or string predicate.

{
  "set": [
    {
      "created": "2016-01-15T0:00:00.000Z",
      "name": "first"
    },
    {
      "created": "2017-01-17T0:00:00.000Z",
      "name": "second"
    },
    {
      "created": "",
      "name": "third"
    }
  ]
}

3. Try to alter predicate to: created: datetime @index(year) .
4. Fails

The problem is that "" will never parse to a Go time.Time. We need to convert "" to zero time value.