-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Return error instead of panic when geo data is corrupted #4318
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 2 files at r1.
Reviewable status: 1 of 2 files reviewed, 1 unresolved discussion (waiting on @mangalaman93 and @pawanrawal)
query/query.go, line 399 at r1 (raw file):
if err != nil { // This can happen when a mutation ingests corrupt data into the database. return v, errors.Wrap(err, "Error while interpreting appropriate type from binary")
Is it possible to show in error message that we failed to convert into this type by inferring the typeID.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a small comment but otherwise it
Reviewed 2 of 2 files at r1.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @mangalaman93 and @pawanrawal)
query/query3_test.go, line 2259 at r1 (raw file):
}` _, err = dg.NewReadOnlyTxn().Query(ctx, query) require.Contains(t, err.Error(), "wkb: unknown byte order: 1111011")
So why does this fail? Maybe add a comment.
Also, if you can't query the data, should the mutate be allowed to succeed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 2 of 2 files at r1.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @balajijinnah and @mangalaman93)
query/query.go, line 399 at r1 (raw file):
Previously, balajijinnah (balaji) wrote…
Is it possible to show in error message that we failed to convert into this type by inferring the typeID.
We should add the attr and the type name to the error message. Also would be good to have an error log for this since this is a serious enough issue.
query/query3_test.go, line 2259 at r1 (raw file):
}` _, err = dg.NewReadOnlyTxn().Query(ctx, query) require.Contains(t, err.Error(), "wkb: unknown byte order: 1111011")
Are we messing up the byte order while storing the data? Little/big endian somewhere?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @balajijinnah, @martinmr, and @pawanrawal)
query/query.go, line 399 at r1 (raw file):
Previously, pawanrawal (Pawan Rawal) wrote…
We should add the attr and the type name to the error message. Also would be good to have an error log for this since this is a serious enough issue.
Added attr
to error information
query/query3_test.go, line 2259 at r1 (raw file):
Previously, martinmr (Martin Martinez Rivera) wrote…
So why does this fail? Maybe add a comment.
Also, if you can't query the data, should the mutate be allowed to succeed?
Done.
query/query3_test.go, line 2259 at r1 (raw file):
Previously, pawanrawal (Pawan Rawal) wrote…
Are we messing up the byte order while storing the data? Little/big endian somewhere?
Nope, we are storing string json data in geo value which should be marshalled using wkb.Marshal
function. This example shows how we could potentially use GeoVal
API but probably shouldn't.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @balajijinnah, @martinmr, and @pawanrawal)
query/query3_test.go, line 2259 at r1 (raw file):
Previously, mangalaman93 (Aman Mangal) wrote…
Done.
Ideally, mutation shouldn't success but this will happen for all the data types. Adding a check in mutation will probably slow down the mutation. It makes sense to look into it though.
9a00c31
to
2beb44d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 2 of 2 files at r2.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @balajijinnah, @martinmr, and @pawanrawal)
2beb44d
to
479c5e9
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 2 files at r2, 1 of 1 files at r3.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @balajijinnah, @martinmr, and @pawanrawal)
479c5e9
to
cc0c887
Compare
Fixes #3740
We can still add corrupted data into the database. I think, there should be separate effort to fix that. Adding corrupt data is possible for other types as well. This PR ensures that alpha doesn't crash when corrupt data is encountered while querying.
This change is