-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
importccl: set roachpb.Value checksums when creating data #24128
Conversation
Review status: 0 of 2 files reviewed at latest revision, 1 unresolved discussion, some commit checks pending. pkg/ccl/importccl/csv.go, line 711 at r1 (raw file):
Adding the call to Comments from Reviewable |
Yes, the test I added produces the error in the bug. Review status: 0 of 2 files reviewed at latest revision, 1 unresolved discussion, some commit checks pending. pkg/ccl/importccl/csv.go, line 711 at r1 (raw file): Previously, nvanbenschoten (Nathan VanBenschoten) wrote…
Sadly this function (writeRocksDB) is only called in the local version of import, so adding it here would also mean adding it in a second place for the distributed version. The place where it is now is best because that's the latest point before the implementations diverge. Comments from Reviewable |
How does this interact with #24126? Do we still need this for mixed-version clusters or is it obsolete? Review status: 0 of 2 files reviewed at latest revision, 1 unresolved discussion, some commit checks failed. Comments from Reviewable |
I'm not opposed to closing this PR if #24126 is a sufficient fix. Will that make it into 2.0 or 2.0.1? We could choose to not merge this PR since it'd just be a performance decrease for no value. |
We should merge and cherry-pick this (for 2.0.1, I think) since it will help for mixed-version clusters. Then we can remove the checksum computation at the same time we remove all the others. |
IMPORT was not setting value checksums when generating KV pairs from CSV rows. This results in CPuts on, for example, secondary indexes failing because the checksum doesn't match. Fixes #23984 Release note (bug fix): correctly generate on-disk checksums during IMPORT. If there is existing data created by IMPORT that cannot be recreated by this fixed version of IMPORT, use `cockroach dump` to rewrite any affected tables.
IMPORT was not setting value checksums when generating KV pairs from
CSV rows. This results in CPuts on, for example, secondary indexes
failing because the checksum doesn't match.
Fixes #23984
Release note (bug fix): correctly generate on-disk checksums during
IMPORT. If there is existing data created by IMPORT that cannot
be recreated by this fixed version of IMPORT, use
cockroach dump
to rewrite any affected tables.