Skip to content

Commit

Permalink
fix test file with corrupted checksum and re-enable checksum validati…
Browse files Browse the repository at this point in the history
…on (#348)
  • Loading branch information
Achille authored Sep 26, 2022
1 parent 67dbe8d commit f45d184
Show file tree
Hide file tree
Showing 2 changed files with 1 addition and 11 deletions.
12 changes: 1 addition & 11 deletions file.go
Original file line number Diff line number Diff line change
Expand Up @@ -632,17 +632,7 @@ func (f *filePages) readPage(header *format.PageHeader, reader *bufio.Reader) (*
headerChecksum := uint32(header.CRC)
bufferChecksum := crc32.ChecksumIEEE(page.data)

// TODO: checksum validation is disabled until we figure out how the
// checksum of TestOpenFile/testdata/delta_length_byte_array.parquet was
// computed.
//
// Note that we still compute the page checksum even if we are not using
// to avoid skewing benchmarks.
//
// https://github.com/apache/parquet-testing/pull/24#issuecomment-1196045050
const validateChecksum = false

if validateChecksum && headerChecksum != bufferChecksum {
if headerChecksum != bufferChecksum {
// The parquet specs indicate that corruption errors could be
// handled gracefully by skipping pages, tho this may not always
// be practical. Depending on how the pages are consumed,
Expand Down
Binary file modified testdata/delta_length_byte_array.parquet
Binary file not shown.

0 comments on commit f45d184

Please sign in to comment.