Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Wrong data returned from getitem (DeepState fuzzing) #95

Closed
agroce opened this issue Dec 5, 2019 · 11 comments
Closed

Wrong data returned from getitem (DeepState fuzzing) #95

agroce opened this issue Dec 5, 2019 · 11 comments

Comments

@agroce
Copy link

agroce commented Dec 5, 2019

This is a minimal test case; making any value smaller causes it to pass.

TRACE: TestCBlosc2.cpp(37): Performing 1 round trips.
TRACE: TestCBlosc2.cpp(40): *******************************   Starting run #0   *******************************
TRACE: TestCBlosc2.cpp(55): Compression task: type_size: 131 ; num_elements: 1 ; buffer_alignment: 32; compression_level: 6; do_shuffle: 0; compressor: blosclz; delta: 1; buffer_size: 131
TRACE: TestCBlosc2.cpp(69): original[0] = 0
TRACE: TestCBlosc2.cpp(69): original[1] = 0
TRACE: TestCBlosc2.cpp(69): original[2] = 0
TRACE: TestCBlosc2.cpp(69): original[3] = 0
TRACE: TestCBlosc2.cpp(69): original[4] = 0
TRACE: TestCBlosc2.cpp(69): original[5] = 0
TRACE: TestCBlosc2.cpp(69): original[6] = 0
TRACE: TestCBlosc2.cpp(69): original[7] = 0
TRACE: TestCBlosc2.cpp(69): original[8] = 0
TRACE: TestCBlosc2.cpp(69): original[9] = 0
TRACE: TestCBlosc2.cpp(69): original[10] = 0
TRACE: TestCBlosc2.cpp(69): original[11] = 0
TRACE: TestCBlosc2.cpp(69): original[12] = 0
TRACE: TestCBlosc2.cpp(69): original[13] = 0
TRACE: TestCBlosc2.cpp(69): original[14] = 0
TRACE: TestCBlosc2.cpp(69): original[15] = 0
TRACE: TestCBlosc2.cpp(69): original[16] = 0
TRACE: TestCBlosc2.cpp(69): original[17] = 0
TRACE: TestCBlosc2.cpp(69): original[18] = 0
TRACE: TestCBlosc2.cpp(69): original[19] = 0
TRACE: TestCBlosc2.cpp(69): original[20] = 0
TRACE: TestCBlosc2.cpp(69): original[21] = 0
TRACE: TestCBlosc2.cpp(69): original[22] = 0
TRACE: TestCBlosc2.cpp(69): original[23] = 0
TRACE: TestCBlosc2.cpp(69): original[24] = 0
TRACE: TestCBlosc2.cpp(69): original[25] = 0
TRACE: TestCBlosc2.cpp(69): original[26] = 0
TRACE: TestCBlosc2.cpp(69): original[27] = 0
TRACE: TestCBlosc2.cpp(69): original[28] = 0
TRACE: TestCBlosc2.cpp(69): original[29] = 0
TRACE: TestCBlosc2.cpp(69): original[30] = 0
TRACE: TestCBlosc2.cpp(69): original[31] = 0
TRACE: TestCBlosc2.cpp(69): original[32] = 0
TRACE: TestCBlosc2.cpp(69): original[33] = 0
TRACE: TestCBlosc2.cpp(69): original[34] = 0
TRACE: TestCBlosc2.cpp(69): original[35] = 0
TRACE: TestCBlosc2.cpp(69): original[36] = 0
TRACE: TestCBlosc2.cpp(69): original[37] = 0
TRACE: TestCBlosc2.cpp(69): original[38] = 0
TRACE: TestCBlosc2.cpp(69): original[39] = 0
TRACE: TestCBlosc2.cpp(69): original[40] = 0
TRACE: TestCBlosc2.cpp(69): original[41] = 0
TRACE: TestCBlosc2.cpp(69): original[42] = 0
TRACE: TestCBlosc2.cpp(69): original[43] = 0
TRACE: TestCBlosc2.cpp(69): original[44] = 0
TRACE: TestCBlosc2.cpp(69): original[45] = 0
TRACE: TestCBlosc2.cpp(69): original[46] = 0
TRACE: TestCBlosc2.cpp(69): original[47] = 0
TRACE: TestCBlosc2.cpp(69): original[48] = 0
TRACE: TestCBlosc2.cpp(69): original[49] = 0
TRACE: TestCBlosc2.cpp(69): original[50] = 0
TRACE: TestCBlosc2.cpp(69): original[51] = 0
TRACE: TestCBlosc2.cpp(69): original[52] = 0
TRACE: TestCBlosc2.cpp(69): original[53] = 0
TRACE: TestCBlosc2.cpp(69): original[54] = 0
TRACE: TestCBlosc2.cpp(69): original[55] = 0
TRACE: TestCBlosc2.cpp(69): original[56] = 0
TRACE: TestCBlosc2.cpp(69): original[57] = 0
TRACE: TestCBlosc2.cpp(69): original[58] = 0
TRACE: TestCBlosc2.cpp(69): original[59] = 0
TRACE: TestCBlosc2.cpp(69): original[60] = 0
TRACE: TestCBlosc2.cpp(69): original[61] = 0
TRACE: TestCBlosc2.cpp(69): original[62] = 0
TRACE: TestCBlosc2.cpp(69): original[63] = 0
TRACE: TestCBlosc2.cpp(69): original[64] = 0
TRACE: TestCBlosc2.cpp(69): original[65] = 0
TRACE: TestCBlosc2.cpp(69): original[66] = 0
TRACE: TestCBlosc2.cpp(69): original[67] = 0
TRACE: TestCBlosc2.cpp(69): original[68] = 0
TRACE: TestCBlosc2.cpp(69): original[69] = 0
TRACE: TestCBlosc2.cpp(69): original[70] = 0
TRACE: TestCBlosc2.cpp(69): original[71] = 0
TRACE: TestCBlosc2.cpp(69): original[72] = 0
TRACE: TestCBlosc2.cpp(69): original[73] = 0
TRACE: TestCBlosc2.cpp(69): original[74] = 0
TRACE: TestCBlosc2.cpp(69): original[75] = 0
TRACE: TestCBlosc2.cpp(69): original[76] = 0
TRACE: TestCBlosc2.cpp(69): original[77] = 0
TRACE: TestCBlosc2.cpp(69): original[78] = 0
TRACE: TestCBlosc2.cpp(69): original[79] = 0
TRACE: TestCBlosc2.cpp(69): original[80] = 0
TRACE: TestCBlosc2.cpp(69): original[81] = 0
TRACE: TestCBlosc2.cpp(69): original[82] = 0
TRACE: TestCBlosc2.cpp(69): original[83] = 0
TRACE: TestCBlosc2.cpp(69): original[84] = 0
TRACE: TestCBlosc2.cpp(69): original[85] = 0
TRACE: TestCBlosc2.cpp(69): original[86] = 0
TRACE: TestCBlosc2.cpp(69): original[87] = 0
TRACE: TestCBlosc2.cpp(69): original[88] = 0
TRACE: TestCBlosc2.cpp(69): original[89] = 0
TRACE: TestCBlosc2.cpp(69): original[90] = 0
TRACE: TestCBlosc2.cpp(69): original[91] = 0
TRACE: TestCBlosc2.cpp(69): original[92] = 0
TRACE: TestCBlosc2.cpp(69): original[93] = 0
TRACE: TestCBlosc2.cpp(69): original[94] = 0
TRACE: TestCBlosc2.cpp(69): original[95] = 0
TRACE: TestCBlosc2.cpp(69): original[96] = 0
TRACE: TestCBlosc2.cpp(69): original[97] = 0
TRACE: TestCBlosc2.cpp(69): original[98] = 0
TRACE: TestCBlosc2.cpp(69): original[99] = 0
TRACE: TestCBlosc2.cpp(69): original[100] = 0
TRACE: TestCBlosc2.cpp(69): original[101] = 0
TRACE: TestCBlosc2.cpp(69): original[102] = 0
TRACE: TestCBlosc2.cpp(69): original[103] = 0
TRACE: TestCBlosc2.cpp(69): original[104] = 0
TRACE: TestCBlosc2.cpp(69): original[105] = 0
TRACE: TestCBlosc2.cpp(69): original[106] = 0
TRACE: TestCBlosc2.cpp(69): original[107] = 0
TRACE: TestCBlosc2.cpp(69): original[108] = 0
TRACE: TestCBlosc2.cpp(69): original[109] = 0
TRACE: TestCBlosc2.cpp(69): original[110] = 0
TRACE: TestCBlosc2.cpp(69): original[111] = 0
TRACE: TestCBlosc2.cpp(69): original[112] = 0
TRACE: TestCBlosc2.cpp(69): original[113] = 0
TRACE: TestCBlosc2.cpp(69): original[114] = 0
TRACE: TestCBlosc2.cpp(69): original[115] = 0
TRACE: TestCBlosc2.cpp(69): original[116] = 0
TRACE: TestCBlosc2.cpp(69): original[117] = 0
TRACE: TestCBlosc2.cpp(69): original[118] = 0
TRACE: TestCBlosc2.cpp(69): original[119] = 0
TRACE: TestCBlosc2.cpp(69): original[120] = 0
TRACE: TestCBlosc2.cpp(69): original[121] = 0
TRACE: TestCBlosc2.cpp(69): original[122] = 0
TRACE: TestCBlosc2.cpp(69): original[123] = 0
TRACE: TestCBlosc2.cpp(69): original[124] = 0
TRACE: TestCBlosc2.cpp(69): original[125] = 0
TRACE: TestCBlosc2.cpp(69): original[126] = 0
TRACE: TestCBlosc2.cpp(69): original[127] = 0
TRACE: TestCBlosc2.cpp(69): original[128] = 1
TRACE: TestCBlosc2.cpp(69): original[129] = 0
TRACE: TestCBlosc2.cpp(69): original[130] = 0
TRACE: TestCBlosc2.cpp(81): # uncompressed bytes = 131
TRACE: TestCBlosc2.cpp(83): # compressed bytes = 34
TRACE: TestCBlosc2.cpp(84): block size = 131
TRACE: TestCBlosc2.cpp(90): typesize: 131; DOSHUFFLE: 0; DOBITSHUFFLE: 0; DODELTA: 8; MEMCPYED: 0
TRACE: TestCBlosc2.cpp(98): compressor: BloscLZ
TRACE: TestCBlosc2.cpp(111): Performing 1 non-buffer-changing actions.
TRACE: TestCBlosc2.cpp(128): Getting 1 from 0
TRACE: TestCBlosc2.cpp(139): [0][0]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][1]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][2]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][3]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][4]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][5]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][6]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][7]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][8]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][9]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][10]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][11]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][12]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][13]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][14]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][15]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][16]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][17]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][18]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][19]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][20]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][21]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][22]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][23]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][24]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][25]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][26]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][27]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][28]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][29]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][30]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][31]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][32]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][33]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][34]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][35]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][36]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][37]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][38]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][39]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][40]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][41]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][42]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][43]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][44]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][45]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][46]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][47]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][48]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][49]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][50]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][51]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][52]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][53]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][54]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][55]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][56]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][57]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][58]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][59]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][60]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][61]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][62]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][63]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][64]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][65]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][66]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][67]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][68]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][69]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][70]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][71]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][72]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][73]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][74]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][75]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][76]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][77]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][78]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][79]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][80]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][81]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][82]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][83]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][84]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][85]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][86]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][87]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][88]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][89]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][90]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][91]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][92]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][93]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][94]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][95]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][96]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][97]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][98]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][99]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][100]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][101]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][102]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][103]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][104]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][105]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][106]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][107]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][108]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][109]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][110]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][111]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][112]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][113]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][114]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][115]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][116]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][117]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][118]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][119]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][120]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][121]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][122]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][123]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][124]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][125]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][126]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][127]: original: 0; items: 0
TRACE: TestCBlosc2.cpp(139): [0][128]: original: 1; items: 1
TRACE: TestCBlosc2.cpp(137): !!! MISMATCH !!!:
TRACE: TestCBlosc2.cpp(139): [0][129]: original: 0; items: 1
TRACE: TestCBlosc2.cpp(139): [0][130]: original: 0; items: 0
CRITICAL: TestCBlosc2.cpp(141): Get returned wrong data for item 0 (getting 1 from 0)
ERROR: Failed: CBlosc2_RoundTrip
ERROR: Test case wrongdata.fail failed
@agroce
Copy link
Author

agroce commented Dec 5, 2019

It looks like the key to this is:

  1. do a compression with type_size: 131 ; num_elements: 1 ; buffer_alignment: 32; compression_level: 6; do_shuffle: 0; compressor: blosclz; delta: 1

  2. data is all zeros except byte 128

  3. call getitem to get 1 item starting at position 0

byte 129 will be a 1 in "items" and a 0 in the original data

As far as I can tell, this is a real bug, don't see any likely API violations

@agroce
Copy link
Author

agroce commented Dec 6, 2019

The same tests all pass on the original cblosc, which strongly suggests this is a cblosc2 bug.

@agroce
Copy link
Author

agroce commented Dec 7, 2019 •
edited
Loading 

include <stdlib.h>
#include <stdio.h>
#include <assert.h>

#include "blosc2.h"

inline static void* blosc_malloc(const size_t alignment, const size_t size) {
  const int32_t clean_value = 0x99;
  void* block = NULL;
  
  block = aligned_alloc(alignment, size);
  
  if (block == NULL) {
    fprintf(stderr, "Error allocating memory!");
    return NULL;
  }
  
  /* Clean the allocated memory before returning. */
  memset(block, clean_value, size);
  
  return block;
}

int main() {
  blosc_init();
  blosc_set_nthreads(1);

  size_t type_size = 131;
  size_t num_elements = 1;
  size_t buffer_alignment = 32;

  blosc_set_compressor("blosclz");
  blosc_set_delta(1);

  size_t buffer_size = type_size * num_elements;

  /* Allocate memory for the test. */
  void* original = blosc_malloc(buffer_alignment, buffer_size);
  void* intermediate = blosc_malloc(buffer_alignment, buffer_size + BLOSC_MAX_OVERHEAD);
  void* items = blosc_malloc(buffer_alignment, buffer_size);
  void* result = blosc_malloc(buffer_alignment, buffer_size);

  /* The test data */
  memset(original, 0, buffer_size);
  ((char*)original)[128] = 1;

  /* Compress the input data and store it in an intermediate buffer.
     Decompress the data from the intermediate buffer into a result buffer. */
  blosc_compress(6, 0, type_size, buffer_size,
		 original, intermediate, buffer_size + BLOSC_MAX_OVERHEAD);
    
  unsigned start_item = 0;
  unsigned num_items = 1;
  int get_result = blosc_getitem(intermediate, start_item, num_items, items);
  assert(get_result == (num_items * type_size));
  for (int i = 0; i < type_size; i++) {
    int original_value = *((char*)original + i);
    int item_value = *((char*)items + i);
    if (original_value != item_value) {
      printf("ERROR: mismatch at %u: %d (original) vs. %d (getitem)\n", i,
	     original_value, item_value);
    }
  }

  blosc_decompress(intermediate, result, buffer_size);
  
  /* The round-tripped data matches the original data when the
     result of memcmp is 0. */
  assert(memcmp(original, result, buffer_size) == 0);

  /* Now that we see the round-trip passed, check the items */
  assert(memcmp(intermediate, items, get_result) == 0);
  
  /* Free allocated memory. */
  free(original);
  free(intermediate);
  free(items);
  free(result);
  
  /* Cleanup blosc resources. */
  blosc_destroy();
}

is a standalone test showing the problem.

user@f150f14d598e:~/deepstate-c-blosc2$ clang -o failget failget.c /home/user/c-blosc2/install/lib/libblosc2.a -I/home/user/c-blosc2/install/include -lpthread
user@f150f14d598e:~/deepstate-c-blosc2$ ./failget 
ERROR: mismatch at 129: 0 (original) vs. 1 (getitem)
failget: failget.c:72: int main(): Assertion `memcmp(intermediate, items, get_result) == 0' failed.
Aborted

@agroce
Copy link
Author

agroce commented Dec 12, 2019

@FrancescAlted you might want to take a look at this; not an API meaning issue, but a data-changing bug not present in old blosc.

@agroce
Copy link
Author

agroce commented Dec 12, 2019

Standalone test case doesn't need deepstate.

@pgoodman
Copy link

@FrancescAlted Do you have an ETA on a bug fix for this?

@FrancescAlted
Copy link
Member

I am pretty swamped lately, but will try to find some time for this during the holidays season. No guarantees though, so any help is appreciated indeed.

@agroce
Copy link
Author

agroce commented Dec 20, 2019

Would more details about when it happens be helpful? The code looks non-trivial to dive into to bugfix/root cause, but I can clarify conditions quite a bit I suspect...

@agroce
Copy link
Author

agroce commented Jan 28, 2020

@FrancescAlted did you ever get a chance to confirm the bug here, if not yet to fix?

@FrancescAlted
Copy link
Member

Yup, it took me a while, but the problem was finally fixed. Thank you for following on this one!

@agroce
Copy link
Author

agroce commented Jan 28, 2020

Great! Will retest today!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@pgoodman @FrancescAlted @agroce