Skip to content

Commit

Permalink
Compress the lossless part of MGARD data with Zstd. It was uncompress…
Browse files Browse the repository at this point in the history
…ed until now. (#4339)

$ ctest -R MGARD.*/.*BP5.Serial
$ find testing/adios2/engine/bp/operations/bp5 -name "*bp" -exec du -sh {} \;
48K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.00001.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.00001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.01.bp
36K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.00001.bp
44K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.0001.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.00001.bp
36K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.0001.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.0001.bp
32K     testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.01.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.00001.bp
32K     testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.001.bp
44K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.01.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.001.bp
32K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.001.bp
44K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.001.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.0001.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.01.bp
44K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.001.bp
44K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.01.bp
32K     testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.0001.bp
32K     testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.00001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.0001.bp
32K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.01.bp
44K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.00001.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.01.bp
44K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.0001.bp

Before this commit:

$ find testing/adios2/engine/bp/operations/bp5 -name "*bp" -exec du -sh {} \;
352K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.00001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.00001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.01.bp
96K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.00001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.0001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.00001.bp
96K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.0001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.0001.bp
100K    testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.01.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.00001.bp
100K    testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.01.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.001.bp
96K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.0001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.01.bp
352K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.001.bp
344K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.01.bp
100K    testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.0001.bp
100K    testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.00001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.0001.bp
96K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.01.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.00001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.01.bp
352K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.0001.bp
  • Loading branch information
pnorbert authored Sep 9, 2024
1 parent 4650dbc commit d1d871b
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion source/adios2/operator/compress/CompressMGARD.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -151,10 +151,13 @@ size_t CompressMGARD::Operate(const char *dataIn, const Dims &blockStart, const
return 0;
}

mgard_x::Config config;
config.lossless = mgard_x::lossless_type::Huffman_Zstd;

PutParameter(bufferOut, bufferOutOffset, true);
void *compressedData = bufferOut + bufferOutOffset;
mgard_x::compress(mgardDim, mgardType, mgardCount, tolerance, s, errorBoundType, dataIn,
compressedData, sizeOut, true);
compressedData, sizeOut, config, true);
bufferOutOffset += sizeOut;

return bufferOutOffset;
Expand Down

0 comments on commit d1d871b

Please sign in to comment.