Cross Compatibility With LZ4 CLI (& Others) ?

[dchaumette]

I seem to be unable to compress with python-lz4 and decompress with Google's bundled LZ4 CLI and vice-versa.

Decompressing errors are as per below:

• python-lz4: ValueError: corrupt input at byte 10
• LZ4 CLI: Error 44 : Unrecognized header : file cannot be decoded

Could this have anything to do with this container format issue?

Ran on: Windows; LZ4 CLI using binaries and also built from latest (r116) sources; python-lz4 first built from github and then with latest LZ4 source files added.

[darkdragn]

This version of lz4 for python doesn't have the lz4io incorporated. In other words, it won't write a header with the compressed result, and should not be written directly to a file without something else building the container. That's where the issue is. The python version isn't expecting the header, but the c++ version is. If you check out my fork, I incorporated the lz4io side of the house, and provided compressFile and decompressFile methods. They work for me on Linux. If you want to please retry your results with my fork, and let me know how it goes, I would appreciate it!

https://github.com/darkdragn/python-lz4

[dchaumette]

Sorry for the delay and thanks for the response.

In the meantime this was put on hold on my side and I haven't looked into it anymore. Should I get the time or should it come back on the radar I'll try compressFile() and decompressFile() and will let you know how it goes.

I'll close the issue until then I suppose, this isn't truly speaking a defect.
Thanks

[alexandre-mbm]

@darkdragn, I use Arch Linux and try this:

$ mv firefox-bookmarks-backup.jsonlz4 firefox-bookmarks-backup.json.lz4
$ lz4 -d firefox-bookmarks-backup.json.lz4 
Decoding file test.json 
Error 44 : Unrecognized header : file cannot be decoded

I want to decompress only, I don't want to write a new tool in Python. What I do?

---

#!/usr/bin/env python
import lz4
lz4.decompressFileDefault("test.json.lz4")

Don't work. Tested with branches r121 and r122io. The result test.json is 0 byte.

[Cyan4973]

I suspect your situation is related to this bug report :
https://bugzilla.mozilla.org/show_bug.cgi?id=818587

Summary : .jsonlz4 files are not .lz4 compatible. They use some Mozilla-specific headers.

If you want these files to be lz4-conformant, or Mozilla to provide a decoder compatible with their specific format, you'll have to direct your requests towards Mozilla.

[alexandre-mbm]

It seems that I need to use Gecko.