zipped files decompress much larger than the original files

[nalbion]

There is a 105MB file within the lambda-lib.zip archive at node-oracledb-for-lambda. When I use gulp-zip (which uses yazl) to add it to a new zip archive, and then unzip that archive the 105MB file has grown to 140MB.

Also, when submitting a yazl-generated zip containing binary libraries to AWS Lambda, an error is thrown:

ELF file's phentsize not the expected size

(this was the cause of oracle/node-oracledb#468)

[nalbion]

...as a work-around I have replaced gulp-zip with gulp-shell:

//.pipe(zip(taskName + '.zip'))
.pipe(shell('zip dist/' + taskName + '.zip <%= file.relative %>'))

[thejoshwolfe]

i'm looking into this issue. so far i've found that the file appears to be 54MB, not 104MB. I'm not sure this matters.

Using yazl alone, I am not able to reproduce your issue:

~/tmp$ md5sum lambda-lib.zip 
8cd1d56db9d485139cd74af252b92738  lambda-lib.zip
~/tmp$ node ~/yazl/examples/zip.js lambda-lib.zip -o tmp.zip
finalSize prediction: 55722438
~/tmp$ unzip tmp.zip 
Archive:  tmp.zip
replace lambda-lib.zip? [y]es, [n]o, [A]ll, [N]one, [r]ename: r
new name: lambda-lib-out.zip
 extracting: lambda-lib-out.zip      
~/tmp$ md5sum lambda-lib-out.zip 
8cd1d56db9d485139cd74af252b92738  lambda-lib-out.zip

I'm trying to test the usecase with gulp-zip, but I must admit I'm an outsider to the gulp club. After about 10 minutes, this is my best attempt at reproducing your issue:

in gulpfile.js:

const gulp = require('gulp');
const zip = require('gulp-zip');

gulp.task('default', () => {
    return gulp.src("../labmda-lib.zip")
        .pipe(zip('gulp.zip'))
        .pipe(gulp.dest('dist'));
});

the from the command line:

$ npm install gulp
...
$ npm install gulp-zip
...
$ ./node_modules/.bin/gulp 
[18:55:20] Using gulpfile ~/tmp/test/gulpfile.js
[18:55:20] Starting 'default'...
[18:55:20] Finished 'default' after 12 ms
$ ls
gulpfile.js  node_modules

If you can help me get a reproducible test case, it would help me resolve this issue. (I will be unavailable for the next few hours, however.) Otherwise, I will look into a gulp tutorial or something to try to learn what the above example is missing.

[nalbion]

You need to unzip lambda-lib.zip first and then operate on the libociei.so file:

$ unzip lambda-lib.zip
$ ls -alsh libociei.so

const gulp = require('gulp');
const zip = require('gulp-zip');

gulp.task('default', () => {
    return gulp.src('libociei.so')
        .pipe(zip('gulp.zip'))
        .pipe(gulp.dest('dist'));
});

$ mv libociei.so libociei.so.original
$ gulp
$ unzip dist/gulp.zip
$ ls -alsh libociei.so*

[thejoshwolfe]

when i follow your steps, i get the correct output. can you tell me what versions of everything you're using? here's what i've got:

[email protected]
[email protected]

[andrewrk]

Node.js version might be worth noting too.

[thejoshwolfe]

$ node --version
v4.2.6
$ npm --version
3.5.2

[nalbion]

Ah - I'm afraid I'm going to have to cop the blame for this one. I've actually got an extra line in my gulpfile:

gulp.task('default', () => {
    return gulp.src('libociei.so')
        .pipe(require('gulp-replace')(/\$\{email.password\}/g, process.env.EMAIL_PASSWORD))
        .pipe(zip('gulp.zip'))
        .pipe(gulp.dest('dist'));
});

...When I comment out the gulp-replace line the files are identical after being unzipped. It's unlikely to me that Oracle's library contains the text ${email.password} - and enough times that would explain a 35MB increase in file size, but I suppose the lesson for me here is to be careful to avoid doing a search & replace of text in non-text files.

[jofan]

For completeness: gulp-replace has a "skipBinary" option that defaults to false. Set that to true to avoid trying replacement in binary files such as images.