S3 won't upload app files

[bilby91]

Hello,

I'm trying to make the publishing feature work in our project but I'm not sure if I understood correctly the behaviour.

We are building application for mac, win and linux. We are using S3 as the publishing provider.

From my understanding, after I run npm run release (setuped as build -mwl --publish always) I expect to have all the packaged files on the specified bucket in S3. The thing is, only latest-mac.json and latest.yml are published to S3. I'm missing the dmg, exe and AppImage files.

I'm doing something wrong or maybe there is something I'm not understanding.

Any help is more than welcome! Thanks and this project rocks!

Versions:

• electron-builder 15.1.1
• electron-publisher-s3 15.0.1

Status

• Sample project is working if bucket path is set.
• My application is not working with any configuration
• Tried using yarn instead of npm and didn't worked

[develar]

Please try 15.1.1

[bilby91]

@develar After updating I got the same behaviour. Logs show activity like if the file was being uploaded but there is nothing in the bucker, just the files I mentioned before.

[develar]

@bilby91 Hmm... it works for me. Ok, I will try exactly as you do build -mwl BTW, have you tried only build -m?

[develar]

Do you build on macOS?

[bilby91]

@develar Will try with -m only. Yes, I'm building in macOS.

[bilby91]

@develar Same with -m only. The latest-mac.json is getting updated BTW.

[develar]

Is it possible to provide your config?

[develar]

And please set env DEBUG=electron-builder and attach log of the terminal output.

[bilby91]

Sure, let me prepare the config and run the command with the DEBUG env var.

[bilby91]

@develar Here is the missing information https://gist.github.com/bilby91/157655a4871a1c687e9a88e6c9caf42c

Thanks!

[bilby91]

@develar Any thought on what is happening ? Maybe you can point me to the appropriate peace of code and I can debug it.

[develar]

@bilby91 I suspect that aws sdk doesn't like our progress stream. I am going to use aws sdk progress API instead of general handler. Probably I will do it today/tomorrow.

https://github.com/electron-userland/electron-builder/blob/master/packages/electron-publisher-s3/src/s3Publisher.ts#L28

BTW, please ensure that you use electron-publisher-s3 15.0.1

[bilby91]

@develar I just verified that is not working for electron-publisher-s3 15.0.1. Have you been able to reproduce the problem ?

[develar]

Have you been able to reproduce the problem ?

No. Works for me. In any case it is better to use progress API from aws sdk. When it will be implemented, I will ask you to try again.

Also, you can try to build app on CI server (e.g. Travis).

[develar]

You can check using my sample project:

1. Download onshape-desktop-shell.zip and unarchive.
2. cd onshape-desktop-shell && yarn
3. Add your publish config to build in the package.json

"publish": {
    "provider": "s3",
    "bucket": "electron-builder-test",
    "path": "ff"
  },

4. yarn release

[bilby91]

@develar Sure, let me try.

[bilby91]

@develar Same happen with your sample project. Only yml and json files were uploaded.

[bilby91]

@develar If I provide a custom path it seems to be working! I'm going to check in my application now.

[develar]

If I provide a custom path it seems to be working!

@bilby91 Nice. I will check this case.

[bilby91]

@develar It's more strange now :(. It worked for your sample app if I changed the bucket path, didn't work on my application with the same publishing configuration.

UPDATE

• Tried using yarn instead of npm and didn't worked

[mediaslav]

same issue here, only json files were uploaded

[develar]

Sorry that this critical issue is not yet fixed. I really hope to fix it on this weekend.

[mediaslav]

@develar can you please point where to look for this issue in code?

[develar]

@mediaslav https://github.com/electron-userland/electron-builder/blob/master/packages/electron-publisher-s3/src/s3Publisher.ts In any case we must use aws sdk progress API instead of our generic implementation. It should be fixed before investigating this issue.

[bilby91]

After updating electron-builder to the lastest version (16.0.1) I got the app files uploaded but no the latest-mac.json.

Work you are planning in the weekend can solve this too ?

Thanks!

[mediaslav]

Looks like its common problem with uploading big files to S3, and best way is to use multipart uploads with retry on error. Maybe it makes sense to use lib like https://github.com/faceleg/node-s3-client/ for that

[develar]

@mediaslav I was hoping that AWS-SDK doesn't *** in the latest versions, but it seems you are right. Thanks for the link to updated s3 client.

[bilby91]

@mediaslav @develar Makes a lot of sense based on the behaviour I'm having!

It was like my internet connection got totally broken when I published my artifacts :(

[bilby91]

Awesome! Will try it out!

[bilby91]

@develar When could we expect a new release ?

[develar]

@bilby91 In 3 seconds, 16.6.2 (next release).

[bilby91]

🙌🏻

[Maddoc42]

@develar the release does not seem to be on npm yet, is it published somewhere else? Big thanks for fixing this btw! 👍

[develar]

@Maddoc42 It is next release. Not latest. Just set version to 16.7.0 (both packages)

[Maddoc42]

@develar thanks, that worked :)