Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't upload file to device with other-files feature with gs:// link #1971

Closed
veronikashavrova opened this issue May 26, 2021 · 2 comments · Fixed by #2075
Closed

Can't upload file to device with other-files feature with gs:// link #1971

veronikashavrova opened this issue May 26, 2021 · 2 comments · Fixed by #2075
Assignees
@pawelpasterz
Labels
Bug Tiger 🐯

Comments

@veronikashavrova
Copy link

Describe the bug
Can't upload file to device with other-files feature with gs:// link
Tried only on android

To Reproduce

  1. Create flank.yml with property
 other-files: 
    /sdcard/eicar.zip: gs://<bucket>/eicar.zip

Error:
java.nio.file.NoSuchFileException: gs:/<bucket>/eicar.zip
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
	at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
	at java.base/java.nio.file.Files.newByteChannel(Files.java:370)
	at java.base/java.nio.file.Files.newByteChannel(Files.java:421)
	at java.base/java.nio.file.Files.readAllBytes(Files.java:3205)
	at ftl.run.platform.android.UploadOtherFilesKt.upload(UploadOtherFiles.kt:30)
	at ftl.run.platform.android.UploadOtherFilesKt$uploadOtherFiles$2$invokeSuspend$$inlined$map$lambda$1.invokeSuspend(UploadOtherFiles.kt:17)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

This is different from error when file actually doesn't exist:
'gs://<bucket>/eicarr.zip' from otherFiles doesn't exist

Apk from same bucket uploaded successfully:

  additional-apks:
    - gs://<bucket>/eicar.apk

Same file uploads correctly with absolute path:

  other-files: 
    /sdcard/eicar.zip: /Users/<user>/Downloads/eicar.zip

With gcloud same file uploaded successfully:
--other-files /sdcard/eicar.zip=gs://<bucket>/eicar.zip"

Expected behavior
File should be uploaded successfully

Details (please complete the following information):
Using https://github.com/Flank/flank/releases/download/v21.05.0/flank.jar (didn't tried other versions)

Additional context
As a workaround i'm downloading file via wget to local folder on CI and pass local path to other-files
Using android example from https://flank.github.io/flank/
PS: other-files property has incorrect syntax (missing colon) please fix.
@pawelpasterz
Copy link
Contributor

pawelpasterz commented May 26, 2021 •
edited
Loading

Thanks for posting this issue!
Looks like flank is missing one / in gs path 🤔
We will take a look at it shortly.

@jan-goral
Copy link
Contributor

To verify

@pawelpasterz pawelpasterz self-assigned this Jul 9, 2021
@pawelpasterz pawelpasterz mentioned this issue Jul 9, 2021
Merged
@mergify mergify bot closed this as completed in #2075 Jul 14, 2021
mergify bot pushed a commit that referenced this issue Jul 14, 2021
@pawelpasterz
fix: Do not upload/read files when path starts with gs:// (#2075)
f44162e 
Fixes #1971 

When user provides `gs://` path in `other-files` we should not upload it again.
```
Files.readAllBytes(Paths.get(file))
```
will never succeed when the path starts with `gs://` 

## Test Plan
> How do we know the code works?

1. [flank test bucket](https://console.cloud.google.com/storage/browser/flank-open-source.appspot.com/test?project=flank-open-source&pageState=(%22StorageObjectListTable%22:(%22f%22:%22%255B%255D%22))&prefix=&forceOnObjectsSortingFiltering=false)
1. run `./gradlew flankFullRun`
1. prepare config with:
    ```
    other-files:
        /sdcard/[some file name]: gs://[path to file uploaded to the bucket, you can reuse existing one]
    ```
1. run tests
1. go to result bucket and check if there is a file from config under `/artifacts/sdcard/` path
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pawelpasterz pawelpasterz

Labels
Bug Tiger 🐯
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: Do not upload/read files when path starts with gs:// Flank/flank
3 participants
@pawelpasterz @jan-goral @veronikashavrova