Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SUP-1372 fix compressed #91

Merged
merged 14 commits into from
Sep 14, 2023
Merged

SUP-1372 fix compressed #91

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
c3b1a0d
Added check for file/directory to compressed
HugeIRL Sep 5, 2023
ec91690
Updated tests to create files and remove them for testing compressed
HugeIRL Sep 5, 2023
23f438a
Updated README.md version numbers
HugeIRL Sep 5, 2023
4b4d2e0
Moved compression path check to a different location
HugeIRL Sep 6, 2023
7b8cf9e
Fixed tests and added a new path existence test for compression
HugeIRL Sep 6, 2023
b46c1a4
Updated compression check to fail gracefully and be more detailed
HugeIRL Sep 6, 2023
2969b4d
Updated compression test
HugeIRL Sep 6, 2023
370b6bd
Update compression path check to use -n
HugeIRL Sep 6, 2023
fbee13c
Removed compression expression wrap
HugeIRL Sep 7, 2023
3447a39
Updated compression test, add new compression ignore-missing test
HugeIRL Sep 7, 2023
6b257d2
Removed mocked glob pathing in upload compressed tests
HugeIRL Sep 7, 2023
ea94cb1
Add clarification when using compressed with ignore-missing
pzeballos Sep 13, 2023
13f3aa1
Update tests title
pzeballos Sep 13, 2023
3359ae1
Update tests title
pzeballos Sep 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 13 additions & 13 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ This functionality duplicates the [artifact_paths](https://buildkite.com/docs/pi
steps:
- command: ...
plugins:
- artifacts#v1.9.0:
- artifacts#v1.9.1:
upload: "log/**/*.log"
```

Expand All @@ -20,7 +20,7 @@ You can specify multiple files/globs to upload as artifacts:
steps:
- command: ...
plugins:
- artifacts#v1.9.0:
- artifacts#v1.9.1:
upload: [ "log/**/*.log", "debug/*.error" ]
```

Expand All @@ -30,7 +30,7 @@ And even rename them before uploading them (can not use globs here though, sorry
steps:
- command: ...
plugins:
- artifacts#v1.9.0:
- artifacts#v1.9.1:
upload:
- from: log1.log
to: log2.log
Expand All @@ -47,7 +47,7 @@ eg: uploading a public file when using S3
steps:
- command: ...
plugins:
- artifacts#v1.9.0:
- artifacts#v1.9.1:
upload: "coverage-report/**/*"
s3-upload-acl: public-read
```
Expand All @@ -57,7 +57,7 @@ eg: uploading a private file when using GS
steps:
- command: ...
plugins:
- artifacts#v1.9.0:
- artifacts#v1.9.1:
upload: "coverage-report/**/*"
gs-upload-acl: private
```
Expand All @@ -70,7 +70,7 @@ This downloads artifacts matching globs to the local filesystem. See [downloadin
steps:
- command: ...
plugins:
- artifacts#v1.9.0:
- artifacts#v1.9.1:
download: "log/**/*.log"
```

Expand All @@ -80,7 +80,7 @@ You can specify multiple files/patterns:
steps:
- command: ...
plugins:
- artifacts#v1.9.0:
- artifacts#v1.9.1:
download: [ "log/**/*.log", "debug/*.error" ]
```

Expand All @@ -90,7 +90,7 @@ Rename particular files after downloading them:
steps:
- command: ...
plugins:
- artifacts#v1.9.0:
- artifacts#v1.9.1:
download:
- from: log1.log
to: log2.log
Expand All @@ -102,7 +102,7 @@ And even do so from different builds/steps:
steps:
- command: ...
plugins:
- artifacts#v1.9.0:
- artifacts#v1.9.1:
step: UUID-DEFAULT
build: UUID-DEFAULT-2
download:
Expand Down Expand Up @@ -145,7 +145,7 @@ When uploading, the file or directory specified in the `upload` option will be c
steps:
- command: ...
plugins:
- artifacts#v1.9.0:
- artifacts#v1.9.1:
upload: "log/my-folder"
compressed: logs.zip
```
Expand All @@ -156,7 +156,7 @@ When downloading, this option states the actual name of the artifact to be downl
steps:
- command: ...
plugins:
- artifacts#v1.9.0:
- artifacts#v1.9.1:
download: "log/file.log"
compressed: logs.tgz
```
Expand All @@ -177,7 +177,7 @@ Skip uploading if the main command failed with exit code 147:
steps:
- command: ...
plugins:
- artifacts#v1.9.0:
- artifacts#v1.9.1:
upload: "log/*.log"
skip-on-status: 147
```
Expand All @@ -188,7 +188,7 @@ Alternatively, skip artifact uploading on exit codes 1 and 5:
steps:
- command: ...
plugins:
- artifacts#v1.9.0:
- artifacts#v1.9.1:
upload: "log/*.log"
skip-on-status:
- 1
Expand Down
15 changes: 11 additions & 4 deletions hooks/post-command
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,12 +128,19 @@ if [[ "${SINGULAR_UPLOAD_OBJECT}" == "true" ]]; then

if [[ "${COMPRESSED}" == "true" ]]; then
echo "~~~ Compressing ${path} to ${BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED}"
"${compress[@]}" "${path}"
path=" ${BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED}"
if [[ ! -e "${path}" ]]; then
echo "+++ 🚨 Unable to compress artifact, '${path}' may not exist or is an empty directory"
path=""
else
"${compress[@]}" "${path}"
path=" ${BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED}"
fi
fi

echo "~~~ Uploading artifacts ${EXTRA_MESSAGE}"
bk_agent "${args[@]}" "${path}"
if [[ -n "${path}" ]]; then
echo "~~~ Uploading artifacts ${EXTRA_MESSAGE}"
bk_agent "${args[@]}" "${path}"
fi
elif [[ "${COMPRESSED}" == "true" ]]; then
final_paths=()
index=0
Expand Down
52 changes: 51 additions & 1 deletion tests/upload-compressed.bats
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,20 @@ load "${BATS_PLUGIN_PATH}/load.bash"
# Uncomment to enable stub debug output:
# export BUILDKITE_AGENT_STUB_DEBUG=/dev/tty

@test "Compression fails silently when there is nothing to compress" {
export BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD="non_existent_file.txt"
export BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED="file.zip"

run "$PWD/hooks/post-command"

assert_success
assert_output --partial "Unable to compress artifact, 'non_existent_file.txt' may not exist or is an empty directory"
refute_output --partial "Uploading artifacts"

unset BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD
unset BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED
}

@test "Invalid compressed format" {
export BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD="file.log"
export BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED="file.rar"
Expand All @@ -23,6 +37,8 @@ load "${BATS_PLUGIN_PATH}/load.bash"
export BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD="*.log"
export BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED="file.zip"

touch "*.log"

stub buildkite-agent \
"artifact upload \* : echo uploaded \$3"

Expand All @@ -41,12 +57,16 @@ load "${BATS_PLUGIN_PATH}/load.bash"

unset BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD
unset BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED

rm "*.log"
}

@test "Single value tgz" {
export BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD="*.log"
export BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED="file.tgz"

touch "*.log"

stub buildkite-agent \
"artifact upload \* : echo uploaded \$3"

Expand All @@ -65,6 +85,8 @@ load "${BATS_PLUGIN_PATH}/load.bash"

unset BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD
unset BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED

rm "*.log"
}

@test "Single file zip with relocation" {
Expand Down Expand Up @@ -142,6 +164,8 @@ load "${BATS_PLUGIN_PATH}/load.bash"
export BUILDKITE_PLUGIN_ARTIFACTS_JOB="12345"
export BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED="file.zip"

touch "*.log"

stub buildkite-agent \
"artifact upload --job \* \* : echo uploaded \$5 with --job \$4"

Expand All @@ -162,13 +186,17 @@ load "${BATS_PLUGIN_PATH}/load.bash"
unset BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD
unset BUILDKITE_PLUGIN_ARTIFACTS_JOB
unset BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED

rm "*.log"
}

@test "Single value tgz with job" {
export BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD="*.log"
export BUILDKITE_PLUGIN_ARTIFACTS_JOB="12345"
export BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED="file.tgz"

touch "*.log"

stub buildkite-agent \
"artifact upload --job \* \* : echo uploaded \$5 with --job \$4"

Expand All @@ -189,6 +217,8 @@ load "${BATS_PLUGIN_PATH}/load.bash"
unset BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD
unset BUILDKITE_PLUGIN_ARTIFACTS_JOB
unset BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED

rm "*.log"
}

@test "Multiple artifacts zip" {
Expand All @@ -197,6 +227,10 @@ load "${BATS_PLUGIN_PATH}/load.bash"
export BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD_2="baz.log"
export BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED="file.zip"

touch /tmp/foo.log
touch bar.log
touch baz.log

stub buildkite-agent \
"artifact upload \* : echo uploaded \$3"

Expand All @@ -220,6 +254,10 @@ load "${BATS_PLUGIN_PATH}/load.bash"
unset BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD_1
unset BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD_2
unset BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED

rm /tmp/foo.log
rm bar.log
rm baz.log
}

@test "Multiple artifacts tgz" {
Expand All @@ -228,6 +266,10 @@ load "${BATS_PLUGIN_PATH}/load.bash"
export BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD_2="baz.log"
export BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED="file.tgz"

touch /tmp/foo.log
touch bar.log
touch baz.log

stub buildkite-agent \
"artifact upload \* : echo uploaded \$3"

Expand All @@ -251,6 +293,10 @@ load "${BATS_PLUGIN_PATH}/load.bash"
unset BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD_1
unset BUILDKITE_PLUGIN_ARTIFACTS_UPLOAD_2
unset BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED

rm /tmp/foo.log
rm bar.log
rm baz.log
}

@test "Multiple artifacs zip some relocation" {
Expand All @@ -261,6 +307,8 @@ load "${BATS_PLUGIN_PATH}/load.bash"
export BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED="file.zip"

touch /tmp/foo.log
touch bar.log
touch baz.log

stub buildkite-agent \
"artifact upload \* : echo uploaded \$3"
Expand All @@ -282,6 +330,8 @@ load "${BATS_PLUGIN_PATH}/load.bash"
assert [ -e /tmp/foo2.log ]
assert [ ! -e /tmp/foo.log ]
rm /tmp/foo2.log
rm bar.log
rm baz.log

unstub buildkite-agent
unstub zip
Expand Down Expand Up @@ -365,4 +415,4 @@ load "${BATS_PLUGIN_PATH}/load.bash"
unset BUILDKITE_PLUGIN_ARTIFACTS_DOWNLOAD_0_FROM
unset BUILDKITE_PLUGIN_ARTIFACTS_DOWNLOAD_0_TO
unset BUILDKITE_PLUGIN_ARTIFACTS_COMPRESSED
}
}