Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws-test: Drop need for parameters, just use latest/ #312

Merged
merged 3 commits into from
Sep 27, 2018
Merged

aws-test: Drop need for parameters, just use latest/ #312

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3a0c39c
cloud: Don't actually call it -tested.json until it is
jlebon Sep 25, 2018
efab6cf
aws-test: Drop need for parameters, just use latest/
jlebon Sep 25, 2018
a5a1f58
aws-test: Stop using `/srv`
jlebon Sep 26, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions Jenkinsfile.aws-test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
def NODE = "rhcos-jenkins"
def AWS_REGION = "us-east-1"

// this var conveniently refers to a location on the server as well as the
// local dir we sync to/from
// location on the server we'll rsync to/from our $WORKSPACE
def images = "/srv/rhcos/output/images"

node(NODE) {
Expand All @@ -22,15 +21,21 @@ node(NODE) {
return
}

// We're only ever triggered by the cloud job, so we know the latest build is in latest/
// We immediately resolve it back to the specific images/ dir
def version
try {
utils.inside_assembler_container("-v /srv:/srv") {
utils.inside_assembler_container("") {
stage("Sync In") {
withCredentials([
string(credentialsId: params.ARTIFACT_SERVER, variable: 'ARTIFACT_SERVER'),
sshUserPrivateKey(credentialsId: params.ARTIFACT_SSH_CREDS_ID, keyFileVariable: 'KEY_FILE'),
]) {
sh "mkdir -p ${dirpath}"
utils.rsync_file_in(ARTIFACT_SERVER, KEY_FILE, "${dirpath}/aws-${AWS_REGION}-smoketested.json")
utils.rsync_file_in_dest(ARTIFACT_SERVER, KEY_FILE, "${images}/cloud/latest/meta.json", "${WORKSPACE}/meta.json")
version = utils.sh_capture("jq -r '.[\"ostree-version\"]' ${WORKSPACE}/meta.json")
// resolve to original dir to avoid races in the next rsync in
def imgv = utils.sh_capture("jq -r '.[\"image-version\"]' ${WORKSPACE}/meta.json")
utils.rsync_file_in_dest(ARTIFACT_SERVER, KEY_FILE, "${images}/cloud/${imgv}/aws-${AWS_REGION}.json", "${WORKSPACE}/aws-${AWS_REGION}.json")
}
}

Expand All @@ -44,22 +49,17 @@ node(NODE) {
string(credentialsId: params.AWS_CI_ACCOUNT, variable: 'AWS_CI_ACCOUNT'),
string(credentialsId: params.S3_PUBLIC_BUCKET, variable: 'S3_PUBLIC_BUCKET'),
]) {
def ami_intermediate = utils.sh_capture("jq -r .HVM ${WORKSPACE}/aws-${AWS_REGION}.json")
currentBuild.description = "version=${version} ami=${ami_intermediate}"
sh """
# Do testing with intermediate aws image passed in by cloud job
if ! kola -b rhcos -p aws --aws-type ${aws_type} --tapfile rhcos-aws.tap --aws-ami ${ami_intermediate} --aws-region ${AWS_REGION} -j ${NUM_VMS} run; then
if ! kola -b rhcos -p aws --aws-type t2.small --tapfile rhcos-aws.tap --aws-ami ${ami_intermediate} --aws-region ${AWS_REGION} -j ${NUM_VMS} run; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we wanted to keep the configurable for potential future use, but again I'm ok with changing it to be hard defined to simplify things

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh OK, in that case we can make it a proper parameter. Though until then, I'd rather not make define_properties() more complex until we need it?

exit 1
fi

# Tests pass, tag the json in the artifact server to a persistent location
# and give launch permissions to OpenShift CI
export AWS_DEFAULT_REGION=${AWS_REGION}
if [ ! -e ${dirpath}/aws-${AWS_REGION}-smoketested.json ]; then
echo "Cannot find smoketested json artifact."
exit 1
fi

cp ${dirpath}/aws-${AWS_REGION}-smoketested.json ${images}/aws-${AWS_REGION}-tested.json
aws ec2 create-tags \
--resources ${ami_intermediate} \
--tags rhcos_tag=alpha
Expand All @@ -69,7 +69,7 @@ node(NODE) {

# Upload the json file to a public location
aws s3 cp --acl public-read \
${images}/aws-${AWS_REGION}-tested.json \
${WORKSPACE}/aws-${AWS_REGION}-tested.json \
s3://${S3_PUBLIC_BUCKET}/aws-${AWS_REGION}-tested.json
"""
}
Expand All @@ -85,7 +85,7 @@ node(NODE) {
string(credentialsId: params.ARTIFACT_SERVER, variable: 'ARTIFACT_SERVER'),
sshUserPrivateKey(credentialsId: params.ARTIFACT_SSH_CREDS_ID, keyFileVariable: 'KEY_FILE'),
]) {
utils.rsync_file_out(ARTIFACT_SERVER, KEY_FILE, "${images}/aws-${AWS_REGION}-tested.json")
utils.rsync_file_out_dest(ARTIFACT_SERVER, KEY_FILE, "${WORKSPACE}/aws-${AWS_REGION}.json", "${images}/aws-${AWS_REGION}-tested.json")
}
}
}
Expand Down
10 changes: 2 additions & 8 deletions Jenkinsfile.cloud
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@ node(NODE) {
string(credentialsId: params.AWS_CI_ACCOUNT, variable: 'AWS_CI_ACCOUNT'),
]) {
sh """
amijson=${dirpath}/aws-${AWS_REGION}-smoketested.json
amijson=${dirpath}/aws-${AWS_REGION}.json
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, we decided to add the smoketested tag since we run rhcos.basic tests in the cloud job before we generate the ami, thus the generated image should at least boot, can ssh, etc. I'm ok with changing it, but that's the original reasoning we had behind it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we don't upload it at all if it doesn't pass the smoketest, I think I'm OK with omitting it. Kinda bikeshed though, I'm fine keeping it if someone feels strongly about it.

ore aws upload --region ${AWS_REGION} \
--ami-name 'rhcos_dev_${commit[0..6]}' \
--ami-description 'Red Hat CoreOS ${version} (${commit})' \
Expand Down Expand Up @@ -251,13 +251,7 @@ node(NODE) {
parallel par_stages; par_stages = [:]

// Build the job responsible for testing and publishing the ami
def ami_intermediate = utils.sh_capture("jq -r .HVM ${dirpath}/aws-${AWS_REGION}-smoketested.json")
build job: 'coreos-rhcos-aws-test', wait: false, parameters: [
string(name: 'ami_intermediate', value: "${ami_intermediate}"),
string(name: 'version', value: "${version}"),
string(name: 'dirpath', value: "${dirpath}"),
string(name: 'aws_type', value: "t2.small")
]
build job: 'coreos-rhcos-aws-test', wait: false
}
} catch (Throwable e) {
currentBuild.result = 'FAILURE'
Expand Down
8 changes: 8 additions & 0 deletions pipeline-utils.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,10 +120,18 @@ def rsync_file_in(server, key, file) {
rsync_file(key, "${server}:${file}", file)
}

def rsync_file_in_dest(server, key, srcfile, destfile) {
rsync_file(key, "${server}:${srcfile}", destfile)
}

def rsync_file_out(server, key, file) {
rsync_file(key, file, "${server}:${file}")
}

def rsync_file_out_dest(server, key, srcfile, destfile) {
rsync_file(key, srcfile, "${server}:${destfile}")
}

def rsync_file(key, from_file, to_file) {
sh """
rsync -Hlpt --stats \
Expand Down