Skip to content

Commit

Permalink
[INF-118] Updated to 1.0.8
Browse files Browse the repository at this point in the history
  • Loading branch information
@mcdermottsailthru
mcdermottsailthru committed Oct 18, 2023
1 parent 099b5be commit 91a9504
Show file tree
Hide file tree
Showing 8 changed files with 67,633 additions and 289,548 deletions.
41 changes: 39 additions & 2 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,15 @@ The only required input is `project-name`.
that CodeBuild requires.
By default, the action uses the buildspec file location
that you configured in the CodeBuild project.
1. **compute-type-override** (optional) :
The name of a compute type for this build that overrides the one specified
in the build project.
1. **environment-type-override** (optional) :
A container type for this build that overrides the one specified in the
build project.
1. **image-override** (optional) :
The name of an image for this build that overrides the one specified
in the build project.
1. **env-vars-for-codebuild** (optional) :
A comma-separated list of the names of environment variables
that the action passes from GitHub Actions to CodeBuild.
Expand All @@ -37,6 +46,31 @@ The only required input is `project-name`.
the one defined here replaces the one in the CodeBuild project.
For a list of CodeBuild environment variables, see

1. **update-interval** (optional) :
Update interval as seconds for how often the API is called to check on the status.

A higher value mitigates the chance of hitting API rate-limiting especially when
running many instances of this action in parallel, but also introduces a larger
potential time overhead (ranging from 0 to update interval) for the action to
fetch the build result and finish.

Lower value limits the potential time overhead worst case but it may hit the API
rate-limit more often, depending on the use-case.

The default value is 30.

1. **update-back-off** (optional) :
Base back-off time in seconds for the update interval.

When API rate-limiting is hit the back-off time, augmented with jitter, will be
added to the next update interval.
E.g. with update interval of 30 and back-off time of 15, upon hitting the rate-limit
the next interval for the update call will be 30 + random_between(0, 15 _ 2 \*\* 0))
seconds and if the rate-limit is hit again the next interval will be
30 + random_between(0, 15 _ 2 \*\* 1) and so on.

The default value is 15.

### Outputs

1. **aws-build-id** : The CodeBuild build ID of the build that the action ran.
Expand Down Expand Up @@ -136,7 +170,7 @@ the only CodeBuild Run input you need to provide is the project name.
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-2
- name: Run CodeBuild
uses: aws-actions/aws-codebuild-run-build@v1.0.3
uses: aws-actions/aws-codebuild-run-build@v1
with:
project-name: CodeBuildProjectName
```
Expand All @@ -158,10 +192,13 @@ this will overwrite them.
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-2
- name: Run CodeBuild
uses: aws-actions/aws-codebuild-run-build@v1.0.3
uses: aws-actions/aws-codebuild-run-build@v1
with:
project-name: CodeBuildProjectName
buildspec-override: path/to/buildspec.yaml
compute-type-override: compute-type
environment-type-override: environment-type
image-override: ecr-image-uri
env-vars-for-codebuild: |
custom,
requester,
Expand Down
20 changes: 18 additions & 2 deletions action.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,28 @@ inputs:
buildspec-override:
description: 'Buildspec Override'
required: false
compute-type-override:
description: 'The name of a compute type for this build that overrides the one specified in the build project.'
required: false
environment-type-override:
description: 'A container type for this build that overrides the one specified in the build project.'
required: false
image-override:
description: 'The name of an image for this build that overrides the one specified in the build project.'
required: false
env-vars-for-codebuild:
description: 'Comma separated list of environment variables to send to CodeBuild'
required: false
update-interval:
description: 'How often the action calls the API for updates'
required: false
update-back-off:
description: 'Base back-off time for the update calls for API if rate-limiting is encountered'
required: false

outputs:
aws-build-id:
description: 'The AWS CodeBuild Build ID for this build.'
runs:
using: 'node12'
main: 'dist/index.js'
using: 'node16'
main: 'dist/index.js'
162 changes: 130 additions & 32 deletions code-build.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,27 +20,41 @@ function runBuild() {
// get a codeBuild instance from the SDK
const sdk = buildSdk();

const inputs = githubInputs();

const config = (({ updateInterval, updateBackOff }) => ({
updateInterval,
updateBackOff,
}))(inputs);

// Get input options for startBuild
const params = inputs2Parameters(githubInputs());
const params = inputs2Parameters(inputs);

return build(sdk, params);
return build(sdk, params, config);
}

async function build(sdk, params) {
async function build(sdk, params, config) {
// Start the build
const start = await sdk.codeBuild.startBuild(params).promise();

// Wait for the build to "complete"
return waitForBuildEndTime(sdk, start.build);
return waitForBuildEndTime(sdk, start.build, config);
}

async function waitForBuildEndTime(sdk, { id, logs }, nextToken) {
const {
codeBuild,
cloudWatchLogs,
wait = 1000 * 30,
backOff = 1000 * 15,
} = sdk;
async function waitForBuildEndTime(
sdk,
{ id, logs },
{ updateInterval, updateBackOff },
seqEmptyLogs,
totalEvents,
throttleCount,
nextToken
) {
const { codeBuild, cloudWatchLogs } = sdk;

totalEvents = totalEvents || 0;
seqEmptyLogs = seqEmptyLogs || 0;
throttleCount = throttleCount || 0;

// Get the CloudWatchLog info
const startFromHead = true;
Expand All @@ -55,7 +69,12 @@ async function waitForBuildEndTime(sdk, { id, logs }, nextToken) {
// The CloudWatchLog _may_ not be set up, only make the call if we have a logGroupName
logGroupName &&
cloudWatchLogs
.getLogEvents({ logGroupName, logStreamName, startFromHead, nextToken })
.getLogEvents({
logGroupName,
logStreamName,
startFromHead,
nextToken,
})
.promise(),
]).catch((err) => {
errObject = err;
Expand All @@ -70,16 +89,24 @@ async function waitForBuildEndTime(sdk, { id, logs }, nextToken) {
if (errObject) {
//We caught an error in trying to make the AWS api call, and are now checking to see if it was just a rate limiting error
if (errObject.message && errObject.message.search("Rate exceeded") !== -1) {
//We were rate-limited, so add `backOff` seconds to the wait time
let newWait = wait + backOff;
// We were rate-limited, so add backoff with Full Jitter, ref: https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/
let jitteredBackOff = Math.floor(
Math.random() * (updateBackOff * 2 ** throttleCount)
);
let newWait = updateInterval + jitteredBackOff;
throttleCount++;

//Sleep before trying again
await new Promise((resolve) => setTimeout(resolve, newWait));

// Try again from the same token position
return waitForBuildEndTime(
{ ...sdk, wait: newWait },
{ ...sdk },
{ id, logs },
{ updateInterval: newWait, updateBackOff },
seqEmptyLogs,
totalEvents,
throttleCount,
nextToken
);
} else {
Expand All @@ -92,20 +119,48 @@ async function waitForBuildEndTime(sdk, { id, logs }, nextToken) {
const [current] = batch.builds;
const { nextForwardToken, events = [] } = cloudWatch;

// GetLogEvents can return partial/empty responses even when there is data.
// We wait for two consecutive empty log responses to minimize false positive on EOF.
// Empty response counter starts after any logs have been received, or when the build completes.
if (events.length == 0 && (totalEvents > 0 || current.endTime)) {
seqEmptyLogs++;
} else {
seqEmptyLogs = 0;
}
totalEvents += events.length;

// stdout the CloudWatchLog (everyone likes progress...)
// CloudWatchLogs have line endings.
// I trim and then log each line
// to ensure that the line ending is OS specific.
events.forEach(({ message }) => console.log(message.trimEnd()));

// We did it! We can stop looking!
if (current.endTime && !events.length) return current;
// Stop after the build is ended and we've received two consecutive empty log responses
if (current.endTime && seqEmptyLogs >= 2) {
return current;
}

// More to do: Sleep for a few seconds to avoid rate limiting
await new Promise((resolve) => setTimeout(resolve, wait));
// If never throttled and build is complete, halve CWL polling delay to minimize latency
await new Promise((resolve) =>
setTimeout(
resolve,
current.endTime && throttleCount == 0
? updateInterval / 2
: updateInterval
)
);

// Try again
return waitForBuildEndTime(sdk, current, nextForwardToken);
return waitForBuildEndTime(
sdk,
current,
{ updateInterval, updateBackOff },
seqEmptyLogs,
totalEvents,
throttleCount,
nextForwardToken
);
}

function githubInputs() {
Expand All @@ -117,7 +172,7 @@ function githubInputs() {
// So I use the raw ENV.
// There is a complexity here because for pull request
// the GITHUB_SHA value is NOT the correct value.
// See: https://github.com/sailthru/aws-codebuild-run-build/issues/36
// See: https://github.com/aws-actions/aws-codebuild-run-build/issues/36
const sourceVersion =
process.env[`GITHUB_EVENT_NAME`] === "pull_request"
? (((payload || {}).pull_request || {}).head || {}).sha
Expand All @@ -127,19 +182,44 @@ function githubInputs() {
const buildspecOverride =
core.getInput("buildspec-override", { required: false }) || undefined;

const computeTypeOverride =
core.getInput("compute-type-override", { required: false }) || undefined;

const environmentTypeOverride =
core.getInput("environment-type-override", { required: false }) ||
undefined;
const imageOverride =
core.getInput("image-override", { required: false }) || undefined;

const envPassthrough = core
.getInput("env-vars-for-codebuild", { required: false })
.split(",")
.map((i) => i.trim())
.filter((i) => i !== "");

const updateInterval =
parseInt(
core.getInput("update-interval", { required: false }) || "30",
10
) * 1000;
const updateBackOff =
parseInt(
core.getInput("update-back-off", { required: false }) || "15",
10
) * 1000;

return {
projectName,
owner,
repo,
sourceVersion,
buildspecOverride,
computeTypeOverride,
environmentTypeOverride,
imageOverride,
envPassthrough,
updateInterval,
updateBackOff,
};
}

Expand All @@ -150,8 +230,15 @@ function inputs2Parameters(inputs) {
repo,
sourceVersion,
buildspecOverride,
computeTypeOverride,
environmentTypeOverride,
imageOverride,
envPassthrough = [],
} = inputs;

const sourceTypeOverride = "GITHUB";
const sourceLocationOverride = `https://github.com/${owner}/${repo}.git`;

const environmentVariablesOverride = Object.entries(process.env)
.filter(
([key]) => key.startsWith("GITHUB_") || envPassthrough.includes(key)
Expand All @@ -162,36 +249,47 @@ function inputs2Parameters(inputs) {
// This way the GitHub events can manage the builds.
return {
projectName,
sourceVersion,
sourceTypeOverride,
sourceLocationOverride,
buildspecOverride,
computeTypeOverride,
environmentTypeOverride,
imageOverride,
environmentVariablesOverride,
};
}

function buildSdk() {
const codeBuild = new aws.CodeBuild({
customUserAgent: "sailthru/aws-codebuild-run-build",
customUserAgent: "aws-actions/aws-codebuild-run-build",
});

const cloudWatchLogs = new aws.CloudWatchLogs({
customUserAgent: "sailthru/aws-codebuild-run-build",
customUserAgent: "aws-actions/aws-codebuild-run-build",
});

assert(
codeBuild.config.credentials && cloudWatchLogs.config.credentials,
"No credentials. Try adding @sailthru/configure-aws-credentials earlier in your job to set up AWS credentials."
"No credentials. Try adding @aws-actions/configure-aws-credentials earlier in your job to set up AWS credentials."
);

return { codeBuild, cloudWatchLogs };
}

function logName(Arn) {
const [logGroupName, logStreamName] = Arn.split(":log-group:")
.pop()
.split(":log-stream:");
if (logGroupName === "null" || logStreamName === "null")
return {
logGroupName: undefined,
logStreamName: undefined,
};
return { logGroupName, logStreamName };
const logs = {
logGroupName: undefined,
logStreamName: undefined,
};
if (Arn) {
const [logGroupName, logStreamName] = Arn.split(":log-group:")
.pop()
.split(":log-stream:");
if (logGroupName !== "null" && logStreamName !== "null") {
logs.logGroupName = logGroupName;
logs.logStreamName = logStreamName;
}
}
return logs;
}
Loading

0 comments on commit 91a9504

Please sign in to comment.