Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implementation of running a lambda in ecs #1

Merged
merged 14 commits into from
Mar 9, 2018

Conversation

sethvincent
Copy link
Contributor

@sethvincent sethvincent commented Mar 5, 2018

This is based in part on https://git.earthdata.nasa.gov/projects/CUMULUS/repos/ecs-lambda-runner/browse

Updated to work with step function tasks using sf.getActivityTask and related step functions API methods.

@sethvincent
Copy link
Contributor Author

sethvincent commented Mar 5, 2018

When given enough time, ecs service creation will fail with:

Service <arn> did not stabilize.

Copy link

@scisco scisco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

made two comments.

- run: |
docker build -t developmentseed/cumulus:lambda-runner .
docker login -u scisco -p $DOCKER_PASSWORD
docker push developmentseed/cumulus:lambda-runner
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's push this to the cumuluss org on docker hub?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to be exact, push it to: cumuluss/cumulus-ecs-task

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, what is our plan for versioning here? I think we should not release with latest tag and start versioning this image.

@scisco
Copy link

scisco commented Mar 5, 2018

@sethvincent the service ARN is probably not stabilized because it lacks the correct permissions. You have to make sure that your ec2 instance has the right to download the zip file of a given lambda.

Also you have to make sure the base image has the latest version of python 2.7

@sethvincent
Copy link
Contributor Author

sethvincent commented Mar 7, 2018

This branch can now work with a couple of outside changes:

  • changing cumulus-message-adapter-js to use an env var to specify the cumulus-message-adapter location
  • changing iam/cloudformation.template.yml in the deployment to add - states:SendTaskHeartbeat in the EcsRole allow actions

@sethvincent sethvincent changed the title wip implementation of running a lambda in ecs implementation of running a lambda in ecs Mar 7, 2018
@sethvincent
Copy link
Contributor Author

sethvincent commented Mar 8, 2018

I've updated this with:

  • circleci improvements that publish to npm and docker hub when a PR gets merged that bumps the version in package.json
  • revises the dockerfile
  • adds .eslintrc.json from cumulus-nasa/cumulus
  • README.md updates with instructions that will help with reviewing and testing this PR
  • a bunch of lint and bug fixes and a slight refactor from when I first opened this PR

Copy link

@scisco scisco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

It'd be useful to also add a CHANGELOG

@sethvincent sethvincent merged commit 3b8620b into master Mar 9, 2018
@scisco scisco deleted the cumulus-182-ecs-task branch March 30, 2018 21:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants