Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify vcpu and memory as resourceRequirements in AWS Batch executor #95

Merged
merged 1 commit into from
Jun 6, 2024

Conversation

aotimme
Copy link
Contributor

@aotimme aotimme commented May 23, 2024

When trying to use redun with AWS Batch on Fargate using the existing AWSBatchExecutor, we get the following error:

botocore.errorfactory.ClientException: An error occurred (ClientException) when calling the SubmitJob operation: vcpus is not applicable for Fargate. Use resource requirements instead.

As it turns out, it looks like specifying vcpu and memory directly on containerProperties is now deprecated (see here for vcpu and here for memory) in favor of specifying them in the resourceRequirements array, so that's the update provided here.

This update will allow running on Fargate with the AWSBatchExecutor.

@mattrasmus
Copy link
Collaborator

Hi @aotimme thanks for sharing this PR! We'll test it out and merge it in.

Copy link
Collaborator

@mattrasmus mattrasmus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We tested this internally and it works with existing workflows. We'll also do a follow up PR to also update the get_job_details() function to the new resources style.

Thanks again for contributing this fix!

@mattrasmus mattrasmus merged commit fd9479d into insitro:main Jun 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants