Skip to content

Commit

Permalink
feat(client-batch): This release adds Batch support for configuration…
Browse files Browse the repository at this point in the history 
… of multicontainer jobs in ECS, Fargate, and EKS. This support is available for all types of jobs, including both array jobs and multi-node parallel jobs.
  • Loading branch information
awstools committed Feb 28, 2024
1 parent 1ebc3aa commit edbdca7
Show file tree
Hide file tree
Showing 16 changed files with 3,637 additions and 1,362 deletions.
104 changes: 59 additions & 45 deletions clients/client-batch/src/commands/CreateComputeEnvironmentCommand.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,35 +28,42 @@ export interface CreateComputeEnvironmentCommandOutput extends CreateComputeEnvi

/**
* @public
* <p>Creates an Batch compute environment. You can create <code>MANAGED</code> or <code>UNMANAGED</code> compute
* environments. <code>MANAGED</code> compute environments can use Amazon EC2 or Fargate resources.
* <code>UNMANAGED</code> compute environments can only use EC2 resources.</p>
* <p>In a managed compute environment, Batch manages the capacity and instance types of the compute resources
* within the environment. This is based on the compute resource specification that you define or the <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-launch-templates.html">launch template</a> that you
* specify when you create the compute environment. Either, you can choose to use EC2 On-Demand Instances and EC2 Spot
* Instances. Or, you can use Fargate and Fargate Spot capacity in your managed compute environment. You can
* optionally set a maximum price so that Spot Instances only launch when the Spot Instance price is less than a
* specified percentage of the On-Demand price.</p>
* <p>Creates an Batch compute environment. You can create <code>MANAGED</code> or
* <code>UNMANAGED</code> compute environments. <code>MANAGED</code> compute environments can
* use Amazon EC2 or Fargate resources. <code>UNMANAGED</code> compute environments can only use
* EC2 resources.</p>
* <p>In a managed compute environment, Batch manages the capacity and instance types of the
* compute resources within the environment. This is based on the compute resource specification
* that you define or the <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-launch-templates.html">launch template</a> that you
* specify when you create the compute environment. Either, you can choose to use EC2 On-Demand
* Instances and EC2 Spot Instances. Or, you can use Fargate and Fargate Spot capacity in
* your managed compute environment. You can optionally set a maximum price so that Spot
* Instances only launch when the Spot Instance price is less than a specified percentage of the
* On-Demand price.</p>
* <note>
* <p>Multi-node parallel jobs aren't supported on Spot Instances.</p>
* </note>
* <p>In an unmanaged compute environment, you can manage your own EC2 compute resources and have flexibility with how
* you configure your compute resources. For example, you can use custom AMIs. However, you must verify that each of
* your AMIs meet the Amazon ECS container instance AMI specification. For more information, see <a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/container_instance_AMIs.html">container instance AMIs</a> in the
* <i>Amazon Elastic Container Service Developer Guide</i>. After you created your unmanaged compute environment, you can use the <a>DescribeComputeEnvironments</a> operation to find the Amazon ECS cluster that's associated with it. Then, launch
* your container instances into that Amazon ECS cluster. For more information, see <a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/launch_container_instance.html">Launching an Amazon ECS container instance</a> in the
* <i>Amazon Elastic Container Service Developer Guide</i>.</p>
* <p>In an unmanaged compute environment, you can manage your own EC2 compute resources and
* have flexibility with how you configure your compute resources. For example, you can use
* custom AMIs. However, you must verify that each of your AMIs meet the Amazon ECS container instance
* AMI specification. For more information, see <a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/container_instance_AMIs.html">container instance AMIs</a> in the
* <i>Amazon Elastic Container Service Developer Guide</i>. After you created your unmanaged compute environment,
* you can use the <a>DescribeComputeEnvironments</a> operation to find the Amazon ECS
* cluster that's associated with it. Then, launch your container instances into that Amazon ECS
* cluster. For more information, see <a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/launch_container_instance.html">Launching an Amazon ECS container
* instance</a> in the <i>Amazon Elastic Container Service Developer Guide</i>.</p>
* <note>
* <p>To create a compute environment that uses EKS resources, the caller must have permissions to call
* <code>eks:DescribeCluster</code>.</p>
* <p>To create a compute environment that uses EKS resources, the caller must have
* permissions to call <code>eks:DescribeCluster</code>.</p>
* </note>
* <note>
* <p>Batch doesn't automatically upgrade the AMIs in a compute environment after it's created. For example, it
* also doesn't update the AMIs in your compute environment when a newer version of the Amazon ECS optimized AMI is
* available. You're responsible for the management of the guest operating system. This includes any updates and
* security patches. You're also responsible for any additional application software or utilities that you install on
* the compute resources. There are two ways to use a new AMI for your Batch jobs. The original method is to complete
* these steps:</p>
* <p>Batch doesn't automatically upgrade the AMIs in a compute environment after it's
* created. For example, it also doesn't update the AMIs in your compute environment when a
* newer version of the Amazon ECS optimized AMI is available. You're responsible for the management
* of the guest operating system. This includes any updates and security patches. You're also
* responsible for any additional application software or utilities that you install on the
* compute resources. There are two ways to use a new AMI for your Batch jobs. The original
* method is to complete these steps:</p>
* <ol>
* <li>
* <p>Create a new compute environment with the new AMI.</p>
Expand All @@ -71,17 +78,19 @@ export interface CreateComputeEnvironmentCommandOutput extends CreateComputeEnvi
* <p>Delete the earlier compute environment.</p>
* </li>
* </ol>
* <p>In April 2022, Batch added enhanced support for updating compute environments. For more information, see
* <a href="https://docs.aws.amazon.com/batch/latest/userguide/updating-compute-environments.html">Updating compute
* environments</a>. To use the enhanced updating of compute environments to update AMIs, follow these
* rules:</p>
* <p>In April 2022, Batch added enhanced support for updating compute environments. For
* more information, see <a href="https://docs.aws.amazon.com/batch/latest/userguide/updating-compute-environments.html">Updating compute environments</a>.
* To use the enhanced updating of compute environments to update AMIs, follow these
* rules:</p>
* <ul>
* <li>
* <p>Either don't set the service role (<code>serviceRole</code>) parameter or set it to the <b>AWSBatchServiceRole</b> service-linked role.</p>
* <p>Either don't set the service role (<code>serviceRole</code>) parameter or set it to
* the <b>AWSBatchServiceRole</b> service-linked role.</p>
* </li>
* <li>
* <p>Set the allocation strategy (<code>allocationStrategy</code>) parameter to <code>BEST_FIT_PROGRESSIVE</code>,
* <code>SPOT_CAPACITY_OPTIMIZED</code>, or <code>SPOT_PRICE_CAPACITY_OPTIMIZED</code>.</p>
* <p>Set the allocation strategy (<code>allocationStrategy</code>) parameter to
* <code>BEST_FIT_PROGRESSIVE</code>, <code>SPOT_CAPACITY_OPTIMIZED</code>, or
* <code>SPOT_PRICE_CAPACITY_OPTIMIZED</code>.</p>
* </li>
* <li>
* <p>Set the update to latest image version (<code>updateToLatestImageVersion</code>)
Expand All @@ -92,24 +101,29 @@ export interface CreateComputeEnvironmentCommandOutput extends CreateComputeEnvi
* environment.</p>
* </li>
* <li>
* <p>Don't specify an AMI ID in <code>imageId</code>, <code>imageIdOverride</code> (in <a href="https://docs.aws.amazon.com/batch/latest/APIReference/API_Ec2Configuration.html">
* <p>Don't specify an AMI ID in <code>imageId</code>, <code>imageIdOverride</code> (in
* <a href="https://docs.aws.amazon.com/batch/latest/APIReference/API_Ec2Configuration.html">
* <code>ec2Configuration</code>
* </a>), or in the launch
* template (<code>launchTemplate</code>). In that case, Batch selects the latest Amazon ECS optimized AMI that's
* supported by Batch at the time the infrastructure update is initiated. Alternatively, you can specify the AMI ID
* in the <code>imageId</code> or <code>imageIdOverride</code> parameters, or the launch template identified by the
* <code>LaunchTemplate</code> properties. Changing any of these properties starts an infrastructure update. If the
* AMI ID is specified in the launch template, it can't be replaced by specifying an AMI ID in either the
* <code>imageId</code> or <code>imageIdOverride</code> parameters. It can only be replaced by specifying a different
* launch template, or if the launch template version is set to <code>$Default</code> or <code>$Latest</code>, by
* setting either a new default version for the launch template (if <code>$Default</code>) or by adding a new version
* to the launch template (if <code>$Latest</code>).</p>
* </a>), or in the launch template
* (<code>launchTemplate</code>). In that case, Batch selects the latest Amazon ECS
* optimized AMI that's supported by Batch at the time the infrastructure update is
* initiated. Alternatively, you can specify the AMI ID in the <code>imageId</code> or
* <code>imageIdOverride</code> parameters, or the launch template identified by the
* <code>LaunchTemplate</code> properties. Changing any of these properties starts an
* infrastructure update. If the AMI ID is specified in the launch template, it can't be
* replaced by specifying an AMI ID in either the <code>imageId</code> or
* <code>imageIdOverride</code> parameters. It can only be replaced by specifying a
* different launch template, or if the launch template version is set to
* <code>$Default</code> or <code>$Latest</code>, by setting either a new default version
* for the launch template (if <code>$Default</code>) or by adding a new version to the
* launch template (if <code>$Latest</code>).</p>
* </li>
* </ul>
* <p>If these rules are followed, any update that starts an infrastructure update causes the AMI ID to be
* re-selected. If the <code>version</code> setting in the launch template (<code>launchTemplate</code>) is set to
* <code>$Latest</code> or <code>$Default</code>, the latest or default version of the launch template is evaluated up
* at the time of the infrastructure update, even if the <code>launchTemplate</code> wasn't updated.</p>
* <p>If these rules are followed, any update that starts an infrastructure update causes the
* AMI ID to be re-selected. If the <code>version</code> setting in the launch template
* (<code>launchTemplate</code>) is set to <code>$Latest</code> or <code>$Default</code>, the
* latest or default version of the launch template is evaluated up at the time of the
* infrastructure update, even if the <code>launchTemplate</code> wasn't updated.</p>
* </note>
* @example
* Use a bare-bones client and the command you need to make an API call.
Expand Down
13 changes: 7 additions & 6 deletions clients/client-batch/src/commands/CreateJobQueueCommand.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,13 @@ export interface CreateJobQueueCommandOutput extends CreateJobQueueResponse, __M

/**
* @public
* <p>Creates an Batch job queue. When you create a job queue, you associate one or more compute environments to the
* queue and assign an order of preference for the compute environments.</p>
* <p>You also set a priority to the job queue that determines the order that the Batch scheduler places jobs onto
* its associated compute environments. For example, if a compute environment is associated with more than one job
* queue, the job queue with a higher priority is given preference for scheduling jobs to that compute
* environment.</p>
* <p>Creates an Batch job queue. When you create a job queue, you associate one or more
* compute environments to the queue and assign an order of preference for the compute
* environments.</p>
* <p>You also set a priority to the job queue that determines the order that the Batch
* scheduler places jobs onto its associated compute environments. For example, if a compute
* environment is associated with more than one job queue, the job queue with a higher priority
* is given preference for scheduling jobs to that compute environment.</p>
* @example
* Use a bare-bones client and the command you need to make an API call.
* ```javascript
Expand Down
9 changes: 6 additions & 3 deletions clients/client-batch/src/commands/DeleteComputeEnvironmentCommand.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,12 @@ export interface DeleteComputeEnvironmentCommandOutput extends DeleteComputeEnvi
/**
* @public
* <p>Deletes an Batch compute environment.</p>
* <p>Before you can delete a compute environment, you must set its state to <code>DISABLED</code> with the <a>UpdateComputeEnvironment</a> API operation and disassociate it from any job queues with the <a>UpdateJobQueue</a> API operation. Compute environments that use Fargate resources must terminate all
* active jobs on that compute environment before deleting the compute environment. If this isn't done, the compute
* environment enters an invalid state.</p>
* <p>Before you can delete a compute environment, you must set its state to
* <code>DISABLED</code> with the <a>UpdateComputeEnvironment</a> API operation and
* disassociate it from any job queues with the <a>UpdateJobQueue</a> API operation.
* Compute environments that use Fargate resources must terminate all active jobs on that
* compute environment before deleting the compute environment. If this isn't done, the compute
* environment enters an invalid state.</p>
* @example
* Use a bare-bones client and the command you need to make an API call.
* ```javascript
Expand Down
8 changes: 5 additions & 3 deletions clients/client-batch/src/commands/DeleteJobQueueCommand.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,12 @@ export interface DeleteJobQueueCommandOutput extends DeleteJobQueueResponse, __M

/**
* @public
* <p>Deletes the specified job queue. You must first disable submissions for a queue with the <a>UpdateJobQueue</a> operation. All jobs in the queue are eventually terminated when you delete a job queue.
* The jobs are terminated at a rate of about 16 jobs each second.</p>
* <p>Deletes the specified job queue. You must first disable submissions for a queue with the
* <a>UpdateJobQueue</a> operation. All jobs in the queue are eventually terminated
* when you delete a job queue. The jobs are terminated at a rate of about 16 jobs each
* second.</p>
* <p>It's not necessary to disassociate compute environments from a queue before submitting a
* <code>DeleteJobQueue</code> request.</p>
* <code>DeleteJobQueue</code> request.</p>
* @example
* Use a bare-bones client and the command you need to make an API call.
* ```javascript
Expand Down
3 changes: 2 additions & 1 deletion clients/client-batch/src/commands/DeregisterJobDefinitionCommand.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ export interface DeregisterJobDefinitionCommandOutput extends DeregisterJobDefin

/**
* @public
* <p>Deregisters an Batch job definition. Job definitions are permanently deleted after 180 days.</p>
* <p>Deregisters an Batch job definition. Job definitions are permanently deleted after 180
* days.</p>
* @example
* Use a bare-bones client and the command you need to make an API call.
* ```javascript
Expand Down
6 changes: 3 additions & 3 deletions clients/client-batch/src/commands/DescribeComputeEnvironmentsCommand.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,9 @@ export interface DescribeComputeEnvironmentsCommandOutput
/**
* @public
* <p>Describes one or more of your compute environments.</p>
* <p>If you're using an unmanaged compute environment, you can use the <code>DescribeComputeEnvironment</code>
* operation to determine the <code>ecsClusterArn</code> that you launch your Amazon ECS container instances
* into.</p>
* <p>If you're using an unmanaged compute environment, you can use the
* <code>DescribeComputeEnvironment</code> operation to determine the
* <code>ecsClusterArn</code> that you launch your Amazon ECS container instances into.</p>
* @example
* Use a bare-bones client and the command you need to make an API call.
* ```javascript
Expand Down
Loading

0 comments on commit edbdca7

Please sign in to comment.