aws-ecs-patterns: Allow ScheduledFargateTask and ScheduledEc2Task to run on a public subnet

[floehopper]

Allow ScheduledFargateTask and ScheduledEc2Task to run in a public subnet via a configuration option.

Use Case

I'm currently using the ScheduledFargateTask class to run a number of simple tasks. The tasks need access to the internet, but there's no need for them to be accessible from the internet. I could run the tasks on a private subnet, but this would mean I would need either a NAT Gateway (expensive) or run a NAT Instance on EC2 (maintenance/complexity overhead).

Since the tasks only run for a few minutes every week I'm willing to sacrifice the extra security that a private subnet provides in favour of a simpler/cheaper system where the tasks run on a public subnet.

However, currently ScheduledFargateTask will only run a task if its VPC has a private subnet - if there is no private subnet available, an error is reported. I would like to be able to run my tasks on a VPC defined along the following lines:

    const vpc = new ec2.Vpc(stack, 'Vpc', {
      maxAzs: 1,
      natGateways: 0,
      subnetConfiguration: [
        { name: 'public', cidrMask: 24, subnetType: SubnetType.PUBLIC }
      ],
    });
    const cluster = new ecs.Cluster(stack, 'EcsCluster', { vpc });

Proposed Solution

Currently the addTaskDefinitionToEventTarget method on ScheduledTaskBase instantiates the underlying EcsTask, but there is no way to specify the subnetSelection property of that EcsTask and so it defaults to { subnetType: ec2.SubnetType.PRIVATE }.

I propose that we add a subnetSelection property to the ScheduledTaskBaseProps interface and supply that property when instantiating the underlying EcsTask. This new property would default to { subnetType: ec2.SubnetType.PRIVATE } thus retaining the existing behaviour.

Other

• In this comment @skinny85 mentioned that this is a gap in the ScheduledFargateTask construct.

• In this comment @moofish32 suggested that using only a public subnet is a reasonable option in some circumstances.

• Reading between the lines, it sounds as if others commenting on this issue might welcome this functionality.

• 👋 I may be able to implement this feature request

• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[pepastach]

I'm facing exactly the same problem, so thanks @floehopper for a great bug report!

Were you able to work around that? I need to run a ~1 hour task once a month, so provisioning a NAT gateway is out of question. Have you considered creating aFargateService and starting it from a lambda function? Or perhaps a step function with Fargate task might work as well.

Thanks

[floehopper]

@pepastach

I'm facing exactly the same problem, so thanks @floehopper for a great bug report!

Thanks. It's good to know I'm not the only one!

Were you able to work around that?

Yes. I'm currently working round the problem by defining my own ScheduledFargateTaskOnPublicSubnet class that extends ScheduledFargateTask and overrides the addTaskDefinitionToEventTarget method to instantiate the EcsTask with the appropriate subnetSelection.

Have you considered creating a FargateService and starting it from a lambda function?

I hadn't considered this, but now you mention it, it might be a sensible approach in my use cases, because I'm already creating a FargateTaskDefinition in order to be able to pass Secrets Manager secrets directly to the container. I suppose it's a case of how much extra boilerplate would I need to do it that way...?

Or perhaps a step function with Fargate task might work as well.

I'm not very familiar with Step functions, so I'm not clear how this would help.

[floehopper]

@SoManyHs @pkandasamy91 I checked the "I may be able to implement this feature request" checkbox in the issue description as per the contribution guidelines. Is it worth me submitting a PR for this issue or are you unlikely to accept it...?

[pepastach]

@floehopper

Yes. I'm currently working round the problem by defining my own ScheduledFargateTaskOnPublicSubnet class that extends ScheduledFargateTask and overrides the addTaskDefinitionToEventTarget method to instantiate the EcsTask with the appropriate subnetSelection.

I define my VPC as follows:

def vpc(self) -> aws_ec2.Vpc:
    return aws_ec2.Vpc(
        self, 'VPC',
        cidr='172.16.0.0/28',  # 172.16.0.1 -> 172.16.0.14
        subnet_configuration=[
            aws_ec2.SubnetConfiguration(
                name='Public',
                subnet_type=aws_ec2.SubnetType.PUBLIC
            )
        ],
        max_azs=1,
        nat_gateways=0
    )

Then the class you suggested:

class ScheduledFargateTaskOnPublicSubnet(aws_ecs_patterns.ScheduledFargateTask):
    def __init__(self, scope: core.Construct, id: str, *,
                 schedule: aws_applicationautoscaling.Schedule,
                 scheduled_fargate_task_definition_options: Optional[aws_ecs_patterns.ScheduledFargateTaskDefinitionOptions] = None,
                 scheduled_fargate_task_image_options: Optional[aws_ecs_patterns.ScheduledFargateTaskImageOptions] = None,
                 cluster: Optional[aws_ecs.ICluster] = None,
                 desired_task_count: Optional[jsii.Number] = None,
                 vpc: Optional[aws_ec2.IVpc] = None) -> None:
        super().__init__(
            scope, id,
            schedule=schedule,
            scheduled_fargate_task_definition_options=scheduled_fargate_task_definition_options,
            scheduled_fargate_task_image_options=scheduled_fargate_task_image_options,
            cluster=cluster,
            desired_task_count=desired_task_count,
            vpc=vpc
        )

    def _add_task_definition_to_event_target(self, task_definition: aws_ecs.TaskDefinition) -> aws_events_targets.EcsTask:
        event_rule_target = aws_events_targets.EcsTask(
            cluster=self.cluster,
            task_definition=task_definition,
            subnet_selection=aws_ec2.SubnetSelection(subnet_type=aws_ec2.SubnetType.PUBLIC),
            task_count=self.desired_task_count
        )
        self.event_rule().add_target(event_rule_target)

        return event_rule_target

I'm still getting:

jsii.errors.JavaScriptError:
  Error: There are no 'Private' subnet groups in this VPC. Available types: Public
      at Vpc.selectSubnetObjectsByType (/private/var/folders/9b/2y0vwzz52_d7sypzy0yxjmrc0000gn/T/jsii-kernel-O2htDY/node_modules/@aws-cdk/aws-ec2/lib/vpc.js:147:19)
      at Vpc.selectSubnetObjects (/private/var/folders/9b/2y0vwzz52_d7sypzy0yxjmrc0000gn/T/jsii-kernel-O2htDY/node_modules/@aws-cdk/aws-ec2/lib/vpc.js:120:25)
      at Vpc.selectSubnets (/private/var/folders/9b/2y0vwzz52_d7sypzy0yxjmrc0000gn/T/jsii-kernel-O2htDY/node_modules/@aws-cdk/aws-ec2/lib/vpc.js:70:30)
      at EcsTask.bind (/private/var/folders/9b/2y0vwzz52_d7sypzy0yxjmrc0000gn/T/jsii-kernel-O2htDY/node_modules/@aws-cdk/aws-events-targets/lib/ecs-task.js:64:57)
      at Rule.addTarget (/private/var/folders/9b/2y0vwzz52_d7sypzy0yxjmrc0000gn/T/jsii-kernel-O2htDY/node_modules/@aws-cdk/aws-events/lib/rule.js:68:36)
      at ScheduledFargateTask.addTaskDefinitionToEventTarget (/private/var/folders/9b/2y0vwzz52_d7sypzy0yxjmrc0000gn/T/jsii-kernel-O2htDY/node_modules/@aws-cdk/aws-ecs-patterns/lib/base/scheduled-task-base.js:38:24)
      at new ScheduledFargateTask (/private/var/folders/9b/2y0vwzz52_d7sypzy0yxjmrc0000gn/T/jsii-kernel-O2htDY/node_modules/@aws-cdk/aws-ecs-patterns/lib/fargate/scheduled-fargate-task.js:37:14)
      at /Users/xxx/Library/Caches/pypoetry/virtualenvs/cloud-infra-IEwMe2bo-py3.7/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7762:49
      at Kernel._wrapSandboxCode (/Users/xxx/Library/Caches/pypoetry/virtualenvs/cloud-infra-IEwMe2bo-py3.7/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:8225:19)
      at Kernel._create (/Users/xxx/Library/Caches/pypoetry/virtualenvs/cloud-infra-IEwMe2bo-py3.7/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7762:26)
      at Kernel.create (/Users/xxx/Library/Caches/pypoetry/virtualenvs/cloud-infra-IEwMe2bo-py3.7/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7509:21)
      at KernelHost.processRequest (/Users/xxx/Library/Caches/pypoetry/virtualenvs/cloud-infra-IEwMe2bo-py3.7/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7296:28)
      at KernelHost.run (/Users/xxx/Library/Caches/pypoetry/virtualenvs/cloud-infra-IEwMe2bo-py3.7/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7236:14)
      at Immediate._onImmediate (/Users/xxx/Library/Caches/pypoetry/virtualenvs/cloud-infra-IEwMe2bo-py3.7/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7239:37)
      at processImmediate (internal/timers.js:456:21)

It's interesting, that the ScheduledFargateTaskOnPublicSubnet class is not in the stack trace. It's not called. Even if I raise an exception in the _add_task_definition_to_event_target() method, nothing changes.

Any suggestion? Thanks in advance!

[piradeepk]

@SoManyHs @pkandasamy91 I checked the "I may be able to implement this feature request" checkbox in the issue description as per the contribution guidelines. Is it worth me submitting a PR for this issue or are you unlikely to accept it...?

@floehopper that would be great! We'd be more than happy to review your PR! 💯

[floehopper]

It's interesting, that the ScheduledFargateTaskOnPublicSubnet class is not in the stack trace. It's not called. Even if I raise an exception in the _add_task_definition_to_event_target() method, nothing changes.

Any suggestion? Thanks in advance!

Have you changed your stack definition to instantiate the ScheduledFargateTaskOnPublicSubnet class instead of ScheduledFargateTask?

I'm not super-familiar with Python. Does the overridden method definitely not invoke the superclass version of the method. Given that you've explicitly called super in the init method, I assume not. What happens if you print something from the overridden method and run the cdk synth command? Do you see the printed message?

Otherwise I can't see any obvious differences that look like they might be causing your problem. Here are the relevant bits of code from one of my projects:

    const vpc = new ec2.Vpc(this, 'vpc', {
      natGateways: 0,
      subnetConfiguration: [
        { name: 'public', cidrMask: 24, subnetType: ec2.SubnetType.PUBLIC }
      ],
    });

    const backupTask = new ScheduledFargateTaskOnPublicSubnet(this, 'backupTask', {
      cluster: cluster,
      scheduledFargateTaskDefinitionOptions: {
        taskDefinition: taskDefinition, schedule: schedule
      },
      schedule: schedule
    });

[pepastach]

Here's a working solution in Python. I had to inherit from aws_ecs_patterns.ScheduledTaskBase instead of aws_ecs_patterns.ScheduledFargateTask, because the overridden method _add_task_definition_to_event_target() would never get called.

class ScheduledFargateTaskOnPublicSubnet(aws_ecs_patterns.ScheduledTaskBase, metaclass=jsii.JSIIMeta, jsii_type="@aws-cdk/aws-ecs-patterns.ScheduledFargateTask"):
    def __init__(self, scope: core.Construct, id: str, *,
                 schedule: aws_applicationautoscaling.Schedule,
                 scheduled_fargate_task_definition_options: Optional[aws_ecs_patterns.ScheduledFargateTaskDefinitionOptions] = None,
                 scheduled_fargate_task_image_options: Optional[aws_ecs_patterns.ScheduledFargateTaskImageOptions] = None,
                 cluster: Optional[aws_ecs.ICluster] = None,
                 desired_task_count: Optional[jsii.Number] = None,
                 vpc: Optional[aws_ec2.IVpc] = None) -> None:
        super().__init__(scope, id, schedule=schedule, cluster=cluster, desired_task_count=desired_task_count, vpc=vpc)
        self.task_definition = aws_ecs.FargateTaskDefinition(
            self, 'MyTaskDef',
            memory_limit_mib=scheduled_fargate_task_image_options.memory_limit_mib or 512,
            cpu=scheduled_fargate_task_image_options.cpu or 256
        )
        self.task_definition.add_container(
            'MyContainer',
            image=scheduled_fargate_task_image_options.image,
            command=scheduled_fargate_task_image_options.command,
            environment=scheduled_fargate_task_image_options.environment,
            secrets=scheduled_fargate_task_image_options.secrets,
            logging=scheduled_fargate_task_image_options.log_driver
        )

        self._add_task_definition_to_event_target(self.task_definition)

    @jsii.member(jsii_name="taskDefinition")
    def task_definition(self) -> aws_ecs.FargateTaskDefinition:
        """The Fargate task definition in this construct."""
        return self.task_definition

    @jsii.member(jsii_name="addTaskDefinitionToEventTarget")
    def _add_task_definition_to_event_target(self, task_definition: aws_ecs.TaskDefinition) -> aws_events_targets.EcsTask:
        event_rule_target = aws_events_targets.EcsTask(
            cluster=self.cluster,
            task_definition=task_definition,
            subnet_selection=aws_ec2.SubnetSelection(subnet_type=aws_ec2.SubnetType.PUBLIC),
            task_count=self.desired_task_count
        )
        self.event_rule.add_target(event_rule_target)

        return event_rule_target

And then you can create your VPC with a public subnet only and no NAT gateways:

def vpc(self) -> aws_ec2.Vpc:
    return aws_ec2.Vpc(
        self, 'MyVPC',
        cidr='172.16.0.0/28',  # 172.16.0.1 -> 172.16.0.14
        subnet_configuration=[
            aws_ec2.SubnetConfiguration(
                name='Public',
                subnet_type=aws_ec2.SubnetType.PUBLIC
            )
        ],
        max_azs=1,
        nat_gateways=0
    )

[floehopper]

PR submitted: #6624