Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VPC: allow configuring NAT instances instead of gateways (and a 0 NAT gateways bug) #1305

Closed
0xdevalias opened this issue Dec 7, 2018 · 25 comments
Assignees
Labels
@aws-cdk/aws-ec2 Related to Amazon Elastic Compute Cloud @aws-cdk/aws-ecs-patterns Related to ecs-patterns library docs/generated Related to the generated API Reference documentation feature-request A feature should be added or improved. p0

Comments

@0xdevalias
Copy link
Contributor

When using ec2.VpcNetwork the defaults are to create NAT gateways. I originally scoped this down to just creating a single NAT gateway for my public subnet, and a month later was slogged with a $90 AWS bill, with almost all of that cost attributed to the NAT gateway.

So today I decided to try and rework it to remove the NAT gateway (since my app really doesn't need it anyway). Tried removing the key, but that uses the defaults (and makes more), so I tried setting the key to 0. Example config:

const vpc = new ec2.VpcNetwork(this, 'Tokenized-VPC', {
            natGateways: 0,
            // natGatewayPlacement: {subnetName: 'Public'},
            subnetConfiguration: [
                {
                    cidrMask: 26,
                    name: 'Public',
                    subnetType: ec2.SubnetType.Public,
                },
                {
                    name: 'Application',
                    subnetType: ec2.SubnetType.Private,
                },
            ],
            defaultInstanceTenancy: ec2.DefaultInstanceTenancy.Default,
        });

When I ran cdk diff, I got a number of errors back

Exactly one of [NetworkInterfaceId, VpcPeeringConnectionId, GatewayId, EgressOnlyInternetGatewayId, InstanceId, NatGatewayId] must be specified and not empty
 1/9 | 8:27:39 am | UPDATE_FAILED        | AWS::EC2::Route                       | Foo-VPC/ApplicationSubnet2/DefaultRoute (FooVPCApplicationSubnet2DefaultRoute2325F2C6) Exactly one of [NetworkInterfaceId, VpcPeeringConnectionId, GatewayId, EgressOnlyInternetGatewayId, InstanceId, NatGatewayId] must be specified and not empty
	VpcPrivateSubnet.addDefaultRouteToNAT (/foo/bar/deploy/aws-ec2/node_modules/@aws-cdk/aws-ec2/lib/vpc.js:259:9)
	\_ VpcPrivateSubnet.addDefaultNatRouteEntry (/foo/bar/deploy/aws-ec2/node_modules/@aws-cdk/aws-ec2/lib/vpc.js:316:14)
	\_ VpcNetwork.privateSubnets.forEach /foo/bar/deploy/aws-ec2/node_modules/@aws-cdk/aws-ec2/lib/vpc.js:127:31)
	\_ Array.forEach (<anonymous>)
	\_ new VpcNetwork (/foo/bar/deploy/aws-ec2/node_modules/@aws-cdk/aws-ec2/lib/vpc.js:120:33)
	\_ new TokenizedEC2Stack (/foo/bar/deploy/aws-ec2/bin/tokenized.js:15:21)
	\_ Object.<anonymous> (/foo/bar/deploy/aws-ec2/bin/tokenized.js:243:1)
	\_ Module._compile (internal/modules/cjs/loader.js:689:30)
	\_ Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
	\_ Module.load (internal/modules/cjs/loader.js:599:32)
	\_ tryModuleLoad (internal/modules/cjs/loader.js:538:12)
	\_ Function.Module._load (internal/modules/cjs/loader.js:530:3)
	\_ Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
	\_ startup (internal/bootstrap/node.js:279:19)
	\_ bootstrapNodeJSCore (internal/bootstrap/node.js:696:3)

This implies that there isn't good support currently for when NAT gateways is 0 (may need to improve checks around things there), and as best as I could tell skimming the docs, there isn't a great way to use VpcNetwork without a NAT gateway.

Presumably I can use the override methods to 'reach in' and patch those keys manually, probably setting GatewayId/EgressOnlyInternetGatewayId it will probably work, but I was wondering if there is currently a 'better' solution than that when:

  • I have a public subnet that I just want to connect directly to the net (and control connections through security groups, etc)
  • I have a private subnet that only requires egress

It may be that most people using CDK have requirements greater than mine and/or don't mind about the NAT gateway costs, but I feel like someone just playing around may be shockingly surprised at how much $$ the defaults end up costing them. Maybe some doco changes to call this out more explicitly? And/or an example that supports using methods other than the NAT gateway (eg. as mentioned above, or an example that shows how to set it up using the old way with a NAT instance so we can run it on a micro for tiny workloads without costing the world)

@0xdevalias 0xdevalias changed the title Error: Setting natGateways: 0 in ec2.VpcNetwork Error: Setting natGateways: 0 in ec2.VpcNetwork (+huge AWS costs from NAT Gateways) Dec 7, 2018
@rix0rrr rix0rrr added 📚 documentation @aws-cdk/aws-ec2 Related to Amazon Elastic Compute Cloud labels Dec 8, 2018
@rix0rrr
Copy link
Contributor

rix0rrr commented Dec 8, 2018

Thanks for reporting the issue about 0 NAT gateways, that's definitely something we should be looking into.

My initial thought when you said you didn't want any NAT gateways was, you can make the Private subnets Isolated instead. That will not create any NAT gateways, since they won't have internet routing requirements for non-public subnets.

But then you mentioned you DO want egress from a private subnet, but without a NAT gateway.

I'm not an expert on VPCs, but I'm not entirely sure how that would even work. Are you talking about one-way UDP or ICMP packets only? Because surely TCP needs bidirectional packet flows (if only for ACKs) and so you'd need a NAT gateway to map between publicly routable and private IP addresses...?

@rix0rrr rix0rrr added bug This issue is a bug. 📚 documentation and removed 📚 documentation labels Dec 8, 2018
@0xdevalias
Copy link
Contributor Author

0xdevalias commented Dec 8, 2018

My assumption was that it would allow connections originating from the VPC outward (including TCP), but not the other way around. Looking at the doco, that seems not to hold strictly true:

IPv6 addresses are globally unique, and are therefore public by default. If you want your instance to be able to access the Internet, but you want to prevent resources on the Internet from initiating communication with your instance, you can use an egress-only Internet gateway.

In particular, it's only for IPv6, and recommends a NAT gateway for IPv4.

The concept of the NAT gateway is great, the reality of the price is kind of terrible for smaller scale deployments.

The 'old way' of achieving this (at least for me) pre-NAT gateway was to setup a NAT instance:

So I think what would be really nice here, is to have an easy wrapper around creating a NAT instance, and being able to use/specify it as part of the VPC Network, along with associated doco updates to point out the costs and that for smaller deployments that don't require the full power of the NAT gateways, that it might not be the best choice. As a default (and one that will create multiple by default), it's a costly mistake to have to make.

Further reading:

Steps:

  • In the EC2 Launch Instance console, search for amzn-ami-vpc-nat in community instances (or even amzn-ami-vpc-nat-hvm-2018 to be more specific)
    • Right now, this looks to be the latest (at least in ap-southeast-2): amzn-ami-vpc-nat-hvm-2018.03.0.20181116-x86_64-ebs - ami-062c04ec46aecd204
    • It might be cool to have these easily accessible/'look uppable' in the same way Amazon Linux AMI's currently are

The following setup might not be perfectly refined.. but it seems to be a way to implement the NAT instance pattern:

const vpc = new ec2.VpcNetwork(this, 'Tokenized-VPC', {
    natGateways: 0,
    // natGatewayPlacement: {subnetName: 'Public'},
    subnetConfiguration: [
        {
            cidrMask: 26,
            name: 'Public',
            subnetType: ec2.SubnetType.Public,
        },
        {
            name: 'Application',
            subnetType: ec2.SubnetType.Private,
        },
    ],
    defaultInstanceTenancy: ec2.DefaultInstanceTenancy.Default,
});

// Ref: https://awslabs.github.io/aws-cdk/refs/_aws-cdk_aws-ec2.html#@aws-cdk/aws-ec2.SecurityGroup
const natSecurityGroup = new ec2.SecurityGroup(this, 'NATSecurityGroup', {
    vpc,
    groupName: "NATSecurityGroup",
    description: "NAT Instance Security Group",
    allowAllOutbound: true,
});

// TODO: Is this ok?
natSecurityGroup.connections.allowFromAnyIPv4(new ec2.TcpAllPorts());

natSecurityGroup.tags.setTag("Name", natSecurityGroup.path);

if (props.enableSSH) {
    // Allow ssh access
    // Ref: https://awslabs.github.io/aws-cdk/refs/_aws-cdk_aws-ec2.html#allowing-connections
    natSecurityGroup.connections.allowFromAnyIPv4(new ec2.TcpPort(22));
}

// TODO: Add AMI's for other regions here?
// const natImage = new ec2.GenericLinuxImage({
//     "ap-southeast-2": "ami-062c04ec46aecd204", // amzn-ami-vpc-nat-hvm-2018.03.0.20181116-x86_64-ebs
// });

// Ref: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-instance.html
// Ref: https://awslabs.github.io/aws-cdk/refs/_aws-cdk_aws-ec2.html#@aws-cdk/aws-ec2.SecurityGroupProps
const natInstance = new ec2.cloudformation.InstanceResource(this, 'NATInstance', {
    imageId: "ami-062c04ec46aecd204", // ap-southeast-2: amzn-ami-vpc-nat-hvm-2018.03.0.20181116-x86_64-ebs
    instanceType: new ec2.InstanceTypePair(props.ec2NATInstanceClass, props.ec2NATInstanceSize).toString(),
    subnetId: vpc.publicSubnets[0].subnetId,
    securityGroupIds: [natSecurityGroup.securityGroupId],
    sourceDestCheck: false, // Required for NAT
    keyName: "foo-ssh",
});

natInstance.propertyOverrides.tags = [
  {key: "Name", value: `${natInstance.path}`}
];

// Route private subnets through the NAT instance
vpc.privateSubnets.forEach(subnet => {
    const defaultRoute = subnet.findChild('DefaultRoute') as ec2.cloudformation.RouteResource;
    defaultRoute.propertyOverrides.instanceId = natInstance.instanceId;
});

To route other subnet 'layers' with NAT (eg. if you use isolated):

vpc.isolatedSubnets.forEach(subnet => {
    const defaultRoute = subnet.findChild('DefaultRoute') as ec2.cloudformation.RouteResource;
    defaultRoute.propertyOverrides.instanceId = natInstance.instanceId;
});

Later on I have an autoscaling group, and I allow SSH from the NAT gateway:

const appAsg = new autoscaling.AutoScalingGroup(this, 'AppAutoScalingGroup', {
  vpc,
  minSize: 1,
  maxSize: 1,
  desiredCapacity: 1,
  instanceType: new ec2.InstanceTypePair(props.ec2InstanceClass, props.ec2InstanceSize),
  machineImage: amazonLinux2,
  keyName: "foo-ssh",
  vpcPlacement: {
      subnetName: 'Application'
  },
  updateType: autoscaling.UpdateType.ReplacingUpdate,
});

if (props.enableSSH) {
  // Allow ssh access
  // Ref: https://awslabs.github.io/aws-cdk/refs/_aws-cdk_aws-ec2.html#allowing-connections
  appAsg.connections.allowFrom(natSecurityGroup, new ec2.TcpPort(22))
}

const asgLaunchConfig = appAsg.findChild('LaunchConfig') as autoscaling.cloudformation.LaunchConfigurationResource;

asgLaunchConfig.addDependency(natInstance);
asgLaunchConfig.addDependency(appAsg.role);

It would also be nice to have better wrappers around launching single EC2 instances (eg. to setup a bastion host/NAT instance/etc without having to drop to the cloudformation level), that has a similar interface to other higher level wrappers, and auto generates associated security groups/etc.

I guess another alternative for my usecase might be to just put it in public and control access through security groups/etc.

@rix0rrr rix0rrr added the feature-request A feature should be added or improved. label Dec 10, 2018
@rix0rrr rix0rrr changed the title Error: Setting natGateways: 0 in ec2.VpcNetwork (+huge AWS costs from NAT Gateways) VPC: allow configuring NAT instances instead of gateways (and a 0 NAT gateways bug) Dec 10, 2018
@moofish32
Copy link
Contributor

@0xdevalias -- for the cost conscious finding this issue we should also mention cross AZ network fees. If you really want to be least cost and run a single NAT (instance or GW) you should really use only one AZ. Depending on your data usage and instance sizes the cost implications will shift as well.

@eladb eladb removed the bug This issue is a bug. label Dec 17, 2018
@rix0rrr rix0rrr added the gap label Jan 4, 2019
@Doug-AWS Doug-AWS added the docs/generated Related to the generated API Reference documentation label Jan 23, 2019
@sallar
Copy link

sallar commented Feb 4, 2019

Hi @0xdevalias

I tried following your example, but it doesn't work in the newest version of the cdk clients. I did some changes to make the typing work as follows:

    const vpc = new VpcNetwork(stack, "MyApp", {
        natGateways: 0
    });

    const natSecurityGroup = new SecurityGroup(stack, "NATSecurityGroup", {
        vpc,
        groupName: "NATSecurityGroup",
        description: "NAT Instance Security Group",
        allowAllOutbound: true
    });

    natSecurityGroup.tags.setTag("Name", natSecurityGroup.groupName);
    natSecurityGroup.connections.allowFromAnyIPv4(new TcpAllPorts());

    const natInstance = new CfnInstance(stack, "NATInstance", {
        imageId: "ami-d03288a3",
        instanceType: new InstanceTypePair(
            InstanceClass.T2,
            InstanceSize.None
        ).toString(),
        subnetId: vpc.publicSubnets[0].subnetId,
        securityGroupIds: [natSecurityGroup.securityGroupId],
        sourceDestCheck: false, // Required for NAT
        keyName: "myapp-ssh"
    });

    natInstance.propertyOverrides.tags = [
        { key: "Name", value: `${natInstance.stackPath}` }
    ];

    vpc.privateSubnets.forEach(subnet => {
        const defaultRoute = subnet.node.findChild("DefaultRoute") as CfnRoute;
        defaultRoute.propertyOverrides.instanceId = natInstance.instanceId;
    });

This yields the following error:

MyAppInfrastructureStack failed: ValidationError:
Circular dependency between resources: [NATSecurityGroupDB004F3B, MyAppVPCProductionPrivateSubnet2DefaultRoute2D986C86, NATInstance, MyAppPrimaryDBInstance2DD5D4CDB, MyAppVPCProductionPrivateSubnet3DefaultRoute710E04C8, MyAppPrimaryDBInstance171D1C604, MyAppVPCProductionPrivateSubnet1DefaultRouteC69EC64E]
Circular dependency between resources: [NATSecurityGroupDB004F3B, MyAppVPCProductionPrivateSubnet2DefaultRoute2D986C86, NATInstance, MyAppPrimaryDBInstance2DD5D4CDB, MyAppVPCProductionPrivateSubnet3DefaultRoute710E04C8, MyAppPrimaryDBInstance171D1C604, MyAppVPCProductionPrivateSubnet1DefaultRouteC69EC64E]

Any suggestions to how should I remove the default NAT Gateways and instead use my own NAT instance here? The default configuration is too expensive for us.

We are trying to deploy a ECS stack (available via an ALB) which doesn't have much use for so many NAT Gateways.

Thanks a lot

@moofish32
Copy link
Contributor

Hi @sallar -

I'm curious if you actually have a requirement for a NAT GW? If you don't, then you can accomplish this using only public subnets and not use a NAT GW. For example, if I had a very small deployment and I was hyper cost conscious and willing to trade some security layers I would just use public subnets and the Internet Gateway while controlling instance access via security groups. I might use a property override to disable the map public IP on launch as well (depending on some other requirements). You will have given up some ability with NACLs, but I don't see many people that need NACLs versus just security group control levels. If you do have an RDS need, remember that you can use an isolated subnet for RDS which will not require NAT GW either. Many AWS services also support endpoints that can be used to avoid egress for those as well.

Just another thought on how to reduce this cost and keep most of the common security features. Full disclosure I am not an AWS engineer.

@chrnorm
Copy link

chrnorm commented Jun 14, 2019

Based on @sallar's snippet - this stack is working for me on AWS CDK 0.33. It creates a VPC with a t2.nano NAT instance without SSH access. I grabbed the instance ID from this page and looked up my corresponding AWS deployment region for the 'HVM (NAT) EBS-Backed 64-bit' instance.

import cdk = require("@aws-cdk/cdk");
import ec2 = require("@aws-cdk/aws-ec2");

export class VpcStack extends cdk.Stack {
  public readonly vpc: ec2.Vpc;

  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    this.vpc = new ec2.Vpc(this, "VPC", { natGateways: 0 });
    const natSecurityGroup = new ec2.SecurityGroup(this, "NATSecurityGroup", {
      vpc: this.vpc,
      groupName: "NATSecurityGroup",
      description: "NAT Instance Security Group",
      allowAllOutbound: true
    });

    natSecurityGroup.connections.allowFromAnyIPv4(new ec2.TcpAllPorts());

    const natInstance = new ec2.CfnInstance(this, "NATInstance", {
      imageId: "ami-00c1445796bc0a29f",
      instanceType: new ec2.InstanceTypePair(
        ec2.InstanceClass.T2,
        ec2.InstanceSize.Nano
      ).toString(),
      subnetId: this.vpc.publicSubnets[0].subnetId,
      securityGroupIds: [natSecurityGroup.securityGroupId],
      sourceDestCheck: false // Required for NAT
    });

    natInstance.addPropertyOverride("Name", natInstance.stackPath);

    this.vpc.privateSubnets.forEach(subnet => {
      const defaultRoute = subnet.node.findChild(
        "DefaultRoute"
      ) as ec2.CfnRoute;
      defaultRoute.addPropertyOverride("instanceId", natInstance.instanceId);
    });
  }
}

@Pwntus
Copy link

Pwntus commented Jul 24, 2019

It really seems to be something broken with the VPC construct. I need a VPC with a single public subnet and no NAT Gateway.

This is my setup:

    const vpc = new ec2.Vpc(this, 'VPC', {
      maxAzs: 1,
      subnetConfiguration: [{
        cidrMask: 24,
        name: 'PublicSubnet',
        subnetType: ec2.SubnetType.PUBLIC
      }]
    });
    const cluster = new ecs.Cluster(this, 'Cluster', { vpc });

    // Create a scheduled Fargate task
    const collectorTask = new ecs_patterns.ScheduledFargateTask(this, 'FargateTask', {
      cluster,
      image: ecs.ContainerImage.fromRegistry(imageUri),
      schedule: events.Schedule.expression(scheduleExpression)
    });

When the ScheduledFargateTask is created I get the following error:

There are no 'Private' subnets in this VPC. Use a different VPC subnet selection.

@skinny85
Copy link
Contributor

skinny85 commented Jul 24, 2019

@Pwntus I think this might actually be a gap in the ScheduledFargateTask construct.

It's lacking a subnetSelection: ec2.SubnetSelection property. if you had that, you could say subnetSelection: { subnetType: ec2.SubnetType.PUBLIC } when creating it (right now, it's trying to bind to the private subnets of your VPC, which is the default behavior).

@0xdabbad00
Copy link

I have this same problem. I'm trying to deploy a CDK app to spin up an ECS on a nightly job and my biggest cost is going to be the NAT Gateway that I don't need to be using.

@0xdabbad00
Copy link

If it's helpful, my setup is slightly different although it accomplishes much of the same thing. I do:

  // Create network to run everything in
    const vpc = new ec2.Vpc(this, 'CloudMapperVpc', {
      maxAzs: 2,
      natGateways: 1
    });

    const cluster = new ecs.Cluster(this, 'Cluster', { vpc });

    // Define the container application
    const taskDefinition = new ecs.FargateTaskDefinition(this, 'taskDefinition', {});

    taskDefinition.addContainer('cloudmapper-container', {
      image: ecs.ContainerImage.fromAsset('./resources'),
      memoryLimitMiB: 512,
      cpu: 256,
      environment: {
        S3_BUCKET: config['s3_bucket']
      },
      logging: new ecs.AwsLogDriver({
        streamPrefix: 'cloudmapper',
        logRetention: logs.RetentionDays.TWO_WEEKS
      })
    });

   // Create rule to trigger this be run every 24 hours
    new events.Rule(this, "scheduled_run", {
      ruleName: "cloudmapper_scheduler",
      // Run at 2am EST (6am UTC) every night
      schedule: events.Schedule.expression("cron(0 6 * * ? *)"),
      description: "Starts the CloudMapper auditing task every night",
      targets: [new targets.EcsTask({cluster: cluster, taskDefinition: taskDefinition})]
    });

@SoManyHs SoManyHs added the @aws-cdk/aws-ecs-patterns Related to ecs-patterns library label Aug 19, 2019
@jeshan
Copy link

jeshan commented Aug 27, 2019

@0xdabbad00 @Pwntus @sallar

I'm pretty much in the same boat as you and I figured out how to get rid of the nat gateways. The trick is to put a cloudformation condition on the AWS::EC2::Route resource that references the gateway and force it to evaluate to false.
You can do this when nat gateway count = 0.
Here it is in Python syntax:

from aws_cdk.aws_ec2 import Vpc, CfnRoute
from aws_cdk.core import CfnCondition, Fn

vpc = Vpc(self, 'vpc', nat_gateways=0)
exclude_condition = CfnCondition(
    self, 'exclude-default-route-subnet', expression=Fn.condition_equals(True, False)
)
for subnet in vpc.private_subnets:
    for child in subnet.node.children:
        if type(child) == CfnRoute:
            route: CfnRoute = child
            route.cfn_options.condition = exclude_condition # key point here

@rix0rrr I think the above is a pretty reasonable workaround for this issue.

@0xdabbad00
Copy link

I was able to use @jeshan's trick in order to create a VPC without a NAT gateway, but am now unable to create my ECS in a public subnet. It currently runs in a private subnet with no NAT gateway, which means it can't talk to anything. I'm still working on this.

For those using javascript, @jeshan's code works as:

const vpc = new ec2.Vpc(this, 'CloudMapperVpc', {
        maxAzs: 2,
        natGateways: 0
    });

    var exclude_condition = new cdk.CfnCondition(this,
      'exclude-default-route-subnet',
      {
        // Checks if true == false, so this always fails
        expression: cdk.Fn.conditionEquals(true, false)
      }
    );

    for (var subnet of vpc.privateSubnets) {
        for (var child of subnet.node.children) {
            if (child.constructor.name==="CfnRoute") {
              child.cfnOptions.condition = exclude_condition
            }
        }
    }

@0xdabbad00
Copy link

I was able to get the task to run in a public subnet. The trick is that configuration exists where you have the task configured to run, so in my case, it is done from a CloudWatch Event Rule, so the CDK code is:

    // Create rule to trigger this manually
    new events.Rule(this, "manual_run", {
      ruleName: "cloudmapper_manual_run",
      eventPattern: {source: ['cloudmapper']},
      description: "Allows CloudMapper auditing to be manually started",
      targets: [new targets.EcsTask({
        cluster: cluster,
        taskDefinition: taskDefinition,
        subnetSelection: {subnetType: ec2.SubnetType.PUBLIC}
      })]
    });

@rix0rrr rix0rrr added the p0 label Sep 17, 2019
@0xdabbad00
Copy link

@hughevans
Copy link

Thanks, additionally I needed to give my Fargate task a public IP:

const cluster = new ecs.Cluster(this, "Cluster", { vpc });

const fargateTask = new sfn.Task(this, "Run Fargate Task", {
  task: new sfnTasks.RunEcsFargateTask({
    assignPublicIp: true,
    cluster,
    containerOverrides: [
      
    ],
    integrationPattern: sfn.ServiceIntegrationPattern.SYNC,
    taskDefinition,
    subnets: { subnetType: ec2.SubnetType.PUBLIC }
  }),
  resultPath: "$.result"
});

@hoang-innomize
Copy link

I am facing an issue when creating a new VPC is the private subnet is required. Applied @0xdabbad00's suggestion but it still doesn't work

Here is the error I am getting

If you do not want NAT gateways (natGateways=0), make sure you don't configure any PRIVATE subnets in 'subnetConfiguration' (make them PUBLIC or ISOLATED instead)

Here is the code to construct the Vpc

/**
   * Create VPC and Fargate Cluster
   * @param props
   */
  private constructVpc(props: StackProps) {
    // Create VPC to run everything in, but without a NAT gateway.
    // We want to run in a public subnet, but the CDK creates a private subnet
    // by default, which results in the use of a NAT gateway, which costs $30/mo.
    // To avoid that unnecessary charge, we have to create the VPC in a complicated
    // way.
    // This trick was figured out by jeshan in https://github.com/aws/aws-cdk/issues/1305#issuecomment-525474540
    // Normally, the CDK does not allow this because the private subnets have to have
    // a route out, and you can't get rid of the private subnets.
    // So the trick is to remove the routes out.
    // The private subnets remain, but are not usable and have no costs.
    const vpc = new ec2.Vpc(this, 'TradeInCrawlerVpc', {
      maxAzs: 2,
      natGateways: 0
    });

    // Create a condition that will always fail.
    // We will use this in a moment to remove the routes.
    var exclude_condition = new cdk.CfnCondition(
      this,
      'exclude-default-route-subnet',
      {
        // Checks if true == false, so this always fails
        expression: cdk.Fn.conditionEquals(true, false)
      }
    );

    // For the private subnets, add a CloudFormation condition to the routes
    // to cause them to not be created.
    for (const subnet of vpc.privateSubnets) {
      for (const child of subnet.node.children) {
        if (child.constructor.name === 'CfnRoute') {
          (child as CfnRoute).cfnOptions.condition = exclude_condition;
        }
      }
    }

    return vpc;
  }

Does anyone know how to fix this issue?

@jeshan
Copy link

jeshan commented Oct 29, 2019

Don't know if release 1.15 may fix the issue,
It has a bug fix a bit related to this:

https://github.com/aws/aws-cdk/releases/tag/v1.15.0

vpc: additional validation around Subnet Types (#4668) (9a96c37), closes #3704

@hoang-innomize
Copy link

Yeah, I am using latest version 1.15.0

@hoang-innomize
Copy link

Adding subnetConfiguration solve my issue now

new ec2.Vpc(this, 'FargateVPC', {
      cidr: props.cidr || '10.0.0.0/16',
      maxAzs: 1,
      natGateways: 0,
      subnetConfiguration: [
        {
          cidrMask: 24,
          name: 'Public',
          subnetType: ec2.SubnetType.PUBLIC
        }
      ]
    });

@travtarr
Copy link

This seems to be possible with a recent commit #4898 in version 1.16.0 thanks to rix0rrr.

new Vpc(this, `vpc`, {
  cidr: '10.40.0.0/16',
  maxAzs: 2,
  natGateways: 2,
  natGatewayProvider: NatProvider.instance({
    instanceType: InstanceType.of(InstanceClass.T3A, InstanceSize.NANO),
  }),
  gatewayEndpoints: {
    s3: { service: GatewayVpcEndpointAwsService.S3 },
  },
});

@0xdabbad00
Copy link

0xdabbad00 commented Nov 17, 2019

Looks like there are two issues being discussed in this ticket.

  1. Be able to use a NAT instance instead of a NAT Gateway which looks to be solved based on the comment from @travtarr
  2. Be able to not use a NAT instance or NAT gateway, which sounds like it might have been solved based on the comments from @jeshan and @hoang-innomizetech

@rix0rrr
Copy link
Contributor

rix0rrr commented Nov 27, 2019

Be able to use a NAT instance instead of a NAT Gateway

This is definitely resolved.

Be able to not use a NAT instance or NAT gateway

This is also resolved, but you must not request any private subnets (by definition, you must have at least one NAT traversal mechanism for private subnets). You should be creating isolated subnets instead.

Closing this issue. Please reopen if I missed something.

@rix0rrr rix0rrr closed this as completed Nov 27, 2019
@ptitjes
Copy link

ptitjes commented Feb 12, 2020

Hi all. @rix0rrr, I believe the 0 NAT gateway issue is not fixed.

If I do

    const vpc = new Vpc(this, 'Vpc', {
      cidr: '10.0.0.0/16',
      maxAzs: 1,
      natGateways: 0,
    });

I get the following CDK error:

If you do not want NAT gateways (natGateways=0), make sure you don't configure any PRIVATE subnets in 'subnetConfiguration' (make them PUBLIC or ISOLATED instead)

And if I do

    const vpc = new Vpc(this, 'Vpc', {
      cidr: '10.0.0.0/16',
      maxAzs: 1,
      natGateways: 0,
      subnetConfiguration: [
        {
          cidrMask: 24,
          name: 'Public',
          subnetType: SubnetType.PUBLIC,
        }
      ],
    });

I get the following CDK error:

There are no 'Private' subnet groups in this VPC. Available types: Public

What am I missing ?

@harrycollin
Copy link

I seem to be having the same problem.

@adonig
Copy link

adonig commented Dec 15, 2022

I wasn´t able to completely get rid of the private subnets, but this code synthesized to a template without NAT gateways:

        vpc = ec2.Vpc(self, "VPC",
            ip_addresses=ec2.IpAddresses.cidr('10.0.0.0/16'),
            nat_gateways=0,
            subnet_configuration=[
                ec2.SubnetConfiguration(
                    name='Public',
                    cidr_mask=24,
                    subnet_type=ec2.SubnetType.PUBLIC,
                ),
                ec2.SubnetConfiguration(
                    name='Private',
                    cidr_mask=24,
                    subnet_type=ec2.SubnetType.PRIVATE_WITH_EGRESS,
                ),
            ],
        )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-ec2 Related to Amazon Elastic Compute Cloud @aws-cdk/aws-ecs-patterns Related to ecs-patterns library docs/generated Related to the generated API Reference documentation feature-request A feature should be added or improved. p0
Projects
None yet
Development

No branches or pull requests