Use an existing vpc, generate subnets (WIP) #379
Conversation
|
@ozzieba thanks a lot for working on this! I'll review this in once it's ready from your point of view. Also @mumoshu might like to look at this also. In the meantime, just a heads-up - I'm about to land #381 very soon (potentially today), so you will need to rebase. Also, while working on that PR I just opened #383, in case you might want to tackle that in the process, but not to worry. |
|
@ozzieba Hey! Thanks for your efforts! I think this is useful and the approach makes sense. But I want to discuss a bit - mostly regarding the user-case and user-support. Basically, I want to make it clear when to use this feature, and if it's good to have this as a feature, or just include in the documentation. In my experience maintaining another tool called kube-aws, I saw that the more nobs we add, the more users confuse and the more time is taken to trouble-shoot varying use-cases and setups. I'd say that we usually use VPCs and subnets created and configured as we like, manually or via terraform or whatever tool. The user should harmonize the existing vpc, subnets, network ACLs, route tables, internet gateways, NAT gateways and so on. Supporting each of these sounds hard, both for users and maintainers. I believe we should start by adding a catch-all solution to the problem of reusing existing AWS resources. eksctl should be explicitly documented to say its the user's responsibility to properly configure the existing resources(btw I think this is already documented like that), and add more extension points if eksctl's catch-all solution is insufficient. For example, in case your use-case is to reduce the time taken to recreate clusters, can we just provide a reference terraform project to provision vpcs and subnets? We can then add a dedicated section in the doc to use it in combination with |
|
@errordeveloper What do you think? I think including this as a feature definitely makes U/X very good if well maintained. But my experience says the maintenance may be hard. A possible work-around is providing a reference cfn template or terraform project paired with the documentation. I'm not yet sure if it really solves @ozzieba's use-case though! |
|
One more workaround would be to allow eksctl to create vpc and subnets and relevant aws resources only. So that those resources can be reused by later eksctl calls. This covers more use-cases while avoiding to maintain both eksctl and the reference terraform project. |
|
@mumoshu Thanks for looking over this! My basic thinking is that eksctl is meant to allow users to bring up an EKS cluster easily from scratch, without having to worry about details like VPC, subnets, routing, ACLs, etc, unless/until they have a special need for customization. Unfortunately in my particular case I do need to use an existing VPC, so I can mount an existing EFS volume from a GPU instance (see here). I imagine there are other reasons people may want to reuse VPCs, while having everything else auto-generated from scratch. It seems to me this shouldn't add too much of a maintenance burden, since the interface to existing resources (VPC and IGW) is still fairly small. Obviously, I could simply create the subnets etc manually, or use Terraform / Cloudformation; but I think explicitly letting eksctl manage them is likely to be less error-prone and cleaner. In particular, I'd expect that having users manage their own Terraform / Cloudformation templates would result in more issues, like users forgetting to add an InternetGateway or correct routing. Indeed, this is a significant reason to use eksctl in the first place, rather than the AWS-provided Cloudformation scripts. I also expect this somewhat reduces the likelihood of users adding unrelated resources to subnets that should be reserved for EKS. Likewise, letting eksctl clean up unneeded resources when deleting a cluster is convenient. To further reduce errors, eksctl could attempt to check for configurations that won't work, eg missing routing, subnets that already contain other resources, etc, and warn the user before creating the cluster. In extreme cases, eksctl could delete the cloudformation stack while retaining resources that are still needed. |
|
@ozzieba have you tried rebasing yet? |
|
Sorry, it's been a busy week, I hope to get to this in the next few days |
|
I've done the rebase, unfortunately I don't have time today to get testing and docs done properly, will try during the week. |
|
@ozzieba I'm gonna have to close this, if you want to come back to it - please go ahead and open a new PR! :) |
Add standard volume type
Description
Added support for providing --vpc-id without providing subnets, ie the fourth item in #306. Subnets will be auto-generated as if this is a new VPC. To do this, I had to move things around a bit, to cleanly separate management of the subnets and routing from the VPC itself. Also, since only one InternetGateway can be attached to a VPC at once, if one exists for the given VPC it must also be passed to eksctl so it will be used.
Tested with:
If the approach makes sense, I can polish the various edges, eg get the CIDR from the VPC, add handling for default VPC, add test cases and documentation etc. But first let me know if this looks useful.
Checklist
make build)make test)humans.txtfile