Attach EFA SG to network interfaces

[aclevername]

Description

Add the EFA sg to the network cards in the launch template for managed & unmanaged nodes.

Before

 "LaunchTemplate": {
      "Type": "AWS::EC2::LaunchTemplate",
      "Properties": {
        "LaunchTemplateData": {
          "NetworkInterfaces": [
            {
              "DeviceIndex": 0,
              "Groups": [
                {
                  "Fn::ImportValue": "eksctl-jk-efa-cluster::ClusterSecurityGroupId"
                }
              ],
              "InterfaceType": "efa",
              "NetworkCardIndex": 0
            },
            {
              "DeviceIndex": 1,
              "Groups": [
                {
                  "Fn::ImportValue": "eksctl-jk-efa-cluster::ClusterSecurityGroupId"
                }
              ],
              "InterfaceType": "efa",
              "NetworkCardIndex": 1
            },
            {
              "DeviceIndex": 2,
              "Groups": [
                {
                  "Fn::ImportValue": "eksctl-jk-efa-cluster::ClusterSecurityGroupId"
                }
              ],
              "InterfaceType": "efa",
              "NetworkCardIndex": 2
            },
            {
              "DeviceIndex": 3,
              "Groups": [
                {
                  "Fn::ImportValue": "eksctl-jk-efa-cluster::ClusterSecurityGroupId"
                }
              ],
              "InterfaceType": "efa",
              "NetworkCardIndex": 3
            }
          ],

After

"LaunchTemplate": {
      "Type": "AWS::EC2::LaunchTemplate",
      "Properties": {
        "LaunchTemplateData": {
          "NetworkInterfaces": [
            {
              "DeviceIndex": 0,
              "Groups": [
                {
                  "Fn::ImportValue": "eksctl-jk-efa-cluster::ClusterSecurityGroupId"
                },
                {
                  "Ref": "EFASG"
                }
              ],
              "InterfaceType": "efa",
              "NetworkCardIndex": 0
            },
            {
              "DeviceIndex": 1,
              "Groups": [
                {
                  "Fn::ImportValue": "eksctl-jk-efa-cluster::ClusterSecurityGroupId"
                },
                {
                  "Ref": "EFASG"
                }
              ],
              "InterfaceType": "efa",
              "NetworkCardIndex": 1
            },
            {
              "DeviceIndex": 2,
              "Groups": [
                {
                  "Fn::ImportValue": "eksctl-jk-efa-cluster::ClusterSecurityGroupId"
                },
                {
                  "Ref": "EFASG"
                }
              ],
              "InterfaceType": "efa",
              "NetworkCardIndex": 2
            },
            {
              "DeviceIndex": 3,
              "Groups": [
                {
                  "Fn::ImportValue": "eksctl-jk-efa-cluster::ClusterSecurityGroupId"
                },
                {
                  "Ref": "EFASG"
                }
              ],
              "InterfaceType": "efa",
              "NetworkCardIndex": 3
            }
          ],

Checklist

• Added tests that cover your change (if possible)
• Added/modified documentation as required (such as the README.md, or the userdocs directory)
• Manually tested
• Made sure the title of the PR is a good description that can go into the release notes
• (Core team) Added labels for change area (e.g. area/nodegroup) and kind (e.g. kind/improvement)

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

• Backfilled missing tests for code in same general area 🎉
• Refactored something and made the world a better place 🌟

[jyotimahapatra]

In addition to adding the security group, the efa installers also have to be added to launch template user data. Can we add the following userdata to the launch template

        UserData: !Base64
          "Fn::Sub": |
            Content-Type: multipart/mixed; boundary="==BOUNDARY=="
            MIME-Version: 1.0

            --==BOUNDARY==
            Content-Type: text/cloud-boothook; charset="us-ascii"
            cloud-init-per once yum_wget yum install -y wget
            cloud-init-per once wget_efa wget -q --timeout=20 https://s3-us-west-2.amazonaws.com/aws-efa-installer/aws-efa-installer-latest.tar.gz -O /tmp/aws-efa-installer-latest.tar.gz

            cloud-init-per once tar_efa tar -xf /tmp/aws-efa-installer-latest.tar.gz -C /tmp
            pushd /tmp/aws-efa-installer
            cloud-init-per once install_efa ./efa_installer.sh -y -g
            pop /tmp/aws-efa-installer

            cloud-init-per once efa_info /opt/amazon/efa/bin/fi_info -p efa

            --==BOUNDARY==
            Content-Type: text/x-shellscript; charset="us-ascii"

            #!/bin/bash
            set -o xtrace
            EKS_ENDPOINT=${EksEndpoint}
            if [[ ! -z "$EKS_ENDPOINT" ]]; then
                    EKS_ENDPOINT_ARG="--endpoint=$EKS_ENDPOINT"
                    aws eks describe-cluster \
                              --region=${AWS::Region} \
                              $EKS_ENDPOINT_ARG \
                              --name=${ClusterName} \
                              --query 'cluster.{certificateAuthorityData: certificateAuthority.data, endpoint: endpoint}' > /tmp/describe_cluster_result.json
                    CA=$(cat /tmp/describe_cluster_result.json | grep certificateAuthorityData | awk '{print $2}' | sed 's/[,\"]//g')
                    APISERVER_ENDPOINT=$(cat /tmp/describe_cluster_result.json | grep endpoint | awk '{print $2}' | sed 's/[,\"]//g')
                    EKS_OPTIONAL_BOOTSTRAP_ARGS="--b64-cluster-ca $CA  --apiserver-endpoint $APISERVER_ENDPOINT"
            fi
            /etc/eks/bootstrap.sh ${ClusterName} ${BootstrapArguments} $EKS_OPTIONAL_BOOTSTRAP_ARGS
            /opt/aws/bin/cfn-signal --exit-code $? \
                     --stack  ${AWS::StackName} \
                     --resource NodeGroup  \
                     --region ${AWS::Region}
            --==BOUNDARY==--\

[Callisto13]

@jyotimahapatra I can see that we are already adding things to the UserData here and here, are they not appearing in your template when you enable EFA with eksctl?

[jyotimahapatra]

@jyotimahapatra I can see that we are already adding things to the UserData here and here, are they not appearing in your template when you enable EFA with eksctl?

I was only looking at the code before making the comment. I will check the launch template to verify. I will holler if I dont find it. You can disregard my comment.

[Callisto13]

can I ask why we are doing a api.IsEnabled here and passing the value into the makeLaunchTemplateData method, when the method already has some api.IsEnabled going on?

[aclevername]

good spot, moved it.

[Callisto13]

tidy