instance_mapper

Cloud Instance Mapper

This tool creates a score-based mapping of different IaaS VM instance types from EC2 and Azure Compute to Google Cloud Platform instance types. You can also map AWS RDS instances to Google Cloud SQL database types.

When set up correctly with authentication, it will retrieve the latest instance type details from each of the cloud providers's APIs.

Building

• Requires Golang 1.16+

You can build it by running:

go install github.com/GoogleCloudPlatform/professional-services/tools/instance_mapper/cmd/instance_mapper

Configuring

AWS IAM role

Attach the following policies to the desired role: AmazonEC2ReadOnlyAccess and AmazonRDSReadOnlyAccess. Assign a policy for the user or group that allows sts:AssumeRole to the role you created.

Example of configuring the AWS IAM role using aws cli tool:

# aws iam create-user --user-name InstanceMapper
{
    "User": {
        "Path": "/",
        "UserName": "InstanceMapper",
        "UserId": "A123451234512345Y",
        "Arn": "arn:aws:iam::1234567890:user/InstanceMapper"
    }
}
# cat <<EOF >assume-role.json 
{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Principal": { "AWS": "arn:aws:iam::1234567890:user/InstanceMapper" },
          "Action": "sts:AssumeRole"
      }
  ]
}
EOF
# aws iam create-role --role-name Instance-Mapper-Readonly --assume-role-policy-document file://assume-role.json
{
    "Role": {
        "Path": "/",
        "RoleName": "Instance-Mapper-Readonly",
        "RoleId": "A1234512345612345X",
        "Arn": "arn:aws:iam::1234567890:role/Instance-Mapper-Readonly",
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Principal": {
                        "AWS": "arn:aws:iam::1234567890:user/InstanceMapper"
                    },
                    "Action": "sts:AssumeRole"
                }
            ]
        }
    }
}
# aws iam attach-role-policy --role-name Instance-Mapper-Readonly --policy-arn "arn:aws:iam::aws:policy/AmazonRDSReadOnlyAccess"
# aws iam attach-role-policy --role-name Instance-Mapper-Readonly --policy-arn "arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess"
# aws iam create-access-key --user-name InstanceMapper
{
    "AccessKey": {
        "UserName": "InstanceMapper",
        "AccessKeyId": "A123456123456789U",
        "Status": "Active",
        "SecretAccessKey": "This-Is-Your-Secret-Access-Key"
    }
}
# export AWS_ACCESS_KEY_ID="A123456123456789U"
# export AWS_SECRET_ACCESS_KEY="This-Is-Your-Secret-Access-Key"

Running

To get the available command line flags, run:

instance_mapper -help

Example:

instance_mapper 
  -azure-vm \
  -azure-subscription-id 5ed066d4-803b-4d57-bf9e-5a64e5fe3f05 \
  -aws-ec2 \
  -aws-role arn:aws:iam::1234567890:role/Instance-Mapper-Readonly \
  -gcp-project my-project-id > instances.csv

The tool outputs a CSV formatted file on standard out, that looks something like this:

Instance type	Memory	vCPUs	GPUs	GPU type	Total GPU memory	Instance type	Memory	vCPUs	GPUs	GPU type	Total GPU memory	Instance type	Memory	vCPUs	GPUs	GPU type	Total GPU memory	Instance type	Memory	vCPUs	GPUs	GPU type	Total GPU memory
c5d.12xlarge	96.00 GB	48	0		0	n2d-highcpu-96	96.00 GB	96	0		0	n2d-standard-48	192.00 GB	48	0		0	n2d-highmem-48	384.00 GB	48	0		0
c4.large	3.75 GB	2	0		0	e2-medium	4.00 GB	2	0		0	e2-standard-2	8.00 GB	2	0		0	e2-small	2.00 GB	2	0		0
r4.xlarge	30.50 GB	4	0		0	e2-highmem-4	32.00 GB	4	0		0	n2d-highmem-4	32.00 GB	4	0		0	n2-highmem-4	32.00 GB	4	0		0
c6gd.2xlarge	16.00 GB	8	0		0	e2-standard-4	16.00 GB	4	0		0	e2-standard-8	32.00 GB	8	0		0	e2-highcpu-16	16.00 GB	16	0		0
Standard_D1	3.50 GB	1	0		0	n1-standard-1	3.75 GB	1	0		0	t2d-standard-1	4.00 GB	1	0		0	n1-highcpu-4	3.60 GB	4	0		0
Standard_E80is_v4	504.00 GB	80	0		0	n2d-highmem-64	512.00 GB	64	0		0	n2-highmem-64	512.00 GB	64	0		0	n2d-highmem-80	640.00 GB	80	0		0
Standard_M64s_v2	1024.00 GB	64	0		0	m1-ultramem-40	961.00 GB	40	0		0	n2d-highcpu-64	64.00 GB	64	0		0	n2d-standard-64	256.00 GB	64	0		0
Standard_ND40s_v3	672.00 GB	40	8		0	n2d-highmem-80	640.00 GB	80	0		0
Standard_D16s_v3	64.00 GB	16	0		0	e2-standard-16	64.00 GB	16	0		0	n2d-standard-16	64.00 GB	16	0		0	n2-standard-16	64.00 GB	16	0		0

Mapping files

There are two mapping files: gpu-mapping.yaml (for mapping GPU types between cloud providers) and sql-mapping.yaml (for mapping SQL engine types and versions).

Adding pricing information

You can use the public pricelist.json to augment the output with price estimates for the GCP instance types. Download the file and pass the following flags to instance mapper: -gcp-price-list pricelist.json -gcp-price-list-region europe-west4

Customizing matcher

To show the default CEL matchers, run:

instance_mapper -show-cel

You can copy the matchers as basis for your custom files to two files (one for compute instances, one for SQL instances) and specify them using -custom-matcher and -custom-matcher-sql flags.

Limitations

• Azure API does not provide: a GPU type description, shared tenancy support, bare metal, CPU clockspeed