Support DynamoDB autoscaling in table manager

[awh]

As an alternative to #318 & #464, cortex could rely on the the new built-in DynamoDB autoscaling:

https://aws.amazon.com/about-aws/whats-new/2017/06/announcing-amazon-dynamodb-auto-scaling/

In practice this means a configuration option to table manager that prevents it from updating provisioned throughput whilst continuing to create new periodic tables.

[awh]

It looks like you can configure autoscaling for read and write independently, so thinking about it some more, when table manager has autoscaling enabled it should

• Configure read autoscaling on historic tables, but continue setting write IOPS manually to 1
• Configure read and write autoscaling on in-use tables

[tomwilkie]

Autoscaling to reads is unlikely to be that effective as reads are super unpredictable, especially for older tables. I would just stick to autoscaling writes on the head table for now.

[aaron7]

Info: http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/AutoScaling.CLI.html
Autoscaling API: http://docs.aws.amazon.com/sdk-for-go/api/service/applicationautoscaling/

Plan:

• Create a new service role and policy for application auto scaling (via terraform). The ARN will be passed to dynamodb.autoscaling.role-arn
• The table manager creates the table with the existing default throughput write-throughput and read-throughput
• We could have the following configuration to allow different target utilisation for the index and chunk tables:
  dynamodb.periodic-table.write-scale-out-cooldown, dynamodb.periodic-table.write-scale-in-cooldown, dynamodb.periodic-table.write-scale-target-value
dynamodb.chunk-table.write-scale-out-cooldown, dynamodb.chunk-table.write-scale-in-cooldown, dynamodb.chunk-table.write-scale-target-value
• The table manager uses the role ARN and the applicationautoscaling API to register the new table writes to auto scale and set the write scaling policy with the above config.
• The table manager unsets the scaling policy for the old table and deregisters the table for autoscaling (scaling down to 1)

[tomwilkie]

Would be good to get #463 merged before these changes, as they will conflict.