Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add launchtype check before setting platform version #1041

Merged
merged 9 commits into from
May 27, 2020

Conversation

bvtujo
Copy link
Contributor

@bvtujo bvtujo commented May 27, 2020

Checks the launchtype before setting platform version. This reverts a breaking change in compose service up introduced in 1.19.0.

Addresses issue #1040


Enter [N/A] in the box, if an item is not applicable to your change.

Testing

  • [ x] Unit tests passed
  • [... ] Integration tests passed
  • Listed manual checks and their outputs in the comments (example)
  • Link to issue or PR for the integration tests:

Documentation

  • Contacted our doc writer
  • Updated our README

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Copy link
Contributor

@SoManyHs SoManyHs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good -- can we include manual test out put plz?

log.Warn("Learn more: https://aws.amazon.com/blogs/containers/aws-fargate-launches-platform-version-1-4/")
platformVersion = aws.String("1.4.0")
var platformVersion *string
if launchType == "FARGATE" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add a unit test (perhaps add assertions on existing EC2 service tests) which ensure that platform version is not set when launchType != FARGATE?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might be nice to have a separate unit test, like TestEFSWithEC2. We can do this by setting the launchtype in the CommandConfig within the createServiceTest helper:

Copy link
Contributor

@SoManyHs SoManyHs May 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eg:

func TestCreateWithEFS_EC2(t *testing.T) {
	flagSet := flag.NewFlagSet("ecs-cli-up", 0)

	createServiceTest(
		t,
		flagSet,
		&config.CommandConfig{LaunchType: config.LaunchTypeEC2},
		ecsParamsWithFargateEFSVolume(),
		func(input *ecs.CreateServiceInput) {
			launchType := input.LaunchType
			assert.Equal(t, config.LaunchTypeEC2, aws.StringValue(launchType), "launch type is not ec2")
			platformVersion := input.PlatformVersion

			assert.Nil(t, platformVersion)
		},
		ecsSettingDisabled,
	)
}

@bvtujo
Copy link
Contributor Author

bvtujo commented May 27, 2020

manual output for launching ec2 service without EFS via compose service up
docker-compose.yml

version: '3'
services:
  web:
    image: acct.dkr.ecr.us-west-2.amazonaws.com/otter:latest
    ports:
      - "80:80"
    logging:
      driver: awslogs
      options:
        awslogs-group: otter
        awslogs-region: us-west-2
        awslogs-stream-prefix: web

ecs-params.yml

task_definition:
  task_execution_role: ecsTaskExecutionRole
  ecs_network_mode: awsvpc
  task_size:
    mem_limit: 1.0GB
    cpu_limit: 512
  
run_params:
  network_configuration:
    awsvpc_configuration:
      subnets:
        - "subnet-1234"
        - "subnet-5678"
      security_groups:
        - "sg-1234"
./ecs-cli compose service up --aws-profile system-test --cluster-config -ec2-tutorial-config --launch-type EC2
INFO[0000] Using ECS task definition                     TaskDefinition="oaas:20"
INFO[0000] Updated the ECS service with a new task definition. Old containers will be stopped automatically, and replaced with new ones  desiredCount=1 force-deployment=false service=oaas
INFO[0010] (service oaas) stopped 1 pending tasks.       timestamp="2020-05-27 18:48:41 +0000 UTC"
INFO[0010] (service oaas) has started 1 tasks: (task c596d2ac-d6ae-4db1-897a-f754d9abd6e2).  timestamp="2020-05-27 18:48:41 +0000 UTC"
INFO[0071] Service status                                desiredCount=1 runningCount=1 serviceName=oaas
INFO[0071] ECS Service has reached a stable state        desiredCount=1 runningCount=1 serviceName=oaas

@bvtujo
Copy link
Contributor Author

bvtujo commented May 27, 2020

manual test of fargate service with efs volume
ecs-params.yml

version: 1
task_definition:
  task_execution_role: ecsTaskExecutionRole
  ecs_network_mode: awsvpc
  task_size:
    mem_limit: 1.0GB
    cpu_limit: 512
  efs_volumes:
    - name: "myEFSVolume"
      filesystem_id: "fs-fedc8554"
      root_directory: /
      transit_encryption: DISABLED
      iam: DISABLED
run_params:
  network_configuration:
    awsvpc_configuration:
      subnets:
        - "subnet-1234"
        - "subnet-5678"
      security_groups:
        - "sg-1234"
      assign_public_ip: "ENABLED"

docker-compose.yml

version: '3'
services:
  web:
    image: acct.dkr.ecr.us-west-2.amazonaws.com/otter:latest
    volumes:
      - myEFSVolume:/mount/efs
    ports:
      - "80:80"
    logging:
      driver: awslogs
      options:
        awslogs-group: otter
        awslogs-region: us-west-2
        awslogs-stream-prefix: web
volumes:
  myEFSVolume:
➜  oaas git:(ecs-cli) ✗ ./ecs-cli compose service up --aws-profile system-test --cluster-config -fargate-tutorial-config --launch-type FARGATE
INFO[0000] Using ECS task definition                     TaskDefinition="oaas:11"
INFO[0000] Updated the ECS service with a new task definition. Old containers will be stopped automatically, and replaced with new ones  desiredCount=1 force-deployment=false service=oaas
INFO[0000] Service status                                desiredCount=1 runningCount=1 serviceName=oaas
INFO[0020] (service oaas) has started 1 tasks: (task f6bf3318-fc65-4782-a099-144d69ca200d).  timestamp="2020-05-27 19:19:42 +0000 UTC"
INFO[0066] Service status                                desiredCount=1 runningCount=2 serviceName=oaas
INFO[0137] Service status                                desiredCount=1 runningCount=1 serviceName=oaas
INFO[0137] (service oaas) has stopped 1 running tasks: (task fcc33890-1157-4b81-8b52-72d238be622d).  timestamp="2020-05-27 19:21:39 +0000 UTC"
INFO[0147] ECS Service has reached a stable state        desiredCount=1 runningCount=1 serviceName=oaas

@bvtujo
Copy link
Contributor Author

bvtujo commented May 27, 2020

integ test run:


1 | [Container] 2020/05/27 21:28:35 Waiting for agent ping
-- | --
2 | [Container] 2020/05/27 21:28:37 Waiting for DOWNLOAD_SOURCE
3 | [Container] 2020/05/27 21:28:44 Phase is DOWNLOAD_SOURCE
4 | [Container] 2020/05/27 21:28:44 CODEBUILD_SRC_DIR=/codebuild/output/src239619880/src/github.com/aws/amazon-ecs-cli
5 | [Container] 2020/05/27 21:28:44 YAML location is /codebuild/readonly/buildspec.yml
6 | [Container] 2020/05/27 21:28:44 Processing environment variables
7 | [Container] 2020/05/27 21:28:44 Decrypting parameter store environment variables
8 | [Container] 2020/05/27 21:28:44 Moving to directory /codebuild/output/src239619880/src/github.com/aws/amazon-ecs-cli
9 | [Container] 2020/05/27 21:28:44 Registering with agent
10 | [Container] 2020/05/27 21:28:44 Phases found in YAML: 1
11 | [Container] 2020/05/27 21:28:44  BUILD: 1 commands
12 | [Container] 2020/05/27 21:28:44 Phase complete: DOWNLOAD_SOURCE State: SUCCEEDED
13 | [Container] 2020/05/27 21:28:44 Phase context status code:  Message:
14 | [Container] 2020/05/27 21:28:44 Entering phase INSTALL
15 | [Container] 2020/05/27 21:28:44 Phase complete: INSTALL State: SUCCEEDED
16 | [Container] 2020/05/27 21:28:44 Phase context status code:  Message:
17 | [Container] 2020/05/27 21:28:44 Entering phase PRE_BUILD
18 | [Container] 2020/05/27 21:28:44 Phase complete: PRE_BUILD State: SUCCEEDED
19 | [Container] 2020/05/27 21:28:44 Phase context status code:  Message:
20 | [Container] 2020/05/27 21:28:44 Entering phase BUILD
21 | [Container] 2020/05/27 21:28:44 Running command make integ-test
22 | Installing dependencies...
23 | go get github.com/wadey/gocovmerge
24 | Building ecs-cli.test...
25 | env -i PATH=$PATH GOPATH=$(go env GOPATH) GOROOT=$(go env GOROOT) GOCACHE=$(go env GOCACHE) \
26 | go test -coverpkg ./ecs-cli/modules/... -c -tags testrunmain -o ./bin/local/ecs-cli.test ./ecs-cli
27 | Running integration tests...
28 | go test -timeout 60m -tags integ -v ./ecs-cli/integ/e2e/...
29 | === RUN   TestCreateClusterWithEC2Task
30 | --- PASS: TestCreateClusterWithEC2Task (473.19s)
31 | configure.go:53: Created config ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-ec2-tutorial-config
32 | up.go:68: Created cluster ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-ec2-tutorial-cluster in stack amazon-ecs-cli-setup-ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-ec2-tutorial-cluster
33 | ecs.go:43: Number of container instances mismatch, wanted = 2, got = 0
34 | runner.go:98: Current timestamp=2020-05-27 21:32:38.145992707 +0000 UTC m=+181.999394054, sleeping for 15s
35 | ecs.go:43: Number of container instances mismatch, wanted = 2, got = 1
36 | runner.go:98: Current timestamp=2020-05-27 21:32:53.178565744 +0000 UTC m=+197.031967081, sleeping for 15s
37 | ecs.go:51: Cluster ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-ec2-tutorial-cluster has 2 instances
38 | ec2_task_test.go:98: Created /tmp/docker-compose-404704587.yml successfully
39 | compose.go:70: Created containers for e2e-ec2-tutorial-taskdef
40 | ecs.go:75: Cluster ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-ec2-tutorial-cluster has 1 tasks
41 | ps.go:36: Project e2e-ec2-tutorial-taskdef has 2 running containers
42 | compose.go:141: Scaled the task e2e-ec2-tutorial-taskdef to 2
43 | ecs.go:75: Cluster ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-ec2-tutorial-cluster has 2 tasks
44 | ps.go:36: Project e2e-ec2-tutorial-taskdef has 4 running containers
45 | compose.go:200: Deleted task e2e-ec2-tutorial-taskdef
46 | ecs.go:75: Cluster ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-ec2-tutorial-cluster has 0 tasks
47 | down.go:49: Deleted cluster ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-ec2-tutorial-cluster
48 | cfn.go:50: Success: stack amazon-ecs-cli-setup-ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-ec2-tutorial-cluster does not exist
49 | down.go:52: Deleted stack amazon-ecs-cli-setup-ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-ec2-tutorial-cluster
50 | === RUN   TestCreateClusterWithFargateService
51 | --- PASS: TestCreateClusterWithFargateService (314.71s)
52 | configure.go:42: Created config ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-fargate-tutorial-config
53 | up.go:68: Created cluster ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-fargate-tutorial-cluster in stack amazon-ecs-cli-setup-ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-fargate-tutorial-cluster
54 | fargate_service_test.go:86: Created /tmp/docker-compose-324706710.yml successfully
55 | fargate_service_test.go:117: Created /tmp/ecs-params-305371645.yml successfully
56 | compose.go:102: Created service with name ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-e2e-fargate-test-service
57 | compose.go:280: Container is RUNNING: e3f59304-57ae-40e7-a095-e024baf474b5/wordpress  RUNNING  52.38.148.122:80->80/tcp  ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-e2e-fargate-test-service:1  UNKNOWN
58 | compose.go:113: Project ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-e2e-fargate-test-service has 1 running containers
59 | compose.go:173: Scaled the service ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-e2e-fargate-test-service to 2 tasks
60 | compose.go:280: Container is RUNNING: 80fd9728-8e26-43f9-8541-51486abb9fdb/wordpress  RUNNING  34.220.110.148:80->80/tcp  ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-e2e-fargate-test-service:1  UNKNOWN
61 | compose.go:280: Container is RUNNING: e3f59304-57ae-40e7-a095-e024baf474b5/wordpress  RUNNING  52.38.148.122:80->80/tcp   ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-e2e-fargate-test-service:1  UNKNOWN
62 | compose.go:113: Project ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-e2e-fargate-test-service has 2 running containers
63 | compose.go:232: Deleted service ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-e2e-fargate-test-service
64 | down.go:49: Deleted cluster ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-fargate-tutorial-cluster
65 | cfn.go:50: Success: stack amazon-ecs-cli-setup-ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-fargate-tutorial-cluster does not exist
66 | down.go:52: Deleted stack amazon-ecs-cli-setup-ecs-cli-on-demand-integ-test-71b45f73-2f51-4d7b-84f5-cdefe9116c2d-fargate-tutorial-cluster
67 | === RUN   TestECSLocal
68 | === RUN   TestECSLocal/clean_state
69 | === RUN   TestECSLocal/from_task_def
70 | --- PASS: TestECSLocal (65.36s)
71 | --- PASS: TestECSLocal/clean_state (0.11s)
72 | --- PASS: TestECSLocal/from_task_def (65.25s)
73 | PASS


@@ -256,7 +301,7 @@ func TestCreateWithTaskPlacement(t *testing.T) {
Type: aws.String("binpack"),
},
}

assert.Nil(t, input.LaunchType, "launch type should be nil")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to check for launch type and PV?

@mergify mergify bot merged commit 3deafb3 into aws:master May 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants