add support for ECS Service Connect

[slootmaekersdirk]

Is your feature request related to a problem? Please describe.
At the moment inter service communication is based on DNS lookup which is not ideal

Describe the solution you'd like
Amazon ECS Service Connect allows to communicate between ECS services in an easy way, not using DNS lookup

Additional context
What we need to add is

1. when creating the cluster, specify the --service-connect-defaults parameter
2. while creating the service add the following properties:

• "namespace"
• "services":
  [
  {
  "portName": "test-port",
  "clientAliases": [
  {
  "port": 80,
  "dnsName": "test"
  }
  ]
  }
  ]
  -> now we can access this service from another service through http://test:80

[JohnPreston]

Hello :)
Thanks for opening this FR. In the meantime I'd recommend to make sure to use x-cloudmap to create DNS entries etc. for your services.
ECS Connect is an "easy build" for what you'd get with Cloud Map and AppMesh iirc
Will look into it.

[slootmaekersdirk]

Hello John,

did you find time for this FR? We're about to deply our solution, and this part is still a manual configuration job, after the deploy.

thx
Dirk

[JohnPreston]

Hello
I did, given that cloudmap etc. is already in compose-x it shouldn't be too much work to implement, I will need to test it out etc. though.
That's probably something that will come under the x-network settings to logically map services.

[slootmaekersdirk]

Hello John,

is this FR already in the 0.25 rc?

and , if so, how can I use it?

thx
Dirk

[JohnPreston]

Hi @slootmaekersdirk
That is not in the 0.25, that will be a 1.x thing
The Connect Config is actually quite involved, although I already have all the components in place to make the integration easy, I need to play with finding a good UX for users to configure it.

If you have any suggestion of what you feel the format should look like please do tell.

Otherwise, I might simply allow to define the entire property CFN style in x-ecs_connect or x-network.ecs_connect, but that means more work for the user.

PS: You can already achieve the http://test by using x-cloudmap today without any of the connect faff, ** if ** you set the VPC search domain to include the cloudmap domain.

[JohnPreston]

Hello John,

is this FR already in the 0.25 rc?

and , if so, how can I use it?

thx Dirk

What are the properties that you today configure via the UI or in some other way? What does the connect config look like when you do ecs describe-services ?

[slootmaekersdirk]

Hello John, For me the format of the syntax of ecs describe-services Is enough for me, we don’t need a UX at the moment Dirk From: John Preston ***@***.***> Date: Tuesday, 5 March 2024 at 07:28 To: compose-x/ecs_composex ***@***.***> Cc: Dirk Slootmaekers ***@***.***>, Mention ***@***.***> Subject: Re: [compose-x/ecs_composex] add support for ECS Service Connect (Issue #727) Hi @slootmaekersdirk<https://github.com/slootmaekersdirk> That is not in the 0.25, that will be a 1.x thing The Connect Config<https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ecs-service-serviceconnectconfiguration.html> is actually quite involved, although I already have all the components in place to make the integration easy, I need to play with finding a good UX for users to configure it. If you have any suggestion of what you feel the format should look like please do tell. Otherwise, I might simply allow to define the entire property CFN style in x-ecs_connect or x-network.ecs_connect, but that means more work for the user. — Reply to this email directly, view it on GitHub<#727 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACNLQBTD4HC7XR5DAIIFRFLYWVQYTAVCNFSM6AAAAABAPJJFGSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZYGA2DSOJVGM>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[JohnPreston]

I am adding the necessary to make it work in the 1.0 branch as this requires things that weren't originally in the docker compose specification but now are
EDIT: Likely to be an early access feature in 1.0

[slootmaekersdirk]

Hello John,

please keep me up te date, when it is available for testing

thx
Dirk

[JohnPreston]

Planned for 1.1.0 :)

PS: Next week I am at a summit, but should be done and delivered by end of March :)
EDIT: insert T&C that say I have no legal obligations to, limited to availability :D

[JohnPreston]

@slootmaekersdirk
I have it done in a new feature branch, however, want to check with you that I am doing the right thing.
I have the service connect configured, it created an entry in cloudmap but not DNS, API calls only, which is odd.
Then the other thing is, I don't see how you declare that service A wants to connect to service B 🤔
I am missing something?

See #745

[slootmaekersdirk]

Hello John,
 
The whole idea of the connect service is not to go via a DNS, but acting as a side car for each service, so a direct communication between services is possible.
 
To allow 2 side communication you should (when creating the service):
 
"serviceConnectConfiguration": {
        "enabled": true,
        "namespace": "svc-namespace",
        "services": [
            {
                "portName": "testtask",
                "clientAliases": [
                    {
                        "port": 80
                    }
                ],
            }
        ],
        "logConfiguration": {
            "logDriver": "awslogs",
            "options": {
                "awslogs-group": "/ecs/service-connect-proxy",
                "awslogs-region": "region",
                "awslogs-stream-prefix": "service-connect-proxy"
            }
        }
    },
 
Here we create a namespace called svc-namespace I can access this taks via http://testtask:80
all services in the same namespace can communicate with eachother
Dirk

[JohnPreston]

Hey @slootmaekersdirk
This is now implemented in #745 - if you would please test for me.
I tested with the YELB demo app that has been re-used in multiple places to demo connect, and I could get the UI online, vote, and get the stats going up/down
Let me know how you get on.

[JohnPreston]

Implemented in #745

[JohnPreston]

See https://pypi.org/project/ecs_composex/1.1.0rc1/