Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Review ASG Sizes #9322

Closed
Tracked by #8345
ioannakok opened this issue Oct 25, 2023 · 3 comments
Closed
Tracked by #8345

Review ASG Sizes #9322

ioannakok opened this issue Oct 25, 2023 · 3 comments
Assignees
@georgeblahblah @alinaboghiu @arelra @ioannakok
Milestone
Health

Comments

@ioannakok
Copy link
Contributor

ioannakok commented Oct 25, 2023 •
edited
Loading

We're currently using a minimum of 15 and a maximum of 60 instances in production:

dotcom-rendering/dotcom-rendering/cloudformation.yml

Lines 72 to 78 in e234547

StageMap:
PROD:
MinCapacity: 15
MaxCapacity: 60
CODE:
MinCapacity: 1
MaxCapacity: 4

Do these limits allow enough headroom for our additional traffic, scaling requirements, and RiffRaff deploys?
@ioannakok ioannakok mentioned this issue Oct 25, 2023
Closed
@github-project-automation github-project-automation bot moved this to Triage in WebX Team Oct 26, 2023
@alinaboghiu alinaboghiu moved this from Triage to Planned in WebX Team Oct 26, 2023
@ioannakok ioannakok moved this from Planned to In progress in WebX Team Oct 31, 2023
ioannakok added a commit that referenced this issue Oct 31, 2023
@ioannakok @georgeblahblah @arelra
Decrease minCapacity of instances to 25
1367910 
`minCapacity` was bumped from 15 to 30 in #8724. After the fronts migration was completed, it seemed that 15 instances weren't holding up with the new traffic we were sending. That was a hotfix and there is upcoming work to split our stacks (#8351) but until we reach this point we would like to see whether we could handle the traffic with 25 instances. This change is part of the research we're doing to optimise how we scale: #9322.

Co-authored-by: George B <[email protected]>
Co-authored-by: Ravi <[email protected]>
@ioannakok ioannakok mentioned this issue Oct 31, 2023
Merged
ioannakok added a commit that referenced this issue Oct 31, 2023
@ioannakok @georgeblahblah @arelra
Decrease minCapacity of instances to 25
648d1cd 
`minCapacity` was bumped from 15 to 30 in #8724. After the fronts migration was completed, it seemed that 15 instances weren't holding up with the new traffic we were sending. That was a hotfix and there is upcoming work to split our stacks (#8351) but until we reach this point we would like to see whether we could handle the traffic with 25 instances. This change is part of the research we're doing to optimise how we scale: #9322.

Co-authored-by: George B <[email protected]>
Co-authored-by: Ravi <[email protected]>
ioannakok added a commit that referenced this issue Nov 1, 2023
@ioannakok @georgeblahblah @arelra
Decrease minCapacity of instances to 25
7a1a0e4 
`minCapacity` was bumped from 15 to 30 in #8724. After the fronts migration was completed, it seemed that 15 instances weren't holding up with the new traffic we were sending. That was a hotfix and there is upcoming work to split our stacks (#8351) but until we reach this point we would like to see whether we could handle the traffic with 25 instances. This change is part of the research we're doing to optimise how we scale: #9322.

Co-authored-by: George B <[email protected]>
Co-authored-by: Ravi <[email protected]>
@ioannakok
Copy link
Contributor Author

ioannakok commented Nov 1, 2023 •
edited
Loading

PR that reduced minCapacity to 27 was merged on 1/11:

@ioannakok
Copy link
Contributor Author

Progress so far: https://docs.google.com/document/d/1FjDANYnddGrB3JnXdMjoHnie2kB7fL3MgJachCVZHVg/edit

@ioannakok ioannakok moved this from In progress to Planned in WebX Team Nov 7, 2023
@ioannakok ioannakok moved this from Planned to Backlog in WebX Team Nov 21, 2023
@alinaboghiu alinaboghiu moved this from Backlog to Planned in WebX Team Dec 13, 2023
@alinaboghiu alinaboghiu moved this from Planned to In progress in WebX Team Dec 13, 2023
@alinaboghiu alinaboghiu moved this from In Progress to This Sprint in WebX Team Dec 19, 2023
@ioannakok ioannakok mentioned this issue Dec 20, 2023
Closed
@ioannakok ioannakok moved this from This Sprint to In Progress in WebX Team Dec 21, 2023
@alinaboghiu alinaboghiu moved this from In Progress to Review in WebX Team Jan 10, 2024
@alinaboghiu alinaboghiu moved this from Review to In Progress in WebX Team Jan 10, 2024
@alinaboghiu alinaboghiu moved this from In Progress to Review in WebX Team Jan 10, 2024
@alinaboghiu
Copy link
Member

https://github.com/guardian/dotcom-rendering/blob/main/dotcom-rendering/docs/architecture/029-scaling-policy-decision-record.md

@github-project-automation github-project-automation bot moved this from Review to Done in WebX Team Jan 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@georgeblahblah georgeblahblah

@alinaboghiu alinaboghiu

@arelra arelra

@ioannakok ioannakok

Labels
None yet
Projects
WebX Team
Archived in project
Milestone
Health
Development

No branches or pull requests
5 participants
@georgeblahblah @alinaboghiu @arelra @ioannakok @JamieB-gu