Implement per-route options in Cloud Controller #4080

hoffmaen · 2024-11-08T15:20:26Z

A short explanation of the proposed change:

This Pull-Request adds per-route options to the Cloud Controller. Support is added for the following APIs:

POST /v3/routes
GET /v3/routes
PATCH /v3/routes

Per-route options support is also added to the manifest.

An explanation of the use cases your change solves

Users / app developers want to define load-balancing algorithms per route, instead of being forced to use the globally defined algorithm.

Links to any other associated PRs

RFC for per-route options: https://github.com/cloudfoundry/community/blob/main/toc/rfc/rfc-0027-generic-per-route-features.md
capi-release issue: Implement per-route options in Cloud Controller capi-release#482
I have reviewed the contributing guide
I have viewed, signed, and submitted the Contributor License Agreement
I have made this pull request to the main branch
I have run all the unit tests using bundle exec rake
I have run CF Acceptance Tests

jochenehret

I'll test the proposed changes on a dev landscape.

jochenehret · 2024-11-22T12:09:14Z

I've tested the new feature on a dev landscape and everything works as expected. There are however a few corner cases that we should discuss:

1. "options" with null value

This request is accepted:

cf curl -X PATCH /v3/routes/<guid> -d '{"options": null}'

It doesn't change anything. Is that intended or should the request be rejected?

2. Valid option key with null value

cf curl -X PATCH /v3/routes/<guid> -d '{"options": {"lb_algo": null}}'

Accepted and sets the "options" field to {} (empty map). Shouldn't this be rejected?

3. Create new route without "options"

Use the CF CLI to create a new route:

cf -v create-route <domain> --hostname <host>

This will send a POST request:

cf curl -X POST /v3/routes -d ...

Now the "option" field in the database is set to "null" (text "null"). For existing routes the option field is initially null (matches "where is null"). If found no problems with the "null" text, but I guess it would be better to set the "options" value to null for new routes.

4. "options" in app manifest

Pushing an app with manifest works:

version: 1
applications:
- name: test
  routes:
  - route: example.com
    options:
      loadbalancing-algorithm: round-robin

For configuration with null values, we can observe the same behaviour as above:
Setting "options" to null doesn't change anything.
Replacing "round-robin" with null and pushing again sets the options to {} (empty map).

app/messages/manifest_routes_update_message.rb

jochenehret · 2024-11-22T14:45:20Z

Did you plan to create a new acceptance test for this feature? You could add it to this capi-bara-test:
https://github.com/cloudfoundry/capi-bara-tests/blob/fff7cf1f121c13e58c06e1c9d5cfec0e887f724c/baras/manifest.go#L141

hoffmaen · 2024-11-26T10:27:22Z

Thanks for your review @jochenehret

1. "options" with null value

This request is accepted:
cf curl -X PATCH /v3/routes/<guid> -d '{"options": null}'
It doesn't change anything. Is that intended or should the request be rejected?

You found a recently introduced bug, we fix it. According to the tracking issue, this request should unset all options (which it does now again).

2. Valid option key with null value
cf curl -X PATCH /v3/routes/<guid> -d '{"options": {"lb_algo": null}}'
Accepted and sets the "options" field to {} (empty map). Shouldn't this be rejected?

No, this is the expected behavior, as documented in the tracking issue.

3. Create new route without "options"

Use the CF CLI to create a new route:
cf -v create-route <domain> --hostname <host>
This will send a POST request:
cf curl -X POST /v3/routes -d ...
Now the "option" field in the database is set to "null" (text "null"). For existing routes the option field is initially null (matches "where is null"). If found no problems with the "null" text, but I guess it would be better to set the "options" value to null for new routes.

There is another post in the tracking issue. @Gerg argues for always providing the options field, even if no options are set. If we adopt his idea, we could replace null (or 'null' accidentally) with an empty hash.

4. "options" in app manifest

Pushing an app with manifest works:
version: 1
applications:
- name: test
  routes:
  - route: example.com
    options:
      loadbalancing-algorithm: round-robin
For configuration with null values, we can observe the same behaviour as above: Setting "options" to null doesn't change anything. Replacing "round-robin" with null and pushing again sets the options to {} (empty map).

We fixed this - unsetting options in general or loadbalancing-algorithm in particular is not possible any more.

jochenehret

I've retested the PR with the latest changes and found no issues!

This commit adds a configurable load-balancing algorithm as a first example of per-route options. It adds the 'options' field to the route object in the V3 API, and the app manifest. The options field is an object storing key-value pairs, with 'lb_algo' being the only supported key for now. The supported load-balancing algorithms are 'round-robin' and 'least-connections'. The options field is introduced as a column in the database route table, and forwarded to the Diego backend. Co-authored-by: Alexander Nicke <[email protected]> See: cloudfoundry/capi-release#482 See: https://github.com/cloudfoundry/community/blob/main/toc/rfc/rfc-0027-generic-per-route-features.md

Add documentation for the route options object, and its supported fields.

Overwrite behaviour for route options is now fixed and tested: Existing options are not modified if options is nil, {} or not provided A single option (e.g. loadbalancing-algorithm) can be removed by setting its value to nil adjust test for manifest push: options {key:nil} should not modify the existing value

options default: {} API: options is not nullable specific option is additive specific option is nullable empty hash does not change anything (additive) get empty options -> {} manifest: options and specific option is nullable, but no-op

Changes in cloud_controller_ng: - Implement per-route options in Cloud Controller PR: cloudfoundry/cloud_controller_ng#4080 Author: Clemens Hoffmann <[email protected]> Author: Alexander Nicke <[email protected]> - Bump Ruby to 3.2.6 PR: cloudfoundry/cloud_controller_ng#4127 Author: Seth Boyles <[email protected]>

sethboyles · 2024-12-10T18:22:32Z

The new options field seems to be breaking acceptance tests: https://concourse.app-runtime-interfaces.ci.cloudfoundry.org/teams/capi-team/pipelines/capi/jobs/gyro-exp-tests/builds/49

  [FAILED] Expected

      <string>: applications:

          - buildpacks:

              - ruby_buildpack

            env:

              foo: qux

              snack: walnuts

            lifecycle: buildpack

            metadata:

              annotations:

                  contact: [email protected] [email protected]

                  cougar: mellencamp

              labels:

                  juice: cherry

                  potato: yams

            name: BARA-2-APP-22ae327aa21a2346

            processes:

              - disk_quota: 1024M

                health-check-http-endpoint: /env

                health-check-type: http

                instances: 2

                log-rate-limit-per-second: -1

                memory: 300M

                readiness-health-check-type: process

                timeout: 75

                type: web

              - disk_quota: 1024M

                health-check-type: process

                instances: 0

                log-rate-limit-per-second: -1

                memory: 256M

                readiness-health-check-type: process

                type: worker

            routes:

              - options: {}

                protocol: http1

                route: BARA-2-ROUTE-9d3fa91d95c7fa2d.cf.gyro.app-runtime-interfaces.ci.cloudfoundry.org

            stack: cflinuxfs4

  to match YAML of

      <string>: applications:

          - buildpacks:

              - ruby_buildpack

            env:

              foo: qux

              snack: walnuts

            lifecycle: buildpack

            metadata:

              annotations:

                  contact: [email protected] [email protected]

                  cougar: mellencamp

              labels:

                  juice: cherry

                  potato: yams

            name: BARA-2-APP-22ae327aa21a2346

            processes:

              - disk_quota: 1024M

                health-check-http-endpoint: /env

                health-check-type: http

                instances: 2

                log-rate-limit-per-second: -1

                memory: 300M

                readiness-health-check-type: process

                timeout: 75

                type: web

              - disk_quota: 1024M

                health-check-type: process

                instances: 0

                log-rate-limit-per-second: -1

                memory: 256M

                readiness-health-check-type: process

                type: worker

            routes:

              - protocol: http1

                route: BARA-2-ROUTE-9d3fa91d95c7fa2d.cf.gyro.app-runtime-interfaces.ci.cloudfoundry.org

            stack: cflinuxfs4


  first mismatched key: "applications"[0]."routes"[0]

  In [It] at: /go/src/github.com/cloudfoundry/capi-bara-tests/baras/manifest.go:248 @ 12/10/24 12:42:54.278

This might require an update to cf-acceptance-tests

hoffmaen · 2024-12-11T11:45:04Z

@sethboyles Fixed with cloudfoundry/capi-bara-tests#106.

hoffmaen force-pushed the per-route-options branch from 4ce58eb to d5e369c Compare November 13, 2024 13:22

a18e force-pushed the per-route-options branch from 33d7efb to 35d8b72 Compare November 19, 2024 11:48

hoffmaen marked this pull request as ready for review November 21, 2024 09:37

jochenehret self-requested a review November 22, 2024 09:00

jochenehret reviewed Nov 22, 2024

View reviewed changes

app/messages/manifest_routes_update_message.rb Outdated Show resolved Hide resolved

hoffmaen force-pushed the per-route-options branch 2 times, most recently from 7f6cc57 to af076fe Compare November 25, 2024 19:04

hoffmaen force-pushed the per-route-options branch 3 times, most recently from 9be6eb8 to 7eaa6fc Compare December 4, 2024 08:43

maxmoehl mentioned this pull request Dec 4, 2024

Implement per-route options in the CF CLI cloudfoundry/cli#3314

Open

hoffmaen force-pushed the per-route-options branch 3 times, most recently from 05041d7 to 1a5eba6 Compare December 5, 2024 20:13

jochenehret self-requested a review December 9, 2024 14:46

jochenehret approved these changes Dec 9, 2024

View reviewed changes

a18e and others added 7 commits December 9, 2024 16:19

Add route-options documentation

9b3ea30

Add documentation for the route options object, and its supported fields.

Adjust behaviour to new decisions:

1d058dd

options default: {} API: options is not nullable specific option is additive specific option is nullable empty hash does not change anything (additive) get empty options -> {} manifest: options and specific option is nullable, but no-op

Remove route option validations from manifest_route

5567fcd

Rename 'lb_algo' and 'loadbalancing-algorithm' to 'algorithm'

cd7d06a

Disallow null in manifest; Cleanup error outputs

97440e0

hoffmaen force-pushed the per-route-options branch from 1a5eba6 to 97440e0 Compare December 9, 2024 15:20

jochenehret merged commit 05e617f into cloudfoundry:main Dec 10, 2024
15 checks passed

hoffmaen mentioned this pull request Dec 11, 2024

Add route options to manifest cloudfoundry/capi-bara-tests#106

Merged

a18e mentioned this pull request Dec 18, 2024

Update RFC 0027: Unify load balancing property name cloudfoundry/community#1039

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement per-route options in Cloud Controller #4080

Implement per-route options in Cloud Controller #4080

hoffmaen commented Nov 8, 2024 •

edited

Loading

jochenehret left a comment

jochenehret commented Nov 22, 2024

jochenehret commented Nov 22, 2024

hoffmaen commented Nov 26, 2024

jochenehret left a comment

sethboyles commented Dec 10, 2024

hoffmaen commented Dec 11, 2024

Implement per-route options in Cloud Controller #4080

Implement per-route options in Cloud Controller #4080

Conversation

hoffmaen commented Nov 8, 2024 • edited Loading

A short explanation of the proposed change:

An explanation of the use cases your change solves

Links to any other associated PRs

jochenehret left a comment

Choose a reason for hiding this comment

jochenehret commented Nov 22, 2024

jochenehret commented Nov 22, 2024

hoffmaen commented Nov 26, 2024

jochenehret left a comment

Choose a reason for hiding this comment

sethboyles commented Dec 10, 2024

hoffmaen commented Dec 11, 2024

hoffmaen commented Nov 8, 2024 •

edited

Loading