Updates for latest releases

.circleci/config.yml

@@ -348,7 +348,7 @@ commands:
     parameters:
       "version":
         type: string
-        default: "1.15"
+        default: "1.19.1"
     steps:
       - run:
           name: "Install Go << parameters.version >>"

.circleci/config.yml.d/generic_util.yml

@@ -79,7 +79,7 @@ commands:
     parameters:
       "version":
         type: string
-        default: "1.15"
+        default: "1.19.1"
     steps:
       - run:
           name: "Install Go << parameters.version >>"

.circleci/yq.d/go.mod

@@ -1,6 +1,6 @@
 module github.com/datawire/build-aux/bin-go/yq
 
-go 1.17
+go 1.19
 
 require github.com/mikefarah/yq/v4 v4.25.1
 

CHANGELOG.md

@@ -77,16 +77,186 @@ Please see the [Envoy documentation](https://www.envoyproxy.io/docs/envoy/latest
 
 ## RELEASE NOTES
 
+## [3.2.0] 2022-09-27
+[3.2.0]: https://github.com/datawire/edge-stack/releases/v3.2.0
+
+## Ambassador Edge Stack
+
+- Change: The envoy version included in Ambassador Edge Stack has been upgraded from 1.22 to the latest
+  patch release of 1.23. This provides Ambassador Edge Stack with the latest security patches,
+  performances enhancments, and features offered by the envoy proxy.
+
+- Change: Changes to label matching will change how `Hosts` are associated with `Mappings`. There was a bug
+  with label selectors that was causing `Hosts` to be incorrectly being associated with more
+  `Mappings` than intended. If any single label from the selector was matched then the `Host` would
+  be associated with the `Mapping`. Now it has been updated to correctly only associate a `Host`
+  with a `Mapping` if _all_ labels required by the selector are present. This brings the
+  `mappingSelector` field in-line with how label selectors are used in Kubernetes. To avoid
+  unexpected behaviour after the upgrade, add all labels that Hosts have in their `mappingSelector`
+  to `Mappings` you want to associate with the `Host`. You can opt-out of the new behaviour by
+  setting the environment variable `DISABLE_STRICT_LABEL_SELECTORS` to `"true"` (default:
+  `"false"`). (Thanks to <a href="https://github.com/f-herceg">Filip Herceg</a> and <a
+  href="https://github.com/dynajoe">Joe Andaverde</a>!).      
+
+- Feature: Previously the `Host` resource could only use secrets that are in the namespace as the Host. The
+  `tlsSecret` field in the Host has a new subfield `namespace` that will allow the use of secrets
+  from different namespaces.
+
+- Change: Set `AMBASSADOR_EDS_BYPASS` to `true` to bypass EDS handling of endpoints and have endpoints be
+  inserted to clusters manually. This can help resolve with `503 UH` caused by certification
+  rotation relating to a delay between EDS + CDS. The default is `false`.     
+
+- Bugfix: Distinct services with names that are the same in the first forty characters will no longer be
+  incorrectly mapped to the same cluster. ([#4354])
+
+- Feature: By default, when Envoy is unable to communicate with the configured RateLimitService then it will
+  allow traffic through. The `RateLimitService` resource now exposes the <a
+  href="https://www.envoyproxy.io/docs/envoy/v1.23.0/configuration/http/http_filters/rate_limit_filter">failure_mode_deny</a>
+  option. Set `failure_mode_deny: true`, then Envoy will deny traffic when it is unable to
+  communicate to the RateLimitService returning a 500.
+
+- Bugfix: Previously, setting the `stats_name` for the `TracingService`, `RateLimitService` or the
+  `AuthService` would have no affect because it was not being properly passed to the Envoy cluster
+  config. This has been fixed and the `alt_stats_name` field in the cluster config is now set
+  correctly. (Thanks to <a href="https://github.com/psalaberria002">Paul</a>!)
+
+- Feature: The `AMBASSADOR_RECONFIG_MAX_DELAY` env var can be optionally set to batch changes for the
+  specified non-negative window period in seconds before doing an Envoy reconfiguration. Default is
+  "1" if not set.
+
+- Bugfix: If a `Host` or `TLSContext` contained a hostname with a `:` when using the diagnostics endpoints
+  `ambassador/v0/diagd` then an error would be thrown due to the parsing logic not being able to
+  handle the extra colon. This has been fixed and Ambassador Edge Stack will not throw an error when
+  parsing envoy metrics for the diagnostics user interface.
+
+- Feature: It is now possible to set `custom_tags` in the `TracingService`. Trace tags can be set based on
+  literal values, environment variables, or request headers. (Thanks to <a
+  href="https://github.com/psalaberria002">Paul</a>!) ([#4181])
+
+- Bugfix: Ambassador Edge Stack 2.0.0 introduced a bug where a `TCPMapping` that uses SNI, instead of using
+  the hostname glob in the `TCPMapping`, uses the hostname glob in the `Host` that the TLS
+  termination configuration comes from.
+
+- Bugfix: Ambassador Edge Stack 2.0.0 introduced a bug where a `TCPMapping` that terminates TLS must have a
+  corresponding `Host` that it can take the TLS configuration from. This was semi-intentional, but
+  didn't make much sense.  You can now use a `TLSContext` without a `Host`as in Ambassador Edge
+  Stack 1.y releases, or a `Host` with or without a `TLSContext` as in prior 2.y releases.
+
+- Bugfix: Prior releases of Ambassador Edge Stack had the arbitrary limitation that a `TCPMapping` cannot be
+  used on the same port that HTTP is served on, even if TLS+SNI would make this possible. 
+  Ambassador Edge Stack now allows `TCPMappings` to be used on the same `Listener` port as HTTP
+  `Hosts`, as long as that `Listener` terminates TLS.
+
+- Security: Updated Golang to 1.19.1 to address the CVEs: CVE-2022-27664, CVE-2022-32190.
+
+- Bugfix: Previously, Ambassador Edge Stack would incorrectly include empty fields when converting a
+  FilterPolicy or ExternalFilter between versions. This would cause undesired state to be persisted
+  in k8s which would lead to validation issues when trying to kubectl apply the custom resource.
+  This fixes these issues to ensure the correct data is being persisted and roundtripped properly
+  between CRD versions.
+
+- Feature: You may now define (on supported IDPs) a `postLogoutRedirectURI` to your `Oauth2` filter.  This
+  will allow you to redirect to a specific URI upon logging out. However, in order to achieve this
+  you must  define your IDP logout URL to `https:{{host}}/.ambassador/oauth2/post-logout-redirect`.
+  Upon logout  Ambassador Edge Stack will redirect to the custom URI which will then redirect to the
+  URI you have defined in `postLogoutRedirectURI`.
+
+[#4354]: https://github.com/emissary-ingress/emissary/issues/4354
+[#4181]: https://github.com/emissary-ingress/emissary/pull/4181
+
+## [3.1.0] 2022-08-01
+[3.1.0]: https://github.com/datawire/edge-stack/releases/v3.1.0
+
+## Ambassador Edge Stack
+
+- Feature: A new `Fitler` has been added to support validating APIKey's on incoming requests. The new
+  `APIKeyFilter` when applied with a `FilterPolicy` will check to  see if the incoming requests has
+  a valid API Key in the request header. Ambassador Edge Stack uses Kubernetes `Secret`'s to lookup
+  valid keys for authorizing requests.
+
+- Feature: Emissary-ingress has been taught to watch for APIKey secrets when Ambassador Edge Stack is running
+  and makes them available to be used with the new `APIKeyFilter`.
+
+- Feature: A new opt-in feature flag has been added that allows Ambassador Edge Stack to use a new Redis 
+  driver when storing state between requests for the OAuth2 Filter. The new driver has  better
+  connection pool handling, shares connections and supports the Redis RESP3 protocol.  Set
+  `AES_REDIS_EXPERIMENTAL_DRIVER_ENABLED=true` to enable the experimental feature. Most of the
+  standard Redis configuration fields (e.g.`REDIS_*`) can be used with the driver. Howeever, due to
+  the drivers better connection handling the new driver no longer supports setting 
+  `REDIS_SURGE_LIMIT_INTERVAL`, `REDIS_SURGE_LIMIT_AFTER`, `REDIS_SURGE_POOL_SIZE`,
+  `REDIS_SURGE_POOL_DRAIN_INTERVAL` and these will be ignored.
+Note: Other Ambassador Edge Stack
+  features such as the `RateLimitService` will continue to use the current Redis driver and in
+  future releases we plan to roll out the new driver for those features as well.
+
+- Change: If Ambassador Edge Stack is running then Emissary-ingress ensures that only a single
+  RateLimitService is active. If a user doesn't provide one or provides an invalid one then a
+  synthetic RateLimitService will be injected. If the `protocol_version` field is not set or set to
+  an invalid value then it will automatically get upgraded `protocol_version: v3`. 
+This matches the
+  existing behavior that was introduced in Ambassador Edge Stack v3.0.0 for the  `AuthService`. For
+  new installs a valid `RateLimitService` will be added but this  change ensures a smooth upgrade
+  from Ambassador Edge Stack to v2.3.Z to v3.Y for users who use the manifest in a GitOps scenario.
+
+- Feature: The agent is now able to parse api contracts using swagger 2, and to convert them to OpenAPI 3,
+  making them available for use in the dev portal.
+
+- Change: In the standard published `.yaml` files, the `Module` resource enables serving remote client
+  requests to the `:8877/ambassador/v0/diag/` endpoint. The associated Helm chart release also now
+  enables it by default.
+
+- Bugfix: When an `OAuth2` filter sets cookies for a `protectedOrigin`, it should set a cookie's "Secure"
+  flag to true for `https://` origins and false for `http://` origins.  However, for filters with
+  multiple origins, it set the cookie's flag based on the first origin listen in the Filter, rather
+  than the origin that the cookie is actually for.
+
+- Bugfix: When an `OAuth2` filter with multiple `protectedOrigins` needs to adjust the cookies for an active
+  login (which only happens when using a refresh token), it would erroneously redirect the web
+  browser to the last origin listed, rather than returning to the original URL.  This has been
+  fixed.
+
+- Bugfix: Previously, the `OAuth2` filter's known endpoints `/.ambassador/oauth2/logout` and
+  `/.ambassador/oauth2/multicookie` did not understand CORS or CORS preflight request which would
+  cause the browser to reject the request. This has now been fixed and these endpoints will attach
+  the appropriate CORS headers to the response.
+
+- Bugfix: A regression was introduced in 2.3.0 causing the agent to miss some of the metrics coming from
+  emissary ingress before sending them to Ambassador cloud. This issue has been resolved to ensure
+  that all the nodes composing the emissary ingress cluster are reporting properly.
+
+- Bugfix: Previously, we would inject an upstream route for acme-challenge that was targeting the localhost
+  auth service cluster. This route is injected to make Envoy configuration happy and the AuthService
+  that is shipped with Ambassador Edge Stack will handle it properly. However, if the cluster name
+  is longer than 60 characters due to a long namespace, etc... then Ambassador Edge Stack will
+  truncate it and make  sure it is unique. When this happens the name of the cluster assigned to the
+  acme-challenge route would get out-of-sync and would introduce invalid Envoy configuration. 
+To
+  avoid this Ambassador Edge Stack will now inject a route that returns a direct `404` response
+  rather than pointing at an arbitrary cluster. This matches existing behavior and is a transparent
+  change to the user.
+
+- Security: Updated Golang to 1.17.12 to address the CVEs: CVE-2022-23806, CVE-2022-28327, CVE-2022-24675,
+  CVE-2022-24921, CVE-2022-23772.
+
+- Security: Updated Curl to 7.80.0-r2 to address the CVEs: CVE-2022-32207, CVE-2022-27782, CVE-2022-27781,
+  CVE-2022-27780.
+
+- Security: Updated openSSL-dev to 1.1.1q-r0 to address CVE-2022-2097.
+
+- Security: Updated ncurses to 1.1.1q-r0 to address CVE-2022-29458
+
+- Security: Upgrade jwt-go to latest commit to resolve CVE-2020-26160.
+
 ## [3.0.0] 2022-06-29
 [3.0.0]: https://github.com/datawire/edge-stack/releases/v3.0.0
 
 ## Ambassador Edge Stack
 
 - Change: Ambassador Edge Stack is now built on top of Emissary-ingress 3.0.0 which updates Envoy Proxy from
-  v1.17 to v1.22. This provides Ambassador Edge Stack with the latest  security patches,
-  performances enhancments, and features offered by Envoy Proxy.  One notable change that will
-  effect users is the removal of support for  the V2 xDS tranport protocol. See the Emissary-ingress
-  changelog for more details.
+  v1.17 to v1.22. This provides Ambassador Edge Stack with the latest security patches, performances
+  enhancments, and features offered by Envoy Proxy. One notable change that will effect users is the
+  removal of support for the V2 xDS tranport protocol. See the Emissary-ingress changelog for more
+  details.
 
 - Change: In Envoy Proxy 1.18, two behavior changes were made in the way headers are attached to request.
   First, the `:scheme` header is now attached to upstream requests over HTTP/1.1 to align with
@@ -100,9 +270,64 @@ Please see the [Envoy documentation](https://www.envoyproxy.io/docs/envoy/latest
   transport protocol.
 
 - Change: Since Ambassador Edge Stack no longer supports the xDS V2 transport protocol, the default Helm
-  Charts and Manifest explicilty set `protocol_version` to `v3` for  the `RateLimitService` and
+  Charts and Manifest explicilty set `protocol_version` to `v3` for the `RateLimitService` and
   `AuthService` provided by Ambassador Edge Stack.
 
+## [2.4.0] 2022-09-19
+[2.4.0]: https://github.com/datawire/edge-stack/releases/v2.4.0
+
+## Ambassador Edge Stack
+
+- Feature: Previously the `Host` resource could only use secrets that are in the namespace as the Host. The
+  `tlsSecret` field in the Host has a new subfield `namespace` that will allow the use of secrets
+  from different namespaces.
+
+- Change: Set `AMBASSADOR_EDS_BYPASS` to `true` to bypass EDS handling of endpoints and have endpoints be
+  inserted to clusters manually. This can help resolve with `503 UH` caused by certification
+  rotation relating to a delay between EDS + CDS. The default is `false`.
+
+- Bugfix: Previously, Ambassador Edge Stack would incorrectly include empty fields when converting a
+  FilterPolicy or ExternalFilter between versions. This would cause undesired state to be persisted
+  in k8s which would lead to validation issues when trying to kubectl apply the custom resource.
+  This fixes these issues to ensure the correct data is being persisted and roundtripped properly
+  between CRD versions.
+
+## [2.3.2] 2022-08-01
+[2.3.2]: https://github.com/datawire/edge-stack/releases/v2.3.2
+
+## Ambassador Edge Stack
+
+- Bugfix: When an `OAuth2` filter sets cookies for a `protectedOrigin`, it should set a cookie's "Secure"
+  flag to true for `https://` origins and false for `http://` origins.  However, for filters with
+  multiple origins, it set the cookie's flag based on the first origin listen in the Filter, rather
+  than the origin that the cookie is actually for.
+
+- Bugfix: When an `OAuth2` filter with multiple `protectedOrigins` needs to adjust the cookies for an active
+  login (which only happens when using a refresh token), it would erroneously redirect the web
+  browser to the last origin listed, rather than returning to the original URL.  This has been
+  fixed.
+
+- Bugfix: Previously, the `OAuth2` filter's known endpoints `/.ambassador/oauth2/logout` and
+  `/.ambassador/oauth2/multicookie` did not understand CORS or CORS preflight request which would
+  cause the browser to reject the request. This has now been fixed and these endpoints will attach
+  the appropriate CORS headers to the response.
+
+- Bugfix: A regression was introduced in 2.3.0 causing the agent to miss some of the metrics coming from
+  emissary ingress before sending them to Ambassador cloud. This issue has been resolved to ensure
+  that all the nodes composing the emissary ingress cluster are reporting properly.
+
+- Security: Updated Golang to 1.17.12 to address the CVEs: CVE-2022-23806, CVE-2022-28327, CVE-2022-24675,
+  CVE-2022-24921, CVE-2022-23772.
+
+- Security: Updated Curl to 7.80.0-r2 to address the CVEs: CVE-2022-32207, CVE-2022-27782, CVE-2022-27781,
+  CVE-2022-27780.
+
+- Security: Updated openSSL-dev to 1.1.1q-r0 to address CVE-2022-2097.
+
+- Security: Updated ncurses to 1.1.1q-r0 to address CVE-2022-29458
+
+- Security: Upgrade jwt-go to latest commit to resolve CVE-2020-26160.
+
 ## [2.3.1] 2022-06-09
 [2.3.1]: https://github.com/datawire/edge-stack/releases/v2.3.1
 

Makefile

@@ -71,7 +71,8 @@ generate:
 
 generate-clean:
 	rm -rf $(generate/files)
-.PHONY: generate
+	rm -rf $(EDGE_STACK_HOME)/charts/edge-stack/charts/
+.PHONY: generate-clean
 
 $(EDGE_STACK_HOME)/CHANGELOG.md: $(EDGE_STACK_HOME)/docs/CHANGELOG.tpl $(EDGE_STACK_HOME)/docs/releaseNotes.yml
 	docker run --rm \

VERSION

@@ -1 +1 @@
-3.0.0
+3.2.0

charts/edge-stack/CHANGELOG.md

@@ -2,11 +2,24 @@
 
 This file documents all notable changes to Edge Stack Helm Chart. The release
 numbering uses [semantic versioning](http://semver.org).
+## v8.2.0
+
+- Update Edge Stack chart image to version v3.2.0: [CHANGELOG](https://github.com/datawire/edge-stack/blob/master/CHANGELOG.md)
+
+## v8.1.0
+
+- Update Edge Stack chart image to version v3.1.0: [CHANGELOG](https://github.com/datawire/edge-stack/blob/master/CHANGELOG.md)
 
 ## v8.0.0
 
 - Update Edge Stack chart image to version v3.0.0: [CHANGELOG](https://github.com/datawire/edge-stack/blob/master/CHANGELOG.md)
 
+## v7.5.0
+- Update Edge Stack chart image to version v2.4.0: [CHANGELOG](https://github.com/datawire/edge-stack/blob/master/CHANGELOG.md)
+
+## v7.4.2
+- Update Edge Stack chart image to version v2.3.2: [CHANGELOG](https://github.com/datawire/edge-stack/blob/master/CHANGELOG.md)
+
 ## v7.4.1
 
 - Update Edge Stack chart image to version v2.3.1: [CHANGELOG](https://github.com/datawire/edge-stack/blob/master/CHANGELOG.md)

charts/edge-stack/Chart.lock

@@ -1,6 +1,6 @@
 dependencies:
 - name: emissary-ingress
   repository: https://s3.amazonaws.com/datawire-static-files/charts
-  version: 8.0.0
-digest: sha256:5cf1b088272dfcb672dbab5471946c3d3bd2b5d924a669863676593f68c50d91
-generated: "2022-06-28T13:33:23.110421-05:00"
+  version: 8.2.0
+digest: sha256:f46f64fd6e4be3020311d3931b9db273b776ad13e078ea781bddebc79bfab1eb
+generated: "2022-09-27T15:43:43.215815585-04:00"

charts/edge-stack/Chart.yaml

@@ -1,8 +1,8 @@
 apiVersion: v2
-appVersion: 3.0.0
+appVersion: 3.2.0
 description: A Helm chart for Ambassador Edge Stack
 name: edge-stack
-version: 8.0.0
+version: 8.2.0
 # TODO: change these to whatever the appropriate things are
 icon: https://www.getambassador.io/images/logo.png
 home: https://www.getambassador.io/
@@ -27,5 +27,5 @@ maintainers:
 engine: gotpl
 dependencies:
   - name: emissary-ingress
-    version: 8.0.0
+    version: 8.2.0
     repository: https://s3.amazonaws.com/datawire-static-files/charts

charts/edge-stack/templates/aes-redis.yaml

@@ -90,6 +90,14 @@ spec:
         imagePullPolicy: {{ .Values.redis.image.pullPolicy }}
         resources:
         {{- toYaml .Values.redis.resources | nindent 10 }}
+        {{- if .Values.redis.containerArgs }}
+        args:
+        {{- toYaml .Values.redis.containerArgs | nindent 10 }}
+        {{- end }}
+      {{- if .Values.redis.imagePullSecrets }}
+      imagePullSecrets:
+      {{- toYaml .Values.redis.imagePullSecrets | nindent 8 }}
+      {{- end }}
       restartPolicy: Always
       {{- with .Values.redis.nodeSelector }}
       nodeSelector:

charts/edge-stack/values.yaml

@@ -70,7 +70,7 @@ emissary-ingress: # +doc-gen:break
 
   image:
     repository: docker.io/datawire/aes
-    tag: 3.0.0
+    tag: 3.2.0
     pullPolicy: IfNotPresent
 
 rbac:
@@ -150,6 +150,14 @@ redis:
   nodeSelector: {}
   affinity: {}
   tolerations: {}
+  # Arguments for the redis container
+  containerArgs: {}
+  #  - arg1
+  #  - arg2
+  # Secrets used for pulling the redis image from a private repo
+  imagePullSecrets: {}
+  #  - name: example-secret-1
+  #  - name: example-secret-2
 
 
 # Configures the AuthService that ships with the Ambassador Edge Stack.