Status of testing of Apache Airflow Helm Chart 1.9.0rc1

[jedcunningham]

We are kindly requesting that contributors to Apache Airflow Helm Chart 1.9.0rc1 help test the RC.

Please let us know by commenting if the issue is addressed in the latest RC.

• Enable trigger logging in webserver (#27758): @dstandish
• Helm deployment fails when postgresql.nameOverride is used (#29214): @evanrosebrook @amoghrajesh
  Linked issues:
  • Helm deployment fails when postgresql.nameOverride is used (#22790)
• Adding possibility for annotations in logs pvc (#29270): @do-it-tim @amoghrajesh
  Linked issues:
  • annotations in logs pvc (#28745)
• Bugfix/global volumes in pod template file (#29295): @JulesTriomphe
  Linked issues:
  • Bugfix/global volumes in pod template file and formatting (#29288)
  • ExtraVolumeMounts in sidecars and initContainers (#27420)
• Fix airflow_dags_mount formatting (#29296): @JulesTriomphe @jedcunningham
• Fix webserver.service.ports formatting (#29297): @JulesTriomphe @jedcunningham
• Add ttlSecondsAfterFinished to migrateDatabaseJob and createUserJob (#29314): @calebwoofenden @alexandermalyga
  Linked issues:
  • Helm chart tries to patch immutable Job resources on helm upgrade (#27561)
• Template extra volumes in helm chart (#29357): @planoe
• Add gitSync optional env description (#29378): @1yangsh
• Add log groomer sidecar to triggerer service (#29392): @dstandish
• Add webserver service.type NodePort Description. (#29460): @jcpark3797
• Fix bitnami/postgresql default username and password (#29478): @dnskr
• fix: mount kerberos.keytab to worker when enable (#29526): @mikegit21 @npsables
  Linked issues:
  • Keytab file is not mounted in worker-kerberos container (#24369)
• Fix adding annotations for dag persistence PVC (#29622): @amoghrajesh
  Linked issues:
  • Fix adding annotations for dag persistence PVC (#29621)
• Make Liveness/Readiness Probe timeouts configurable for PgBouncer Exporter (#29752): @JeremieDoctrine @nuclear0wl
  Linked issues:
  • Pgbouncer metrics exporter restarts (#28891)
• fix: update pgbouncer and pgbouncer-exporter images (#29792): @evanasse
• updates values.yaml config.kubernetes to config.kubernetes_executor (#29818): @N-ickJones
  Linked issues:
  • Chart config section doesn't add kubernetes_executor to airflow.cfg (#29817)
• Remove replicas if KEDA is enabled (#29838): @mabrikan
• Add note about k8s version support (#29884): @jedcunningham
• Reformat chart templates (#29917): @dnskr
• Switch to newer versions of pgbouncer and pgbouncer exporter in chart (#29919): @evanasse @potiuk
  Linked issues:
  • fix: update pgbouncer and pgbouncer-exporter images (#29792)
• Reformat chart templates part 2 (#29941): @dnskr
• Introducing hostAliases for airflow webserver and scheduler (#30051): @funes79 @amoghrajesh
  Linked issues:
  • hostAliases for scheduler and webserver (#29957)
• Update scripts to make our pgbouncer-related scripts multiplatform (#30054): @potiuk
  Linked issues:
  • Build our supporting images for chart in multi-platform versions (#29967)
• Helm chart controller annotations (#30126): @bnguyennhon
• feat: add extra annotations pgbouncer (#30168): @dim-ops
• Added support for using SHA digest of Docker images. (#30214): @elongl
  Linked issues:
  • Support for providing SHA digest for the image in Helm chart (#30200)
• Chart: Block extra properties in image config (#30217): @jedcunningham
• Add additional annotations to the extra configmap and secrets (#30303): @sudeepgupta90
• Reformat chart templates part 3 (#30312): @dnskr

Thanks to all who contributed to the release (probably not a complete list!):
@mikegit21 @planoe @funes79 @evanasse @jcpark3797 @evanrosebrook @nuclear0wl @calebwoofenden @do-it-tim @dstandish @elongl @JulesTriomphe @dim-ops @potiuk @1yangsh @JeremieDoctrine @sudeepgupta90 @N-ickJones @alexandermalyga @npsables @mabrikan @jedcunningham @bnguyennhon @dnskr @amoghrajesh

[alexandermalyga]

#29314 works as expected!

[dnskr]

Fix bitnami/postgresql default username and password (#29478) - tested, works as expected.

Not sure it is possible to additionally test reformatting changes:

• Reformat chart templates (#29917)
• Reformat chart templates part 2 (#29941)
• Reformat chart templates part 3 (#30312)

[jedcunningham]

Not sure it is possible to additionally test reformatting changes

Yep, good call. I've marked them as done. Thanks!

[sudeepgupta90]

#30303 working as expected

…

On Tue, Apr 11, 2023 at 9:22 PM Jed Cunningham ***@***.***> wrote: Body We are kindly requesting that contributors to Apache Airflow Helm Chart 1.9.0rc1 <https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.9.0rc1> help test the RC. Please let us know by commenting if the issue is addressed in the latest RC. - Enable trigger logging in webserver (#27758) <#27758>: @dstandish <https://github.com/dstandish> - Helm deployment fails when postgresql.nameOverride is used (#29214) <#29214>: @evanrosebrook <https://github.com/evanrosebrook> @amoghrajesh <https://github.com/amoghrajesh> Linked issues: - Helm deployment fails when postgresql.nameOverride is used (#22790) <#22790> - Adding possibility for annotations in logs pvc (#29270) <#29270>: @do-it-tim <https://github.com/do-it-tim> @amoghrajesh <https://github.com/amoghrajesh> Linked issues: - annotations in logs pvc (#28745) <#28745> - Bugfix/global volumes in pod template file (#29295) <#29295>: @JulesTriomphe <https://github.com/JulesTriomphe> Linked issues: - Bugfix/global volumes in pod template file and formatting (#29288) <#29288> - ExtraVolumeMounts in sidecars and initContainers (#27420) <#27420> - Fix airflow_dags_mount formatting (#29296) <#29296>: @JulesTriomphe <https://github.com/JulesTriomphe> @jedcunningham <https://github.com/jedcunningham> - Fix webserver.service.ports formatting (#29297) <#29297>: @JulesTriomphe <https://github.com/JulesTriomphe> @jedcunningham <https://github.com/jedcunningham> - Add ttlSecondsAfterFinished to migrateDatabaseJob and createUserJob (#29314) <#29314>: @calebwoofenden <https://github.com/calebwoofenden> @alexandermalyga <https://github.com/alexandermalyga> Linked issues: - Helm chart tries to patch immutable Job resources on helm upgrade (#27561) <#27561> - Template extra volumes in helm chart (#29357) <#29357>: @planoe <https://github.com/planoe> - Add gitSync optional env description (#29378) <#29378>: @1yangsh <https://github.com/1yangsh> - Add log groomer sidecar to triggerer service (#29392) <#29392>: @dstandish <https://github.com/dstandish> - Add webserver service.type NodePort Description. (#29460) <#29460>: @jcpark3797 <https://github.com/jcpark3797> - Fix bitnami/postgresql default username and password (#29478) <#29478>: @dnskr <https://github.com/dnskr> - fix: mount kerberos.keytab to worker when enable (#29526) <#29526>: @mikegit21 <https://github.com/mikegit21> @npsables <https://github.com/npsables> Linked issues: - Keytab file is not mounted in worker-kerberos container (#24369) <#24369> - Fix adding annotations for dag persistence PVC (#29622) <#29622>: @amoghrajesh <https://github.com/amoghrajesh> Linked issues: - Fix adding annotations for dag persistence PVC (#29621) <#29621> - Make Liveness/Readiness Probe timeouts configurable for PgBouncer Exporter (#29752) <#29752>: @JeremieDoctrine <https://github.com/JeremieDoctrine> @nuclear0wl <https://github.com/nuclear0wl> Linked issues: - Pgbouncer metrics exporter restarts (#28891) <#28891> - fix: update pgbouncer and pgbouncer-exporter images (#29792) <#29792>: @evanasse <https://github.com/evanasse> - updates values.yaml config.kubernetes to config.kubernetes_executor (#29818) <#29818>: @N-ickJones <https://github.com/N-ickJones> Linked issues: - Chart config section doesn't add kubernetes_executor to airflow.cfg (#29817) <#29817> - Remove replicas if KEDA is enabled (#29838) <#29838>: @mabrikan <https://github.com/mabrikan> - Add note about k8s version support (#29884) <#29884>: @jedcunningham <https://github.com/jedcunningham> - Reformat chart templates (#29917) <#29917>: @dnskr <https://github.com/dnskr> - Switch to newer versions of pgbouncer and pgbouncer exporter in chart (#29919) <#29919>: @evanasse <https://github.com/evanasse> @potiuk <https://github.com/potiuk> Linked issues: - fix: update pgbouncer and pgbouncer-exporter images (#29792) <#29792> - Reformat chart templates part 2 (#29941) <#29941>: @dnskr <https://github.com/dnskr> - Introducing hostAliases for airflow webserver and scheduler (#30051) <#30051>: @funes79 <https://github.com/funes79> @amoghrajesh <https://github.com/amoghrajesh> Linked issues: - hostAliases for scheduler and webserver (#29957) <#29957> - Update scripts to make our pgbouncer-related scripts multiplatform (#30054) <#30054>: @potiuk <https://github.com/potiuk> Linked issues: - Build our supporting images for chart in multi-platform versions (#29967) <#29967> - Helm chart controller annotations (#30126) <#30126>: @bnguyennhon <https://github.com/bnguyennhon> - feat: add extra annotations pgbouncer (#30168) <#30168>: @dim-ops <https://github.com/dim-ops> - Added support for using SHA digest of Docker images. (#30214) <#30214>: @elongl <https://github.com/elongl> Linked issues: - Support for providing SHA digest for the image in Helm chart (#30200) <#30200> - Chart: Block extra properties in image config (#30217) <#30217>: @jedcunningham <https://github.com/jedcunningham> - Add additional annotations to the extra configmap and secrets (#30303) <#30303>: @sudeepgupta90 <https://github.com/sudeepgupta90> - Reformat chart templates part 3 (#30312) <#30312>: @dnskr <https://github.com/dnskr> Thanks to all who contributed to the release (probably not a complete list!): @mikegit21 <https://github.com/mikegit21> @planoe <https://github.com/planoe> @funes79 <https://github.com/funes79> @evanasse <https://github.com/evanasse> @jcpark3797 <https://github.com/jcpark3797> @evanrosebrook <https://github.com/evanrosebrook> @nuclear0wl <https://github.com/nuclear0wl> @calebwoofenden <https://github.com/calebwoofenden> @do-it-tim <https://github.com/do-it-tim> @dstandish <https://github.com/dstandish> @elongl <https://github.com/elongl> @JulesTriomphe <https://github.com/JulesTriomphe> @dim-ops <https://github.com/dim-ops> @potiuk <https://github.com/potiuk> @1yangsh <https://github.com/1yangsh> @JeremieDoctrine <https://github.com/JeremieDoctrine> @sudeepgupta90 <https://github.com/sudeepgupta90> @N-ickJones <https://github.com/N-ickJones> @alexandermalyga <https://github.com/alexandermalyga> @npsables <https://github.com/npsables> @mabrikan <https://github.com/mabrikan> @jedcunningham <https://github.com/jedcunningham> @bnguyennhon <https://github.com/bnguyennhon> @dnskr <https://github.com/dnskr> @amoghrajesh <https://github.com/amoghrajesh> Committer - I acknowledge that I am a maintainer/committer of the Apache Airflow project. — Reply to this email directly, view it on GitHub <#30585>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB5XLD2ANKYD2MFG2KHUFGTXAV43XANCNFSM6AAAAAAW2O37J4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[jcpark3797]

#29460 working as expected! Thanks for calling.😎

[potiuk]

The images used are multiplatform: #30054 #29919 (and newer versions) 👍

[mabrikan]

Can confirm that #29838 is working as expected.

Default

With worker.keda.enabled

[potiuk]

Comment: There is an interesting PR about default settings for probe timeouts, that seems to be following better the defintion for those probes https://github.com/apache/airflow/pull/30609/files and seem that the default settings are rejected by some Kubernetes deployments.

It's not a regression, so we can add it in the next release but maybe it's a good idea to include them. Also there is an interesting (and safe) change https://github.com/apache/airflow/pull/30609/files just added to add log groomer to dag file processor when run standalone - again, not a regression, so we can defer it but, if we decide to go rc2, might be a valuable addition for 1.9.0.

[JulesTriomphe]

#29295, #29296 and #29297 work as expected

[evanasse]

#29792 pgbouncer images are working as expected.

[jedcunningham]

Good call out on those 2 @potiuk, and I agree those are safe. That said, I'd rather get the release out today. I'm happy to do another release cycle in a few weeks to get those out sooner than later though.

[potiuk]

Fine for me

[jedcunningham]

1.9.0 is (almost) out! Thanks everyone for taking the time to test the RC!