v0.12.0
Secure Kubernetes
This is the first minor release that includes "breaking" security hardening additions. Specifically, we are now shipping etcd v3 w/ TLS communications enforced between both client/server (k8s cluster and etcd API, and between etcd peer nodes themselves. This is an important cluster security story, but its current implementation does not provide automatic backwards-compatible cluster operations for clusters deployed with prior versions of acs-engine. In practice this means upgrade operations against preëxisting clusters will not work with v0.12.0
.
Also, etcdctl
commands requiring auth (e.g., etcdctl cluster-health
) now require sudo
privileges to properly establish communications with the protected private key.
On that note we'll be calling out security-related items going forward!
Generic Kubernetes Configuration Interfaces
This release introduces generic configuration interfaces for Kubernetes cluster deployments for the kubelet, controller-manager, and apiserver run-time components. These conveniences (1) allow for more explicit configuration declarations, where appropriate, and more importantly (2) enable user-provided configuration values for the various Kubernetes components without changing acs-engine code (where acs-engine itself does not enforce an opinionated requirement).
This change also includes some breaking changes for existing kubernetesConfig
property usage patterns using the vlabs api model paradigm:
HardEvictionThreshold
,NodeStatusUpdateFrequency
, andNonMasqueradeCidr
properties have been moved to the newkubernetesConfig.kubeletConfig
configuration object. E.g.:
"kubernetesConfig": {
<...>
"kubeletConfig": {
"--eviction-hard": "memory.available<250Mi,nodefs.available<20%,nodefs.inodesFree<10%",
"--node-status-update-frequency": "1m",
"--non-masquerade-cidr": "10.0.0.0/8"
}
<...>
}
CtrlMgrNodeMonitorGracePeriod
,CtrlMgrPodEvictionTimeout
, andCtrlMgrRouteReconciliationPeriod
properties have been moved to the newkubernetesConfig.controllerManagerConfig
configuration object. E.g.:
"kubernetesConfig": {
<...>
"controllerManagerConfig": {
"--node-monitor-grace-period": "40s",
"--pod-eviction-timeout": "5m0s",
"--route-reconciliation-period": "10s"
}
<...>
}
KUBERNETES SECURITY 🔐
- etcd v3 w/ TLS (PRs #1934, #1929, #1958 @CecileRobertMichon)
- secure communication between API server and kubelet (PR #1978 @pidah)
- etcd encryption at rest (PR #1973 @pidah)
- add DenyEscalatingExec admission controller (PR #1961 @pidah)
- https dashboard (v1.9 only) (PR #1947 @karataliu)
- disable profiling (PRs #1940, #1941 @brendanburns)
- disable read-only port & don't keep terminated pod volumes (PR #1942 @brendanburns)
- add CIS-recommended security options (PR #1989 @JunSun17)
FEATURES: 🌈
- Kubernetes v1.9.0 and v1.9.1 support (PRs #1893, #2006 @jackfrancis)
- Kubernetes v1.8.6 support (PR #1979 @jackfrancis)
- Kubernetes v1.7.12 support (PR #1983 @jackfrancis)
- Generic Kubernetes configuration interfaces for kubelet, apiserver, controller-manager, and cloud-controller-manager (PRs #1854, #1960, #2012, #2017, #2030, #2033, #2034 @jackfrancis)
- Enable AAD groups (PR #2037 @mirthy)
- Updated kube-addon-manager to v6.5 (PR #1982 @jackfrancis)
- custom bootstrap URLs for DC/OS (PR #1952 @yakman2020)
- custom Windows image for DC/OS (PR #2004 @yakman2020)
- Prometheus-grafana extension on k8s agent nodes (PR #1959 @ritazh)
- enable RBAC by default for Kubernetes clusters (PR #1962 @jackfrancis)
- persist journald logs on Kubernetes nodes (PR #1956 @feiskyer)
- enable default flexvolume plugin directory on Windows (Kubernetes) (PRs #1967, #1991 @andy)
- enable hostport functionality in Kubernetes (PR #1999 @jackfrancis)
- enable upgrade to v1.9 (PR #1997 @dmitsh)
- update to 16.04.201801050 Ubuntu image (PR #2031 @jackfrancis)
BUGS FIXED: 🐞
- Kubernetes addons cleanup follow-up (PR #1965, @JunSun17)
- Add timeout to Kubernetes upgrade (PR #1986 @JackQuincy)
- Handle nil service (PR #1990 @CecileRobertMichon)
- Handle scale operations where len of vms is < 1 (PR #2059 @CecileRobertMichon)
DEV IMPROVEMENTS/CHORES: 🔧
- Simplify Kubernetes addons template implementation (PR #1946 @JunSun17)
- Rationalize vendor/ directory (PR #2040 @ultimateboy)
- Maintain Windows k8s image build script (PRs #1963, #1981, #1994 @JiangtianLi)
- Update GCR URLs (PRs #1964, #2042 @mboersma)
- Update az constants (PR #1984 @jackfrancis)
- Rationalize common utils libs (PR #2010, @feiskyer)
- Update translation files (PR #2026 @JiangtianLi)
- Windows prefix for DC/OS hostnames (PR #2023 @yakman2020)
- Docs updates!
- PR #1951 @jackfrancis
- PR #1933 @jackfrancis
- PR #2041 @jackfrancis
- CI/E2E maintenance/improvements!
- PR #1921 @jchauncey
- PR #1953 @jackfrancis
- PR #2001 @jackfrancis
- PR #2009 @jackfrancis
- PR #2011 @jchauncey