-
Notifications
You must be signed in to change notification settings - Fork 558
Support Windows Server 2019 and make it default #4299
Support Windows Server 2019 and make it default #4299
Conversation
Codecov Report
@@ Coverage Diff @@
## master #4299 +/- ##
==========================================
+ Coverage 55.43% 55.45% +0.02%
==========================================
Files 109 109
Lines 16050 16053 +3
==========================================
+ Hits 8897 8902 +5
+ Misses 6369 6368 -1
+ Partials 784 783 -1 |
Well, I didn't break Linux :) |
The clusters seem to deploy ok.
Adding better error logging so I can fix the tests I broke. |
Pods start, services work
|
I believe the windows e2e tests use this file for cluster creation: https://github.com/Azure/acs-engine/blob/master/examples/e2e-tests/kubernetes/windows/hybrid/definition.json Which is why the tests are failing, as it doesn't have a sku. Will want to make sure to default a sku if not provided in the cluster definition json. |
Thanks @jsturtevant - will look into that. It looks like the SKU is currently defined in the ARM template which affects the deployment, but isn't known by the acs-engine tests. |
Looks like I'm on a good track. I need to schedule a test pass on Windows Server version 1803 to make sure I didn't break those tests.
The |
/lgtm |
This is a problem - looking into it
It fails with
|
🤦♂️ |
Alright, think I found the last reference to Windows OS version **ci/circleci: k8s-windows-1.11-release-e2e — Your tests passed on CircleCI! ** |
/hold |
/remove hold Tests seem ok on 1803! Ran 22 of 32 Specs in 1479.677 seconds |
@@ -119,6 +123,35 @@ type Status struct { | |||
ContainerStatuses []ContainerStatus `json:"containerStatuses"` | |||
} | |||
|
|||
// ReplaceContainerImageFromFile loads in a YAML, finds the image: line, and replaces it with the value of containerImage | |||
func ReplaceContainerImageFromFile(filename, containerImage string) (string, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jackfrancis @CecileRobertMichon @tariq1890 - any feedback here? if there's another place where you're manipulating existing YAML I could use it instead. Otherwise this is good to merge.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also - I chose to do this rather than changing the YAML to a template. The docs use the same YAML so I wanted it to still work with kubectl apply …
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
kubectl apply
is a patch operation correct? Can we not just initialize a pod object and perform a patch operation on the existing cluster using client-go
?
_, err = tmpFile.Write([]byte(outString)) | ||
return tmpFile.Name(), err | ||
} | ||
|
||
// CreatePodFromFile will create a Pod from file with a name | ||
func CreatePodFromFile(filename, name, namespace string, sleep, duration time.Duration) (*Pod, error) { | ||
cmd := exec.Command("kubectl", "apply", "-f", filename) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tariq1890 kubectl apply
can actually substitute as kubectl create
. In this test it's used in place of create. I'd like to avoid creating a deployment, then patching it because if the wrong thing is deployed first - it will still be pulled which can be a 10GB mistake
d54a812
to
5d8989b
Compare
@PatrickLang did I mess up the iis scale tests during rebase? they seem to be taking a long time... |
nevermind, these really do take a long time :) Thanks for seeing this through @PatrickLang! |
* add support for k8s v1.12.0-rc.1 (Azure#3872) * Adding DeleteApp func to AzureClient and returning appObjectID in CreateApp (Azure#3869) * Update AKS base image to 0.15.0 (Azure#3870) * Disable AKS VHD for sovereign clouds (Azure#3874) * disable outbound internet check (Azure#3878) * Optimizing template conditional blocks in K8s templates (Azure#3871) * Enforce windows password complexity requirements in acs-engine client… (Azure#3854) * Enforce windows password complexity requirements in acs-engine client. Azure#2407 Added Windows Agent password complexity check as per guidelines in https://docs.microsoft.com/en-us/windows/security/threat-protection/security-policy-settings/password-must-meet-complexity-requirements. This will prevent generation of arm templates if password complexity for windows vm does not meet the complexity requirements. * Addressing review comments: 1). Adding negative test cases to ensure passwords whose complexity is not met are getting rejected. 2). Error message reworded to convey the password complexity requirements enforced currently by the implemented regex * Adding two test cases - password with 0 length and password same as username * Replace deprecated Azure SDK method calls (Azure#3881) * Adding test case for Generate Cluster ID (Azure#3879) * Add availability zone support for masters (Azure#3864) * use dockerhub.akscn.io in mooncake (Azure#3887) * Update docs for AZ (Azure#3886) * Update docs for AZ * Address comments * E2E - enable focused tests (Azure#3885) * re-enable CSE 50 (Azure#3892) * add support for k8s v1.12.0-rc.2 (Azure#3893) See https://github.com/kubernetes/kubernetes/releases/tag/v1.12.0-rc.2 * Handle iterated subtest execution correctly (Azure#3894) * freshen ubuntu image (Azure#3898) * Actually allow cloudprovider rate limit / backoff disabling (Azure#3891) * Extracting property values to make ARM output variables accessible (Azure#3877) * Fixing unreported gosimple lints (Azure#3901) * Update azure-sdk-for-go to v21.0.0 (Azure#3884) * Update azure-sdk-for-go to v20.2.0 See https://github.com/Azure/azure-sdk-for-go/releases/tag/v20.2.0 * Update azure-sdk-for-go to v21.0.0 See https://github.com/Azure/azure-sdk-for-go/releases/tag/v21.0.0 * Windows dns connectivity - e2e tests (Azure#3760) * Add shortcuts for some common command-line arguments (Azure#3904) * Add zip package to VHD image (Azure#3912) * Add zip package to VHD image * Alphabetize packages to install with apt-get * Update AKS base image to 0.16.0 (Azure#3913) * Stop ginkgo tests after first failure (Azure#3922) * Perform JSON escaping of strings (Azure#3919) * removed duplication of shellQuote function and added test cases. (Azure#3927) * Change 'windowsVersion' to 'imageVersion' in docs for deploying specific windows version (Azure#3928) * Add support for Kubernetes 1.9.11 (Azure#3934) See https://github.com/kubernetes/kubernetes/releases/tag/v1.9.11 * Simplify some upgrader version cases (Azure#3924) * Use `echo -n` to skip adding newline to external command output (Azure#3940) * Add warning message for VMSS master deployments (Azure#3936) * Add Kubernetes 1.12.0 to VHD image (Azure#3942) * Migrating Get Addon By Name and Get Container Index By Name methods (Azure#3938) * Fix accidentally shadowed variable in upgrade cluster. (Azure#3943) * Docs Docs Docs! Adding windowsAgent apimodel parameters (Azure#3939) * change default value for osImage (Azure#3944) * Add support for Kubernetes 1.12.0 (Azure#3918) * ip-masq-agent as addon (Azure#3916) * Update AKS base image to 0.17.0 (Azure#3949) Adds support binaries for Kubernetes 2.0.0 and 1.9.11. * Move utility methods to the helper package (Azure#3948) * exit 3 means resource group doesn’t exist (Azure#3954) * AKS distro is for Kubernetes only (Azure#3951) * use westus2 for swarm tests (Azure#3956) * add basic distro tests for swarm, swarmmode, dcos (Azure#3957) * Update go-dev tools image for go 1.11.1 (Azure#3947) * Refactor VM prefix to template functions (Azure#3925) * Migrating cloud spec config to api package (Azure#3953) * Accelerated networking for Windows (Azure#3908) * Add support for Kubernetes 1.12.1 (Azure#3963) * Cleanup Packer directory after VHD build (Azure#3964) * can't move the same file twice (Azure#3965) * sudo sudo sudo (Azure#3967) * retain existing AKS SNAT implementation (Azure#3966) * create cgroups needed by kubelet's --system-reserved and --kube-reserved flags (Azure#3915) * Dont set default distro when OSType is Windows (Azure#3950) * vmss needs systemConf too (Azure#3970) * update AKS VHD image to ver 0.18.0 (Azure#3969) * Fix urls to gofi.sh (Azure#3973) * Strengthen unit tests for cluster ID (Azure#3972) * optimize customData payload by removing comments (Azure#3971) * bump default from 1.8 to 1.10 (Azure#3946) * Using local rand object to generate cluster ids (Azure#3978) * Update node-labels to 1.6+ standard (Azure#3980) * E2E: retry kubectl delete job (Azure#3981) * E2E: actually fail when no InternalIP, ssh master tweaks, delete retries (Azure#3982) * 1.12 uses coredns (Azure#3987) * Refactor: Moving set defaults logic from package acsengine to package api (Azure#3974) * Enable the kubelet-monitor systemd unit (Azure#3983) * k8s component tests should happen before api tests (Azure#3991) * add kubernetes1.12 example (Azure#3992) * gosimple fixes (Azure#3993) * Azure CNI v1.0.12 (Azure#3989) * bump etcd version (Azure#3975) * swarmm = swarm mode (Azure#3995) * Update apiversion to make it consistent in k8s templates (Azure#3909) * E2E: set stability iterations to 10 by default (Azure#3997) * kube-dns 1.14.13 for k8s 1.8 and up (Azure#4004) * update kubernetes-dashboard to 1.10 (Azure#4005) * only schedule coredns pods on a linux node (Azure#4014) * add coredns image reference to components versions map (Azure#3998) * Remove redundant exechealthz references (Azure#4012) * health-monitor script doesn’t require docker (Azure#4028) * Updating the tag for omsagent container to use the latest production tag (Azure#4015) * Replace docker images with the official releases. (Azure#4026) * Fix linter errors reported by gosimple (Azure#4031) * Split Windows setup scripts, prepare for cleanup and multiple CRI (Azure#3994) * Image version bump (Azure#4033) * Doc style, minor updates pass (Azure#4017) * acsengine and deploy pass * Clean up the main README * pass over the kubernetes walkthrough doc - I think maybe just azure specific bits should stay here? * Review changes * Fix typo in prometheus-grafana-k8s extension (Azure#4039) * Add support for Kubernetes 1.10.9 (Azure#4040) See https://github.com/kubernetes/kubernetes/releases/tag/v1.10.9 * Add support for Kubernetes v1.13.0-alpha.1 (Azure#4036) * Fix the Authorization and ManagedIdentity api versions (Azure#4048) * schedule ip-masq-agent on masters (Azure#4049) * delay docker and kubelet health monitors for 30 mins (Azure#4050) * Don't block on Kubernetes installation cleanup operations (Azure#4056) * update to latest AKS VHD image (Azure#4054) * set default masterSubnet value for custom VNET (Azure#4058) * Updating oms agent tag to use the latest tag released (Azure#4059) * Don't test k8s 1.8 or 1.9 in CircleCI (Azure#4061) * Don't test k8s 1.8 or 1.9 in CircleCI * Add k8s 1.13 jobs to build_and_test_master task Also reordered the jobs so that maintainers are less likely to forget about adding both Linux and Windows jobs. * Azure CNI 1.0.12 should be in VHD image (Azure#4067) * 16.04:latest by default for Ubuntu distro flows (Azure#4068) * E2E: enable pod-svc connection test (Azure#4062) * E2E: reuse long-running apache pod, HPA stabilization (Azure#4073) * Update doc: keyvault-flexvol addon default flag (Azure#4072) * use latest AKS VHD (Azure#4074) * E2E: general hardening (Azure#4079) * test scale down as well (Azure#4087) * remove unused nsg for AKS (Azure#4085) * fix error log message (Azure#4088) * Fix issue where kubernetesDashboard params weren't being added despite e enabling the dashboard addon (Azure#4084) * use latest tag for flexvol versions (Azure#4091) * set FailureActions for docker, kubelet, kubeproxy (Azure#3905) * no more default stability test iterations (Azure#4095) * Update vmss master EncryptionWithExternalKms with userassignedidentity (Azure#4082) * update azure-npm to v1.0.13 (Azure#4094) * apt lock hygiene (Azure#4081) * Add userassignedidentity for EncryptionWithExternalKms (Azure#4089) * use azk8s.cn instead of akscn.io (Azure#4099) * Fix calico for k8s 1.12 (Azure#4090) * enable user-configurable Azure CNI URL (Azure#4097) * Fix standard lb with vmss master (Azure#4101) * Don't require vm tags (Azure#4100) * Moby container runtime (Azure#3896) * minor template optimization in kubernetesmastervarsvmss.t (Azure#4112) * Fix potential nil pointer dereference when VM tags are empty (Azure#4117) * add resilience to nvidia driver install/config (Azure#4113) * don’t timeout for apt (Azure#4121) * only install GPU if docker-engine (Azure#4122) * Make --profiling user configurable (Azure#4114) * suppressing sensitive openssl output (Azure#4123) * Configure Docker Version on Windows (Azure#4119) Tests passed. merging. thanks! * disable kubelet health monitor (Azure#4127) * Add support for Kubernetes v1.13.0-alpha.2 (Azure#4128) See https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.13.md#v1130-alpha2 * Add support for Kubernetes 1.11.4 (Azure#4130) See https://github.com/kubernetes/kubernetes/releases/tag/v1.11.4 * adjust pagefile size (Azure#4098) * Add support for Kubernetes 1.12.2 (Azure#4131) See https://github.com/kubernetes/kubernetes/releases/tag/v1.12.2 * Add external custom yaml for manifests (Azure#4092) * add AKSDockerEngine distro (Azure#4120) * Adding DOCKER_API_VERSION workaround (Azure#4141) * Adding DOCKER_API_VERSION workaround * Fix extra character added * restore exechealthz references (Azure#4145) * re-use ILB test deployment (Azure#4147) * Enable k8s features by default (Azure#4133) * Enable k8s features by default * Add test * Optimize CSE + FeatureFlags option for run in background (Azure#4104) * add VHD images w/ k8s 1.11.4 and k8s 1.12.2 (Azure#4146) * use china mirror in binary downloading (Azure#4137) * Make windows binary url configurable (Azure#4103) * Move the role assignment to the ARM template and fix api versions (Azure#4032) * Merging kubernetesmastervarsvmss into kubernetesmastervars (Azure#4116) * virtualNetworkName is needed for vmss masters (Azure#4159) * vmss masters listen on firstConsecutiveStaticIP (Azure#4162) * rationalize addons/kube-system e2e checks (Azure#4166) * vmss masters customvnet dependson lb (Azure#4167) * Remove unreachable NSG code (Azure#4164) * move k8s specific params to params_k8s.go (Azure#4156) * delete empty file (Azure#4180) * add Skip functionality for skipped tests (Azure#4181) * Output kernel version during VHD build (Azure#4176) * updated VHDs for aks and aks-docker-engine distros (Azure#4178) * distinct outbound test for mooncake clusters (Azure#4169) * update dependencies to point to latest k8s api release (Azure#4157) * Pre-pull hyperkube in VHD (Azure#4174) * use gcr.azk8s.cn for ip-masq-agent on Azure China (Azure#4190) * remove unnecessary bytes (Azure#4187) * cleanUpContainerImages (Azure#4195) * Add AKS container images to VHD build script (Azure#4194) * Update ip assignment and cert gen for vmss masters (Azure#4193) * Azure CNI v1.0.13 (Azure#4197) * Enable upgrade to next supported Kubernetes version (Azure#3968) * reduce customData overhead via streamlined boilerplate (Azure#4183) * Fix 1.8 cluster config (Azure#4200) * More validations for custom vnet and vmss masters (Azure#4199) * fix cilium cluster config (Azure#4202) * Calico support for azure-vnet-ipam (Azure#4154) * Update VHD image to 2018.11.06 (Azure#4201) * move kubeserviceCidr params to windowsparams tpl (Azure#4203) * Only cleanup AKS container images if cluster is not a hosted master cluster (Azure#4204) * 2 units of errata (Azure#4205) * setting default Images in addon defaults instead of params_k8s.go (Azure#4208) * mount xtables lock in proxy (Azure#4210) * test outbound for URLs that we know we need (Azure#4211) * imagePullPolicy: IfNotPresent for all versioned containers (Azure#4212) * don’t save _output as artifacts (Azure#4214) * fix standard lb (Azure#4217) * ensure N series clusters get aks-docker-engine (Azure#4221) * ensure addon image is overwritten during upgrade (Azure#4224) * Update to Docker 18.09 for Windows (Azure#4223) * ensure validate-dns job doesn’t already exist before creating (Azure#4230) * remove empty customdata yml file (Azure#4231) * adding back in double quotes one at a time (Azure#4235) * azure npm addon has differently named pods (Azure#4237) * E2E: ensure long-running-apache hpa doesn’t already exist before creating (Azure#4232) * Adding c:\tmp as needed to pass Kubernetes tests (Azure#4240) * up image to 1108 (Azure#4239) * Add no outbound internet feature flag (Azure#4222) * update azure-const.sh with new location of azure constants python file (Azure#4247) * Tigera Technical Advisory TTA-2018-001 (Azure#4244) * Enable pre-rendering of Container addons (Azure#4218) * Make orchestrator command Windows aware (Azure#4142) * Enable multiple Windows vmss agent pools - refactor pool names (Azure#3907) * consistent use of kubernetes image base (Azure#4233) * remove extraneous sed statements for mooncake (Azure#4253) * Add exechealthz to 1.12/13 section as 1.11 or earlier (Azure#4252) * append bug means we aren’t cleaning up! (Azure#4255) * *string PrincipalID needs to be nil-guarded (Azure#4258) * fix retrycmd_if_failure: $retries should be $r (Azure#4263) * Add DockerEngine feature flag (Azure#4262) * updating azureconst and adding PB6 skus (Azure#4265) * Fix outbound connection check for master VMSS (Azure#4267) * use mcr repos and disable smb flexvol addon (Azure#4266) * bash func definition needs () without “function” (Azure#4269) * Replace docker engine feature flag by existing cloud spec (Azure#4270) * remove DockerEngine FeatureFlag (Azure#4275) * E2E: rationalize node check + kube-system check, no kms (Azure#4273) * install gpu drivers before extracting hyperkube (Azure#4276) * [docs] Add documentation for GPU w/ docker-engine (Azure#4268) * Windows e2e scale up / down test Fixes#3632 (Azure#4264) * remove dead code. (Azure#4282) * remove one extra english paragraph in zh-cn readme. (Azure#4281) * rollback k8s client-go deps to v7.0.0 (Azure#4291) * Fix issue caused by updating azure.json (Azure#4279) * enable typha and add horizontal autoscaler (Azure#4290) * feat(perf): Invoke-WebRequest much slower then browser download (Azure#4294) * Set progresspreference to avoid progress bar and speed up downloads (Azure#4300) * Ensure we do have an error before testing it (Azure#4301) * update client-go to v9 (Azure#4296) * Update to Azure-CNI v1.0.14 (Azure#4297) * Make AvailabilitySet profile for master use Availability Zones (Azure#4286) * Updates from aks-engine spike (Azure#4302) * Fix prow set up * e2e changes * removing openshift artifacts * accelerated networking rationalization, with tests * remove additional sed statements for ip-masq addons * Update go-dev tools image for go 1.11.2 * remove unused azconst methods * add support PB6 vm skus * update azure_const unit test * update tiller versions in the recent versions of kubernetes * VSTS VHD pipeline hosted ubuntu pool * azureconst cruft * scale: persist scale down in api model * Add support for Kubernetes 1.11.5 * Fix docker-engine install in VHD pipeline * remove IsOpenShift from E2E * replace premature aks-engine reference * make validate-headers doesn’t exist, revert rename * all outbound checks are retried (Azure#4304) * fix bunch of warnings for arm templates. (Azure#4285) * Adding doc on how to set Azure CNI versions (Azure#4293) * Support Windows Server 2019 and make it default (Azure#4299) * fix malformed clusterautoscaler yaml bug (Azure#4322) * update kubernetes api to 1.12.3 (Azure#4315) * Prune non-go files from vendoring (Azure#4320) * Prune non-go files from vendoring * Work around errors from gosimple linter * Bump cluster-autoscaler to recommended version for 1.11.5 (Azure#4314) * Add kubelet system-reserved on Windows (Azure#3999) * Add system-reserved on Windows * Remove extra quotes * Add system-reserved on Windows * Update to match usage in #69960 * Re-escape quotes * Just add system reserved as planned at 2Gb * Bump VHD version to 2018.11.28 (Azure#4323) * Add test for docker based workflow (ContainerInventory) (Azure#4198) * Add copyright headers to source files (Azure#4324) * Add Copyright header * Add Copyright header to more files * Rearrange finicky package comments and enforce validate-headers in CI * Remove some extraneous diffs * Rename Makefile target to be more descriptive * deprecation notice (Azure#4335) * Use 2018.12.03 VHD images (Azure#4333) * we need newline (Azure#4341) * [BUG] orchestratorVersion should not get changed for ACS scale apiVersion 2017-07-01 (Azure#4346) * Enable Azure CNI 1.0.15 (Azure#4361) * clarified docs (Azure#4362) * chore: add config for "stale" bot service (Azure#4364) * chore: add config for "stale" bot service * fix: make PRs stale after a week This repo is deprecated and shouldn't be getting any PRs. * fix: rename stale config to have ".yml" extension
What this PR does / why we need it:
Windows Server 2019 was released on Azure a few weeks ago. This PR adds support for 2019 and makes it the default
This also updates the tests to work on 1803 and 1809/2019.
If applicable: