Support vmss in kubernetes deployment

[JiangtianLi]

Is this a request for help?:

---

Is this an ISSUE or FEATURE REQUEST? (choose one):
Feature

What version of acs-engine?:

---

Orchestrator and version (e.g. Kubernetes, DC/OS, Swarm)

What happened:
Kubernetes currently only supports availability set in agent pool.

What you expected to happen:
Support vmss in kubernetes agent pool.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know:

[JiangtianLi]

@feiskyer

[feiskyer]

vmss support will be added in kubernetes v1.10. To support vmss, acs-engine should ensure

• master nodes are still running in AvailabilitySet
• agent nodes are running in ScaleSet

And /etc/kubernetes/azure.json should be changed for both master and agent nodes:

{
  ...
  "vmType":"vmss",
  // remove primaryAvailabilitySetName
  "primaryScaleSetName":"<vmss-name>"
}

[timkock]

What is weird is that the kubernetes/autoscaler documentation references VMSS enabled kubernetes clusters... am I missing something or is this the edge or are they referring to some sort of managed cluster that runs vmss 😜

[feiskyer]

@timkock Yep, Azure support in autoscaler is also targeted v1.10, should be released together.

Meanwhile, some customers are already validating features with their self-deployed clusters.

[yfernando]

@feiskyer Could I check if it is going to support cross-zone ScaletSets when you mentioned "agent nodes are running in ScaleSet"

[feiskyer]

Could I check if it is going to support cross-zone ScaletSets when you mentioned "agent nodes are running in ScaleSet"

@yfernando It should support this, but I haven't validated it yet.

[forrestcoward]

I'm a bit confused, and sorry if this is a silly question, but are there changes that need to happen in ACS-Engine to support VMSS, or is this simply waiting for Kubernetes 1.10?

I want to run the Kubernetes autoscaler on a VMSS cluster created via ACS-Engine and I'm trying to figure out what configuration I need to set aside from targeting 1.10

[forrestcoward]

Ah okay. Nevermind. I tried to target VMSS but got:

�[36mINFO�[0m[0000] Error returned by LoadContainerService: VirtualMachineScaleSets are not supported with Kubernetes since Kubernetes requires the ability to attach/detach disks.  To fix specify "AvailabilityProfile":"AvailabilitySet"

[feiskyer]

@forrestcoward We are waiting both. @JiangtianLi will work on acs-engine part and kubernetes v1.10 will be released in next few days.