Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for Issue #2141 #2155

Merged
merged 4 commits into from
Jan 24, 2018
Merged

Fix for Issue #2141 #2155

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c87bb2f
Fix for Issue #2141
brutus333 Jan 12, 2018
8c45c88
Fix for Issue #2141 - added policy file
brutus333 Jan 12, 2018
3125f93
Added disable_volume_zone_conflict variable
brutus333 Jan 18, 2018
a4d1423
Renamed variable from disable_volume_zone_conflict to volume_cross_zo…
brutus333 Jan 23, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions inventory/group_vars/k8s-cluster.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,3 +179,8 @@ local_volumes_enabled: false
## Supplementary addresses that can be added in kubernetes ssl keys.
## That can be usefull for example to setup a keepalived virtual IP
# supplementary_addresses_in_ssl_keys: [10.0.0.1, 10.0.0.2, 10.0.0.3]

## Running on top of openstack vms with cinder enabled may lead to unschedulable pods due to NoVolumeZoneConflict restriction in kube-scheduler.
## See https://github.com/kubernetes-incubator/kubespray/issues/2141
## Set this variable to true to get rid of this issue
disable_volume_zone_conflict: false
3 changes: 3 additions & 0 deletions roles/kubernetes/master/defaults/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,3 +80,6 @@ scheduler_custom_flags: []
# kubeadm settings
# Value of 0 means it never expires
kubeadm_token_ttl: 0

## Variable for influencing kube-scheduler behaviour
disable_volume_zone_conflict: false
9 changes: 8 additions & 1 deletion roles/kubernetes/master/tasks/static-pod-setup.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,13 @@

- meta: flush_handlers

- name: Write kube-scheduler policy file
template:
src: kube-scheduler-policy.yaml.j2
dest: "{{ kube_config_dir }}/kube-scheduler-policy.yaml"
tags:
- kube-scheduler

- name: Write kube-scheduler kubeconfig
template:
src: kube-scheduler-kubeconfig.yaml.j2
Expand Down Expand Up @@ -39,4 +46,4 @@
tags:
- kube-controller-manager

- meta: flush_handlers
- meta: flush_handlers
18 changes: 18 additions & 0 deletions roles/kubernetes/master/templates/kube-scheduler-policy.yaml.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"kind" : "Policy",
"apiVersion" : "v1",
"predicates" : [
Copy link
Contributor

@bradbeam bradbeam Jan 17, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels like these are things that would be worthwhile to abstract into variables, but I'm not super familiar with the use case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if there are of any use outside Openstack deployment scenario. In Openstack you may have different AZ setup for compute and storage, not sure if there is any other cloud with the same behavior.
And usually you don't want to disable default scheduler restrictions, do you?

{"name" : "PodFitsHostPorts"},
{"name" : "PodFitsResources"},
{"name" : "NoDiskConflict"},
{"name" : "MatchNodeSelector"},
{"name" : "HostName"}
],
"priorities" : [
{"name" : "LeastRequestedPriority", "weight" : 1},
{"name" : "BalancedResourceAllocation", "weight" : 1},
{"name" : "ServiceSpreadingPriority", "weight" : 1},
{"name" : "EqualPriority", "weight" : 1}
],
"hardPodAffinitySymmetricWeight" : 10
}
13 changes: 13 additions & 0 deletions roles/kubernetes/master/templates/manifests/kube-scheduler.manifest.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ spec:
- scheduler
- --leader-elect=true
- --kubeconfig={{ kube_config_dir }}/kube-scheduler-kubeconfig.yaml
{% if cloud_provider == 'openstack' and disable_volume_zone_conflict %}
- --policy-config-file={{ kube_config_dir }}/kube-scheduler-policy.yaml
{% endif %}
- --profiling=false
- --v={{ kube_log_level }}
{% if kube_feature_gates %}
Expand Down Expand Up @@ -62,6 +65,11 @@ spec:
- mountPath: "{{ kube_config_dir }}/kube-scheduler-kubeconfig.yaml"
name: kubeconfig
readOnly: true
{% if cloud_provider == 'openstack' and disable_volume_zone_conflict %}
- mountPath: "{{ kube_config_dir }}/kube-scheduler-policy.yaml"
name: kube-scheduler-policy
readOnly: true
{% endif %}
volumes:
- name: ssl-certs-host
hostPath:
Expand All @@ -77,3 +85,8 @@ spec:
- name: kubeconfig
hostPath:
path: "{{ kube_config_dir }}/kube-scheduler-kubeconfig.yaml"
{% if cloud_provider == 'openstack' and disable_volume_zone_conflict %}
- name: kube-scheduler-policy
hostPath:
path: "{{ kube_config_dir }}/kube-scheduler-policy.yaml"
{% endif %}