Modify GCI mounter to enable NFSv3 #36610

jingxu97 · 2016-11-10T22:14:38Z

In order to make NFSv3 work, mounter needs to start rpcbind daemon. This
change modify mounter's Dockerfile and mounter script to start the
rpcbind daemon if it is not running on the host.

After this change, need to make push the image and update the sha number in Changelog.

This change is

vishh · 2016-11-11T20:58:43Z

cluster/gce/gci/mounter/Dockerfile

@@ -16,5 +16,5 @@ FROM ubuntu:xenial
 MAINTAINER [email protected]

 RUN apt-get update && apt-get install -y netbase nfs-common=1:1.2.8-9ubuntu12 glusterfs-client=3.7.6-1ubuntu1
-
+ADD startmounter.sh /startmounter.sh
 ENTRYPOINT ["/bin/mount"]


You need to change the entrypoint

I noticed this will be overwritten by rkt exec command. But I will change to make sure.

vishh · 2016-11-11T20:59:39Z

cluster/gce/gci/mounter/Dockerfile

@@ -16,5 +16,5 @@ FROM ubuntu:xenial
 MAINTAINER [email protected]

 RUN apt-get update && apt-get install -y netbase nfs-common=1:1.2.8-9ubuntu12 glusterfs-client=3.7.6-1ubuntu1
-
+ADD startmounter.sh /startmounter.sh


You also need to update the -exec argument passed to rkt in the mounter script to point to this script.

That's right. Somehow it is not in the commit. I will add it.

jbeda · 2016-11-15T22:12:18Z

@vishh -- looks like you got this. If you want my eyes let me know.

vishh · 2016-11-16T23:07:19Z

@jbeda Yup. I got this! Thanks!

k8s-ci-robot · 2016-11-17T07:51:51Z

Jenkins GCI GCE e2e failed for commit f6f3eb6d522b77dfee0eef25e06e1e9f5c6ef2ea. Full PR test history.

The magic incantation to run this job again is @k8s-bot gci gce e2e test this. Please help us cut down flakes by linking to an open flake issue when you hit one in your PR.

k8s-ci-robot · 2016-11-17T07:51:53Z

Jenkins Kubemark GCE e2e failed for commit f6f3eb6d522b77dfee0eef25e06e1e9f5c6ef2ea. Full PR test history.

The magic incantation to run this job again is @k8s-bot kubemark e2e test this. Please help us cut down flakes by linking to an open flake issue when you hit one in your PR.

k8s-ci-robot · 2016-11-17T07:52:21Z

Jenkins GCE e2e failed for commit f6f3eb6d522b77dfee0eef25e06e1e9f5c6ef2ea. Full PR test history.

The magic incantation to run this job again is @k8s-bot cvm gce e2e test this. Please help us cut down flakes by linking to an open flake issue when you hit one in your PR.

k8s-ci-robot · 2016-11-17T07:52:51Z

Jenkins GCE etcd3 e2e failed for commit f6f3eb6d522b77dfee0eef25e06e1e9f5c6ef2ea. Full PR test history.

The magic incantation to run this job again is @k8s-bot gce etcd3 e2e test this. Please help us cut down flakes by linking to an open flake issue when you hit one in your PR.

k8s-ci-robot · 2016-11-17T07:58:52Z

Jenkins CRI GCE Node e2e failed for commit f6f3eb6d522b77dfee0eef25e06e1e9f5c6ef2ea. Full PR test history.

The magic incantation to run this job again is @k8s-bot cri node e2e test this. Please help us cut down flakes by linking to an open flake issue when you hit one in your PR.

k8s-ci-robot · 2016-11-17T08:00:22Z

Jenkins GCE Node e2e failed for commit f6f3eb6d522b77dfee0eef25e06e1e9f5c6ef2ea. Full PR test history.

The magic incantation to run this job again is @k8s-bot node e2e test this. Please help us cut down flakes by linking to an open flake issue when you hit one in your PR.

k8s-ci-robot · 2016-11-17T08:04:54Z

Jenkins GKE smoke e2e failed for commit f6f3eb6d522b77dfee0eef25e06e1e9f5c6ef2ea. Full PR test history.

The magic incantation to run this job again is @k8s-bot cvm gke e2e test this. Please help us cut down flakes by linking to an open flake issue when you hit one in your PR.

k8s-ci-robot · 2016-11-17T08:05:23Z

Jenkins GCI GKE smoke e2e failed for commit f6f3eb6d522b77dfee0eef25e06e1e9f5c6ef2ea. Full PR test history.

The magic incantation to run this job again is @k8s-bot gci gke e2e test this. Please help us cut down flakes by linking to an open flake issue when you hit one in your PR.

jingxu97 · 2016-11-17T22:42:50Z

@k8s-bot gci gke e2e test this

jingxu97 · 2016-11-18T03:19:56Z

@vishh @mtaufen I added NFSv3 test to this PR and it passed! PTAL.

vishh · 2016-11-18T06:26:50Z

cluster/gce/gci/mounter/startmounter.sh

+   echo "Starting rpcbind: /sbin/rpcbind -w"
+   /sbin/rpcbind -w
+fi
+echo `/etc/init.d/rpcbind status`


We will be running rpcbind even when we don''t mount nfs. Is that the intended behavior?

Since this mounter script will be called only for nfs type or glusterfs, it should be ok in my option. Any concerns?

If glusterfs does not rely on rpcbind (I don't think it does, but would need to check) then ideally we would only start rpcbind when someone tries to mount nfs, correct? That said, if parsing the args and figuring out what fs type the user is trying to mount is too ugly, I can live with this as-is.

(Since we're pre-warming the mounter, this means we're always starting rpcbind on every node, which we shouldn't do)

wrt offline convo with @jingxu97 and @saad-ali we can get around the pre-warm issue by not starting rpcbind when the arg list is empty

@mtaufen @saad-ali , modified the code to address this comment. PTAL

@mtaufen @saad-ali , modified the code to address this comment. PTAL

Am I missing something? It looks like RPC bind is started if it is not already started regardless of args?

The change for checking the empty input parameters is in cluster/gce/gci/mounter/mounter

vishh · 2016-11-18T06:27:30Z

test/e2e/common/volumes.go

+			config := VolumeTestConfig{
+				namespace:   namespace.Name,
+				prefix:      "nfs",
+				serverImage: "gcr.io/google_containers/volume-nfs:0.8",


How do we know this is testing v3?

In line 421, I put /exports in the mount path, so this is testing v3. Also in serverPorts, I also have to put 20048 in serverports for v3 (which is not required by v4)

Does it use a different server image then the v4 test? Maybe we should maintain two seperate nfs-server bianries in gcr.io, one for each version?

In order to make NFSv3 work, mounter needs to start rpcbind daemon. This change modify mounter's Dockerfile and mounter script to start the rpcbind daemon if it is not running on the host. After this change, need to make push the image and update the sha number in Changelog.

k8s-ci-robot · 2016-11-22T00:43:45Z

Jenkins Bazel Build failed for commit 2a8d89e. Full PR test history.

The magic incantation to run this job again is @k8s-bot bazel test this. Please help us cut down flakes by linking to an open flake issue when you hit one in your PR.

saad-ali · 2016-11-22T19:51:19Z

cluster/gce/gci/mounter/startmounter.sh

+   echo "Starting rpcbind: /sbin/rpcbind -w"
+   /sbin/rpcbind -w
+fi
+echo `/etc/init.d/rpcbind status`


@mtaufen @saad-ali , modified the code to address this comment. PTAL

Am I missing something? It looks like RPC bind is started if it is not already started regardless of args?

saad-ali · 2016-11-22T19:52:09Z

test/e2e/common/volumes.go

+			config := VolumeTestConfig{
+				namespace:   namespace.Name,
+				prefix:      "nfs",
+				serverImage: "gcr.io/google_containers/volume-nfs:0.8",


Does it use a different server image then the v4 test? Maybe we should maintain two seperate nfs-server bianries in gcr.io, one for each version?

saad-ali · 2016-11-22T19:53:28Z

cluster/gce/gci/mounter/Makefile

@@ -12,7 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.

-TAG=v2
+TAG=v3


Did you already push this container to gcr.io?

Spoke offline. Already done.

saad-ali · 2016-11-23T02:45:24Z

LGTM

k8s-github-robot · 2016-11-23T06:53:02Z

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

k8s-github-robot · 2016-11-23T07:38:50Z

Automatic merge from submit-queue

This PR is a retry for PR kubernetes#36610

googlebot added the cla: yes label Nov 10, 2016

jingxu97 mentioned this pull request Nov 10, 2016

Modify GCI mounter to enable NFSv3 #36609

Closed

k8s-github-robot assigned jbeda Nov 10, 2016

k8s-github-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. release-note-label-needed do-not-merge DEPRECATED. Indicates that a PR should not merge. Label can only be manually applied/removed. labels Nov 10, 2016

grodrigues3 removed the do-not-merge DEPRECATED. Indicates that a PR should not merge. Label can only be manually applied/removed. label Nov 11, 2016

k8s-github-robot added the do-not-merge DEPRECATED. Indicates that a PR should not merge. Label can only be manually applied/removed. label Nov 11, 2016

apelisse removed the do-not-merge DEPRECATED. Indicates that a PR should not merge. Label can only be manually applied/removed. label Nov 11, 2016

jingxu97 force-pushed the Nov/nfsv3 branch from 4239b6a to a5ec54a Compare November 11, 2016 18:25

vishh reviewed Nov 11, 2016

View reviewed changes

jingxu97 force-pushed the Nov/nfsv3 branch from 3279f96 to b52a562 Compare November 11, 2016 23:42

jbeda assigned vishh and unassigned jbeda Nov 15, 2016

vishh assigned mtaufen Nov 16, 2016

jingxu97 force-pushed the Nov/nfsv3 branch from 89200b6 to f6f3eb6 Compare November 17, 2016 07:27

jingxu97 force-pushed the Nov/nfsv3 branch from f6f3eb6 to db47030 Compare November 17, 2016 18:19

jingxu97 force-pushed the Nov/nfsv3 branch 2 times, most recently from 68206a0 to 86fc77f Compare November 18, 2016 01:34

vishh reviewed Nov 18, 2016

View reviewed changes

jingxu97 added the sig/storage Categorizes an issue or PR as relevant to SIG Storage. label Nov 18, 2016

jingxu97 force-pushed the Nov/nfsv3 branch from 86fc77f to 2a8d89e Compare November 22, 2016 00:43

k8s-github-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 22, 2016

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Nov 22, 2016

jingxu97 added this to the v1.5 milestone Nov 22, 2016

jingxu97 removed the release-note-label-needed label Nov 22, 2016

k8s-github-robot added the release-note-label-needed label Nov 22, 2016

saad-ali reviewed Nov 22, 2016

View reviewed changes

saad-ali added release-note Denotes a PR that will be considered when it comes time to generate release notes. lgtm "Looks good to me", indicates that a PR is ready to be merged. and removed release-note-label-needed labels Nov 23, 2016

k8s-github-robot merged commit e801fcf into kubernetes:master Nov 23, 2016

piosz mentioned this pull request Nov 23, 2016

Revert "Modify GCI mounter to enable NFSv3" #37375

Merged

jingxu97 mentioned this pull request Nov 23, 2016

[k8s.io] Pod Disks Should schedule a pod w/ a RW PD, gracefully remove it, then schedule it on another host [Slow] {Kubernetes e2e suite} #28283

Closed

jingxu97 mentioned this pull request Nov 28, 2016

Modify GCI mounter to enable NFSv3 #37582

Merged

jingxu97 added a commit to jingxu97/kubernetes that referenced this pull request Nov 29, 2016

Modify GCI mounter to enable NFSv3

80f2e58

This PR is a retry for PR kubernetes#36610

chentao1596 mentioned this pull request Dec 5, 2016

WIP:kubelet: support multi-headers when getting pod from HTTP source #38089

Closed

Modify GCI mounter to enable NFSv3 #36610

Modify GCI mounter to enable NFSv3 #36610

Conversation

jingxu97 commented Nov 10, 2016 • edited by k8s-oncall Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbeda commented Nov 15, 2016

vishh commented Nov 16, 2016

k8s-ci-robot commented Nov 17, 2016

k8s-ci-robot commented Nov 17, 2016

k8s-ci-robot commented Nov 17, 2016

k8s-ci-robot commented Nov 17, 2016

k8s-ci-robot commented Nov 17, 2016

k8s-ci-robot commented Nov 17, 2016

k8s-ci-robot commented Nov 17, 2016

k8s-ci-robot commented Nov 17, 2016

jingxu97 commented Nov 17, 2016

jingxu97 commented Nov 18, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

k8s-ci-robot commented Nov 22, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saad-ali commented Nov 23, 2016

k8s-github-robot commented Nov 23, 2016

k8s-github-robot commented Nov 23, 2016

jingxu97 commented Nov 10, 2016 •

edited by k8s-oncall

Loading