Provided base ceph-rbd CSI driver example

[ryanmickler]

No description provided.

[hashicorp-cla]

All committers have signed the CLA.

[tgross]

This looks great @ryanmickler! Thanks for the contribution!

[tgross]

cc'ing @angrycub as a heads up

[ryanmickler]

whoops. i realize the title of the Readme is 'Openstack Ceph-CSI Plugin'. Probably shouldn't have mentioned Openstack there (I borrowed the format from the Openstack Cinder example for consistency)

[frederikbosch]

@ryanmickler Does this mean that Nomad is able to use Ceph RBD via CSI now? Despite that not all CSI features are implemented?

[ryanmickler]

@frederikbosch Yes it does, but I feel more testing is required. The volume creation features arent implemented yet however.

[frederikbosch]

@ryanmickler Thanks for the quick reply. So you would not take it into production if I understand correctly? Maybe I do not understand CSI well enough, but if there is no volume creation, what does that mean? That I have to create the volume manually before I can use it in a job?

[ryanmickler]

@frederikbosch Yeah, I wouldn't say its production ready yet, primarly because of the need to manually generate the correctly formatted CSI ID for each volume post creation. I do the volume creation via ceph cli interface - not ideal

[tomiles]

@ryanmickler
Can you detail how you preprovision your volumes in ceph? Do you then register the volume with the specifically formatted CSI ID as a static volume or not? And is the volume/image name in ceph the full CSI ID or just the objectUUID as in your terraform script.

I seem to be overlooking something, because using the latest nomad release and your example I can’t get it to work.

When I don’t register the volume as static in nomad it can’t find the image in ceph.
When I register it as a static volume it can find the correct volume/image in ceph and maps the rbd device on the client node. But it never gets mounted and the job stays stuck in “Pending” status.

[kriestof]

I also can not get it work (but with cephfs). If somebody could help please take a look here.

[ryanmickler]

sorry for the slow response.
I only ever got one volume to mount correctly with CSI this way, and it for it I had to manually generate the CSI ID. There was too much diff between my production code and what I submitted here so my CSI ID generation script is really lacking and untested (as i mention above)

[github-actions]

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.