Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support manila fileshares (cephfs) #344

Merged
merged 22 commits into from
Jan 31, 2024
Merged

Support manila fileshares (cephfs) #344

merged 22 commits into from
Jan 31, 2024

Conversation

sjpb
Copy link
Collaborator

@sjpb sjpb commented Dec 14, 2023

  1. Adds general support for multiple manila fileshares using CephFS protocol. See groups, group_vars and role for manila.

  2. In .caas environment for Azimuth:
    a. Storage for home directories is now configurable using bool cluster_home_manila_share:

    • If true, a manila share is created and deleted with the cluster. In this case the share type must be defined with cluster_home_manila_share_type unless the a default share type is defined in OpenStack.
    • If false, an OpenStack volume attached to control node and NFS exported to cluster (pre-PR behaviour)

    In both cases home_volume_size defines the size (in GB) of the relevant storage

    b. A project share can be enabled by setting bool cluster_project_manila_share: true. This is expected to already exist (i.e. its lifecycle is not tied to the cluster). Its name is defined by cluster_project_manila_share_name, defaulting to azimuth-project-share. See environments/.caas/inventory/group_vars/all/manila.yml.

  3. In skeleton terraform (used by new cookiecutter environments and stackhpc CI environment):
    a. home_volume_size can be set to 0 to not create a home volume. This allows e.g. a manila share to be externally defined or defined in additional terraform
    b. Manila client (default nautilus version) installed in fat image.

@sjpb
Copy link
Collaborator Author

sjpb commented Dec 14, 2023

Checked share works ok after reboot

@sjpb sjpb force-pushed the feat/manila branch 6 times, most recently from 0ed5076 to 80a8222 Compare January 16, 2024 14:38
@sjpb
Copy link
Collaborator Author

sjpb commented Jan 16, 2024

Fat image build w/ (default nautilus) manila client: https://github.com/stackhpc/ansible-slurm-appliance/actions/runs/7543193442

@sjpb
Copy link
Collaborator Author

sjpb commented Jan 16, 2024

Fat image build: https://github.com/stackhpc/ansible-slurm-appliance/actions/runs/7544263443 openhpc-240116-1604-b3563a08

@sjpb sjpb marked this pull request as ready for review January 17, 2024 10:07
@sjpb sjpb requested a review from a team as a code owner January 17, 2024 10:07
@sjpb
Copy link
Collaborator Author

sjpb commented Jan 17, 2024

Example additional config on arcus rcp-cloud-portal-demo:

azimuth_caas_cluster_templates_overrides:
  slurm-manila:
    gitUrl: https://github.com/stackhpc/ansible-slurm-appliance.git
    gitVersion: feat/manila
    uiMetaUrl: https://raw.githubusercontent.com/stackhpc/ansible-slurm-appliance/feat/manila/environments/.caas/ui-meta/slurm-infra-manila.yml
    playbook: ansible/site.yml
    extraVars:
      cluster_image: 6a7b4a9c-b142-4a35-8590-d8950d28123c # "openhpc-240116-1604-b3563a08" # https://github.com/stackhpc/ansible-slurm-appliance/pull/344
      login_flavor_name: vm.ska.cpu.general.small
      control_flavor_name: vm.ska.cpu.general.small
      cluster_state_volume_size: 40
      cluster_home_manila_share: true
      cluster_home_manila_share_type: ceph01_cephfs # no default defined in arcus rcp-cloud-portal-demo at least
      cluster_project_manila_share: true
    envVars:
      # Normally set through environment's activate script:
      ANSIBLE_INVENTORY: environments/common/inventory,environments/.caas/inventory # NB: Relative to runner project dir

Copy link
Member

@JohnGarbutt JohnGarbutt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My only real worry is clashing with this change:
#348

ansible/roles/cluster_infra/templates/resources.tf.j2 Outdated Show resolved Hide resolved
ansible/roles/cluster_infra/templates/resources.tf.j2 Outdated Show resolved Hide resolved
environments/.caas/ui-meta/slurm-infra-manila.yml Outdated Show resolved Hide resolved
@sjpb sjpb requested a review from JohnGarbutt January 18, 2024 10:35
@sjpb sjpb requested a review from JohnGarbutt January 30, 2024 11:39
Copy link
Member

@JohnGarbutt JohnGarbutt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove that ui-meta, after that consider the approved, awesome work on getting this sorted!

environments/.caas/ui-meta/slurm-infra-manila.yml Outdated Show resolved Hide resolved
Copy link
Member

@JohnGarbutt JohnGarbutt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thank you

@sjpb sjpb merged commit f3f60b9 into main Jan 31, 2024
1 check failed
@sjpb sjpb deleted the feat/manila branch January 31, 2024 09:10
@mkjpryor mkjpryor restored the feat/manila branch February 1, 2024 16:22
@sjpb sjpb mentioned this pull request Feb 7, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants