Initial support for Host Volumes #5923

endocrimes · 2019-07-04T15:32:04Z

This is the minimum viable work for #5377.

Subsequent PRs to the f-host-volumes branch will include support for features like getting volume status from the API, further improvements to driver support, validating volumes on client start, etc.

Example Client Config

client {
  host_volume "tmp-dir" {
    path = "/tmp"
  }
}

Example Jobspec

job "example" {
  datacenters = ["dc1"]

  type = "service"

  group "cache" {
    count = 1

    volume "tmp" {
      type = "host"

      config {
        source = "tmp-dir"
      }
    }

    task "redis" {
      driver = "docker"

      config {
        image = "redis"
      }

      volume_mount {
        volume           = "tmp"
        destination      = "/tmp"
      }

      resources {
        cpu    = 500 # 500 MHz
        memory = 256 # 256MB

        network {
          mbits = 10
          port  "db"  {}
        }
      }
    }
  }
}

command/agent/testdata/basic.hcl

jobspec/test-fixtures/basic.hcl

notnoop

I did a quick skim of the PR very quickly and made few comments. I can roughly see how the pieces fit together and see the relevancy of changes, but having comments on the newly added structs and stating the convention for how volume vs mount vs host volume is used will help a lot in following the code.

I'll review again my morning! Thanks!

client/allocrunner/taskrunner/volume_hook.go

notnoop · 2019-07-29T16:03:58Z

client/allocrunner/taskrunner/volume_hook.go

+	return result
+}
+
+func (h *volumeHook) hostVolumeMountConfigurations(vmounts []*structs.VolumeMount, volumes map[string]*structs.VolumeRequest, client map[string]*structs.ClientHostVolumeConfig) ([]*drivers.MountConfig, error) {


I'm getting lost tracking the the distinction between volume mount, volume config, mount config, and volume mount config :). I'd suggest adding a comment somewhere to explain what each type is meant to represent.

Also, I'd love to clarify the parameter names here, client. Maybe naming the parameter based on source of value would be useful if they are all volumes of some kind.

ok - So I have a better handle now - Let me phrase them as best as I understand and have me confirm:

volume is the filesystem thingy that is unit of scheduling to be assigned to an alloc

host volume is the client configuration of a volume. HostVolume fields and variables in this PR refer to client config values of type ClientHostVolumeConfig (or derived from it) and never to the volumes instances themselves.

volume mount refers to the task level configuration for how the volume is mounted to the task filesystem.

mount config refers to the concrete OS bind specification that executors will use to actually bind mount?

Is this roughly correct? Would be nice to call that out in the struct documentation?

client/allocrunner/taskrunner/volume_hook.go

client/client.go

langmartin

This is just a first pass. I have an additional question: what actually causes the volume to be mounted in the task directory?

plugins/drivers/driver.go

client/allocrunner/taskrunner/volume_hook.go

notnoop

I did one more round of review. I have a high level question about the feasibility logic and the distinction between Volume and VolumeRequest.

I think I have a better handle on the names now, but I believe this PR would benefit greatly from better having documentation for the new types and methods, specially ones in structs package.

I think I'll need one more review cycle before I feel I grock this PR well.

Also, I don't expect to be addressed in this review, but wondering how the feasibility logic will evolve when we support non-host-volumes that may have different feasability logic? In this iteration, the schedule is aware of the volume type requirements, but I guess it will not be with pluggable volume types?

nomad/structs/volumes.go

scheduler/feasible.go

nomad/structs/volumes.go

scheduler/feasible.go

nomad/structs/volumes.go

notnoop · 2019-07-31T15:22:38Z

client/allocrunner/taskrunner/volume_hook.go

+	return result
+}
+
+func (h *volumeHook) hostVolumeMountConfigurations(vmounts []*structs.VolumeMount, volumes map[string]*structs.VolumeRequest, client map[string]*structs.ClientHostVolumeConfig) ([]*drivers.MountConfig, error) {


ok - So I have a better handle now - Let me phrase them as best as I understand and have me confirm:

volume is the filesystem thingy that is unit of scheduling to be assigned to an alloc

host volume is the client configuration of a volume. HostVolume fields and variables in this PR refer to client config values of type ClientHostVolumeConfig (or derived from it) and never to the volumes instances themselves.

volume mount refers to the task level configuration for how the volume is mounted to the task filesystem.

mount config refers to the concrete OS bind specification that executors will use to actually bind mount?

Is this roughly correct? Would be nice to call that out in the struct documentation?

command/agent/config.go

nomad/structs/volumes.go

preetapan · 2019-08-02T16:43:02Z

scheduler/stack_oss.go

 	// Create the feasibility wrapper which wraps all feasibility checks in
 	// which feasibility checking can be skipped if the computed node class has
 	// previously been marked as eligible or ineligible. Generally this will be
 	// checks that only needs to examine the single node to determine feasibility.
 	jobs := []FeasibilityChecker{s.jobConstraint}
-	tgs := []FeasibilityChecker{s.taskGroupDrivers, s.taskGroupConstraint, s.taskGroupDevices}
+	tgs := []FeasibilityChecker{s.taskGroupDrivers, s.taskGroupConstraint, s.taskGroupHostVolumes, s.taskGroupDevices}
 	s.wrappedChecks = NewFeasibilityWrapper(ctx, s.quota, jobs, tgs)


Noting that this needs to happen in nomad enterprise as well.

preetapan · 2019-08-02T16:46:59Z

scheduler/stack_oss.go

 	// Create the feasibility wrapper which wraps all feasibility checks in
 	// which feasibility checking can be skipped if the computed node class has
 	// previously been marked as eligible or ineligible. Generally this will be
 	// checks that only needs to examine the single node to determine feasibility.
 	jobs := []FeasibilityChecker{s.jobConstraint}
-	tgs := []FeasibilityChecker{s.taskGroupDrivers, s.taskGroupConstraint, s.taskGroupDevices}
+	tgs := []FeasibilityChecker{s.taskGroupDrivers, s.taskGroupConstraint, s.taskGroupHostVolumes, s.taskGroupDevices}
 	s.wrappedChecks = NewFeasibilityWrapper(ctx, s.quota, jobs, tgs)

 	// Filter on distinct host constraints.


IS this worth adding an e2e test for (in a separate PR)?

Maybe? - I doubt it tho tbh - there's v little new behaviour overall.

Co-Authored-By: Mahmood Ali <[email protected]>

github-actions · 2023-02-05T02:18:35Z

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

endocrimes force-pushed the dani/rfc-host-volumes branch from 48b1a56 to 773e560 Compare July 4, 2019 15:34

endocrimes commented Jul 4, 2019

View reviewed changes

command/agent/testdata/basic.hcl Outdated Show resolved Hide resolved

jobspec/test-fixtures/basic.hcl Outdated Show resolved Hide resolved

endocrimes force-pushed the dani/rfc-host-volumes branch 6 times, most recently from 25eae45 to 39037d7 Compare July 23, 2019 13:07

endocrimes changed the base branch from master to f-host-volumes July 23, 2019 13:12

endocrimes changed the title ~~[WIP] Initial support for Host Volumes~~ Initial support for Host Volumes Jul 23, 2019

endocrimes force-pushed the dani/rfc-host-volumes branch from 39037d7 to b21861b Compare July 23, 2019 16:55

endocrimes requested review from cgbaker and langmartin July 24, 2019 11:44

endocrimes force-pushed the dani/rfc-host-volumes branch from 474d7c5 to a456a69 Compare July 24, 2019 12:08

endocrimes mentioned this pull request Jul 25, 2019

HostVolumes: Include volume info in nomad node status #6011

Merged

endocrimes force-pushed the dani/rfc-host-volumes branch from a456a69 to 51963fe Compare July 25, 2019 13:54

This was referenced Jul 25, 2019

HostVolumes: Support For ACLs #6012

Closed

HostVolumes: Support for ACLs #6013

Closed

endocrimes force-pushed the dani/rfc-host-volumes branch 4 times, most recently from 5935e7a to 4a1d99c Compare July 25, 2019 14:49

endocrimes added this to the 0.10.0 milestone Jul 25, 2019

endocrimes added the theme/storage label Jul 25, 2019

endocrimes force-pushed the dani/rfc-host-volumes branch from 4a1d99c to d2ee5fc Compare July 25, 2019 15:07

endocrimes requested a review from schmichael July 25, 2019 22:46

notnoop reviewed Jul 29, 2019

View reviewed changes

langmartin reviewed Jul 30, 2019

View reviewed changes

plugins/drivers/driver.go Show resolved Hide resolved

client/allocrunner/taskrunner/volume_hook.go Show resolved Hide resolved

notnoop reviewed Jul 31, 2019

View reviewed changes

notnoop reviewed Aug 1, 2019

View reviewed changes

nomad/structs/volumes.go Outdated Show resolved Hide resolved

endocrimes removed request for schmichael and cgbaker August 1, 2019 11:58

preetapan reviewed Aug 2, 2019

View reviewed changes

nomad/structs/volumes.go Outdated Show resolved Hide resolved

preetapan reviewed Aug 2, 2019

View reviewed changes

endocrimes and others added 10 commits August 5, 2019 13:43

jobspec: Add Volume and VolumeMount declarations

3729b0c

structs: Add declarations of basic structs for volume support

33ff8e7

client: Add parsing and registration of HostVolume configuration

553e573

api: Allow submission of jobs with volumes

f5dba35

scheduler: Add a feasability checker for Host Vols

2c8b16d

client: Add volume_hook for mounting volumes

c088324

Update scheduler/feasible.go

0e2683b

Co-Authored-By: Mahmood Ali <[email protected]>

structs: Unify Volume and VolumeRequest

5c8699b

HostVolumeConfig: Source -> Path

6136839

structs: Document VolumeMount

22b16de

endocrimes force-pushed the dani/rfc-host-volumes branch from bd34a58 to 22b16de Compare August 5, 2019 11:56

schmichael requested a review from langmartin August 7, 2019 15:58

langmartin approved these changes Aug 8, 2019

View reviewed changes

endocrimes merged commit a087e8e into f-host-volumes Aug 9, 2019

endocrimes mentioned this pull request Aug 9, 2019

Host Volumes Support: Rollup Edition #6100

Merged

DingoEatingFuzz mentioned this pull request Sep 21, 2019

UI support for Host Volumes #6362

Closed

github-actions bot locked as resolved and limited conversation to collaborators Feb 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial support for Host Volumes #5923

Initial support for Host Volumes #5923

endocrimes commented Jul 4, 2019 •

edited

Loading

notnoop left a comment

notnoop Jul 29, 2019

notnoop Jul 31, 2019

langmartin left a comment

notnoop left a comment

notnoop Jul 31, 2019

preetapan Aug 2, 2019

preetapan Aug 2, 2019

endocrimes Aug 3, 2019 •

edited

Loading

github-actions bot commented Feb 5, 2023

Initial support for Host Volumes #5923

Initial support for Host Volumes #5923

Conversation

endocrimes commented Jul 4, 2019 • edited Loading

Example Client Config

Example Jobspec

notnoop left a comment

Choose a reason for hiding this comment

notnoop Jul 29, 2019

Choose a reason for hiding this comment

notnoop Jul 31, 2019

Choose a reason for hiding this comment

langmartin left a comment

Choose a reason for hiding this comment

notnoop left a comment

Choose a reason for hiding this comment

notnoop Jul 31, 2019

Choose a reason for hiding this comment

preetapan Aug 2, 2019

Choose a reason for hiding this comment

preetapan Aug 2, 2019

Choose a reason for hiding this comment

endocrimes Aug 3, 2019 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Feb 5, 2023

endocrimes commented Jul 4, 2019 •

edited

Loading

endocrimes Aug 3, 2019 •

edited

Loading