Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add new generate options for setting cpu/mem #175

Merged
merged 1 commit into from
Sep 6, 2016
Merged

add new generate options for setting cpu/mem #175

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions cmd/ocitools/generate.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,19 @@ var generateFlags = []cli.Flag{
cli.StringFlag{Name: "template", Usage: "base template to use for creating the configuration"},
cli.StringSliceFlag{Name: "label", Usage: "add annotations to the configuration e.g. key=value"},
cli.IntFlag{Name: "oom-score-adj", Usage: "oom_score_adj for the container"},
cli.Uint64Flag{Name: "linux-cpu-shares", Usage: "the relative share of CPU time available to the tasks in a cgroup"},
cli.Uint64Flag{Name: "linux-cpu-period", Usage: "the CPU period to be used for hardcapping (in usecs)"},
cli.Uint64Flag{Name: "linux-cpu-quota", Usage: "the allowed CPU time in a given period (in usecs)"},
cli.Uint64Flag{Name: "linux-realtime-runtime", Usage: "the time realtime scheduling may use (in usecs)"},
cli.Uint64Flag{Name: "linux-realtime-period", Usage: "CPU period to be used for realtime scheduling (in usecs)"},
cli.StringFlag{Name: "linux-cpus", Usage: "CPUs to use within the cpuset (default is to use any CPU available)"},
cli.StringFlag{Name: "linux-mems", Usage: "list of memory nodes in the cpuset (default is to use any available memory node)"},
cli.Uint64Flag{Name: "linux-mem-limit", Usage: "memory limit (in bytes)"},
cli.Uint64Flag{Name: "linux-mem-reservation", Usage: "memory reservation or soft limit (in bytes)"},
cli.Uint64Flag{Name: "linux-mem-swap", Usage: "total memory limit (memory + swap) (in bytes)"},
cli.Uint64Flag{Name: "linux-mem-kernel-limit", Usage: "kernel memory limit (in bytes)"},
cli.Uint64Flag{Name: "linux-mem-kernel-tcp", Usage: "kernel memory limit for tcp (in bytes)"},
cli.Uint64Flag{Name: "linux-mem-swappiness", Usage: "how aggressive the kernel will swap memory pages (Range from 0 to 100)"},
}

var generateCommand = cli.Command{
Expand Down Expand Up @@ -324,6 +337,58 @@ func setupSpec(g *generate.Generator, context *cli.Context) error {
g.SetLinuxResourcesOOMScoreAdj(context.Int("oom-score-adj"))
}

if context.IsSet("linux-cpu-shares") {
g.SetLinuxResourcesCPUShares(context.Uint64("linux-cpu-shares"))
}

if context.IsSet("linux-cpu-period") {
g.SetLinuxResourcesCPUPeriod(context.Uint64("linux-cpu-period"))
}

if context.IsSet("linux-cpu-quota") {
g.SetLinuxResourcesCPUQuota(context.Uint64("linux-cpu-quota"))
}

if context.IsSet("linux-realtime-runtime") {
g.SetLinuxResourcesCPURealtimeRuntime(context.Uint64("linux-realtime-runtime"))
}

if context.IsSet("linux-realtime-period") {
g.SetLinuxResourcesCPURealtimePeriod(context.Uint64("linux-realtime-period"))
}

if context.IsSet("linux-cpus") {
g.SetLinuxResourcesCPUCpus(context.String("linux-cpus"))
}

if context.IsSet("linux-mems") {
g.SetLinuxResourcesCPUMems(context.String("linux-mems"))
}

if context.IsSet("linux-mem-limit") {
g.SetLinuxResourcesMemoryLimit(context.Uint64("linux-mem-limit"))
}

if context.IsSet("linux-mem-reservation") {
g.SetLinuxResourcesMemoryReservation(context.Uint64("linux-mem-reservation"))
}

if context.IsSet("linux-mem-swap") {
g.SetLinuxResourcesMemorySwap(context.Uint64("linux-mem-swap"))
}

if context.IsSet("linux-mem-kernel-limit") {
g.SetLinuxResourcesMemoryKernel(context.Uint64("linux-mem-kernel-limit"))
}

if context.IsSet("linux-mem-kernel-tcp") {
g.SetLinuxResourcesMemoryKernelTCP(context.Uint64("linux-mem-kernel-tcp"))
}

if context.IsSet("linux-mem-swappiness") {
g.SetLinuxResourcesMemorySwappiness(context.Uint64("linux-mem-swappiness"))
}

var sd string
var sa, ss []string

Expand Down
39 changes: 39 additions & 0 deletions man/ocitools-generate.1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,45 @@ read the configuration from `config.json`.
**--label**=[]
Add annotations to the configuration e.g. key=value.

**--linux-cpu-shares**=CPUSHARES
Specifies a relative share of CPU time available to the tasks in a cgroup.

**--linux-cpu-period**=CPUPERIOD
Specifies a period of time in microseconds for how regularly a cgroup's access to CPU resources should be reallocated (CFS scheduler only).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

“CFS only” seems like we want a --host-specific guard to error out if someone tries to set it with --host-specific when they aren't using a CFS scheduler. But after reading over sched(7) and sched_setscheduler(2), I'm not quite sure how to do that. It looks like you check for a SCHED_OTHER policy, but sched_getscheduler is per-thread. Is the scheduler policy really set on a per-thread basis? Is a host-specific check just supposed to look for internal consistency (e.g. the config doesn't set both a CFS-only and a realtime-only parameter)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wking , I think there are two ways we can use to determine the current scheduler.

  1. Read the kernel config file, refer to https://en.wikipedia.org/wiki/Completely_Fair_Scheduler:
[hmeng@localhost boot]$ grep CONFIG_SCHED_AUTOGROUP /boot/config-4.6.4-201.fc23.x86_64 
CONFIG_SCHED_AUTOGROUP=y
  1. check the /sys/fs/cgroup/cpu dir to see whether cpu.cfs_period_us exists or not.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Fri, Aug 05, 2016 at 11:58:48AM -0700, hmeng-19 wrote:

@wking , I think there are two ways we can use to determine the current scheduler.

  1. Read the kernel config file, refer to https://en.wikipedia.org/wiki/Completely_Fair_Scheduler:
[hmeng@localhost boot]$ grep CONFIG_SCHED_AUTOGROUP /boot/config-4.6.4-201.fc23.x86_64 
CONFIG_SCHED_AUTOGROUP=y
  1. check the /sys/fs/cgroup/cpu dir to see whether cpu.cfs_period_us exists or not.

These both sound like “does your kernel support that scheduler”, but
if scheduler policy is per-thread, then how do you know the container
will use that available scheduler (and not another available
scheduler)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wking , @mrunalp , do correct me if I am wrong.
I always thought the scheduling policy is a system-level config. It is decided when you built your kernel. Right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Fri, Aug 05, 2016 at 08:25:27PM -0700, hmeng-19 wrote:

+--linux-cpu-period=CPUPERIOD

  • Specifies a period of time in microseconds for how regularly a cgroup's access to CPU resources should be reallocated (CFS scheduler only).

@wking , @mrunalp , do correct me if I am wrong.
I always thought the scheduling policy is a system-level config. It
is decided when you built your kernel. Right?

That was my impression before poking around in the man pages today.
But see also 1, which has:

Each process or thread is controlled by an associated scheduling
policy and priority…

And 2 has:

If at least one RT task is running, no other SCHED_OTHER task will
be allowed to run in any CPU.

which makes it fairly clear that you can have threads/processes with
different schedulers running concurrently. I'm not sure how that
works for the cgroup settings though. Maybe you can set them all
(both real-time and CFS parameters) and the kernel will apply the
appropriate limits to each task depending on its policy? In that
case, configs that specify settings for multiple schedulers are fair
game.


**--linux-cpu-quota**=CPUQUOTA
Specifies the total amount of time in microseconds for which all tasks in a cgroup can run during one period.

**--linux-realtime-runtime**=REALTIMERUNTIME
Specifies a period of time in microseconds for the longest continuous period in which the tasks in a cgroup have access to CPU resources.

**--linux-realtime-period**=REALTIMEPERIOD
Sets the CPU period to be used for realtime scheduling (in usecs). Same as **--linux-cpu-period** but applies to realtime scheduler only.

**--linux-cpus**=CPUS
Sets the CPUs to use within the cpuset (default is to use any CPU available).

**--linux-mems**=MEMS
Sets the list of memory nodes in the cpuset (default is to use any available memory node).

**--linux-mem-limit**=MEMLIMIT
Sets the limit of memory usage in bytes.

**--linux-mem-reservation**=MEMRESERVATION
Sets the soft limit of memory usage in bytes.

**--linux-mem-swap**=MEMSWAP
Sets the total memory limit (memory + swap) in bytes.

**--linux-mem-kernel-limit**=MEMKERNELLIMIT
Sets the hard limit of kernel memory in bytes.

**--linux-mem-kernel-tcp**=MEMKERNELTCP
Sets the hard limit of kernel TCP buffer memory in bytes.

**--linux-mem-swappiness**=MEMSWAPPINESS
Sets the swappiness of how the kernel will swap memory pages (Range from 0 to 100).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're going to specify a range here, we probably want code in SetLinuxResourcesMemorySwappiness to enforce it (e.g. erroring out for values > 100).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wking , good points. In fact, there are several other fields having ranges we can enforce in the generate library.
Do you mind of opening an issue on it? I will fix it in a separate PR. This PR just aims to add the command options.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wking, @mrunalp , I am not sure how much overlap ocitools/generate and ocitools/validate should have in these fields.
We may want to have a wider reflection about the functionalities of these two ocitools subcommands. Anyway, I think opening an issue should a good start.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Fri, Aug 05, 2016 at 10:49:39AM -0700, hmeng-19 wrote:

+--linux-mem-swappiness=MEMSWAPPINESS

  • Sets the swappiness of how the kernel will swap memory pages (Range from 0 to 100).

@wking , good points. In fact, there are several other fields having ranges we can enforce in the generate library.
Do you mind of opening an issue on it?

Filed as #186.


**--mount**=*PATH*
Use a mount namespace where *PATH* is an existing mount namespace file
to join. The special *PATH* empty-string creates a new namespace.
Expand Down