Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Still got an occasional "permission denied" error while cgroup frozen failed. #3803

Closed
jiusanzhou opened this issue Apr 3, 2023 · 2 comments · Fixed by #3804
Closed

Still got an occasional "permission denied" error while cgroup frozen failed. #3803

jiusanzhou opened this issue Apr 3, 2023 · 2 comments · Fixed by #3804

Comments

@jiusanzhou
Copy link
Contributor

jiusanzhou commented Apr 3, 2023

Description

Currenty we have frozen the container to avoid the container get
an occasional "permission denied" error, while the systemd's application of device
rules is done disruptively.

But when the processes in the container can not be frozen over 2 seconds, we still update the cgroup which resulting the container get
an occasional "permission denied" error in some cases.

The code in libcontainer/cgroups/systemd/v1.go

func (m *LegacyManager) Set(r *configs.Resources) error {
	if needsFreeze {
		if err := m.doFreeze(configs.Frozen); err != nil {
			// If freezer cgroup isn't supported, we just warn about it.
			logrus.Infof("freeze container before SetUnitProperties failed: %v", err)
		}
	}
	setErr := setUnitProperties(m.dbus, unitName, properties...)
}

btw, 2 seconds set in libcontainer/cgroups/fs/freezer.go

func (s *FreezerGroup) Set(path string, r *configs.Resources) (Err error) {
	for i := 0; i < 1000; i++ {
		time.Sleep(10 * time.Microsecond)
	}
}

Steps to reproduce the issue

  1. Kubelet and containerd, which with systemd v1 as cgroup driver;
  2. Start a container which start a process that reads the urandom device 1 time per second;
  3. Start some process with D state in the container.

Describe the results you received and expected

Avoid an occasional "permission denied" error.

What version of runc are you using?

1.1.2

Host OS information

No response

Host kernel information

No response

@kolyshkin
Copy link
Contributor

What version of runc are you using?
1.0.2

This is an old version. Can you please retry with the latest released runc version, 1.1.5?

@jiusanzhou
Copy link
Contributor Author

Sorry I gave a wrong version, it's actually 1.1.2
According to the latest code, I think this issue still exists.

jiusanzhou added a commit to jiusanzhou/runc that referenced this issue Apr 6, 2023
jiusanzhou added a commit to jiusanzhou/runc that referenced this issue Apr 6, 2023
jiusanzhou added a commit to jiusanzhou/runc that referenced this issue Apr 7, 2023
In code we have frozen the cgroup to avoid the processes get
an occasional "permission denied" error, while the systemd's application of device
rules is done disruptively. When the processes in the container can not
be frozen over 2 seconds (which defined in fs/freezer.go),
we still update the cgroup which resulting the container get an occasional
"permission denied" error in some cases.

Return error directly without updating cgroup, when freeze fails.

Fixes: opencontainers#3803

Signed-off-by: Zoe <[email protected]>
jiusanzhou added a commit to jiusanzhou/runc that referenced this issue Apr 7, 2023
In code we have frozen the cgroup to avoid the processes get
an occasional "permission denied" error, while the systemd's application of device
rules is done disruptively. When the processes in the container can not
be frozen over 2 seconds (which defined in fs/freezer.go),
we still update the cgroup which resulting the container get an occasional
"permission denied" error in some cases.

Return error directly without updating cgroup, when freeze fails.

Fixes: opencontainers#3803

Signed-off-by: Zoe <[email protected]>
jiusanzhou added a commit to jiusanzhou/runc that referenced this issue Apr 17, 2023
In code we have frozen the cgroup to avoid the processes get
an occasional "permission denied" error, while the systemd's application of device
rules is done disruptively. When the processes in the container can not
be frozen over 2 seconds (which defined in fs/freezer.go),
we still update the cgroup which resulting the container get an occasional
"permission denied" error in some cases.

Return error directly without updating cgroup, when freeze fails.

Fixes: opencontainers#3803

Signed-off-by: Zoe <[email protected]>
jiusanzhou added a commit to jiusanzhou/runc that referenced this issue Apr 27, 2023
In code we have frozen the cgroup to avoid the processes get
an occasional "permission denied" error, while the systemd's application of device
rules is done disruptively. When the processes in the container can not
be frozen over 2 seconds (which defined in fs/freezer.go),
we still update the cgroup which resulting the container get an occasional
"permission denied" error in some cases.

Return error directly without updating cgroup, when freeze fails.

Fixes: opencontainers#3803

Signed-off-by: Zoe <[email protected]>
jiusanzhou added a commit to jiusanzhou/runc that referenced this issue Apr 29, 2023
In code we have frozen the cgroup to avoid the processes get
an occasional "permission denied" error, while the systemd's application of device
rules is done disruptively. When the processes in the container can not
be frozen over 2 seconds (which defined in fs/freezer.go),
we still update the cgroup which resulting the container get an occasional
"permission denied" error in some cases.

Return error directly without updating cgroup, when freeze fails.

Fixes: opencontainers#3803

Signed-off-by: Zoe <[email protected]>
jiusanzhou added a commit to jiusanzhou/runc that referenced this issue Jun 12, 2023
In code we have frozen the cgroup to avoid the processes get
an occasional "permission denied" error, while the systemd's application of device
rules is done disruptively. When the processes in the container can not
be frozen over 2 seconds (which defined in fs/freezer.go),
we still update the cgroup which resulting the container get an occasional
"permission denied" error in some cases.

Return error directly without updating cgroup, when freeze fails.

Fixes: opencontainers#3803

Signed-off-by: Zoe <[email protected]>
jiusanzhou added a commit to jiusanzhou/runc that referenced this issue Jun 30, 2023
In code we have frozen the cgroup to avoid the processes get
an occasional "permission denied" error, while the systemd's application of device
rules is done disruptively. When the processes in the container can not
be frozen over 2 seconds (which defined in fs/freezer.go),
we still update the cgroup which resulting the container get an occasional
"permission denied" error in some cases.

Return error directly without updating cgroup, when freeze fails.

Fixes: opencontainers#3803

Signed-off-by: Zoe <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants