Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use seccomp instead of setsid() to workaround CVE-2017-5226 #150

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

alexlarsson
Copy link
Collaborator

The setsid() workaround of
#143 is problematic,
because it e.g. breaks shell job control for bubblewrap instances.
So, instead we use a seccomp approach based on:
util-linux/util-linux@8e49250
However, since we don't want to pull in any more dependencies into
the setuid binary we pre-compile the seccomp code during the build.

If libseccomp is not available on your architecture, we still support
the old fix with --disable-seccomp-tty-fix.

This fixes #147

The setsid() workaround of
containers#143 is problematic,
because it e.g. breaks shell job control for bubblewrap instances.
So, instead we use a seccomp approach based on:
 util-linux/util-linux@8e49250
However, since we don't want to pull in any more dependencies into
the setuid binary we pre-compile the seccomp code during the build.

If libseccomp is not available on your architecture, we still support
the old fix with --disable-seccomp-tty-fix.

This fixes containers#147
@alexlarsson
Copy link
Collaborator Author

@smcv I tested this with your test.c from #142 and it seems to work.

@cgwalters
Copy link
Collaborator

Hmm. But this still leaves bwrap users on other arches with that behavior, which we'd still have to then document how to work around in examples, etc.

Note that util-linux reverted their seccomp change; the commit message doesn't document extensively why though.

@cgwalters
Copy link
Collaborator

I'm not opposed to this though if you think it's worth it (which I guess you do having written the patch 😄 )

@alexlarsson
Copy link
Collaborator Author

Its definately worth it, I've updated flatpak to the bubblewrap 0.1.6 in master, and it is completely useless on the command line. Whatever you do you end up with multiple processes reading from the terminal and you have to start a new terminal to find which one to kill. Its a complete show-stopper imho,

@alexlarsson
Copy link
Collaborator Author

Are there any arches that don't have seccomp yet though? If so its probably just a matter of time before they get it.

Also, I don't see how you can work around any issues with this...

@cgwalters
Copy link
Collaborator

You can work around it by creating a new pty and having the shell use that as the controlling terminal. I don't have a handy one liner for this, but e.g. running tmux seems to work just fine.

@alexlarsson
Copy link
Collaborator Author

You can work around the sh: cannot set terminal process group (-1): Inappropriate ioctl for device" error, but you can't work around the fact that if you e.g. run bwrap --bind / / sh, then ctrl-C then you get two shell reading the input from the same terminal with intermingled output. Trying to use flatpak for development and debugging just one day with this makes it obvious that it won't work at all.

@alexlarsson
Copy link
Collaborator Author

For instance, you can't do flatpak run org.my.App and expect ctrl-C to kill it.

@cgwalters
Copy link
Collaborator

We could address the Ctrl-C UX by having a process outside of the container that acts as a lifecycle bind to init inside the container. Basically both watch a pipe, and if one exits, the other does.

@alexlarsson
Copy link
Collaborator Author

That's just a single example though. The general issue is that it doesn't behave like a UNIX command.

@alexlarsson
Copy link
Collaborator Author

I.E it doesn't get sigstop when it writer to the try when backgrounded, etc


if test "x$enable_seccomp_tty_fix" = "xno"; then
AC_DEFINE([DISABLE_SECCOMP], [1],
[Define if using seccomp])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if not using

generate_seccomp_LDADD = $(LIBSECCOMP_LIBS) $(SELINUX_LIBS)

seccomp-filter.h: generate-seccomp$(EXEEXT)
./generate-seccomp$(EXEEXT) > seccomp-filter.h
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work in a cross-compilation scenario? Offhand it seems we'd generate rules for the wrong arch?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it will fail to cross-compile because the built generate-seccomp will fail to run.

@cgwalters
Copy link
Collaborator

What UNIX/commandline issues wouldn't be addressed by having an "outer init"? (I realized there's no need for a pipe, the "outer init" could just have the "container init" as a child process, and the "outer init" watches it via SIGCHILD, and container init should use PR_SET_PDEATHSIG)

@cgwalters
Copy link
Collaborator

Eh, SIGSTOP. Yeah, true. We'd have to go to proxying read/write and signals across. I guess really we'd end up with a userspace pty emulator, just without TIOCSTI.

@alexlarsson
Copy link
Collaborator Author

We could have an outer init that proxies stdin/stdout/stderror and signals (SIGSTOP/SIGCONT, etc), but it would never quite get the right thing, because e.g. /dev/tty will not work properly inside it. It also seems very complex and easy to get wrong.

@cgwalters
Copy link
Collaborator

I think for /dev/tty we could argue that bwrap-using software like flatpak should implement a shell command which would inject code into the container to allocate a pty and run a shell with it.

@cgwalters
Copy link
Collaborator

The scope for "outer init" would be a lot smaller if we specified that we didn't support ttys (e.g. TIOCGWINSZ); basically if you couldn't run vi/emacs/tmux/etc inside.

@alexlarsson
Copy link
Collaborator Author

we still have to proxy things like canonical/raw mode, etc, no?

@cgwalters
Copy link
Collaborator

Although, "outer init + setsid()" still seems viable for the basic "ctrl-c'able" case. Not sure how many people would initially notice the SIGSTOP/backgroundable bit.

@cgwalters
Copy link
Collaborator

cgwalters commented Jan 16, 2017

OK so...why explicitly generate the rules at build time? (And do you agree it breaks cross compilation?) Is there other precedent for this? systemd for example doesn't seem to do this.

@alexlarsson
Copy link
Collaborator Author

Because I don't want to load essentially a compiler into a setuid binary.

@smcv
Copy link
Collaborator

smcv commented Jan 16, 2017

we could argue that bwrap-using software like flatpak should implement a shell command which would inject code into the container to allocate a pty and run a shell with it

Here is that code: xterm :-P

(or an xterm clone with lighter-weight dependencies)

@cgwalters
Copy link
Collaborator

It is a compiler of sorts, but it's operating on known, static trusted input. I'm not sure it's worth breaking cross builds for this, though I admit to not actively maintaining a cross-built system right now myself.

@alexlarsson
Copy link
Collaborator Author

Last fixup drops the generate at build-time

@cgwalters
Copy link
Collaborator

One other concern that pops to mind - we're still vulnerable without the ptrace-after-seccomp patches, which are in 4.8, but probably not backported to kernels like the CentOS7/RHEL one.

@alexlarsson
Copy link
Collaborator Author

For flatpak we disable ptrace by default unless you run with -d or grant "developer" permissions.

@alexlarsson
Copy link
Collaborator Author

This is also disabled with ptrace, but i believe that by itself should be ok, because the ptrace-after-seccomp issue is after ptrace has been enabled, no?

@cgwalters
Copy link
Collaborator

cgwalters commented Jan 16, 2017

So...what one could argue here is we should really have a --disable-setsid runtime option. Then, this seccomp filter could live in flatpak, where it also knows that it has a filter to disable ptrace.

Conceptually, the CVE then isn't in bwrap - it's in any program which is using bwrap with a pty connected to separate security domains. There are other ways to fix the issue externally - for example, just don't provide a pty to the child process - if it's a background daemon, connect it to the e.g. systemd journal. Or, per above, install a seccomp filter in your software.

One argument that the setsid() invocation shouldn't have been added to bwrap at all is the fact that we support not passing --unshare-pid. If one does that it's obviously trivial to kill processes outside.

Another really important example is we support providing /home into the container, which may have ssh/etc key material...

So there's all of this "best practice" stuff that needs to live outside. Hence then, why don't we add a command line option --disable-setsid to allow programs to opt-in to disabling it?

@alexlarsson
Copy link
Collaborator Author

That sounds good to me, although --disable-setsid is perhaps inverted, something like --new-session might be better (or you could just run setsid bwrap ...

@cgwalters
Copy link
Collaborator

cgwalters commented Jan 16, 2017

The reason I phrased it as --disable-setsid is theoretically (very very theoretically...) there is some software out there relying on the fact we added setsid() in 0.1.6.

That said...I think I'm convincing myself we should do this:

  1. Revert the addition of setsid() (or as you suggest make it an option - why not, it's convenient)
  2. Change demos/bubblewrap-shell.sh to do it along with other hardening (unsharing all of the namespaces, explicitly not inheriting /home, perhaps too scrubbing the environment? (one often has auth tokens there))
  3. Change flatpak to add the seccomp filter, and point users at that as a best practice if you want to retain a pty

@alexlarsson
Copy link
Collaborator Author

By 1) do you mean invert its meaning? Or rely on apps to use setsid(1)?

@smcv
Copy link
Collaborator

smcv commented Jan 16, 2017

I'd prefer --disable-setsid (or maybe turn its meaning around and call it --keep-terminal) rather than --enable-setsid, so that the simplest possible use is the most-sandboxed, and you punch holes; and, in particular, so that people using bwrap for non-Flatpak things that we don't know about are protected against contained processes typing into their terminals.

That's consistent with how bwrap does filesystems (if you don't do any --bind then you don't get to see those files), although admittedly not consistent with the --unshare-foo family of options.

@alexlarsson
Copy link
Collaborator Author

Have you tried to use bwarp with setsid though? Its extremely easy to get into very confusing situations where the terminal is essentially unusable.

cgwalters added a commit to cgwalters/bubblewrap that referenced this pull request Jan 16, 2017
In discussion in containers#150
it was noted that most of the bwrap command line tends towards "closed
by default, request open".  But the `--unshare` options are inverse.

Now, I suspect in practice there's only one namespace that most users
will care about, which is the network namespace.  There are very useful
programs to build on both cases.

I think everything else (pid, ipc, uts) people will want as a group.
Any cases that are unusual enough to want to turn one of them off
can still fall back to the previous bwrap behavior of explicitly
unsharing.  They're likely to be security sensitive enough
that if a new namespace were added, it would make sense to evaluate
the tool.

But again I think most users will want all namespaces, with the network one as a
primary "enable it" option.
@alexlarsson
Copy link
Collaborator Author

So, the reason I dislike --disable-setsid is, if we ignore for the moment the CVE, that we're introducing a new default that changes the semantics of the sandbox. Suddenly we're making something that worked (such as flatpak) and essentially break it if you update bubblewrap (because, with bubblewrap 0.1.6 doing development with flatpak is basically broken).

I think most users of bubblewrap want as-secure-as-possible, but don't break my app. However, this is really really hard to guarantee for random apps, so the only guaranteed way is to add sandboxing features as default open, ask for limit.

For example, I can add a --disable-setsid to bubblewrap, and then use that from flatpak. However, this means the next flatpak has to require the newer bubblewrap (will fail with the old one with unknown switch). My plan was to do a stable bugfix-only release of flatpak and hope stable distros (like Debian 9) could just pick it up. However, thay may be foiled by having to rely on a new bubblewrap that adds new features (the switch).

@alexlarsson
Copy link
Collaborator Author

@cgwalters #154 has an approach like you suggested above instead. Then I'll add the seccomp rule to flatpak instead.

@rh-atomic-bot
Copy link

☔ The latest upstream changes (presumably a6e1516) made this pull request unmergeable. Please resolve the merge conflicts.

cgwalters added a commit to cgwalters/bubblewrap that referenced this pull request Jan 17, 2017
In discussion in containers#150
it was noted that most of the bwrap command line tends towards "closed
by default, request open".  But the `--unshare` options are inverse.

Now, I suspect in practice there's only one namespace that most users
will care about, which is the network namespace.  There are very useful
programs to build on both cases.

I think everything else (pid, ipc, uts) people will want as a group.
Any cases that are unusual enough to want to turn one of them off
can still fall back to the previous bwrap behavior of explicitly
unsharing.  They're likely to be security sensitive enough
that if a new namespace were added, it would make sense to evaluate
the tool.

But again I think most users will want all namespaces, with the network one as a
primary "enable it" option.
rh-atomic-bot pushed a commit that referenced this pull request Jan 17, 2017
In discussion in #150
it was noted that most of the bwrap command line tends towards "closed
by default, request open".  But the `--unshare` options are inverse.

Now, I suspect in practice there's only one namespace that most users
will care about, which is the network namespace.  There are very useful
programs to build on both cases.

I think everything else (pid, ipc, uts) people will want as a group.
Any cases that are unusual enough to want to turn one of them off
can still fall back to the previous bwrap behavior of explicitly
unsharing.  They're likely to be security sensitive enough
that if a new namespace were added, it would make sense to evaluate
the tool.

But again I think most users will want all namespaces, with the network one as a
primary "enable it" option.

Closes: #153
Approved by: alexlarsson
@cgwalters
Copy link
Collaborator

It's still an open question a bit to me whether we want to add any seccomp to bwrap itself.

@hartwork
Copy link
Contributor

If this gets picked up later, please note that it's not just TIOCSTI but also TIOCLINUX, see https://github.com/jwilk/ttyjack for proof and details. For anyone who also likes to play with a related seccomp filter and e.g. pass it to bubblewrap via --seccomp without need to create a BPF program by hand, please see https://github.com/hartwork/antijack .

}

if (seccomp_rule_add (ctx, SCMP_ACT_ERRNO(EPERM), SCMP_SYS(ioctl), 1,
SCMP_A1(SCMP_CMP_EQ, (int)TIOCSTI)) < 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would need SCMP_CMP_MASKED_EQ rather than SCMP_CMP_EQ to not re-introduce CVE-2019-10063. Sending TIOCSTI + 0x100000000 (eight zeros) can be used for a test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

setsid() workaround in 0.1.6 breaks interactive terminals
5 participants