Convert to standard container infrastructure

[brson]

Playpen is not well-maintained and is very security sensitive. Let's not leave ourselves open to security problems by using an unmaintained sandbox and instead use docker or similar.

[edunham]

The architecture used by codepad might be of interest here.
I agree that we'll be able to achieve a lower overall level of risk with comparable performance by refactoring the playpen to use a more comprehensively tested sandboxing solution.

When this was brought up in IRC, there were some concerns about how "containers aren't sandboxes", but actually in our case the sandbox is built on the same underlying kernel features as Docker.

[thestinger]

How is https://github.com/thestinger/playpen not well-maintained? It's the only thing that has kept https://play.rust-lang.org/ from being hacked despite no one doing the work to keep it secure. It's not going to be Playpen's fault when it does inevitably get hacked. If it used an OS container like Docker, it would have been owned months ago. Also hard to see how a ptrace sandbox would be a step forward over seccomp with namespaces with a ro root directly (not that the seccomp filter in this repository is a good one).

[thestinger]

The kernel is from January so it's vulnerable to every local root exploit found this year. Why isn't it hacked yet? AFAICT, nearly all of them aren't exploitable from within the sandbox despite the permissive whitelist. They would certainly be exploitable from Docker.

I don't spend much time working on Playpen so it doesn't have important features like parameter filtering + support for automatically adding those rules via the existing learning mode. My use case for it was the IRC and web eval stuff I made for Rust (this) and for fairly obvious reasons I don't care about that anymore. It's not like Docker is ever going to provide features like that anyway.

[thestinger]

And if I was still maintaining it... it would be using an up-to-date grsecurity/PaX kernel, not an unhardened kernel from 7 months ago.

[edunham]

@thestinger, you bring up a number of good points. As an aside, I disagree with your apparent assumption that the Play host hasn't been hacked yet; all I can say for certain from its current status is that nobody has both gained access and chosen to vandalize it.

I agree with @brson that the current playpen setup is not a viable long-term solution for 2 main reasons:

First, although playpen.c itself is distro-agnostic, rust-playpen depends on using an Arch environment [clarification: Pacstrap and Pacman] for provisioning sandboxes. Arch was a fine choice for the playpen in Rust's early days -- "the contributor who's building this prefers it" is all the reason that an experimental, toy environment needs for picking a distro. However, the project has grown a lot since then. Users depend on the playpen's availability, especially for teaching others about Rust. Arch's paradigm of constant updates and relatively-frequent reboots for maintaining a working system takes unnecessary effort to circumvent, compared to distros intended for "enterprise" production use.

Having multiple distros in Rust's production infrastructure effectively doubles the amount of effort that updates, maintenance, and security testing require. If only one person ever needed to touch the infrastructure, it could potentially make sense to standardize on Arch, but if the author or sole maintainer gets hit by a bus the systems need to be relatively easy for others to take over and keep secure.

This alone doesn't imply any need to move away from playpen.c itself, which as you point out appears to have been almost singlehandedly preventing exploits of an otherwise-vulnerable machine.

Secondly, as you mentioned in your second comment, nobody is particularly interested in developing playpen.c. I appreciate the time you've spent updating it with security fixes and minor patches, but as your own comment makes clear, it doesn't seem like you're particularly interested in continuing to work on it any more than necessary. My personal opinion is that the playpen script's security will be fine as long as you're actively keeping an eye on it. If you were unavailable for an extended period of time for any reason, ensuing CVEs could affect it in unexpected ways and I have no guarantee that anyone with the systems programming expertise to understand every line of playpen.c would notice the problem then report or fix it.

It's true that vulnerabilities are exposed and fixed more often in code that gets examined by more people, but that doesn't imply that unexamined code is magically free of flaws. I'm interested in finding a sandboxing solution that's maintained by a larger community not because playpen.c is currently malfunctioning, but in the interests of future-proofing the playpen infrastructure. It's neither fair to you nor wise of us to assume you'll permanently keep maintaining a one-off script that you wrote to support a project with which you're no longer actively involved.

[thestinger]

depends on using an Arch environment for provisioning sandboxes.

That's not true.

that an experimental, toy environment needs for picking a distro.

It's not an "experimental", "toy environment" but that's certainly an apt description of the Rust language. Arch uses the latest stable releases of packages without downstream modifications after pushing them through testing repositories. It's rare for security bug fixes to be assigned a CVE, and distributions like Debian don't even backport all of the ones that are assigned one. Few distributions offer hardened kernels and that's the single most important requirement for securing something like this. There are better suited distributions for the task like Alpine Linux, but Arch works pretty well: up-to-date stable releases, official grsecurity kernel with PaX integration, etc.

Arch's paradigm of constant updates and relatively-frequent reboots for maintaining a working system takes unnecessary effort to circumvent, compared to distros intended for "enterprise" production use.

The frequency of reboots is driven by kernel updates, and the rate is the same across distributions. Arch doesn't require more reboots on a server. If you aren't frequently rebooting machines, then the infrastructure is insecure due to mismanagement (exceptions: kpatch, kexec, etc.).

Having multiple distros in Rust's production infrastructure effectively doubles the amount of effort that updates, maintenance, and security testing require. If only one person ever needed to touch the infrastructure, it could potentially make sense to standardize on Arch, but if the author or sole maintainer gets hit by a bus the systems need to be relatively easy for others to take over and keep secure.

I don't really see what any of this has to do with Playpen...

Secondly, as you mentioned in your second comment, nobody is particularly interested in developing playpen.c. I appreciate the time you've spent updating it with security fixes and minor patches, but as your own comment makes clear, it doesn't seem like you're particularly interested in continuing to work on it any more than necessary. My personal opinion is that the playpen script's security will be fine as long as you're actively keeping an eye on it. If you were unavailable for an extended period of time for any reason, ensuing CVEs could affect it in unexpected ways and I have no guarantee that anyone with the systems programming expertise to understand every line of playpen.c would notice the problem then report or fix it.

There's a difference between developing nice new features that aren't available anywhere else (auto-learning system call parameter filtering rules for flag parameters, etc.) and maintaining it. I'm certainly going to be maintaining it. I am just not going to be spending a lot of time making it into a more general purpose sandbox or making it portable to other APIs for managing control groups.

Playpen has never had a security vulnerability (unless you count the timeout being inaccurate in the past) and has no open bugs. It has been reviewed by various people other than myself. It's very simple and easy to audit because the project has very few features.

It's true that vulnerabilities are exposed and fixed more often in code that gets examined by more people, but that doesn't imply that unexamined code is magically free of flaws. I'm interested in finding a sandboxing solution that's maintained by a larger community not because playpen.c is currently malfunctioning, but in the interests of future-proofing the playpen infrastructure. It's neither fair to you nor wise of us to assume you'll permanently keep maintaining a one-off script that you wrote to support a project with which you're no longer actively involved.

Well, good luck with that. It's pretty clear that you folks don't know what you're doing when it comes to stuff like this (or language/library development, really).

Stop spreading FUD about my projects by referring to them as "unmaintained" or implying that they are insecure / unaudited without you having any clue about it. Thanks.

[thestinger]

I don't maintain Playpen because it's used by https://play.rust-lang.org/, I maintain it because it's a useful sandboxing tool. It's not a "one off script" for https://play.rust-lang.org/. I would not have made any commits or releases this year if it was.

[brson]

@thestinger Sorry for suggesting that your sandbox was not well maintained. What I meant was that this project, rust-playpen is unmaintained. I know your sandbox is very high quality, but nobody else knows how to keep rust-playpen up to date.

[DemiMarie]

Is the kernel maintained now?

[DemiMarie]

i.e. is an up-to-date kernel on each server

[alexcrichton]

Ah yes this is now done as we've migrated to using docker as a container backend. @DemiMarie right now we're running a vanilla Ubuntu Xenial AMI, so it's as up-to-date as that is! PRs and suggestions are of course always welcome to continue to improve along this front as well.