Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core,scripts: When no cachedir is enabled in unified-core, disable FUSE #1591

Closed
wants to merge 4 commits into from

Conversation

cgwalters
Copy link
Member

This is prep for running inside (unprivileged) Kube containers
as they exist today: #1329

Sadly FUSE today uses a suid binary that ends up wanting CAP_SYS_ADMIN.
I think there's some work on FUSE-in-containers but I'm not sure of
the current status.

What rofiles-fuse here is doing here is protecting is the hardlinked
repo imports. But if --cachedir isn't specified, that repository
gets thrown away anyways. So there's no real value to using FUSE
here.

Also since nothing is cached, disable the devino cache.

Down the line ideally we gain the capability to detect if either
unprivileged overlayfs/FUSE are available. Then if --cachedir
is specified we can make things work.

@cgwalters
Copy link
Member Author

Part of #1332

* anyways.
*/
g_autoptr(GKeyFile) config = ostree_repo_copy_config (self->pkgcache_repo);
g_key_file_set_boolean (config, "rpmostree", "enable-fuse", FALSE);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we just tell the core directly not to use FUSE instead of adding the magical bit to the repo config? Feels a bit underhanded.

@@ -524,6 +524,11 @@ rpmostree_context_set_repos (RpmOstreeContext *self,
{
g_set_object (&self->ostreerepo, base_repo);
g_set_object (&self->pkgcache_repo, pkgcache_repo);

self->enable_fuse =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor/bikeshed: would enable_rofiles be a nicer name for this? Makes the correspondence to RPMOSTREE_BWRAP_MUTATE_ROFILES more obvious.

@jlebon
Copy link
Member

jlebon commented Sep 27, 2018

OK, this looks good! One concern: do we need to worry about files which map to the same file object but are transformed into different objects by scripts? I'm especially thinking about degenerate cases like empty files.

@jlebon
Copy link
Member

jlebon commented Sep 27, 2018

Hmm, one potential way to handle this in libostree would be a flag to force copy if st_nlink > 1 so only the first checkout is a hard link. Or maybe making it part of a higher level "throwaway repo" concept.

@cgwalters
Copy link
Member Author

OK, we relied on fuse though to do the /usr/etc ➡️ /etc mapping. If we don't have that, I guess we'll need to do the "rename when executing a script" dance like we do elsewhere.

@cgwalters
Copy link
Member Author

One concern: do we need to worry about files which map to the same file object but are transformed into different objects by scripts? I'm especially thinking about degenerate cases like empty files.

I don't understand this at all - since we're not using a devino cache...we'll just end up comitting whatever's in the target right? If the repo is corrupted, oh well, we're throwing it away anyways.

@cgwalters
Copy link
Member Author

Oh...like some case where files are hardlinked by libostree but aren't ordinarily hardlinks, and scripts mutating one don't expect to mutate all of them?

Right...whee, yeah we could teach libostree about that. But, I don't think it should block this, if we hit those cases I figure we can work around it.

@jlebon
Copy link
Member

jlebon commented Sep 27, 2018

Oh...like some case where files are hardlinked by libostree but aren't ordinarily hardlinks, and scripts mutating one don't expect to mutate all of them?

Exactly!

Fair enough, I'm fine for deferring this.

@rh-atomic-bot r+ 36d0c78

@cgwalters
Copy link
Member Author

This now blocks on #1592

@rh-atomic-bot
Copy link

⌛ Testing commit 36d0c78 with merge 4bb4779...

rh-atomic-bot pushed a commit that referenced this pull request Sep 28, 2018
This is prep for running inside (unprivileged) Kube containers
as they exist today: #1329

Sadly FUSE today uses a suid binary that ends up wanting CAP_SYS_ADMIN.
I think there's some work on FUSE-in-containers but I'm not sure of
the current status.

What rofiles-fuse here is doing here is protecting is the hardlinked
repo imports.  But if `--cachedir` isn't specified, that repository
gets thrown away anyways.  So there's no real value to using FUSE
here.

Also since nothing is cached, disable the devino cache.

Down the line ideally we gain the capability to detect if either
unprivileged overlayfs/FUSE are available.  Then if `--cachedir`
is specified we can make things work.

Closes: #1591
Approved by: jlebon
@cgwalters cgwalters changed the title core,scripts: When no cachedir is enabled in unified-core, disable FUSE WIP: core,scripts: When no cachedir is enabled in unified-core, disable FUSE Sep 28, 2018
@cgwalters
Copy link
Member Author

@rh-atomic-bot r-

@cgwalters
Copy link
Member Author

@rh-atomic-bot force

@rh-atomic-bot
Copy link

💔 Test failed - status-atomicjenkins

@rh-atomic-bot
Copy link

☔ The latest upstream changes (presumably 426e16e) made this pull request unmergeable. Please resolve the merge conflicts.

@jlebon jlebon added the WIP label Oct 3, 2018
@cgwalters cgwalters force-pushed the no-cachedir-no-fuse branch from 36d0c78 to 62046c2 Compare October 10, 2018 14:39
@cgwalters cgwalters changed the title WIP: core,scripts: When no cachedir is enabled in unified-core, disable FUSE core,scripts: When no cachedir is enabled in unified-core, disable FUSE Oct 10, 2018
@cgwalters cgwalters removed the WIP label Oct 10, 2018
@cgwalters
Copy link
Member Author

Rebased 🏄‍♂️ and lifting WIP. This is just another step in the puzzle though.

@jlebon
Copy link
Member

jlebon commented Oct 10, 2018

+ sed -e 's/^/# /'
# -00755 0 0 12853032 { [(b'security.selinux', b'system_u:object_r:bin_t:s0')] } /usr/bin/docker-current
+ fatal 'File '\''docker.txt'\'' doesn'\''t match fixed string list '\''system_u:object_r:container_runtime_exec_t:s0'\'''
+ echo File ''\''docker.txt'\''' 'doesn'\''t' match fixed string list ''\''system_u:object_r:container_runtime_exec_t:s0'\'''
File 'docker.txt' doesn't match fixed string list 'system_u:object_r:container_runtime_exec_t:s0'

Hmm, same pkgset as the currently passing master at least as far as SELinux-related things are concerned. Seems like not using rofiles-fuse somehow is causing /usr/bin/docker-current to be mislabeled?

@cgwalters
Copy link
Member Author

While I was working on the code I happened to see this bit:

      /* Necessary for unified core to work with semanage calls in %post, like container-selinux */
      if (!rpmostree_rootfs_fixup_selinux_store_root (tmprootfs_dfd, cancellable, error))
        return FALSE;

Which is a clearly topical comment here; annotate points to:
8c4ffc3

I also noticed, comparing the logs of good vs bad:

good:

container-selinux.post: setsebool:  SELinux is disabled.

bad:

container-selinux.post: setsebool:  SELinux is disabled.
container-selinux.post: libsemanage.semanage_get_lock: Could not get direct read lock at /etc/selinux/targeted/semanage.read.LOCK. (Resource temporarily unavailable).
container-selinux.post: /usr/sbin/semodule:  Failed!

@jlebon
Copy link
Member

jlebon commented Oct 10, 2018

Hmm, that error also echoes memories of #1002.

Hmm, actually it might very well be the same issue:

OK, this looks good! One concern: do we need to worry about files which map to the same file object but are transformed into different objects by scripts? I'm especially thinking about degenerate cases like empty files.

This would be the case for the two lock files: #999 (comment)

@cgwalters
Copy link
Member Author

OK so your comment originally was very prescient 😄

I'll see about adding a libostree API to not hardlink empty files. That should solve most pathological cases.

@cgwalters cgwalters force-pushed the no-cachedir-no-fuse branch from 1585b04 to e8e0c48 Compare October 10, 2018 21:44
@cgwalters
Copy link
Member Author

But yeah ugh:

# find /usr -size 0
/usr/etc/environment
/usr/etc/exports
/usr/etc/machine-id
/usr/etc/motd
/usr/etc/subgid
/usr/etc/subuid
/usr/etc/wvdial.conf
/usr/etc/cups/classes.conf
/usr/etc/cups/client.conf
/usr/etc/cups/lpoptions
/usr/etc/cups/printers.conf
/usr/etc/cups/subscriptions.conf
/usr/etc/kernel/install.d/20-grubby.install
/usr/etc/kernel/install.d/90-loaderentry.install
/usr/etc/pki/ca-trust/extracted/pem/objsign-ca-bundle.pem
/usr/etc/pki/tls/ct_log_list.cnf
/usr/etc/security/opasswd
/usr/etc/selinux/targeted/contexts/files/file_contexts.local
/usr/etc/selinux/targeted/contexts/files/file_contexts.subs
/usr/etc/systemd/dont-synthesize-nobody
/usr/lib/grub/i386-pc/fdt.lst
/usr/lib/locale/locale-archive.tmpl
/usr/lib/python2.7/site-packages/pip/_vendor/html5lib/filters/__init__.py
/usr/lib/python2.7/site-packages/pip/_vendor/urllib3/contrib/__init__.py
/usr/lib/python2.7/site-packages/pip/_vendor/urllib3/contrib/_securetransport/__init__.py
/usr/lib/python2.7/site-packages/pip/_vendor/urllib3/packages/backports/__init__.py
/usr/lib/python2.7/site-packages/pip/operations/__init__.py
/usr/lib/python2.7/site-packages/pkg_resources/_vendor/__init__.py
/usr/lib/python2.7/site-packages/setuptools/_vendor/__init__.py
/usr/lib/python2.7/site-packages/urllib3/contrib/__init__.py
/usr/lib/python2.7/site-packages/urllib3/contrib/_securetransport/__init__.py
/usr/lib/python2.7/site-packages/urllib3/packages/backports/__init__.py
/usr/lib/python3.6/site-packages/asn1crypto/_perf/__init__.py
/usr/lib/python3.6/site-packages/docker/models/__init__.py
/usr/lib/python3.6/site-packages/firewall/__init__.py
/usr/lib/python3.6/site-packages/firewall/core/__init__.py
/usr/lib/python3.6/site-packages/firewall/server/__init__.py
/usr/lib/python3.6/site-packages/justbytes/_util/__init__.py
/usr/lib/python3.6/site-packages/orca/backends/__init__.py
/usr/lib/python3.6/site-packages/orca/scripts/__init__.py
/usr/lib/python3.6/site-packages/pip/_vendor/html5lib/filters/__init__.py
/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/contrib/__init__.py
/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/contrib/_securetransport/__init__.py
/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/packages/backports/__init__.py
/usr/lib/python3.6/site-packages/pip/operations/__init__.py
/usr/lib/python3.6/site-packages/pkg_resources/_vendor/__init__.py
/usr/lib/python3.6/site-packages/registries/__init__.py
/usr/lib/python3.6/site-packages/setuptools/_vendor/__init__.py
/usr/lib/python3.6/site-packages/slip/__init__.py
/usr/lib/python3.6/site-packages/slip/_wrappers/__init__.py
/usr/lib/python3.6/site-packages/urllib3/contrib/__init__.py
/usr/lib/python3.6/site-packages/urllib3/contrib/_securetransport/__init__.py
/usr/lib/python3.6/site-packages/urllib3/packages/backports/__init__.py
/usr/lib64/firefox/chrome.manifest
/usr/lib64/firefox/browser/chrome.manifest
/usr/lib64/python2.7/email/mime/__init__.py
/usr/lib64/python2.7/pydoc_data/__init__.py
/usr/lib64/python3.6/email/mime/__init__.py
/usr/lib64/python3.6/pydoc_data/__init__.py
/usr/lib64/python3.6/urllib/__init__.py
/usr/share/X11/locale/am_ET.UTF-8/XI18N_OBJS
/usr/share/X11/locale/cs_CZ.UTF-8/XI18N_OBJS
/usr/share/X11/locale/el_GR.UTF-8/XI18N_OBJS
/usr/share/X11/locale/fi_FI.UTF-8/XI18N_OBJS
/usr/share/cups/templates/help-trailer.tmpl
/usr/share/cups/templates/de/help-trailer.tmpl
/usr/share/cups/templates/es/help-trailer.tmpl
/usr/share/cups/templates/fr/help-trailer.tmpl
/usr/share/cups/templates/ja/help-trailer.tmpl
/usr/share/cups/templates/pt_BR/help-trailer.tmpl
/usr/share/cups/templates/ru/help-trailer.tmpl
/usr/share/doc/NetworkManager-ssh/ChangeLog
/usr/share/doc/NetworkManager-ssh/NEWS
/usr/share/doc/ibus-libzhuyin/NEWS
/usr/share/doc/python3-slip/dbus/example/import_marker.py
/usr/share/doc/python3-slip-dbus/example/import_marker.py
/usr/share/pki/ca-trust-legacy/ca-bundle.legacy.default.crt
/usr/share/pki/ca-trust-legacy/ca-bundle.legacy.disable.crt
/usr/share/rpm/.dbenv.lock
/usr/share/skk/SKK-JISYO.tmp
/usr/share/xml/iso-codes/iso_3166-3.xml

@cgwalters
Copy link
Member Author

bot, retest this please

@cgwalters
Copy link
Member Author

ostreedev/ostree#1752

@rh-atomic-bot
Copy link

☔ The latest upstream changes (presumably 096f8de) made this pull request unmergeable. Please resolve the merge conflicts.

@cgwalters cgwalters force-pushed the no-cachedir-no-fuse branch 2 times, most recently from a089958 to a6adb0e Compare October 11, 2018 19:02
@cgwalters
Copy link
Member Author

This one needs ostreedev/ostree#1753 too.

@cgwalters
Copy link
Member Author

bot, retest this please

1 similar comment
@cgwalters
Copy link
Member Author

bot, retest this please

@cgwalters
Copy link
Member Author

OK, now passing tests ✔️.

These shouldn't be in the package; the fact that they're empty
files causes libostree to hardlink them which breaks things.
See also coreos#1002
This is basically overriding what happens with `bare-user` mode
OSTree repositories.  I put a lot of thought into avoiding creating
suid files with that mode.

But today this creates a situation where if we don't have a devino
cache, the file will lose its suid bits.

In the end, since we're using the "inaccessible directory" pattern
anyways for rpm-ostree on the host, we don't need to really worry
about transient suid binaries.  And similarly when we're run inside
an existing container, that's also fine.
This is prep for running inside (unprivileged) Kube containers
as they exist today: coreos#1329

Sadly FUSE today uses a suid binary that ends up wanting CAP_SYS_ADMIN.
I think there's some work on FUSE-in-containers but I'm not sure of
the current status.

What rofiles-fuse here is doing here is protecting is the hardlinked
repo imports.  But if `--cachedir` isn't specified, that repository
gets thrown away anyways.  So there's no real value to using FUSE
here.

Also since nothing is cached, disable the devino cache.

We also make use of --force-copy-zerosized that just landed
in libostree: ostreedev/ostree#1752

Down the line ideally we gain the capability to detect if either
unprivileged overlayfs/FUSE are available.  Then if `--cachedir`
is specified we can make things work.
@cgwalters cgwalters force-pushed the no-cachedir-no-fuse branch from 3de3f84 to 3490ce7 Compare October 12, 2018 14:41
@jlebon
Copy link
Member

jlebon commented Oct 12, 2018

@rh-atomic-bot r+ 3490ce7

@rh-atomic-bot
Copy link

⌛ Testing commit 3490ce7 with merge 9107855...

rh-atomic-bot pushed a commit that referenced this pull request Oct 12, 2018
These shouldn't be in the package; the fact that they're empty
files causes libostree to hardlink them which breaks things.
See also #1002

Closes: #1591
Approved by: jlebon
rh-atomic-bot pushed a commit that referenced this pull request Oct 12, 2018
This is basically overriding what happens with `bare-user` mode
OSTree repositories.  I put a lot of thought into avoiding creating
suid files with that mode.

But today this creates a situation where if we don't have a devino
cache, the file will lose its suid bits.

In the end, since we're using the "inaccessible directory" pattern
anyways for rpm-ostree on the host, we don't need to really worry
about transient suid binaries.  And similarly when we're run inside
an existing container, that's also fine.

Closes: #1591
Approved by: jlebon
rh-atomic-bot pushed a commit that referenced this pull request Oct 12, 2018
This is prep for running inside (unprivileged) Kube containers
as they exist today: #1329

Sadly FUSE today uses a suid binary that ends up wanting CAP_SYS_ADMIN.
I think there's some work on FUSE-in-containers but I'm not sure of
the current status.

What rofiles-fuse here is doing here is protecting is the hardlinked
repo imports.  But if `--cachedir` isn't specified, that repository
gets thrown away anyways.  So there's no real value to using FUSE
here.

Also since nothing is cached, disable the devino cache.

We also make use of --force-copy-zerosized that just landed
in libostree: ostreedev/ostree#1752

Down the line ideally we gain the capability to detect if either
unprivileged overlayfs/FUSE are available.  Then if `--cachedir`
is specified we can make things work.

Closes: #1591
Approved by: jlebon
@jlebon
Copy link
Member

jlebon commented Oct 12, 2018

@rh-atomic-bot retry

@rh-atomic-bot
Copy link

⌛ Testing commit 3490ce7 with merge b3f6f25...

rh-atomic-bot pushed a commit that referenced this pull request Oct 12, 2018
This is basically overriding what happens with `bare-user` mode
OSTree repositories.  I put a lot of thought into avoiding creating
suid files with that mode.

But today this creates a situation where if we don't have a devino
cache, the file will lose its suid bits.

In the end, since we're using the "inaccessible directory" pattern
anyways for rpm-ostree on the host, we don't need to really worry
about transient suid binaries.  And similarly when we're run inside
an existing container, that's also fine.

Closes: #1591
Approved by: jlebon
rh-atomic-bot pushed a commit that referenced this pull request Oct 12, 2018
This is prep for running inside (unprivileged) Kube containers
as they exist today: #1329

Sadly FUSE today uses a suid binary that ends up wanting CAP_SYS_ADMIN.
I think there's some work on FUSE-in-containers but I'm not sure of
the current status.

What rofiles-fuse here is doing here is protecting is the hardlinked
repo imports.  But if `--cachedir` isn't specified, that repository
gets thrown away anyways.  So there's no real value to using FUSE
here.

Also since nothing is cached, disable the devino cache.

We also make use of --force-copy-zerosized that just landed
in libostree: ostreedev/ostree#1752

Down the line ideally we gain the capability to detect if either
unprivileged overlayfs/FUSE are available.  Then if `--cachedir`
is specified we can make things work.

Closes: #1591
Approved by: jlebon
@rh-atomic-bot
Copy link

☀️ Test successful - status-atomicjenkins
Approved by: jlebon
Pushing b3f6f25 to master...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants