Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] Enhanced RDoc for FileUtils #78

Merged
merged 9 commits into from
Jun 6, 2022
Merged
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 97 additions & 44 deletions lib/fileutils.rb
Original file line number Diff line number Diff line change
Expand Up @@ -898,13 +898,15 @@ def copy_stream(src, dest)
# | `-- src.txt
# `-- src1.txt
#
# Keyword arguments:
#
# - <tt>force: true</tt> - attempts to force the move;
# if the move includes removing +src+
# (that is, if +src+ and +dest+ are on different devices),
# ignores raised exceptions of StandardError and its descendants.
# - <tt>noop: true</tt> - does not move files.
# - <tt>secure: true</tt> - removes +src+ securely
# by calling FileUtils.remove_entry_secure.
# - <tt>secure: true</tt> - removes +src+ securely;
# see details at FileUtils.remove_entry_secure.
# - <tt>verbose: true</tt> - prints an equivalent command:
#
# FileUtils.mv('src0', 'dest0', noop: true, verbose: true)
Expand Down Expand Up @@ -949,13 +951,28 @@ def mv(src, dest, force: nil, noop: nil, verbose: nil, secure: nil)
alias move mv
module_function :move

# Removes entries at the paths given in +list+,
# which should be a string path or an array of string paths; returns +list+.
jeremyevans marked this conversation as resolved.
Show resolved Hide resolved
#
# With no keyword arguments, removes files at the paths given in +list+:
#
# FileUtils.touch(['src0.txt', 'src0.dat'])
# FileUtils.rm(['src0.dat', 'src0.txt']) # => ["src0.dat", "src0.txt"]
#
# Keyword arguments:
#
# Remove file(s) specified in +list+. This method cannot remove directories.
# All StandardErrors are ignored when the :force option is set.
# - <tt>force: true</tt> - attempts to remove files regardless of permissions;
# ignores raised exceptions of StandardError and its descendants.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous description

All StandardErrors are ignored when the :force option is set.

is actually more descriptive of the current behavior. Currently the only difference between force: true and force: false is that force: true makes any StandardErrors while trying to remove files be ignored. But if a file is impossible to remove due to permissions, the removal will be attempted regardless of force, and the files won't get removed regardless of force.

For what it's worth, I think this behaviour is harmful, I opened a ruby-core ticket at https://bugs.ruby-lang.org/issues/18784. But unless the behavior is changed, the documentation should explain the current behavior.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it was not too clear what I'm suggesting. It's this:

Suggested change
# - <tt>force: true</tt> - attempts to remove files regardless of permissions;
# ignores raised exceptions of StandardError and its descendants.
# - <tt>force: true</tt> - ignores raised exceptions of StandardError and its descendants.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

# - <tt>noop: true</tt> - does not remove files; returns +nil+.
# - <tt>verbose: true</tt> - prints an equivalent command:
#
# FileUtils.rm %w( junk.txt dust.txt )
# FileUtils.rm Dir.glob('*.so')
# FileUtils.rm 'NotExistFile', force: true # never raises exception
# FileUtils.rm(['src0.dat', 'src0.txt'], noop: true, verbose: true)
#
# Output:
#
# rm src0.dat src0.txt
#
# FileUtils.remove is an alias for FileUtils.rm.
#
def rm(list, force: nil, noop: nil, verbose: nil)
list = fu_list(list)
Expand All @@ -971,10 +988,13 @@ def rm(list, force: nil, noop: nil, verbose: nil)
alias remove rm
module_function :remove

# Equivalent to:
#
# FileUtils.rm(list, force: true, **kwargs)
#
# Equivalent to
# See FileUtils.rm for keyword arguments.
#
# FileUtils.rm(list, force: true)
# FileUtils.safe_unlink is an alias for FileUtils.rm_f.
#
def rm_f(list, noop: nil, verbose: nil)
rm list, force: true, noop: noop, verbose: verbose
Expand All @@ -984,24 +1004,48 @@ def rm_f(list, noop: nil, verbose: nil)
alias safe_unlink rm_f
module_function :safe_unlink

# Removes files and directories at the paths given in array +list+;
# returns +list+.
#
# remove files +list+[0] +list+[1]... If +list+[n] is a directory,
# removes its all contents recursively. This method ignores
# StandardError when :force option is set.
# May cause a local vulnerability if not called with keyword argument
# <tt>secure: true</tt>.
jeremyevans marked this conversation as resolved.
Show resolved Hide resolved
#
# FileUtils.rm_r Dir.glob('/tmp/*')
# FileUtils.rm_r 'some_dir', force: true
# For each file path, removes the file at that path:
#
# WARNING: This method causes local vulnerability
jeremyevans marked this conversation as resolved.
Show resolved Hide resolved
# if one of parent directories or removing directory tree are world
# writable (including /tmp, whose permission is 1777), and the current
# process has strong privilege such as Unix super user (root), and the
# system has symbolic link. For secure removing, read the documentation
# of remove_entry_secure carefully, and set :secure option to true.
# Default is <tt>secure: false</tt>.
# FileUtils.touch(['src0.txt', 'src0.dat'])
# FileUtils.rm_r(['src0.dat', 'src0.txt'])
# File.exist?('src0.txt') # => false
# File.exist?('src0.dat') # => false
#
# NOTE: This method calls remove_entry_secure if :secure option is set.
# See also remove_entry_secure.
# For each directory path, recursively removes files and directories:
#
# system('tree --charset=ascii src1')
# src1
# |-- dir0
# | |-- src0.txt
# | `-- src1.txt
# `-- dir1
# |-- src2.txt
# `-- src3.txt
# FileUtils.rm_r('src1')
# File.exist?('src1') # => false
#
# Keyword arguments:
#
# - <tt>force: true</tt> - attempts to remove entries regardless of permissions;
# ignores raised exceptions of StandardError and its descendants.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto regarding force.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also fixed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't seem fixed, no? The sentence about permissions is still there. You did remove some stuff regarding force in other methods that take care of moving files, but I'm not sure how force behaves there, so I'm not sure if that's correct or not.

# - <tt>noop: true</tt> - does not remove entries; returns +nil+.
# - <tt>secure: true</tt> - removes +src+ securely;
# see details at FileUtils.remove_entry_secure.
# - <tt>verbose: true</tt> - prints an equivalent command:
#
# FileUtils.rm_r(['src0.dat', 'src0.txt'], noop: true, verbose: true)
# FileUtils.rm_r('src1', noop: true, verbose: true)
#
# Output:
#
# rm -r src0.dat src0.txt
# rm -r src1
#
def rm_r(list, force: nil, noop: nil, verbose: nil, secure: nil)
list = fu_list(list)
Expand All @@ -1017,13 +1061,16 @@ def rm_r(list, force: nil, noop: nil, verbose: nil, secure: nil)
end
module_function :rm_r

# Equivalent to:
#
# Equivalent to
# FileUtils.rm_r(list, force: true, **kwargs)
#
# FileUtils.rm_r(list, force: true)
# May cause a local vulnerability if not called with keyword argument
# <tt>secure: true</tt>.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a link to the place that explains this vulnerability?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll be reworking all about secure and vulnerabilities, putting it into the doc for the module, and adding links as needed.

#
# WARNING: This method causes local vulnerability.
jeremyevans marked this conversation as resolved.
Show resolved Hide resolved
# Read the documentation of rm_r first.
# See FileUtils.rm_r for keyword arguments.
#
# FileUtils.rmtree is an alias for FileUtils.rm_rf.
#
def rm_rf(list, noop: nil, verbose: nil, secure: nil)
rm_r list, force: true, noop: noop, verbose: verbose, secure: secure
Expand All @@ -1033,21 +1080,29 @@ def rm_rf(list, noop: nil, verbose: nil, secure: nil)
alias rmtree rm_rf
module_function :rmtree

# Securely removes the entry given by +path+,
# which should be the entry for a regular file, a symbolic link,
# or a directory.
#
# Here, "securely" means "avoiding
# {Time-of-check to time-of-use}[https://en.wikipedia.org/wiki/Time-of-check_to_time-of-use]
# vulnerabilities", which can exist when:
#
# This method removes a file system entry +path+. +path+ shall be a
# regular file, a directory, or something. If +path+ is a directory,
# remove it recursively. This method is required to avoid TOCTTOU
# (time-of-check-to-time-of-use) local security vulnerability of rm_r.
# #rm_r causes security hole when:
# - An ancestor directory of the entry at +path+ is world writable;
# such directories include <tt>/tmp</tt>.
# - The directory tree at +path+ includes:
#
# * Parent directory is world writable (including /tmp).
# * Removing directory tree includes world writable directory.
# * The system has symbolic link.
# - A world-writable descendant directory.
# - A symbolic link.
#
# To avoid this security hole, this method applies special preprocess.
# If +path+ is a directory, this method chown(2) and chmod(2) all
# removing directories. This requires the current process is the
# owner of the removing whole directory tree, or is the super user (root).
# To avoid such a vulnerability, this method applies a special pre-process:
#
# - If +path+ is a directory, this method uses
# {chown(2)}[https://man7.org/linux/man-pages/man2/chown.2.html]
# and {chmod(2)}[https://man7.org/linux/man-pages/man2/chmod.2.html]
# in removing directories.
# - The owner of +path+ should be either the current proces
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo:

Suggested change
# - The owner of +path+ should be either the current proces
# - The owner of +path+ should be either the current process

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

# or the super user (root).
#
# WARNING: You must ensure that *ALL* parent directories cannot be
# moved by other untrusted users. For example, parent directories
Expand All @@ -1058,12 +1113,10 @@ def rm_rf(list, noop: nil, verbose: nil, secure: nil)
# user (root) should invoke this method. Otherwise this method does not
# work.
#
# For details of this security vulnerability, see Perl's case:
#
# * https://cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2005-0448
# * https://cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2004-0452
# For details of this security vulnerability, see Perl cases:
#
# For fileutils.rb, this vulnerability is reported in [ruby-dev:26100].
# - {CVE-2005-0448}[https://cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2005-0448].
# - {CVE-2004-0452}[https://cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2004-0452].
#
def remove_entry_secure(path, force = false)
unless fu_have_symlink?
Expand Down