Change install command's default behaviour to upgrade packages by default

docs/quickstart.rst

@@ -11,7 +11,7 @@ Install a package from `PyPI`_:
     [...]
     Successfully installed SomePackage
 
-Install a package already downloaded from `PyPI`_ or got elsewhere.
+Install a package already downloaded from `PyPI`_ or obtained via other means.
 This is useful if the target machine does not have a network connection:
 
 ::
@@ -39,11 +39,11 @@ List what packages are outdated:
   $ pip list --outdated
     SomePackage (Current: 1.0 Latest: 2.0)
 
-Upgrade a package:
+To upgrade an existing package, simply run the install command again:
 
 ::
 
-  $ pip install --upgrade SomePackage
+  $ pip install SomePackage
     [...]
     Found existing installation: SomePackage 1.0
     Uninstalling SomePackage:

docs/reference/pip_install.rst

@@ -758,13 +758,6 @@ Examples
       $ pip install -r requirements.txt
 
 
-#. Upgrade an already installed `SomePackage` to the latest from PyPI.
-
-    ::
-
-      $ pip install --upgrade SomePackage
-
-
 #. Install a local project in "editable" mode. See the section on :ref:`Editable Installs <editable-installs>`.
 
     ::

docs/user_guide.rst

@@ -472,32 +472,29 @@ $ pip install --no-index --find-links=DIR -r requirements.txt
 "Only if needed" Recursive Upgrade
 **********************************
 
-``pip install --upgrade`` is currently written to perform an eager recursive
-upgrade, i.e. it upgrades all dependencies regardless of whether they still
-satisfy the new parent requirements.
+``pip install``, now, defaults to performs a "only if needed" recursive
+upgrade, i.e. it upgrades dependencies only when they no longer satisfy the
+new parent requirements. This was not always the case.
 
-E.g. supposing:
+Before pip 9.0, the upgrade behaviour defaulted to a different "eager"
+upgrade, i.e. it used to upgrade all dependencies regardless of whether they
+already satisfied the new parent requirements. This behaviour was seen as
+problematic and was replaced with the current behaviour in pip 9.0. This
+section contained instructions on how to work around the problematic
+behaviour. The recommended fix for that behaviour, now, is to upgrade to
+pip 9.0 or greater.
 
-* `SomePackage-1.0` requires `AnotherPackage>=1.0`
-* `SomePackage-2.0` requires `AnotherPackage>=1.0` and `OneMorePackage==1.0`
-* `SomePackage-1.0` and `AnotherPackage-1.0` are currently installed
-* `SomePackage-2.0` and `AnotherPackage-2.0` are the latest versions available on PyPI.
+See :issue:`59` and :issue:`3786` for the discussion on this change in
+behaviour.
 
-Running ``pip install --upgrade SomePackage`` would upgrade `SomePackage` *and*
-`AnotherPackage` despite `AnotherPackage` already being satisfied.
 
-pip doesn't currently have an option to do an "only if needed" recursive
-upgrade, but you can achieve it using these 2 steps::
+As an historic note, the fix for the behaviour was::
 
-  pip install --upgrade --no-deps SomePackage
-  pip install SomePackage
+    pip install --upgrade --no-deps SomePackage
+    pip install SomePackage
 
-The first line will upgrade `SomePackage`, but not dependencies like
-`AnotherPackage`.  The 2nd line will fill in new dependencies like
-`OneMorePackage`.
-
-See :issue:`59` for a plan of making "only if needed" recursive the default
-behavior for a new ``pip upgrade`` command.
+A proposal for an ``upgrade-all`` command is being considered as a safer
+alternative to the earlier behaviour of eager upgrading.
 
 
 User Installs
@@ -565,21 +562,13 @@ From within a real python, where ``SomePackage`` *is* installed globally, but is
 
   $ pip install --user SomePackage
   [...]
-  Requirement already satisfied (use --upgrade to upgrade)
-
-  $ pip install --user --upgrade SomePackage
-  [...]
   Successfully installed SomePackage
 
 
 From within a real python, where ``SomePackage`` *is* installed globally, and is the latest version::
 
   $ pip install --user SomePackage
   [...]
-  Requirement already satisfied (use --upgrade to upgrade)
-
-  $ pip install --user --upgrade SomePackage
-  [...]
   Requirement already up-to-date: SomePackage
 
   # force the install
@@ -657,7 +646,7 @@ You can then install from the archive like this::
 
     $ tempdir=$(mktemp -d /tmp/wheelhouse-XXXXX)
     $ (cd $tempdir; tar -xvf /path/to/bundled.tar.bz2)
-    $ pip install --force-reinstall --ignore-installed --upgrade --no-index --no-deps $tempdir/*
+    $ pip install --force-reinstall --ignore-installed --no-index --no-deps $tempdir/*
 
 Note that compiled packages are typically OS- and architecture-specific, so
 these archives are not necessarily portable across machines.

pip/commands/install.py

@@ -67,9 +67,7 @@ def __init__(self, *args, **kw):
             metavar='dir',
             default=None,
             help='Install packages into <dir>. '
-                 'By default this will not replace existing files/folders in '
-                 '<dir>. Use --upgrade to replace existing packages in <dir> '
-                 'with new versions.'
+                 'This will not replace existing files/folders in <dir>.'
         )
 
         cmd_opts.add_option(
@@ -87,17 +85,14 @@ def __init__(self, *args, **kw):
             '-U', '--upgrade',
             dest='upgrade',
             action='store_true',
-            help='Upgrade all specified packages to the newest available '
-                 'version. This process is recursive regardless of whether '
-                 'a dependency is already satisfied.'
+            help='No-op option. Kept for backwards compatibility.'
         )
 
         cmd_opts.add_option(
             '--force-reinstall',
             dest='force_reinstall',
             action='store_true',
-            help='When upgrading, reinstall all packages even if they are '
-                 'already up-to-date.')
+            help='Reinstall all packages even if they are already up-to-date.')
 
         cmd_opts.add_option(
             '-I', '--ignore-installed',
@@ -268,7 +263,6 @@ def run(self, options, args):
                     build_dir=build_dir,
                     src_dir=options.src_dir,
                     download_dir=options.download_dir,
-                    upgrade=options.upgrade,
                     as_egg=options.as_egg,
                     ignore_installed=options.ignore_installed,
                     ignore_dependencies=options.ignore_dependencies,
@@ -368,26 +362,12 @@ def run(self, options, args):
                 for item in os.listdir(lib_dir):
                     target_item_dir = os.path.join(options.target_dir, item)
                     if os.path.exists(target_item_dir):
-                        if not options.upgrade:
-                            logger.warning(
-                                'Target directory %s already exists. Specify '
-                                '--upgrade to force replacement.',
-                                target_item_dir
-                            )
-                            continue
-                        if os.path.islink(target_item_dir):
-                            logger.warning(
-                                'Target directory %s already exists and is '
-                                'a link. Pip will not automatically replace '
-                                'links, please remove if replacement is '
-                                'desired.',
-                                target_item_dir
-                            )
-                            continue
-                        if os.path.isdir(target_item_dir):
-                            shutil.rmtree(target_item_dir)
-                        else:
-                            os.remove(target_item_dir)
+                        logger.warning(
+                            'Skipping target directory %s as it already '
+                            'exists.',
+                            target_item_dir
+                        )
+                        continue
 
                     shutil.move(
                         os.path.join(lib_dir, item),

pip/commands/wheel.py

@@ -107,18 +107,18 @@ def check_required_packages(self):
             "'pip wheel' requires the 'wheel' package. To fix this, run: "
             "pip install wheel"
         )
+
+        need_setuptools_message = (
+            "'pip wheel' requires setuptools >= 0.8 for dist-info support. "
+            "To fix this, run: pip install setuptools>=0.8"
+        )
         pkg_resources = import_or_raise(
             'pkg_resources',
             CommandError,
-            "'pip wheel' requires setuptools >= 0.8 for dist-info support."
-            " To fix this, run: pip install --upgrade setuptools"
+            need_setuptools_message
         )
         if not hasattr(pkg_resources, 'DistInfoDistribution'):
-            raise CommandError(
-                "'pip wheel' requires setuptools >= 0.8 for dist-info "
-                "support. To fix this, run: pip install --upgrade "
-                "setuptools"
-            )
+            raise CommandError(need_setuptools_message)
 
     def run(self, options, args):
         self.check_required_packages()

pip/req/req_set.py

@@ -139,7 +139,7 @@ def prep_for_dist(self):
 
 class RequirementSet(object):
 
-    def __init__(self, build_dir, src_dir, download_dir, upgrade=False,
+    def __init__(self, build_dir, src_dir, download_dir,
                  ignore_installed=False, as_egg=False, target_dir=None,
                  ignore_dependencies=False, force_reinstall=False,
                  use_user_site=False, session=None, pycompile=True,
@@ -169,7 +169,6 @@ def __init__(self, build_dir, src_dir, download_dir, upgrade=False,
         # be combined if we're willing to have non-wheel archives present in
         # the wheelhouse output by 'pip wheel'.
         self.download_dir = download_dir
-        self.upgrade = upgrade
         self.ignore_installed = ignore_installed
         self.force_reinstall = force_reinstall
         self.requirements = Requirements()
@@ -241,6 +240,8 @@ def add_requirement(self, install_req, parent_req_name=None):
         install_req.use_user_site = self.use_user_site
         install_req.target_dir = self.target_dir
         install_req.pycompile = self.pycompile
+        install_req.is_direct = (parent_req_name is None)
+
         if not name:
             # url or path requirement w/o an egg fragment
             self.unnamed_requirements.append(install_req)
@@ -396,17 +397,20 @@ def _check_skip_installed(self, req_to_install, finder):
         # Check whether to upgrade/reinstall this req or not.
         req_to_install.check_if_exists()
         if req_to_install.satisfied_by:
-            skip_reason = 'satisfied (use --upgrade to upgrade)'
-            if self.upgrade:
-                best_installed = False
+            upgrade_allowed = req_to_install.is_direct
+
+            # Is the best version is installed.
+            best_installed = False
+
+            if upgrade_allowed:
                 # For link based requirements we have to pull the
                 # tree down and inspect to assess the version #, so
                 # its handled way down.
                 if not (self.force_reinstall or req_to_install.link):
                     try:
-                        finder.find_requirement(req_to_install, self.upgrade)
+                        finder.find_requirement(
+                            req_to_install, upgrade_allowed)
                     except BestVersionAlreadyInstalled:
-                        skip_reason = 'up-to-date'
                         best_installed = True
                     except DistributionNotFound:
                         # No distribution found, so we squash the
@@ -423,6 +427,17 @@ def _check_skip_installed(self, req_to_install, finder):
                         req_to_install.conflicts_with = \
                             req_to_install.satisfied_by
                     req_to_install.satisfied_by = None
+
+            # Figure out a nice message to say why we're skipping this.
+            if best_installed:
+                skip_reason = 'already up-to-date'
+            elif not upgrade_allowed:
+                # NOTE: Change this message if someday the upgrade strategy
+                #       changes.
+                skip_reason = 'not upgraded as not directly required'
+            else:
+                skip_reason = 'already satisfied'
+
             return skip_reason
         else:
             return None
@@ -443,6 +458,7 @@ def _prepare_file(self,
             return []
 
         req_to_install.prepared = True
+        upgrade_allowed = req_to_install.is_direct
 
         # ###################### #
         # # print log messages # #
@@ -463,7 +479,7 @@ def _prepare_file(self,
                     'req_to_install.satisfied_by is set to %r'
                     % (req_to_install.satisfied_by,))
                 logger.info(
-                    'Requirement already %s: %s', skip_reason,
+                    'Requirement %s: %s', skip_reason,
                     req_to_install)
             else:
                 if (req_to_install.link and
@@ -519,7 +535,7 @@ def _prepare_file(self,
                         % (req_to_install, req_to_install.source_dir)
                     )
                 req_to_install.populate_link(
-                    finder, self.upgrade, require_hashes)
+                    finder, upgrade_allowed, require_hashes)
                 # We can't hit this spot and have populate_link return None.
                 # req_to_install.satisfied_by is None here (because we're
                 # guarded) and upgrade has no impact except when satisfied_by
@@ -609,7 +625,7 @@ def _prepare_file(self,
                 if not self.ignore_installed:
                     req_to_install.check_if_exists()
                 if req_to_install.satisfied_by:
-                    if self.upgrade or self.ignore_installed:
+                    if upgrade_allowed or self.ignore_installed:
                         # don't uninstall conflict if user install and
                         # conflict is not user install
                         if not (self.use_user_site and not
@@ -620,8 +636,7 @@ def _prepare_file(self,
                         req_to_install.satisfied_by = None
                     else:
                         logger.info(
-                            'Requirement already satisfied (use '
-                            '--upgrade to upgrade): %s',
+                            'Requirement already satisfied: %s',
                             req_to_install,
                         )
 

pip/utils/outdated.py

@@ -151,7 +151,7 @@ def pip_version_check(session):
             logger.warning(
                 "You are using pip version %s, however version %s is "
                 "available.\nYou should consider upgrading via the "
-                "'%s install --upgrade pip' command.",
+                "'%s install pip' command.",
                 pip_version, pypi_version, pip_cmd
             )
 

tests/data/packages/README.txt

@@ -108,3 +108,8 @@ requires_wheelbroken_upper
 --------------------------
 Requires wheelbroken and upper - used for testing implicit wheel building
 during install.
+
+require_simple-1.0.tar.gz
+------------------------
+contains "require_simple" package which requires simple>=2.0 - used for testing
+if dependencies are handled correctly.