From bad82c5dcc73657a97c410ad8c16470a68c7142f Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Thu, 3 Mar 2022 16:08:31 +0000 Subject: [PATCH 01/14] Add links to MANIFEST.in docs and clarify data files inclusion --- docs/setuptools.rst | 28 ++++++++++++++++++++++-- docs/userguide/datafiles.rst | 41 ++++++++++++++++++++++++------------ 2 files changed, 53 insertions(+), 16 deletions(-) diff --git a/docs/setuptools.rst b/docs/setuptools.rst index d0fb9a9cec..53cf54b264 100644 --- a/docs/setuptools.rst +++ b/docs/setuptools.rst @@ -21,8 +21,9 @@ Feature Highlights: individually in setup.py * Automatically include all relevant files in your source distributions, - without needing to create a ``MANIFEST.in`` file, and without having to force - regeneration of the ``MANIFEST`` file when your source tree changes. + without needing to create a |MANIFEST.in|_ file, and without having to force + regeneration of the ``MANIFEST`` file when your source tree changes + [#manifest]_. * Automatically generate wrapper scripts or Windows (console and GUI) .exe files for any number of "main" functions in your project. (Note: this is not @@ -211,3 +212,26 @@ set of steps to reproduce. .. _GitHub Discussions: https://github.com/pypa/setuptools/discussions .. _setuptools bug tracker: https://github.com/pypa/setuptools/ + + +---- + + +.. [#manifest] For the most common use cases, ``setuptools`` will automatically + find out which files are necessary for distributing the package. + This includes all pure Python modules in the ``py_modules`` or ``packages`` + configuration and all C sources listed as part of extensions + (it doesn't catch C headers, though). + + More complex packages (e.g. packages that include non-Python files, or that + need to use custom C headers), might still need to specify |MANIFEST.in|_ or + use a plugin like :pypi:`setuptools-scm` or :pypi:`setuptools-svn` + to automatically include files tracked by your Revision Control System. + + Please note that only files **inside the package directory** are included in + the final wheel distribution, by default. See :doc:`userguide/datafiles` for + more information. + + +.. |MANIFEST.in| replace:: ``MANIFEST.in`` +.. _MANIFEST.in: https://packaging.python.org/en/latest/guides/using-manifest-in/ diff --git a/docs/userguide/datafiles.rst b/docs/userguide/datafiles.rst index 69cf36e699..28faa84f54 100644 --- a/docs/userguide/datafiles.rst +++ b/docs/userguide/datafiles.rst @@ -5,11 +5,11 @@ Data Files Support The distutils have traditionally allowed installation of "data files", which are placed in a platform-specific location. However, the most common use case for data files distributed with a package is for use *by* the package, usually -by including the data files in the package directory. +by including the data files **inside the package directory**. -Setuptools offers three ways to specify data files to be included in your -packages. First, you can simply use the ``include_package_data`` keyword, -e.g.:: +Setuptools offers three ways to specify this most common type of data files to +be included in your packages [#datafiles]_. +First, you can simply use the ``include_package_data`` keyword, e.g.:: from setuptools import setup, find_packages setup( @@ -18,9 +18,10 @@ e.g.:: ) This tells setuptools to install any data files it finds in your packages. -The data files must be specified via the distutils' ``MANIFEST.in`` file. +The data files must be specified via the distutils' |MANIFEST.in|_ file. (They can also be tracked by a revision control system, using an appropriate -plugin. See the section below on :ref:`Adding Support for Revision +plugin such as :pypi:`setuptools-scm` or :pypi:`setuptools-svn`. +See the section below on :ref:`Adding Support for Revision Control Systems` for information on how to write such plugins.) If you want finer-grained control over what files are included (for example, @@ -87,12 +88,11 @@ When building an ``sdist``, the datafiles are also drawn from the ``package_name.egg-info/SOURCES.txt`` file, so make sure that this is removed if the ``setup.py`` ``package_data`` list is updated before calling ``setup.py``. -(Note: although the ``package_data`` argument was previously only available in -``setuptools``, it was also added to the Python ``distutils`` package as of -Python 2.4; there is `some documentation for the feature`__ available on the -python.org website. If using the setuptools-specific ``include_package_data`` -argument, files specified by ``package_data`` will *not* be automatically -added to the manifest unless they are listed in the MANIFEST.in file.) +.. note:: + If using the ``include_package_data`` argument, files specified by + ``package_data`` will *not* be automatically added to the manifest unless + they are listed in the |MANIFEST.in|_ file or by a plugin like + :pypi:`setuptools-scm` or :pypi:`setuptools-svn`. __ https://docs.python.org/3/distutils/setupscript.html#installing-package-data @@ -125,11 +125,13 @@ included as a result of using ``include_package_data``. In summary, the three options allow you to: ``include_package_data`` - Accept all data files and directories matched by ``MANIFEST.in``. + Accept all data files and directories matched by |MANIFEST.in|_ or added by + a :ref:` Adding Support for Revision Control Systems`. ``package_data`` Specify additional patterns to match files that may or may - not be matched by ``MANIFEST.in`` or found in source control. + not be matched by ``MANIFEST.in`` or added by + a :ref:` Adding Support for Revision Control Systems`. ``exclude_package_data`` Specify patterns for data files and directories that should *not* be @@ -175,3 +177,14 @@ no supported facility to reliably retrieve these resources. Instead, the PyPA recommends that any data files you wish to be accessible at run time be included in the package. + + +---- + +.. [#datafiles] ``setuptools`` consider a *package data file* any non-Python + file **inside the package directory** (i.e., that co-exists in the same + location as the regular ``.py`` files being distributed). + + +.. |MANIFEST.in| replace:: ``MANIFEST.in`` +.. _MANIFEST.in: https://packaging.python.org/en/latest/guides/using-manifest-in/ From 2f1dffb7b53be6943b98ab73f548c1837f460e68 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Thu, 3 Mar 2022 16:27:01 +0000 Subject: [PATCH 02/14] Add a note about data files being read-only --- docs/userguide/datafiles.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/userguide/datafiles.rst b/docs/userguide/datafiles.rst index 28faa84f54..bfec2afb09 100644 --- a/docs/userguide/datafiles.rst +++ b/docs/userguide/datafiles.rst @@ -166,6 +166,19 @@ a quick example of converting code that uses ``__file__`` to use .. _Importlib Resources: https://docs.python.org/3/library/importlib.html#module-importlib.resources +.. tip:: Files inside the package directory should be *read-only* to avoid a + series of common problems (e.g. when multiple users share a common Python + installation, when the package is loaded from a zip file, or when multiple + instances of a Python application run in parallel). + + If your Python package needs to write to a file for shared data or configuration, + you can use standard platform/OS-specific system directories, such as + ``~/.local/config/$appname`` or ``/usr/share/$appname/$version`` (Linux specific) [#system-dirs]_. + A common approach is to add a read-only template file to the package + directory that is then copied to the correct system directory if no + pre-existing file is found. + + Non-Package Data Files ---------------------- @@ -185,6 +198,9 @@ run time be included in the package. file **inside the package directory** (i.e., that co-exists in the same location as the regular ``.py`` files being distributed). +.. [#system-dirs] These locations can be discovered with the help of + third-party libraries such as :pypi:`platformdirs`. + .. |MANIFEST.in| replace:: ``MANIFEST.in`` .. _MANIFEST.in: https://packaging.python.org/en/latest/guides/using-manifest-in/ From 523e7c7d008bd043ae6ad707a21e004f53aa9531 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Thu, 3 Mar 2022 17:02:26 +0000 Subject: [PATCH 03/14] Modify datafiles docs to emphasize importlib.resources over pkg_resorueces --- docs/conf.py | 3 +++ docs/userguide/datafiles.rst | 24 ++++++++++++------------ 2 files changed, 15 insertions(+), 12 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index e57131b102..0443799dcf 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -199,3 +199,6 @@ ] intersphinx_mapping['pip'] = 'https://pip.pypa.io/en/latest', None +intersphinx_mapping['importlib-resources'] = ( + 'https://importlib-resources.readthedocs.io/en/latest', None +) diff --git a/docs/userguide/datafiles.rst b/docs/userguide/datafiles.rst index bfec2afb09..ce62f3ab84 100644 --- a/docs/userguide/datafiles.rst +++ b/docs/userguide/datafiles.rst @@ -94,7 +94,7 @@ the ``setup.py`` ``package_data`` list is updated before calling ``setup.py``. they are listed in the |MANIFEST.in|_ file or by a plugin like :pypi:`setuptools-scm` or :pypi:`setuptools-svn`. -__ https://docs.python.org/3/distutils/setupscript.html#installing-package-data +.. https://docs.python.org/3/distutils/setupscript.html#installing-package-data Sometimes, the ``include_package_data`` or ``package_data`` options alone aren't sufficient to precisely define what files you want included. For @@ -126,12 +126,12 @@ In summary, the three options allow you to: ``include_package_data`` Accept all data files and directories matched by |MANIFEST.in|_ or added by - a :ref:` Adding Support for Revision Control Systems`. + a :ref:`plugin `. ``package_data`` Specify additional patterns to match files that may or may not be matched by ``MANIFEST.in`` or added by - a :ref:` Adding Support for Revision Control Systems`. + a :ref:`plugin `. ``exclude_package_data`` Specify patterns for data files and directories that should *not* be @@ -156,15 +156,10 @@ Typically, existing programs manipulate a package's ``__file__`` attribute in order to find the location of data files. However, this manipulation isn't compatible with PEP 302-based import hooks, including importing from zip files and Python Eggs. It is strongly recommended that, if you are using data files, -you should use the :ref:`ResourceManager API` of ``pkg_resources`` to access -them. The ``pkg_resources`` module is distributed as part of setuptools, so if -you're using setuptools to distribute your package, there is no reason not to -use its resource management API. See also `Importlib Resources`_ for -a quick example of converting code that uses ``__file__`` to use -``pkg_resources`` instead. - -.. _Importlib Resources: https://docs.python.org/3/library/importlib.html#module-importlib.resources - +you should use :mod:`importlib.resources` to access them. +:mod:`importlib.resources` is available since Python 3.7 and the latest version of +the library is also available via the :pypi:`importlib-resources` backport. +See :doc:`importlib-resources:using` for detailed instructions [#importlib]_. .. tip:: Files inside the package directory should be *read-only* to avoid a series of common problems (e.g. when multiple users share a common Python @@ -201,6 +196,11 @@ run time be included in the package. .. [#system-dirs] These locations can be discovered with the help of third-party libraries such as :pypi:`platformdirs`. +.. [#importlib] Recent versions of :mod:`importlib.resources` available in + Pythons' standard library should be API compatible with + :pypi:`importlib-metadata`. However this might vary depending on which version + of Python is installed. + .. |MANIFEST.in| replace:: ``MANIFEST.in`` .. _MANIFEST.in: https://packaging.python.org/en/latest/guides/using-manifest-in/ From 7d6eac45f32d4908413bf035ad623538f321f0e4 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Thu, 3 Mar 2022 17:37:55 +0000 Subject: [PATCH 04/14] Add section about distributed files to miscellaneous --- docs/setuptools.rst | 19 +++++------------ docs/userguide/miscellaneous.rst | 35 ++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+), 14 deletions(-) diff --git a/docs/setuptools.rst b/docs/setuptools.rst index 53cf54b264..aa638300de 100644 --- a/docs/setuptools.rst +++ b/docs/setuptools.rst @@ -217,20 +217,11 @@ set of steps to reproduce. ---- -.. [#manifest] For the most common use cases, ``setuptools`` will automatically - find out which files are necessary for distributing the package. - This includes all pure Python modules in the ``py_modules`` or ``packages`` - configuration and all C sources listed as part of extensions - (it doesn't catch C headers, though). - - More complex packages (e.g. packages that include non-Python files, or that - need to use custom C headers), might still need to specify |MANIFEST.in|_ or - use a plugin like :pypi:`setuptools-scm` or :pypi:`setuptools-svn` - to automatically include files tracked by your Revision Control System. - - Please note that only files **inside the package directory** are included in - the final wheel distribution, by default. See :doc:`userguide/datafiles` for - more information. +.. [#manifest] The default behaviour for ``setuptools`` will work well for pure + Python packages, or packages with simple C extensions (that don't require + any special C header). See :ref:`Controlling files in the distribution` and + :doc:`userguide/datafiles` for more information about complex scenarios, if + you want to include other types of files. .. |MANIFEST.in| replace:: ``MANIFEST.in`` diff --git a/docs/userguide/miscellaneous.rst b/docs/userguide/miscellaneous.rst index 3df327d795..ad565ed462 100644 --- a/docs/userguide/miscellaneous.rst +++ b/docs/userguide/miscellaneous.rst @@ -94,3 +94,38 @@ correctly when installed as a zipfile, correct any problems if you can, and then make an explicit declaration of ``True`` or ``False`` for the ``zip_safe`` flag, so that it will not be necessary for ``bdist_egg`` to try to guess whether your project can work as a zipfile. + + +.. _Controlling files in the distribution: + +Controlling files in the distribution +------------------------------------- + +For the most common use cases, ``setuptools`` will automatically +find out which files are necessary for distributing the package. +This includes all pure Python modules in the ``py_modules`` or ``packages`` +configuration and all C sources listed as part of extensions +(it doesn't catch C headers, though). + +However, when building more complex packages (e.g. packages that include +non-Python files, or that need to use custom C headers), you might find that +not all files present in your project folder are included in package +distribution archive. In these situations you can use a ``setuptools`` +:ref:`plugin `, such as +:pypi:`setuptools-scm` or :pypi:`setuptools-svn` to automatically include all +files tracked by your Revision Control System to the source distribution +archive (``sdist``). + +.. _Using MANIFEST.in: + +In the case you need fine control over the included files you can also specify +a ``MANIFEST.in`` file at the root of your project with precise +instructions. A comprehensive guide to ``MANIFEST.in`` syntax is available at +the `PyPA's packaging user guide`_. + +Please note that, by default, only files **inside the package directory** are +included in the final ``wheel`` distribution. See :doc:`/userguide/datafiles` for +more information. + + +.. _PyPa's packaging user guide: https://packaging.python.org/en/latest/guides/using-manifest-in/ From e8b418d0b3ccd9b0c70401ef1fe38481980c5578 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Thu, 3 Mar 2022 17:41:13 +0000 Subject: [PATCH 05/14] Add link to MANIFEST.in in quickstart --- docs/userguide/quickstart.rst | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/userguide/quickstart.rst b/docs/userguide/quickstart.rst index 203d620473..61ab7f972b 100644 --- a/docs/userguide/quickstart.rst +++ b/docs/userguide/quickstart.rst @@ -180,7 +180,9 @@ can simply use the ``include_package_data`` keyword: include_package_data = True This tells setuptools to install any data files it finds in your packages. -The data files must be specified via the distutils' ``MANIFEST.in`` file. +The data files must be specified via the distutils' |MANIFEST.in| file +or automatically added by a :ref:`Revision Control System plugin +`. For more details, see :doc:`datafiles` @@ -228,3 +230,7 @@ Resources on Python packaging Packaging in Python can be hard and is constantly evolving. `Python Packaging User Guide `_ has tutorials and up-to-date references that can help you when it is time to distribute your work. + + +.. |MANIFEST.in| replace:: ``MANIFEST.in`` +.. _MANIFEST.in: https://packaging.python.org/en/latest/guides/using-manifest-in/ From c0bdfb66a08e3f5c68e2bc91f33ca6d1a7757511 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Thu, 3 Mar 2022 18:16:24 +0000 Subject: [PATCH 06/14] Add news fragment --- changelog.d/3148.doc.1.rst | 3 +++ changelog.d/3148.doc.2.rst | 4 ++++ 2 files changed, 7 insertions(+) create mode 100644 changelog.d/3148.doc.1.rst create mode 100644 changelog.d/3148.doc.2.rst diff --git a/changelog.d/3148.doc.1.rst b/changelog.d/3148.doc.1.rst new file mode 100644 index 0000000000..af89bde23a --- /dev/null +++ b/changelog.d/3148.doc.1.rst @@ -0,0 +1,3 @@ +Added clarifications about ``MANIFEST.in``, that include links to PyPUG docs +and more prominent mentions to using a revision control system plugin as an +alternative. diff --git a/changelog.d/3148.doc.2.rst b/changelog.d/3148.doc.2.rst new file mode 100644 index 0000000000..f46fb24869 --- /dev/null +++ b/changelog.d/3148.doc.2.rst @@ -0,0 +1,4 @@ +Removed mention to ``pkg_resources`` as the recommended way of accessing data +files, in favour of :doc:`importlib.resources`. +Additionally more emphasis was put on the fact that *package data files* reside +**inside** the *package directory* (and therefore should be *read-only*). From 580801296f57c829a57567006284d312853b1f2d Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Thu, 3 Mar 2022 18:24:10 +0000 Subject: [PATCH 07/14] Add missing link to PyPUG MANIFEST.in docs --- docs/userguide/quickstart.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/userguide/quickstart.rst b/docs/userguide/quickstart.rst index 61ab7f972b..f318362458 100644 --- a/docs/userguide/quickstart.rst +++ b/docs/userguide/quickstart.rst @@ -180,7 +180,7 @@ can simply use the ``include_package_data`` keyword: include_package_data = True This tells setuptools to install any data files it finds in your packages. -The data files must be specified via the distutils' |MANIFEST.in| file +The data files must be specified via the distutils' |MANIFEST.in|_ file or automatically added by a :ref:`Revision Control System plugin `. For more details, see :doc:`datafiles` From f529729ebd858f6d37ba9c6abd319a7cf8e6dc68 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Thu, 3 Mar 2022 18:26:37 +0000 Subject: [PATCH 08/14] Add another missing link to PyPUG --- docs/userguide/datafiles.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/userguide/datafiles.rst b/docs/userguide/datafiles.rst index ce62f3ab84..32f91affd8 100644 --- a/docs/userguide/datafiles.rst +++ b/docs/userguide/datafiles.rst @@ -130,7 +130,7 @@ In summary, the three options allow you to: ``package_data`` Specify additional patterns to match files that may or may - not be matched by ``MANIFEST.in`` or added by + not be matched by |MANIFEST.in|_ or added by a :ref:`plugin `. ``exclude_package_data`` From 7c2b42292fcdb0076630b00ec09778c598a2a1eb Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Thu, 3 Mar 2022 18:28:32 +0000 Subject: [PATCH 09/14] Avoid using a same set of words repeatedly too close --- docs/userguide/datafiles.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/userguide/datafiles.rst b/docs/userguide/datafiles.rst index 32f91affd8..d974a30181 100644 --- a/docs/userguide/datafiles.rst +++ b/docs/userguide/datafiles.rst @@ -157,7 +157,7 @@ order to find the location of data files. However, this manipulation isn't compatible with PEP 302-based import hooks, including importing from zip files and Python Eggs. It is strongly recommended that, if you are using data files, you should use :mod:`importlib.resources` to access them. -:mod:`importlib.resources` is available since Python 3.7 and the latest version of +:mod:`importlib.resources` was added to Python 3.7 and the latest version of the library is also available via the :pypi:`importlib-resources` backport. See :doc:`importlib-resources:using` for detailed instructions [#importlib]_. From d0b9e825fe6c61d4af7b86512dab6ca5ddaafd3c Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Thu, 3 Mar 2022 18:29:33 +0000 Subject: [PATCH 10/14] Emphasize data files should be included inside the package --- docs/userguide/datafiles.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/userguide/datafiles.rst b/docs/userguide/datafiles.rst index d974a30181..e4e94f9840 100644 --- a/docs/userguide/datafiles.rst +++ b/docs/userguide/datafiles.rst @@ -184,7 +184,7 @@ fall back to the platform-specific location for installing data files, there is no supported facility to reliably retrieve these resources. Instead, the PyPA recommends that any data files you wish to be accessible at -run time be included in the package. +run time be included **inside the package**. ---- From 5553c276e8071533d45f99db5c00108e852839ba Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Fri, 4 Mar 2022 13:08:36 +0000 Subject: [PATCH 11/14] Apply suggestions from code review Co-authored-by: Steven Silvester --- docs/userguide/datafiles.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/userguide/datafiles.rst b/docs/userguide/datafiles.rst index e4e94f9840..9817e63913 100644 --- a/docs/userguide/datafiles.rst +++ b/docs/userguide/datafiles.rst @@ -8,7 +8,7 @@ for data files distributed with a package is for use *by* the package, usually by including the data files **inside the package directory**. Setuptools offers three ways to specify this most common type of data files to -be included in your packages [#datafiles]_. +be included in your package's [#datafiles]_. First, you can simply use the ``include_package_data`` keyword, e.g.:: from setuptools import setup, find_packages @@ -18,7 +18,7 @@ First, you can simply use the ``include_package_data`` keyword, e.g.:: ) This tells setuptools to install any data files it finds in your packages. -The data files must be specified via the distutils' |MANIFEST.in|_ file. +The data files must be specified via the |MANIFEST.in|_ file. (They can also be tracked by a revision control system, using an appropriate plugin such as :pypi:`setuptools-scm` or :pypi:`setuptools-svn`. See the section below on :ref:`Adding Support for Revision From 82529755d433de93fcce1c48385dce3cf4003253 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Fri, 4 Mar 2022 15:21:14 +0000 Subject: [PATCH 12/14] Clarify the relationship between wheel <> sdist --- docs/conf.py | 1 + docs/userguide/miscellaneous.rst | 70 ++++++++++++++++++++++---------- 2 files changed, 50 insertions(+), 21 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index 0443799dcf..da4d9f33d5 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -199,6 +199,7 @@ ] intersphinx_mapping['pip'] = 'https://pip.pypa.io/en/latest', None +intersphinx_mapping['PyPUG'] = ('https://packaging.python.org/en/latest/', None) intersphinx_mapping['importlib-resources'] = ( 'https://importlib-resources.readthedocs.io/en/latest', None ) diff --git a/docs/userguide/miscellaneous.rst b/docs/userguide/miscellaneous.rst index ad565ed462..8d494d16f3 100644 --- a/docs/userguide/miscellaneous.rst +++ b/docs/userguide/miscellaneous.rst @@ -101,31 +101,59 @@ whether your project can work as a zipfile. Controlling files in the distribution ------------------------------------- -For the most common use cases, ``setuptools`` will automatically -find out which files are necessary for distributing the package. -This includes all pure Python modules in the ``py_modules`` or ``packages`` -configuration and all C sources listed as part of extensions -(it doesn't catch C headers, though). +For the most common use cases, ``setuptools`` will automatically find out which +files are necessary for distributing the package. +This includes all :term:`pure Python modules ` in the +``py_modules`` or ``packages`` configuration, and the C sources (but not C +headers) listed as part of extensions when creating a :term:`Source +Distribution (or "sdist")`. However, when building more complex packages (e.g. packages that include non-Python files, or that need to use custom C headers), you might find that not all files present in your project folder are included in package -distribution archive. In these situations you can use a ``setuptools`` -:ref:`plugin `, such as -:pypi:`setuptools-scm` or :pypi:`setuptools-svn` to automatically include all -files tracked by your Revision Control System to the source distribution -archive (``sdist``). +:term:`distribution archive `. -.. _Using MANIFEST.in: - -In the case you need fine control over the included files you can also specify -a ``MANIFEST.in`` file at the root of your project with precise -instructions. A comprehensive guide to ``MANIFEST.in`` syntax is available at -the `PyPA's packaging user guide`_. - -Please note that, by default, only files **inside the package directory** are -included in the final ``wheel`` distribution. See :doc:`/userguide/datafiles` for -more information. +In these situations you can use a ``setuptools`` +:ref:`plugin `, +such as :pypi:`setuptools-scm` or :pypi:`setuptools-svn` to automatically +include all files tracked by your Revision Control System into the ``sdist``. +.. _Using MANIFEST.in: -.. _PyPa's packaging user guide: https://packaging.python.org/en/latest/guides/using-manifest-in/ +Alternatively, if you need finer control, you can add a ``MANIFEST.in`` file at +the root of your project. +This file contains instructions that tell ``setuptools`` which files exactly +should be part of the ``sdist`` (or not). +A comprehensive guide to ``MANIFEST.in`` syntax is available at the +:doc:`PyPA's Packaging User Guide `. + +Once the correct files are present in the ``sdist``, they can then be used by +binary extensions during the build process, or included in the final +:term:`wheel ` [#build-process]_ if you configure ``setuptools`` with +``include_package_data=True``. + +.. important:: + Please note that, when using ``include_package_data=True``, only files **inside + the package directory** are included in the final ``wheel``, by default. + + So for example, if you create a :term:`Python project ` that uses + :pypi:`setuptools-scm` and have a ``tests`` directory outside of the package + folder, the ``tests`` directory will be present in the ``sdist`` but not in the + ``wheel`` [#wheel-vs-sdist]_. + + See :doc:`/userguide/datafiles` for more information. + +---- + +.. [#build-process] + You can think about the build process as two stages: first the ``sdist`` + will be created and then the ``whell`` will be produced from that ``sdist``. + +.. [#wheel-vs-sdist] + This happens because the ``sdist`` can contain files that are useful during + development or the build process itself, but not in runtime (e.g. tests, + docs, examples, etc...). + The ``wheel``, on the other hand, is a file format that has been optimized + and ready to be unpacked into a running installation of Python or + :term:`Virtual Environment`. + Therefore it only contains items that are required during runtime. From bb7c45e348fef933da97f514c79d693af8a96283 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Fri, 4 Mar 2022 16:25:22 +0000 Subject: [PATCH 13/14] Apply suggestions from code review Co-authored-by: Steven Silvester --- docs/userguide/miscellaneous.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/userguide/miscellaneous.rst b/docs/userguide/miscellaneous.rst index 8d494d16f3..e545adae5b 100644 --- a/docs/userguide/miscellaneous.rst +++ b/docs/userguide/miscellaneous.rst @@ -147,7 +147,7 @@ binary extensions during the build process, or included in the final .. [#build-process] You can think about the build process as two stages: first the ``sdist`` - will be created and then the ``whell`` will be produced from that ``sdist``. + will be created and then the ``wheel`` will be produced from that ``sdist``. .. [#wheel-vs-sdist] This happens because the ``sdist`` can contain files that are useful during From 1cbe68dce532d4b14f7685bd1332ba4dc838bfca Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Fri, 4 Mar 2022 16:34:57 +0000 Subject: [PATCH 14/14] Update docs/userguide/miscellaneous.rst --- docs/userguide/miscellaneous.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/userguide/miscellaneous.rst b/docs/userguide/miscellaneous.rst index e545adae5b..5fd2f0a8b3 100644 --- a/docs/userguide/miscellaneous.rst +++ b/docs/userguide/miscellaneous.rst @@ -154,6 +154,6 @@ binary extensions during the build process, or included in the final development or the build process itself, but not in runtime (e.g. tests, docs, examples, etc...). The ``wheel``, on the other hand, is a file format that has been optimized - and ready to be unpacked into a running installation of Python or + and is ready to be unpacked into a running installation of Python or :term:`Virtual Environment`. Therefore it only contains items that are required during runtime.