Skip to content

Commit

Permalink
Add tasks of making a new release
Browse files Browse the repository at this point in the history
* Also, move DEVELOPER_NOTES.md to folder docs.
  • Loading branch information
wkliao committed Sep 11, 2024
1 parent 4b59cde commit cdc8ed7
Showing 1 changed file with 53 additions and 4 deletions.
57 changes: 53 additions & 4 deletions DEVELOPER_NOTES.md → docs/DEVELOPER_NOTES.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,55 @@
## Notes for PnetCDF-python developers

This file contains instructions for PnetCDF-Python developers.
Users of PnetCDF-Python please refer to the
[PnetCDF-Python User Guide](https://pnetcdf-python.readthedocs.io/en/latest/).

* [Making A New Release](#making-a-new-release)
* [Library Packaging And Publishing](#library-packaging-and-publishing)

---
### Making A New Release

Below is a list of tasks to be done immediately before making a new release
(must run in the following order).

1. Update the release version string
* Edit file [src/pnetcdf/__init__.py](src/pnetcdf/__init__.py) and change
the following string.
```
__version__ = "1.0.0"
```
2. Update release note
* Edit file [RELEASE_NOTES.md](RELEASE_NOTES.md) to include notes about new
features and other noticeable changes.
* Update the release date.
3. Commit all changes to github repo
* Run command below to upload changes to github.com.
```
git push upstream main
```
4. Create a tag and push it to the gitbub repo
* Run commands below to do so. Replace the proper version string.
```
git tag -a v.1.0.0.pre -m "1.0.0 pre-release"
git push upstream v.1.0.0.pre
```
5. On the browser, click [tags](https://github.com/Parallel-NetCDF/PnetCDF-Python/tags)
* Click the name of tag just pushed.
* Click "Create release from tag".
* Fill in "Release title"
* Fill in "Description field"
* Optionally, click "Set as a pre-release", if this is a pre-release.
* Optionally, click "Generate release notes", which will add a list of all
PRs and new contributors into the Description field..
* Click "Save draft" to save the changes. It can be modified later.
* See all options from [github docs](https://docs.github.com/en/repositories/releasing-projects-on-github/automatically-generated-release-notes)
---
### Library packaging and publishing
### Library Packaging And Publishing
* Currently, pip-install via build distribution is disabled. No wheel files are uploaded to PyPI. Lastest pnetcdf-python package on PyPI: https://pypi.org/project/pnetcdf/
* Packaging: build source distribution and wheel distribution
1. Create virtual env and install PnetCDF-C and all python dependencies as developer installation
Expand Down Expand Up @@ -58,7 +107,7 @@
* Pip-install command’s second go-to option if no matching build distribution is found on PyPI. It will build its own wheel file for the current system. For python libs with C-extension (like pnetcdf-python, netcdf4-python, h5py), wheel installation require their C bindings.
* Invariant to platforms/versions, usually each version of python library has one single source distribution.
2. Build distribution (.whl)
* Pip-install command’s first go-to option by default. For python libs with C-extension (like pnetcdf-python, netcdf4-python, h5py), wheel installation does not require their C bindings.
* Pip-install command’s first go-to option by default. For python libs with C-extension (like pnetcdf-python, netcdf4-python, h5py), wheel installation does not require their C bindings.
* Wheels are platform-specific and python-version specific. Different system used for building and uploading will generate different versions of wheel files on PyPI. To cover most mainstream python versions and operating systems, python libraries (like [numpy 1.25](https://pypi.org/project/numpy/#files)) has 20+ wheels files to cover most mainstream systems (e.g. Linux (x86_64), macOS (x86_64), and Windows (32-bit and 64-bit)) and recent python versions.
* General procedures of building and uploading python library wheels (build distributions) for MacOS and Linux systems
* Python libs with C-extension (like pncpy, netcdf4-python, h5py) requires shared object (.so in linux and .dylibs in mac) files collected from C software installation. When making python library wheels (build distribution), an extra post-processing step (Delocate tool for Mac, auditwheel Tool in linux) is usually performed to copy and store these files in the python package to remove these dependencies. That’s why the user’s pip install with build distribution does not require a C installation as a prerequisite.
Expand All @@ -70,7 +119,7 @@
* Developer install
* Developer installation is mainly managed by `setup.py` and `pyproject.toml`. The former one is the core file to build the package and the latter manages dependencies required by `pip install` command.
* `python setup.py install` builds and installs the python library based on the current python environment and will error out if dependency (e.g. mpi4py) not met. This command is going to be deprecated and is not recommended for modern python library installation.
* `pip install .` works as a wrapper command for the above but it goes further: it automatically handles and install any dependencies listed in `pyproject.toml` file.Need to pay special attention to the dependencies listed in `requires` under `[build-system]` and `dependencies`.
* `pip install .` works as a wrapper command for the above but it goes further: it automatically handles and install any dependencies listed in `pyproject.toml` file.Need to pay special attention to the dependencies listed in `requires` under `[build-system]` and `dependencies`.
* `dependencies` defines python libraries required for running the project and will first check if requirement already satisfied in current environment before installing the latest qualified version of the library.
* `requires` defines libraries required for building the project. `pip install` by default creates and uses isolated building env for building stage which completely ignores current user env. For example, if user already installed mpi4py==3.1.6, "mpi4py>=3.1.4" will still automatically install a mpi4py 4.0.0 in the building env and thereafter use syntax from 4.0.0 to build pnetcdf-python. This caused version mismatch issues between building and running envs when numpy 2.0 and mpi4py 4.0.0 are released. To address this issue, current user guide forces to use current python env for building stage by adding ` -no-build-isolation` arg to `pip install` command.
Expand All @@ -86,4 +135,4 @@
* Current environment variables set (only effective solution found to set env variable at installation):
* `CC`: mpicc.mpich
* `PNETCDF_DIR`: /home/docs/checkouts/readthedocs.org/user_builds/pnetcdf-python/checkouts/latest/_readthedocs/PnetCDF
* `PNETCDF_VER`: 1.12.3
* `PNETCDF_VER`: 1.12.3

0 comments on commit cdc8ed7

Please sign in to comment.