Skip to content

_Dev: Releasing HoloViews

Jump to bottom
Jean-Luc Stevens edited this page Nov 13, 2017 · 5 revisions

Don't forget to shift dev.holoviews.org over to holoviews.org!

Conda-forge

Conda-forge https://github.com/conda-forge provides a github organisation for sharing conda recipes, and a conda channel for handling the built artifacts.

It's pretty neat

https://conda-forge.github.io/#add_recipe provides instructions on adding recipes

Conda forge uses travis and appveyor to manage builds

This means that I can set up my fork to use the same tests, and make use of the testing infrastructure to test prior to putting up PRs. So, I can test a pre-release commit before cutting the release, to make sure it builds usefully

travis is a service and appveyor is a webhook. you need to log in (using github ID) to each and tell them that you want them to be useful. appveyor gives you a URL to ping, travis needs to have the repo turned on

once you've done this, you can pr a branch against your own master and see the test builds running, e.g. https://github.com/marqh/staged-recipes/pull/1

PyPI

PyPI is very unforgiving of mistakes when uploading a release, even if you attempt to rectify your mistakes quickly. It will insist on picking a new filename for the next upload so beware!

Tag the commit as follows:

git tag -a v1.2 -m 'Version 1.2.0'

Note that this is an annotated tag. You can then mark this as a release on github once you've pushed the tags:

git push --tags

Our README is now in markdown and PyPI accepts RST. Conversion with pandoc:

pandoc --from=markdown --to=rst --output=README.rst README.md

Note: Do not use annotated tags for development releases!

If you try to do the release entirely on GitHub, an annotated tag isn't generated which means it will not work properly with param.Version

Instructions

  1. Populate the fake pseudo-packages assets and notebooks and tests inside holoviews.
  2. The first time you want to upload a package you need python setup.py register
  3. You need a ~/.pypirc to upload credentials to PyPI.
  4. To build the compressed archive (make sure to check it!):HOLOVIEWS_RELEASE=True python setup.py sdist --formats=zip
  5. Finally, once you are sure it is all perfectly documented, no typos at all etc and contains vaguely the correct files you can use HOLOVIEWS_RELEASE=True python setup.py sdist --formats=zip upload
  6. Make sure to grep the documentation for the old version number and update it as appropriate.

Conda

Add build.sh:

#!/bin/bash
$PYTHON setup.py --quiet install --single-version-externally-managed --record=record.txt

Update the version in the meta.yml.

Don't forget to build for Python 3 versions!

For instance, run the following (then the same for 3.4 and 2.7):

conda build --python 3.5 .

Make sure to delete cruft such as notebook checkpoints and .DS_Store files from the generate archives. In file roller, you can use the pattern *DS*Store for deletion.

NOTE: In recent conda versions you have to ship .pyc files, __pycache__ directories and other cruft. Do not remove!

DEVELOPMENT VERSIONS

Version names

Our goal is to keep development versions consistent with release versions but also to make it easy to decide on the next development version without having to decide whether the next release version is minor or major. For this reason, the expected convention is to tag with devX where X is a monotonically increasing integer starting from 1 e.g 1.7dev2, 1.8dev1 etc.

The idea is that minor versions typically correspond to critical fixes made shortly after release and so we do not expect development versions between minor versions. With the proposed scheme, there is no issue if 1.7dev1 is released before a minor version such as 1.6.3. Note that although this scheme is expected (and recommended), it isn't strict either. There is no reason a development version such as 1.7.1dev1 wouldn't be allowed but there would need to be a good reason to have such a release.

1. Update setup.py version

This happens first as it is an actual commit. Set the version to the dev name e.g 1.7dev4. This is for people pip installing against the dev release tag.

2. Setting __version__

Set param.Version with a commit SHA and a commit_count (leaving the release tuple alone), for instance:

__version__ = param.Version(release=(1,5,0), fpath=__file__,
                            commit="2f6bbca", commit_count=87,
                            reponame='holoviews')

NOTE: The commit value should be an actual SHA as listed in by git log. The version returned by git describe add 'g' as a prefix so beware!

A short SHA is 7 characters long by default. You can use git describe to get it (dropping the initial 'g'!) or alternatively:

git rev-parse --short HEAD

The commit_count can be quickly found in IPython using:

import holoviews; str(holoviews.__version__)

3. Update meta.yaml

The corresponding meta.yaml should be marked with the appropriate version (see above). In this case 1.6dev1, assuming that this development version hasn't already been released.

4. Git tag

Development version tags should not be annotated tags. For instance, use:

git tag v1.7dev2
git push --tags

Here is the very rough script I use to help automate the dev process. Note, this isn't production quality so use at your own risk!

#!/bin/bash
echo "1. Make sure to update the setup.py version e.g 1.7dev4" 
echo
echo "2. Make sure you are signed in to the right anaconda account"
echo "   anaconda login"
echo
echo "3. Make sure to specify the version (<next major version>devX) to this script e.g 1.7dev2"
DEV_VERSION=$1

echo "4. Make sure that the commit SHA and commit number is set in __version__:"

echo """
__version__ = param.Version(release=(1,5,0), fpath=__file__,
                             commit='22ed445', commit_count=121, reponame='holoviews')"""
echo
echo "Use git describe (without the g prefix!) and holoviews.__version__ to find the appropriate values."
echo
echo "5. Remember to update meta.yaml. E.g:"
echo
# Could grep to ensure match here....
echo "version: $DEV_VERSION"
echo
echo "6. Make sure to tag with a REGULAR tag e.g \`git tag v1.7dev2\` and then \`git push --tags\`"
echo
echo "7. It might be helpful to clear the conda-bld directory in miniconda.."
echo

read -p "Continue (y/n)? " -n 1 -r
echo    # (optional) move to a new line
if [[ $REPLY =~ ^[Yy]$ ]]
then
rm -ri  ../holoviews-dev-$DEV_VERSION
cp -r $(pwd) ../holoviews-dev-$DEV_VERSION
cd ../holoviews-dev-$DEV_VERSION

echo "Moving to directory:"
echo $(pwd)

echo "Removing unwanted .DS_Store files"
# Remove .DS_Store files
find . -name '*.DS_Store' -type f -delete
# Remove __pycache__, *.pyc and checkpoint folders
echo "Removing .pyc files, .ipynb_checkpoints and __pycache__ directories"
find . | \
  grep -E "(__pycache__|\.pyc$|.ipynb_checkpoints)" | \
  xargs rm -rf

echo "Cleaning doc directory"
rm -Rf doc/_build/
rm -Rf doc/nbpublisher
rm -Rf doc/builder
rm -Rf doc/test_data

echo "Building Python 2 version"
conda build --python 2.7 .
echo "Building Python 3.4 version"
conda build --python 3.4 .
echo "Building Python 3.5 version"
conda build --python 3.5 .
echo "Building Python 3.6 version"
conda build --python 3.6 .


# BEWARE! MAY BE /miniconda/ OR /miniconda2/

cd /Users/jstevens/miniconda2/conda-bld/
echo "Moving to directory:"
echo $(pwd)

echo
echo "Make sure to LEAVE cruft such as *.pyc files and __pycache__ files"
echo
read -p "Conda convert (y/n)? " -n 1 -r
echo    # (optional) move to a new line
if [[ $REPLY =~ ^[Yy]$ ]]
then
    conda convert -p all osx-64/holoviews-$DEV_VERSION-py27_0.tar.bz2
    conda convert -p all osx-64/holoviews-$DEV_VERSION-py34_0.tar.bz2
    conda convert -p all osx-64/holoviews-$DEV_VERSION-py35_0.tar.bz2
    conda convert -p all osx-64/holoviews-$DEV_VERSION-py36_0.tar.bz2
fi

read -p "Upload (y/n)? " -n 1 -r
echo    # (optional) move to a new line
if [[ $REPLY =~ ^[Yy]$ ]]
then
    echo "Uploading"
    # Python 2.7
    anaconda upload -l dev osx-64/holoviews-$DEV_VERSION-py27_0.tar.bz2
    anaconda upload -l dev win-64/holoviews-$DEV_VERSION-py27_0.tar.bz2
    anaconda upload -l dev win-32/holoviews-$DEV_VERSION-py27_0.tar.bz2
    anaconda upload -l dev linux-64/holoviews-$DEV_VERSION-py27_0.tar.bz2
    anaconda upload -l dev linux-32/holoviews-$DEV_VERSION-py27_0.tar.bz2
    # Python 3.4
    anaconda upload -l dev osx-64/holoviews-$DEV_VERSION-py34_0.tar.bz2
    anaconda upload -l dev win-64/holoviews-$DEV_VERSION-py34_0.tar.bz2
    anaconda upload -l dev win-32/holoviews-$DEV_VERSION-py34_0.tar.bz2
    anaconda upload -l dev linux-64/holoviews-$DEV_VERSION-py34_0.tar.bz2
    anaconda upload -l dev linux-32/holoviews-$DEV_VERSION-py34_0.tar.bz2
    # Python 3.5
    anaconda upload -l dev osx-64/holoviews-$DEV_VERSION-py35_0.tar.bz2
    anaconda upload -l dev win-64/holoviews-$DEV_VERSION-py35_0.tar.bz2
    anaconda upload -l dev win-32/holoviews-$DEV_VERSION-py35_0.tar.bz2
    anaconda upload -l dev linux-64/holoviews-$DEV_VERSION-py35_0.tar.bz2
    anaconda upload -l dev linux-32/holoviews-$DEV_VERSION-py35_0.tar.bz2
    # Python 3.6
    anaconda upload -l dev osx-64/holoviews-$DEV_VERSION-py36_0.tar.bz2
    anaconda upload -l dev win-64/holoviews-$DEV_VERSION-py36_0.tar.bz2
    anaconda upload -l dev win-32/holoviews-$DEV_VERSION-py36_0.tar.bz2
    anaconda upload -l dev linux-64/holoviews-$DEV_VERSION-py36_0.tar.bz2
    anaconda upload -l dev linux-32/holoviews-$DEV_VERSION-py36_0.tar.bz2

fi

echo "Done"
fi

Empty

Clone this wiki locally