Simplify install #472
Merged
Simplify install #472
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
init-db.sh requires psql to be available. Replacing it with a Python script that uses psycopg relaxes this requirement.
gulp-sass depends on the deprecated node-sass which requires Python 2 to install. This replacement doesn't depend on node-sass.
Use poetry for dependency management. Removes old pip freeze requirement files. Adds a conda environment.yml file. Updates installation and configuation documentation. Replaces gulp-sass with a fork that doesn't depend on the deprecated node-sass which is now problematic to install.
dask>=2.16.0 set the default multiprocessing context to spawn which doesn't play nice with loading Django models. This explicitly set the context back to the former default "fork". Fixes #454.
mypy, flake8, black
Removed references to removed requirements files from GitHub actions. Added poetry. Docs instruct contributors to install poetry. Updated CHANGELOG.md.
Currently vaex is only used to write out arrow files which is provided by the sub-package vaex-arrow. Turned on arrow file output for the basic association test.
marxide
force-pushed
the
simplify-install
branch
from
c599777 to
e8811f1
Compare
marxide
added
cleanup
We should make sure the test-suite uses this same Docker image. It should be fairly straightforward, just point the docker-compose to your Docker image, rather than the Dockerfile (which we can then remove) |
ajstewart
reviewed
Mar 17, 2021
Thanks! I missed that. Fixed in cc5447b. |
ajstewart
reviewed
Mar 24, 2021
Co-authored-by: Adam Stewart <[email protected]>
ajstewart
approved these changes
Mar 24, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I went over the installation and configuration documentation and have attempted to simplify things for users. The main changes are:
init-tools/init-db.shscript with a Python version that usespsycopg2. The original shell script requiredpsqlto be installed which is now no longer required.gulp-sasswith a fork that doesn't depend on the deprecatednode-sasspackage which is difficult to install as it requires Python 2.pyproject.toml. This file is maintained with Poetry. There are several advantages, including: able to specify more flexible version rules (e.g. any release with the same major version); all dependencies can be specified in the same file yet still be grouped (e.g. dev, production); this is still transparent to the user - modern pip versions know what to do withpyproject.tomland will temporarily install poetry during thepip install .process; thepoetry.lockfile contains exact pinned versions so users can be sure they always get a set of dependencies that work.requirements/environment.ymlfor the non-Python dependencies. The docs encourage users to use this as it's simple and cross-platform, but also indicate they can still use OS package managers for these dependencies.vaexwithvaex-arrowsince we currently only use Vaex to write arrow files and the Vaex meta package includes a lot more than that.CREATE_MEASUREMENTS_ARROW_FILES = Truein the basic association test config so that the Vaex use is tested.