Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX-#6587: Use different env files for unidist engine for windows and linux #6588

Merged
merged 4 commits into from
Sep 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -349,7 +349,7 @@ jobs:
- uses: actions/checkout@v3
- uses: ./.github/actions/mamba-env
with:
environment-file: requirements/env_unidist.yml
environment-file: requirements/env_unidist_linux.yml
activate-environment: modin_on_unidist
python-version: ${{matrix.python-version}}
- name: Install HDF5
Expand Down Expand Up @@ -556,7 +556,7 @@ jobs:
- uses: actions/checkout@v3
- uses: ./.github/actions/mamba-env
with:
environment-file: ${{ matrix.execution.name == 'unidist' && 'requirements/env_unidist.yml' || 'environment-dev.yml' }}
environment-file: ${{ matrix.os == 'ubuntu' && matrix.execution.name == 'unidist' && 'requirements/env_unidist_linux.yml' || matrix.os == 'windows' && matrix.execution.name == 'unidist' && 'requirements/env_unidist_win.yml' || 'environment-dev.yml' }}
activate-environment: ${{ matrix.execution.name == 'unidist' && 'modin_on_unidist' || 'modin' }}
python-version: ${{matrix.python-version}}
- name: Install HDF5
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ dependencies:
- pandas>=2.1,<2.2
- numpy>=1.22.4
- unidist-mpi>=0.2.1
- mpich
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why it is not a Linux-specific dependency of unidist-mpi instead...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we added any MPI library as a linux-specific dependency of unidist-mpi, we would jeopardize the option of using a system-provided MPI library.

- fsspec>=2022.05.0
- packaging>=21.0
- psutil>=5.8.0
Expand Down
61 changes: 61 additions & 0 deletions requirements/env_unidist_win.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: modin_on_unidist
channels:
- conda-forge
dependencies:
- pip

# required dependencies
- pandas>=2.1,<2.2
- numpy>=1.22.4
- unidist-mpi>=0.2.1
- msmpi
- fsspec>=2022.05.0
- packaging>=21.0
- psutil>=5.8.0

# optional dependencies
- pyarrow>=7.0.0
- xarray>=2022.03.0
- jinja2>=3.1.2
- scipy>=1.8.1
- s3fs>=2022.05.0
- lxml>=4.8.0
- openpyxl>=3.0.10
- xlrd>=2.0.1
- matplotlib>=3.6.1
- sqlalchemy>=1.4.0,<1.4.46
- pandas-gbq>=0.15.0
- pytables>=3.7.0
# pymssql==2.2.8 broken: https://github.com/modin-project/modin/issues/6429
- pymssql>=2.1.5,!=2.2.8
- psycopg2>=2.9.3
- fastparquet>=0.8.1
- tqdm>=4.60.0
# pandas isn't compatible with numexpr=2.8.5: https://github.com/modin-project/modin/issues/6469
- numexpr<2.8.5

# dependencies for making release
- pygithub>=v1.58.0
- pygit2>=1.9.2

# test dependencies
- coverage>=7.1.0
- moto>=4.1.0
- pytest>=7.3.2
- pytest-cov>=4.0.0
- pytest-xdist>=3.2.0

# code linters
- black>=23.1.0
- flake8>=6.0.0
- flake8-no-implicit-concat>=0.3.4
- flake8-print>=5.0.0
- mypy>=1.0.0
- pandas-stubs>=2.0.0

- pip:
# Fixes breaking ipywidgets changes, but didn't release yet.
- git+https://github.com/modin-project/modin-spreadsheet.git@49ffd89f683f54c311867d602c55443fb11bf2a5
- connectorx>=0.2.6a4
# The `numpydoc` version should match the version installed in the `lint-pydocstyle` job of the CI.
- numpydoc==1.1.0