Merge branch 'branch-0.20' of github.com:rapidsai/cudf into fix-missi…

…ng-column-stream-mr

.github/PULL_REQUEST_TEMPLATE.md

@@ -19,15 +19,18 @@ Here are some guidelines to help the review process go smoothly.
    noted here: https://help.github.com/articles/closing-issues-using-keywords/
 
 5. If your pull request is not ready for review but you want to make use of the
-   continuous integration testing facilities please label it with `[WIP]`.
+   continuous integration testing facilities please mark your pull request as Draft.
+   https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/changing-the-stage-of-a-pull-request#converting-a-pull-request-to-a-draft
 
 6. If your pull request is ready to be reviewed without requiring additional
-   work on top of it, then remove the `[WIP]` label (if present) and replace
-   it with `[REVIEW]`. If assistance is required to complete the functionality,
-   for example when the C/C++ code of a feature is complete but Python bindings
-   are still required, then add the label `[HELP-REQ]` so that others can triage
-   and assist. The additional changes then can be implemented on top of the
-   same PR. If the assistance is done by members of the rapidsAI team, then no
+   work on top of it, then remove it from "Draft" and make it "Ready for Review".
+   https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/changing-the-stage-of-a-pull-request#marking-a-pull-request-as-ready-for-review
+
+   If assistance is required to complete the functionality, for example when the
+   C/C++ code of a feature is complete but Python bindings are still required,
+   then add the label `help wanted` so that others can triage and assist.
+   The additional changes then can be implemented on top of the same PR.
+   If the assistance is done by members of the rapidsAI team, then no
    additional actions are required by the creator of the original PR for this,
    otherwise the original author of the PR needs to give permission to the
    person(s) assisting to commit to their personal fork of the project. If that
@@ -39,10 +42,10 @@ Here are some guidelines to help the review process go smoothly.
    features or make changes out of the scope of those requested by the reviewer
    (doing this just add delays as already reviewed code ends up having to be
    re-reviewed/it is hard to tell what is new etc!). Further, please do not
-   rebase your branch on main/force push/rewrite history, doing any of these
-   causes the context of any comments made by reviewers to be lost. If
-   conflicts occur against main they should be resolved by merging main
-   into the branch used for making the pull request.
+   rebase your branch on the target branch, force push, or rewrite history.
+   Doing any of these causes the context of any comments made by reviewers to be lost.
+   If conflicts occur against the target branch they should be resolved by
+   merging the target branch into the branch used for making the pull request.
 
 Many thanks in advance for your cooperation!
 

CHANGELOG.md

@@ -1,3 +1,7 @@
+# cuDF 0.20.0 (Date TBD)
+
+Please see https://github.com/rapidsai/cudf/releases/tag/v0.20.0a for the latest changes to this development branch.
+
 # cuDF 0.19.0 (Date TBD)
 
 Please see https://github.com/rapidsai/cudf/releases/tag/v0.19.0a for the latest changes to this development branch.

CONTRIBUTING.md

@@ -131,14 +131,14 @@ run each time you commit changes.
 
 Compiler requirements:
 
-* `gcc`     version 5.4+
-* `nvcc`    version 10.0+
-* `cmake`   version 3.14.0+
+* `gcc`     version 9.3+
+* `nvcc`    version 11.0+
+* `cmake`   version 3.18.0+
 
 CUDA/GPU requirements:
 
-* CUDA 10.0+
-* NVIDIA driver 410.48+
+* CUDA 11.0+
+* NVIDIA driver 450.80.02+
 * Pascal architecture or better
 
 You can obtain CUDA from [https://developer.nvidia.com/cuda-downloads](https://developer.nvidia.com/cuda-downloads).
@@ -320,7 +320,7 @@ flag. Below is a list of the available arguments and their purpose:
 | `PYARROW_VERSION` | 1.0.1 | Not supported | set pyarrow version |
 | `CMAKE_VERSION` | newest | >=3.14 | set cmake version |
 | `CYTHON_VERSION` | 0.29 | Not supported | set Cython version |
-| `PYTHON_VERSION` | 3.6 | 3.7 | set python version |
+| `PYTHON_VERSION` | 3.7 | 3.8 | set python version |
 
 ---
 

README.md

@@ -57,35 +57,35 @@ Please see the [Demo Docker Repository](https://hub.docker.com/r/rapidsai/rapids
 
 ### CUDA/GPU requirements
 
-* CUDA 10.1+
-* NVIDIA driver 418.39+
+* CUDA 11.0+
+* NVIDIA driver 450.80.02+
 * Pascal architecture or better (Compute Capability >=6.0)
 
 ### Conda
 
 cuDF can be installed with conda ([miniconda](https://conda.io/miniconda.html), or the full [Anaconda distribution](https://www.anaconda.com/download)) from the `rapidsai` channel:
 
-For `cudf version == 0.18` :
+For `cudf version == 0.19` :
 ```bash
 # for CUDA 10.1
 conda install -c rapidsai -c nvidia -c numba -c conda-forge \
-    cudf=0.18 python=3.7 cudatoolkit=10.1
+    cudf=0.19 python=3.7 cudatoolkit=10.1
 
 # or, for CUDA 10.2
 conda install -c rapidsai -c nvidia -c numba -c conda-forge \
-    cudf=0.18 python=3.7 cudatoolkit=10.2
+    cudf=0.19 python=3.7 cudatoolkit=10.2
 
 ```
 
 For the nightly version of `cudf` :
 ```bash
-# for CUDA 10.1
+# for CUDA 11.0
 conda install -c rapidsai-nightly -c nvidia -c numba -c conda-forge \
-    cudf python=3.7 cudatoolkit=10.1
+    cudf python=3.7 cudatoolkit=11.0
 
-# or, for CUDA 10.2
+# or, for CUDA 11.2
 conda install -c rapidsai-nightly -c nvidia -c numba -c conda-forge \
-    cudf python=3.7 cudatoolkit=10.2
+    cudf python=3.7 cudatoolkit=11.1
 ```
 
 Note: cuDF is supported only on Linux, and with Python versions 3.7 and later.

ci/release/update-version.sh

@@ -47,11 +47,14 @@ function sed_runner() {
 }
 
 # cpp update
-sed_runner 's/'"CUDA_DATAFRAME VERSION .* LANGUAGES"'/'"CUDA_DATAFRAME VERSION ${NEXT_FULL_TAG} LANGUAGES"'/g' cpp/CMakeLists.txt
+sed_runner 's/'"CUDF VERSION .* LANGUAGES"'/'"CUDF VERSION ${NEXT_FULL_TAG} LANGUAGES"'/g' cpp/CMakeLists.txt
 
 # cpp libcudf_kafka update
 sed_runner 's/'"CUDA_KAFKA VERSION .* LANGUAGES"'/'"CUDA_KAFKA VERSION ${NEXT_FULL_TAG} LANGUAGES"'/g' cpp/libcudf_kafka/CMakeLists.txt
 
+# cpp cudf_jni update
+sed_runner 's/'"CUDF_JNI VERSION .* LANGUAGES"'/'"CUDF_JNI VERSION ${NEXT_FULL_TAG} LANGUAGES"'/g' java/src/main/native/CMakeLists.txt
+
 # doxyfile update
 sed_runner 's/PROJECT_NUMBER         = .*/PROJECT_NUMBER         = '${NEXT_FULL_TAG}'/g' cpp/doxygen/Doxyfile
 

conda/environments/cudf_dev_cuda10.1.yml

@@ -11,10 +11,10 @@ dependencies:
   - clang=8.0.1
   - clang-tools=8.0.1
   - cupy>7.1.0,<9.0.0a0
-  - rmm=0.19.*
+  - rmm=0.20.*
   - cmake>=3.14
   - cmake_setuptools>=0.1.3
-  - python>=3.6,<3.8
+  - python>=3.7,<3.9
   - numba>=0.49.0,!=0.51.0
   - numpy
   - pandas>=1.0,<1.3.0dev0
@@ -36,15 +36,14 @@ dependencies:
   - pandoc=<2.0.0
   - cudatoolkit=10.1
   - pip
-  - partd
   - flake8=3.8.3
   - black=19.10
   - isort=5.0.7
   - mypy=0.782
   - typing_extensions
   - pre_commit
-  - dask>=2021.3.1
-  - distributed>=2.22.0
+  - dask==2021.4.0
+  - distributed>=2.22.0,<=2021.4.0
   - streamz
   - dlpack
   - arrow-cpp=1.0.1

conda/environments/cudf_dev_cuda10.2.yml

@@ -11,10 +11,10 @@ dependencies:
   - clang=8.0.1
   - clang-tools=8.0.1
   - cupy>7.1.0,<9.0.0a0
-  - rmm=0.19.*
+  - rmm=0.20.*
   - cmake>=3.14
   - cmake_setuptools>=0.1.3
-  - python>=3.6,<3.8
+  - python>=3.7,<3.9
   - numba>=0.49,!=0.51.0
   - numpy
   - pandas>=1.0,<1.3.0dev0
@@ -36,15 +36,14 @@ dependencies:
   - pandoc=<2.0.0
   - cudatoolkit=10.2
   - pip
-  - partd
   - flake8=3.8.3
   - black=19.10
   - isort=5.0.7
   - mypy=0.782
   - typing_extensions
   - pre_commit
-  - dask>=2021.3.1
-  - distributed>=2.22.0
+  - dask==2021.4.0
+  - distributed>=2.22.0,<=2021.4.0
   - streamz
   - dlpack
   - arrow-cpp=1.0.1

conda/environments/cudf_dev_cuda11.0.yml

@@ -11,10 +11,10 @@ dependencies:
   - clang=8.0.1
   - clang-tools=8.0.1
   - cupy>7.1.0,<9.0.0a0
-  - rmm=0.19.*
+  - rmm=0.20.*
   - cmake>=3.14
   - cmake_setuptools>=0.1.3
-  - python>=3.6,<3.8
+  - python>=3.7,<3.9
   - numba>=0.49,!=0.51.0
   - numpy
   - pandas>=1.0,<1.3.0dev0
@@ -36,15 +36,14 @@ dependencies:
   - pandoc=<2.0.0
   - cudatoolkit=11.0
   - pip
-  - partd
   - flake8=3.8.3
   - black=19.10
   - isort=5.0.7
   - mypy=0.782
   - typing_extensions
   - pre_commit
-  - dask>=2021.3.1
-  - distributed>=2.22.0
+  - dask==2021.4.0
+  - distributed>=2.22.0,<=2021.4.0
   - streamz
   - dlpack
   - arrow-cpp=1.0.1

conda/environments/cudf_dev_cuda11.1.yml

@@ -0,0 +1,67 @@
+# Copyright (c) 2021, NVIDIA CORPORATION.
+
+name: cudf_dev
+channels:
+  - rapidsai
+  - nvidia
+  - rapidsai-nightly
+  - conda-forge
+  - defaults
+dependencies:
+  - clang=8.0.1
+  - clang-tools=8.0.1
+  - cupy>7.1.0,<9.0.0a0
+  - rmm=0.20.*
+  - cmake>=3.14
+  - cmake_setuptools>=0.1.3
+  - python>=3.7,<3.9
+  - numba>=0.49,!=0.51.0
+  - numpy
+  - pandas>=1.0,<1.3.0dev0
+  - pyarrow=1.0.1
+  - fastavro>=0.22.9
+  - notebook>=0.5.0
+  - cython>=0.29,<0.30
+  - fsspec>=0.6.0
+  - pytest
+  - pytest-benchmark
+  - pytest-xdist
+  - sphinx
+  - sphinx_rtd_theme
+  - sphinxcontrib-websupport
+  - nbsphinx
+  - numpydoc
+  - ipython
+  - recommonmark
+  - pandoc=<2.0.0
+  - cudatoolkit=11.1
+  - pip
+  - flake8=3.8.3
+  - black=19.10
+  - isort=5.0.7
+  - mypy=0.782
+  - typing_extensions
+  - pre_commit
+  - dask==2021.4.0
+  - distributed>=2.22.0,<=2021.4.0
+  - streamz
+  - dlpack
+  - arrow-cpp=1.0.1
+  - arrow-cpp-proc * cuda
+  - boost-cpp>=1.72.0
+  - double-conversion
+  - rapidjson
+  - flatbuffers
+  - hypothesis
+  - sphinx-markdown-tables
+  - sphinx-copybutton
+  - mimesis
+  - packaging
+  - protobuf
+  - nvtx>=0.2.1
+  - cachetools
+  - pip:
+      - git+https://github.com/dask/dask.git@main
+      - git+https://github.com/dask/distributed.git@main
+      - git+https://github.com/python-streamz/streamz.git
+      - pyorc

conda/environments/cudf_dev_cuda11.2.yml

@@ -0,0 +1,67 @@
+# Copyright (c) 2021, NVIDIA CORPORATION.
+
+name: cudf_dev
+channels:
+  - rapidsai
+  - nvidia
+  - rapidsai-nightly
+  - conda-forge
+  - defaults
+dependencies:
+  - clang=8.0.1
+  - clang-tools=8.0.1
+  - cupy>7.1.0,<9.0.0a0
+  - rmm=0.20.*
+  - cmake>=3.14
+  - cmake_setuptools>=0.1.3
+  - python>=3.7,<3.9
+  - numba>=0.49,!=0.51.0
+  - numpy
+  - pandas>=1.0,<1.3.0dev0
+  - pyarrow=1.0.1
+  - fastavro>=0.22.9
+  - notebook>=0.5.0
+  - cython>=0.29,<0.30
+  - fsspec>=0.6.0
+  - pytest
+  - pytest-benchmark
+  - pytest-xdist
+  - sphinx
+  - sphinx_rtd_theme
+  - sphinxcontrib-websupport
+  - nbsphinx
+  - numpydoc
+  - ipython
+  - recommonmark
+  - pandoc=<2.0.0
+  - cudatoolkit=11.2
+  - pip
+  - flake8=3.8.3
+  - black=19.10
+  - isort=5.0.7
+  - mypy=0.782
+  - typing_extensions
+  - pre_commit
+  - dask==2021.4.0
+  - distributed>=2.22.0,<=2021.4.0
+  - streamz
+  - dlpack
+  - arrow-cpp=1.0.1
+  - arrow-cpp-proc * cuda
+  - boost-cpp>=1.72.0
+  - double-conversion
+  - rapidjson
+  - flatbuffers
+  - hypothesis
+  - sphinx-markdown-tables
+  - sphinx-copybutton
+  - mimesis
+  - packaging
+  - protobuf
+  - nvtx>=0.2.1
+  - cachetools
+  - pip:
+      - git+https://github.com/dask/dask.git@main
+      - git+https://github.com/dask/distributed.git@main
+      - git+https://github.com/python-streamz/streamz.git
+      - pyorc

conda/recipes/cudf/meta.yaml

@@ -28,7 +28,7 @@ requirements:
     - numba >=0.49.0
     - dlpack
     - pyarrow 1.0.1
-    - libcudf {{ version }}=*_{{ GIT_DESCRIBE_NUMBER }}
+    - libcudf {{ version }}
     - rmm {{ minor_version }}
     - cudatoolkit {{ cuda_version }}
   run:

conda/recipes/cudf_kafka/meta.yaml

@@ -29,12 +29,12 @@ requirements:
     - python
     - cython >=0.29,<0.30
     - setuptools
-    - cudf {{ version }}=*_{{ GIT_DESCRIBE_NUMBER }}
-    - libcudf_kafka {{ version }}=*_{{ GIT_DESCRIBE_NUMBER }}
+    - cudf {{ version }}
+    - libcudf_kafka {{ version }}
   run:
-    - libcudf_kafka {{ version }}=*_{{ GIT_DESCRIBE_NUMBER }}
+    - libcudf_kafka {{ version }}
     - python-confluent-kafka
-    - cudf {{ version }}=*_{{ GIT_DESCRIBE_NUMBER }}
+    - cudf {{ version }}
 
 test:
   requires: