Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] HDBSCAN: metric='precomputed' results in AttributeError #4475

Closed
RichieHakim opened this issue Jan 8, 2022 · 1 comment
Closed

[BUG] HDBSCAN: metric='precomputed' results in AttributeError #4475

RichieHakim opened this issue Jan 8, 2022 · 1 comment
Labels
? - Needs Triage Need team to review and classify bug Something isn't working

Comments

@RichieHakim
Copy link

RichieHakim commented Jan 8, 2022

Describe the bug
Attempting to use the 'precomputed' option for cuml.cluster.HDBSCAN results in an empty AttributeError.

Steps/Code to reproduce bug

hdbscan = cuml.cluster.HDBSCAN(metric="precomputed").fit(distances.toarray())

Error received:

/media/rich/OS/Users/Richard/Linux_stuff_on_OS/conda_envs/envs/rapids-21.12/lib/python3.8/site-packages/cuml/internals/api_decorators.py in inner_with_setters(*args, **kwargs)
    407                                 target_val=target_val)
    408 
--> 409                 return func(*args, **kwargs)
    410 
    411         @wraps(func)

cuml/cluster/hdbscan.pyx in cuml.cluster.hdbscan.HDBSCAN.fit()

cuml/common/base.pyx in cuml.common.base.Base.__getattr__()

AttributeError: 

Expected behavior
The function should take in a distance matrix (square shaped. I'm using the output of a KNN run)

Environment details (please complete the following information):

  • Environment location: [Bare-metal, Docker, Cloud(specify cloud provider)]
  • Linux Distro/Architecture: [Ubuntu 20.04 amd64]
  • GPU Model/Driver: [NVIDIA RTX 3090]
  • CUDA: [11.0]
  • Method of cuDF & cuML install: [Conda]
# packages in environment at /media/rich/OS/Users/Richard/Linux_stuff_on_OS/conda_envs/envs/rapids-21.12:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
abseil-cpp                20210324.2           h9c3ff4c_0    conda-forge
anytree                   2.8.0                    pypi_0    pypi
argon2-cffi               21.3.0                   pypi_0    pypi
argon2-cffi-bindings      21.2.0                   pypi_0    pypi
arrow-cpp                 5.0.0           py38h579a05f_22_cuda    conda-forge
arrow-cpp-proc            3.0.0                      cuda    conda-forge
asttokens                 2.0.5                    pypi_0    pypi
attrs                     21.4.0                   pypi_0    pypi
aws-c-cal                 0.5.11               h95a6274_0    conda-forge
aws-c-common              0.6.2                h7f98852_0    conda-forge
aws-c-event-stream        0.2.7               h3541f99_13    conda-forge
aws-c-io                  0.10.5               hfb6a706_0    conda-forge
aws-checksums             0.1.11               ha31a3da_7    conda-forge
aws-sdk-cpp               1.8.186              hb4091e7_3    conda-forge
backcall                  0.2.0                    pypi_0    pypi
bleach                    4.1.0                    pypi_0    pypi
bokeh                     2.4.2            py38h578d9bd_0    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
ca-certificates           2021.10.26           h06a4308_2  
cachetools                5.0.0              pyhd8ed1ab_0    conda-forge
cffi                      1.15.0                   pypi_0    pypi
click                     8.0.3            py38h578d9bd_1    conda-forge
cloudpickle               2.0.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.4                    pypi_0    pypi
cudatoolkit               11.0.221             h6bb024c_0    nvidia
cudf                      21.12.02        cuda_11_py38_g06540b9b37_0    rapidsai
cuml                      21.12.00        cuda11_py38_g04c4927f3_0    rapidsai
cupy                      9.6.0            py38h57017e2_0    conda-forge
cycler                    0.11.0                   pypi_0    pypi
cython                    0.29.26                  pypi_0    pypi
cytoolz                   0.11.2           py38h497a2fe_1    conda-forge
dask                      2021.11.2          pyhd8ed1ab_0    conda-forge
dask-core                 2021.11.2          pyhd8ed1ab_0    conda-forge
dask-cudf                 21.12.02        cuda_11_py38_g06540b9b37_0    rapidsai
debugpy                   1.5.1                    pypi_0    pypi
decorator                 5.1.1                    pypi_0    pypi
defusedxml                0.7.1                    pypi_0    pypi
distlib                   0.3.4                    pypi_0    pypi
distributed               2021.11.2        py38h578d9bd_0    conda-forge
dlpack                    0.5                  h9c3ff4c_0    conda-forge
entrypoints               0.3                      pypi_0    pypi
executing                 0.8.2                    pypi_0    pypi
faiss-proc                1.0.0                      cuda    rapidsai
fastavro                  1.4.9            py38h497a2fe_0    conda-forge
fastrlock                 0.8              py38h709712a_1    conda-forge
fonttools                 4.28.5                   pypi_0    pypi
freetype                  2.10.4               h0708190_1    conda-forge
fsspec                    2021.11.1          pyhd8ed1ab_0    conda-forge
future                    0.18.2                   pypi_0    pypi
gflags                    2.2.2             he1b5a44_1004    conda-forge
glog                      0.5.0                h48cff8f_0    conda-forge
grpc-cpp                  1.42.0               ha1441d3_1    conda-forge
h5py                      3.6.0                    pypi_0    pypi
h5py-wrapper              1.1.0                    pypi_0    pypi
hdfdict                   0.3.1                    pypi_0    pypi
heapdict                  1.0.1                      py_0    conda-forge
icu                       69.1                 h9c3ff4c_0    conda-forge
imageio                   2.13.5                   pypi_0    pypi
importlib-resources       5.4.0                    pypi_0    pypi
ipykernel                 6.6.1                    pypi_0    pypi
ipython                   7.31.0                   pypi_0    pypi
ipython-genutils          0.2.0                    pypi_0    pypi
ipywidgets                7.6.5                    pypi_0    pypi
jbig                      2.1               h7f98852_2003    conda-forge
jedi                      0.18.1                   pypi_0    pypi
jinja2                    3.0.3              pyhd8ed1ab_0    conda-forge
joblib                    1.1.0              pyhd8ed1ab_0    conda-forge
johnnydep                 1.10                     pypi_0    pypi
jpeg                      9d                   h36c2ea0_0    conda-forge
jsonschema                4.3.3                    pypi_0    pypi
jupyter                   1.0.0                    pypi_0    pypi
jupyter-client            7.1.0                    pypi_0    pypi
jupyter-console           6.4.0                    pypi_0    pypi
jupyter-core              4.9.1                    pypi_0    pypi
jupyterlab-pygments       0.1.2                    pypi_0    pypi
jupyterlab-widgets        1.0.2                    pypi_0    pypi
kiwisolver                1.3.2                    pypi_0    pypi
krb5                      1.19.2               hcc1bbae_3    conda-forge
lcms2                     2.12                 hddcbb42_0    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
lerc                      3.0                  h9c3ff4c_0    conda-forge
libblas                   3.9.0           12_linux64_openblas    conda-forge
libbrotlicommon           1.0.9                h7f98852_6    conda-forge
libbrotlidec              1.0.9                h7f98852_6    conda-forge
libbrotlienc              1.0.9                h7f98852_6    conda-forge
libcblas                  3.9.0           12_linux64_openblas    conda-forge
libcudf                   21.12.02        cuda11_g06540b9b37_0    rapidsai
libcuml                   21.12.00        cuda11_g04c4927f3_0    rapidsai
libcumlprims              21.12.00        cuda11_g0a7f19f_0    nvidia
libcurl                   7.81.0               h2574ce0_0    conda-forge
libcusolver               11.3.2.107           hc875929_0    nvidia
libdeflate                1.8                  h7f98852_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.10               h9b69904_4    conda-forge
libfaiss                  1.7.0           cuda110h8045045_8_cuda    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 11.2.0              h1d223b6_11    conda-forge
libgfortran-ng            11.2.0              h69a702a_11    conda-forge
libgfortran5              11.2.0              h5c6108e_11    conda-forge
libgomp                   11.2.0              h1d223b6_11    conda-forge
libhwloc                  2.3.0                h5e5b7d1_1    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblapack                 3.9.0           12_linux64_openblas    conda-forge
libllvm11                 11.1.0               hf817b99_2    conda-forge
libnghttp2                1.43.0               h812cca2_1    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libopenblas               0.3.18          pthreads_h8fe5266_0    conda-forge
libpng                    1.6.37               h21135ba_2    conda-forge
libprotobuf               3.19.2               h780b84a_0    conda-forge
librmm                    21.12.00        cuda11_g957ad04_0    rapidsai
libssh2                   1.10.0               ha56f1ee_2    conda-forge
libstdcxx-ng              11.2.0              he4da1e4_11    conda-forge
libthrift                 0.15.0               he6d91bd_1    conda-forge
libtiff                   4.3.0                h6f004c6_2    conda-forge
libutf8proc               2.7.0                h7f98852_0    conda-forge
libwebp-base              1.2.1                h7f98852_0    conda-forge
libxml2                   2.9.12               h885dcf4_1    conda-forge
libzlib                   1.2.11            h36c2ea0_1013    conda-forge
llvmlite                  0.37.0           py38h4630a5e_1    conda-forge
locket                    0.2.0                      py_2    conda-forge
lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
markupsafe                2.0.1            py38h497a2fe_1    conda-forge
matplotlib                3.5.1                    pypi_0    pypi
matplotlib-inline         0.1.3                    pypi_0    pypi
mistune                   0.8.4                    pypi_0    pypi
msgpack-python            1.0.3            py38h1fd1430_0    conda-forge
nbclient                  0.5.9                    pypi_0    pypi
nbconvert                 6.4.0                    pypi_0    pypi
nbformat                  5.1.3                    pypi_0    pypi
nccl                      2.11.4.1             h96e36e3_0    conda-forge
ncurses                   6.2                  h58526e2_4    conda-forge
nest-asyncio              1.5.4                    pypi_0    pypi
networkx                  2.6.3                    pypi_0    pypi
notebook                  6.4.6                    pypi_0    pypi
numba                     0.54.1           py38h4bf6c61_0    conda-forge
numpy                     1.20.3           py38h9894fe3_1    conda-forge
nvtx                      0.2.3            py38h497a2fe_1    conda-forge
olefile                   0.46               pyh9f0ad1d_1    conda-forge
opencv-contrib-python     4.5.5.62                 pypi_0    pypi
openjpeg                  2.4.0                hb52868f_1    conda-forge
openssl                   1.1.1l               h7f98852_0    conda-forge
opt-einsum                3.3.0                    pypi_0    pypi
orc                       1.7.2                h1be678f_0    conda-forge
oyaml                     1.0                      pypi_0    pypi
packaging                 21.3               pyhd8ed1ab_0    conda-forge
pandas                    1.3.5            py38h43a58ef_0    conda-forge
pandocfilters             1.5.0                    pypi_0    pypi
parquet-cpp               1.5.1                         2    conda-forge
parso                     0.8.3                    pypi_0    pypi
partd                     1.2.0              pyhd8ed1ab_0    conda-forge
pexpect                   4.8.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pillow                    8.4.0            py38h8e6f84c_0    conda-forge
pip                       21.3.1             pyhd8ed1ab_0    conda-forge
pkginfo                   1.8.2                    pypi_0    pypi
prometheus-client         0.12.0                   pypi_0    pypi
prompt-toolkit            3.0.24                   pypi_0    pypi
protobuf                  3.19.2           py38h709712a_0    conda-forge
psutil                    5.9.0            py38h497a2fe_0    conda-forge
ptxcompiler               0.2.0            py38hb739d79_0    rapidsai
ptyprocess                0.7.0                    pypi_0    pypi
pure-eval                 0.2.1                    pypi_0    pypi
pyarrow                   5.0.0           py38ha746e9d_22_cuda    conda-forge
pycparser                 2.21                     pypi_0    pypi
pygments                  2.11.2                   pypi_0    pypi
pyparsing                 3.0.6              pyhd8ed1ab_0    conda-forge
pyrsistent                0.18.0                   pypi_0    pypi
pytest-runner             5.3.1                    pypi_0    pypi
python                    3.8.12          hb7a2778_2_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python_abi                3.8                      2_cp38    conda-forge
pytz                      2021.3             pyhd8ed1ab_0    conda-forge
pywavelets                1.2.0                    pypi_0    pypi
pyyaml                    5.4.1                    pypi_0    pypi
pyzmq                     22.3.0                   pypi_0    pypi
qtconsole                 5.2.2                    pypi_0    pypi
qtpy                      2.0.0                    pypi_0    pypi
re2                       2021.11.01           h9c3ff4c_0    conda-forge
readline                  8.1                  h46c0cb4_0    conda-forge
rmm                       21.12.00        cuda11_py38_g957ad04_0_no_cma    rapidsai
s2n                       1.0.10               h9b69904_0    conda-forge
scanimage-tiff-reader     1.4.1                    pypi_0    pypi
scikit-image              0.19.1                   pypi_0    pypi
scikit-learn              1.0.2                    pypi_0    pypi
scipy                     1.7.3            py38h56a6a73_0    conda-forge
send2trash                1.8.0                    pypi_0    pypi
setuptools                60.3.1           py38h578d9bd_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sklearn                   0.0                      pypi_0    pypi
snappy                    1.1.8                he1b5a44_3    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
spdlog                    1.8.5                h4bd325d_0    conda-forge
sqlite                    3.37.0               h9cd32fc_0    conda-forge
structlog                 21.5.0                   pypi_0    pypi
tabulate                  0.8.9                    pypi_0    pypi
tblib                     1.7.0              pyhd8ed1ab_0    conda-forge
terminado                 0.12.1                   pypi_0    pypi
testpath                  0.5.0                    pypi_0    pypi
threadpoolctl             3.0.0                    pypi_0    pypi
tifffile                  2021.11.2                pypi_0    pypi
tk                        8.6.11               h27826a3_1    conda-forge
toml                      0.10.2                   pypi_0    pypi
toolz                     0.11.2             pyhd8ed1ab_0    conda-forge
torch                     1.9.0+cu111              pypi_0    pypi
torchaudio                0.9.0                    pypi_0    pypi
torchvision               0.10.0+cu111             pypi_0    pypi
tornado                   6.1              py38h497a2fe_2    conda-forge
tqdm                      4.62.3                   pypi_0    pypi
traitlets                 5.1.1                    pypi_0    pypi
treelite                  2.1.0            py38hdd725b4_0    conda-forge
treelite-runtime          2.1.0                    pypi_0    pypi
typing_extensions         4.0.1              pyha770c72_0    conda-forge
ucx                       1.11.2+gef2bbcf      cuda11.0_0    rapidsai
ucx-proc                  1.0.0                       gpu    rapidsai
ucx-py                    0.23.0          py38_gef2bbcf_0    rapidsai
varname                   0.8.1                    pypi_0    pypi
wcwidth                   0.2.5                    pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
widgetsnbextension        3.5.2                    pypi_0    pypi
wimpy                     0.6                      pypi_0    pypi
xz                        5.2.5                h516909a_1    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
zict                      2.0.0                      py_0    conda-forge
zipp                      3.7.0                    pypi_0    pypi
zlib                      1.2.11            h36c2ea0_1013    conda-forge
zstd                      1.5.1                ha95c52a_0    conda-forge

Additional context

  1. I don't it's relevant, but there also seems to be versioning incompatibilities with the hdbscan library. This is despite the fact that some of the methods in cuML's HDBSCAN require hdbscan. The incompatibility is between the combination of numba, numpy, and scikit-learn
  2. Using metric = 'manhattan' seems to kill my kernel also
@RichieHakim RichieHakim added ? - Needs Triage Need team to review and classify bug Something isn't working labels Jan 8, 2022
@cjnolet
Copy link
Member

cjnolet commented Jan 12, 2022

Thanks for opening an issue about this @RichieHakim. You are correct to notice that the bug is in the documentation- it should not state precomputed is an option for the metric argument because that feature is not yet supported. The only distance currently supported is 'euclidean' and we do plan to support other distances in the future. Since it's a duplicate of #4460, I'm going to close this issue and add a note to #4460 to also include only the euclidean metric is supported in the documentation for now. We have a pending item in #3879 to add support more distances.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
? - Needs Triage Need team to review and classify bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants