Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] Adding SLHC prims. #140

Merged
merged 72 commits into from
Mar 16, 2021
Merged
Show file tree
Hide file tree
Changes from 60 commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
d40ea9e
Moving sparse prims to raft
cjnolet Feb 8, 2021
70c61a8
Pulling some funcitons out of SLHC
cjnolet Feb 8, 2021
d8b3feb
Fixing style
cjnolet Feb 8, 2021
f1fc022
Moving knn graph. Adjusting namespaces
cjnolet Feb 8, 2021
7aa3461
Hiding private elements
cjnolet Feb 8, 2021
e549cb8
Updates
cjnolet Feb 8, 2021
7ddf24b
Merge branch 'imp-018-desymmetrize_mst' into fea-019-slhc_prims
cjnolet Feb 8, 2021
e565cd3
Adding single_linkage
cjnolet Feb 8, 2021
33e559b
Removing typedef in kernel (cuda 10.1 doesn't seem to like it)
cjnolet Feb 9, 2021
376791e
Importing single cub header
cjnolet Feb 9, 2021
fe18ba6
Using proper cub import
cjnolet Feb 9, 2021
8758bd4
Fixing issue in test
cjnolet Feb 9, 2021
7700067
FIxing style
cjnolet Feb 9, 2021
69f0d09
Updating copyright years
cjnolet Feb 10, 2021
878adfa
Merge branch 'imp-019-move_cuml_sparse_prims' into fea-019-slhc_prims
cjnolet Feb 10, 2021
15ffe24
Merge branch 'branch-0.19' into imp-019-move_cuml_sparse_prims
cjnolet Feb 10, 2021
852f6db
Merge branch 'branch-0.19' into imp-019-move_cuml_sparse_prims
cjnolet Feb 11, 2021
39a67d5
Merge branch 'imp-019-move_cuml_sparse_prims' into fea-019-slhc_prims
cjnolet Feb 11, 2021
cba40bd
initialize colors option
divyegala Feb 17, 2021
8f56218
fixing
divyegala Feb 17, 2021
0756bb8
Merge branch 'branch-0.19' into fea-019-slhc_prims
cjnolet Feb 22, 2021
13e08e2
Updating copyright years to 2021
cjnolet Feb 22, 2021
3ccfb6d
Merge branch 'imp-018-desymmetrize_mst' into fea-019-slhc_prims
cjnolet Feb 22, 2021
0a4cd82
Merge branch 'imp-018-desymmetrize_mst' into fea-019-slhc_prims
cjnolet Feb 22, 2021
dfeab1e
Merge branch 'imp-018-desymmetrize_mst' into fea-019-slhc_prims
cjnolet Feb 22, 2021
b734c2d
Merge branch 'branch-0.19' into fea-019-slhc_prims
cjnolet Feb 22, 2021
96562c2
Merge branch 'imp-019-mst_unsymmetric' into fea-019-slhc_prims
cjnolet Feb 24, 2021
1819365
Merge branch 'fea-019-fusedl2nn' into fea-019-slhc_prims
cjnolet Feb 24, 2021
0c02f12
Merge branch 'branch-0.19' into fea-019-slhc_prims
cjnolet Mar 2, 2021
ab934d4
Adding gtest
cjnolet Mar 2, 2021
fc155c0
Moving Mickael's updates over to raft
cjnolet Mar 3, 2021
f6694d3
Updating style
cjnolet Mar 3, 2021
dfdfb49
Merge branch 'imp-019-updating_prims' into fea-019-slhc_prims
cjnolet Mar 3, 2021
83c9abf
Updating style
cjnolet Mar 3, 2021
cc4c203
Removing cuml debugs
cjnolet Mar 3, 2021
ba57d5b
Fixing style
cjnolet Mar 3, 2021
34c8335
Merge branch 'imp-019-updating_prims' into fea-019-slhc_prims
cjnolet Mar 3, 2021
dcc8490
Adding gtest for single linkage. This is going to fail until the knn …
cjnolet Mar 3, 2021
23e516c
SLHC tests are passing when k == n_samples
cjnolet Mar 4, 2021
420c467
updating gtests
cjnolet Mar 4, 2021
97bbdcd
Merge branch 'imp-019-updating_prims' into fea-019-slhc_prims
cjnolet Mar 4, 2021
ef61dae
merge and gtests for restart
divyegala Mar 4, 2021
9784d5c
style check
divyegala Mar 4, 2021
8f3792c
Making progress
cjnolet Mar 4, 2021
498d042
Merge remote-tracking branch 'divye/imp-019-mst_restart' into fea-019…
cjnolet Mar 4, 2021
7f8131c
Updating
cjnolet Mar 4, 2021
b2d7f9c
connected knn graph appears to be working. Need to add assertions in …
cjnolet Mar 4, 2021
5bd914d
Merge remote-tracking branch 'github/branch-0.19' into fea-019-slhc_p…
cjnolet Mar 4, 2021
d3d4414
fixing style
cjnolet Mar 4, 2021
ab5e820
Gtest runs fix_connectivities end-to-end.
cjnolet Mar 4, 2021
ecf2eae
Adding connectivities fix to SLHC
cjnolet Mar 4, 2021
f80b3fd
Adding tests for different prims that enable slhc.
cjnolet Mar 8, 2021
597dfe7
Debugging CI errors
cjnolet Mar 8, 2021
ede30e0
Updating style
cjnolet Mar 8, 2021
fd3fc94
Trying to fix cub issues
cjnolet Mar 9, 2021
fce5e28
Adding proper initialization for mutex
cjnolet Mar 9, 2021
112a80b
Trying to use cub from thrust to fix ci build errors
cjnolet Mar 11, 2021
cd95ae5
Updating knn call to add recently updated param
cjnolet Mar 11, 2021
08b1c04
Merge remote-tracking branch 'github/branch-0.19' into fea-019-slhc_p…
cjnolet Mar 11, 2021
271d3a1
Using updated cub location. trying to remove namespace conflict
cjnolet Mar 11, 2021
e725c35
Pushing up review changes so far
cjnolet Mar 11, 2021
f1394ff
Pushing rest of review updates
cjnolet Mar 11, 2021
6174ecf
Re-enabling explicit cub
cjnolet Mar 11, 2021
58b1692
Hoping updated cub might work
cjnolet Mar 12, 2021
e944800
Updating thrust and cub
cjnolet Mar 12, 2021
03d08ed
Still trying to load new thrust
cjnolet Mar 12, 2021
902d614
Trying out different ortdering
cjnolet Mar 12, 2021
97458d7
Creating internal class for KeyValuePair in hopes to avoid thrust nam…
cjnolet Mar 15, 2021
6c21f0c
Fixing style
cjnolet Mar 15, 2021
e89198b
Removing unecessary timer
cjnolet Mar 16, 2021
3bda071
Cmall change t while loop and threadfence
cjnolet Mar 16, 2021
f9efd32
Style
cjnolet Mar 16, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -213,9 +213,9 @@ set(RAFT_INCLUDE_DIRECTORIES
${FAISS_INCLUDE_DIRS}
${RMM_INCLUDE_DIRS})

if(NOT CUB_IS_PART_OF_CTK)
list(APPEND RAFT_INCLUDE_DIRECTORIES ${CUB_DIR}/src/cub)
endif(NOT CUB_IS_PART_OF_CTK)
#if(NOT CUB_IS_PART_OF_CTK)
# list(APPEND RAFT_INCLUDE_DIRECTORIES ${CUB_DIR}/src/cub)
#endif(NOT CUB_IS_PART_OF_CTK)

if(DEFINED ENV{CONDA_PREFIX})
message(STATUS "Using RMM installation from $ENV{CONDA_PREFIX}")
Expand Down Expand Up @@ -281,6 +281,7 @@ if(BUILD_RAFT_TESTS)
test/sparse/add.cu
test/sparse/convert_coo.cu
test/sparse/convert_csr.cu
test/sparse/connect_components.cu
test/sparse/csr_row_slice.cu
test/sparse/csr_to_dense.cu
test/sparse/csr_transpose.cu
Expand All @@ -290,6 +291,8 @@ if(BUILD_RAFT_TESTS)
test/sparse/distance.cu
test/sparse/filter.cu
test/sparse/knn.cu
test/sparse/knn_graph.cu
test/sparse/linkage.cu
test/sparse/norm.cu
test/sparse/row_op.cu
test/sparse/selection.cu
Expand Down
2 changes: 1 addition & 1 deletion cpp/cmake/Dependencies.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ include(ExternalProject)
if(NOT CUB_IS_PART_OF_CTK)
set(CUB_DIR ${CMAKE_CURRENT_BINARY_DIR}/cub CACHE STRING "Path to cub repo")
ExternalProject_Add(cub
GIT_REPOSITORY https://github.com/thrust/cub.git
GIT_REPOSITORY https://github.com/NVIDIA/cub.git
GIT_TAG 1.8.0
PREFIX ${CUB_DIR}
CONFIGURE_COMMAND ""
Expand Down
48 changes: 48 additions & 0 deletions cpp/include/raft/sparse/hierarchy/common.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
/*
* Copyright (c) 2021, NVIDIA CORPORATION.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

#pragma once

namespace raft {
namespace hierarchy {

enum LinkageDistance { PAIRWISE = 0, KNN_GRAPH = 1 };

/**
* Simple POCO for consolidating linkage results. This closely
* mirrors the trained instance variables populated in
* Scikit-learn's AgglomerativeClustering estimator.
* @tparam value_idx
* @tparam value_t
*/
template <typename value_idx, typename value_t>
struct linkage_output {
value_idx m;
value_idx n_clusters;

value_idx n_leaves;
value_idx n_connected_components;

value_idx *labels; // size: m

value_idx *children; // size: (m-1, 2)
};

struct linkage_output_int_float : public linkage_output<int, float> {};
struct linkage_output__int64_float : public linkage_output<int64_t, float> {};

}; // namespace hierarchy
}; // namespace raft
Loading