Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable edge masking in the remaining primitives #4186

Merged
merged 59 commits into from
Mar 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
0f7860e
update weight_sum_test to match other tests
seunghwak Jan 30, 2024
f22e31b
enalbe edge masking in compute_in|out_weight_sums
seunghwak Jan 30, 2024
d6848f5
test edge masking in mg_per_v_pair_transfomr_dst_nbr_weighted_interse…
seunghwak Jan 30, 2024
51efd36
bug fix
seunghwak Jan 31, 2024
3b814a3
add const to functions that can be const
seunghwak Feb 1, 2024
8f48eb9
update detail::extract_transform_v_frontier_e to support edge masking
seunghwak Feb 2, 2024
df6b4b6
replace uint32_t{0xffffffff} with raft::warp_full_mask()
seunghwak Feb 7, 2024
f1ad089
update/performance tune detail::extract_transform_v_froniter_e with e…
seunghwak Feb 7, 2024
3492ee9
mark fill_edge_src|dst_property.cuh as edge-masking ready
seunghwak Feb 7, 2024
3036bc7
add compute_number_of_edges_with_mask & compute_local_degree_with_mask
seunghwak Feb 7, 2024
fc3b380
add missing include
seunghwak Feb 7, 2024
217ef47
update extract_transform_v_frontier_outgoing_e to support edge masking
seunghwak Feb 7, 2024
6a44b55
update transform_reduce_v_frontier_outgoing_e_by_dst to support edge …
seunghwak Feb 7, 2024
60fd404
update extract_transform_e to support edge masking
seunghwak Feb 7, 2024
c5f556a
Merge branch 'branch-24.04' of https://github.com/rapidsai/cugraph in…
seunghwak Feb 7, 2024
d44c868
clang-format
seunghwak Feb 7, 2024
02de2b7
Merge branch 'branch-24.04' of https://github.com/rapidsai/cugraph in…
seunghwak Feb 8, 2024
f18fdf8
Merge branch 'branch-24.04' of https://github.com/rapidsai/cugraph in…
seunghwak Feb 9, 2024
1ddf922
resolve merge conflicts
seunghwak Feb 9, 2024
0b40837
copyright year
seunghwak Feb 9, 2024
b71f656
clang-format
seunghwak Feb 9, 2024
595ae92
fix compile error
seunghwak Feb 10, 2024
2cf155f
Merge branch 'branch-24.04' of https://github.com/rapidsai/cugraph in…
seunghwak Feb 12, 2024
c164b4c
Merge branch 'branch-24.04' of https://github.com/rapidsai/cugraph in…
seunghwak Feb 15, 2024
1089a60
Merge branch 'branch-24.04' of https://github.com/rapidsai/cugraph in…
seunghwak Feb 20, 2024
0ceef0b
update primitives to support edge masking
seunghwak Feb 21, 2024
0250fe5
Merge branch 'branch-24.04' of https://github.com/rapidsai/cugraph in…
seunghwak Feb 23, 2024
eb39713
code refactoring
seunghwak Feb 23, 2024
9cdddd6
Merge branch 'branch-24.04' of https://github.com/rapidsai/cugraph in…
seunghwak Feb 23, 2024
5760ff2
update primitives to support edge masking
seunghwak Feb 23, 2024
6e65d41
add tests
seunghwak Feb 23, 2024
62fe45e
remove unnecessary target_link_libraries statements
seunghwak Feb 24, 2024
aae5bba
update tests
seunghwak Feb 26, 2024
4732ce9
bug fix (overflow if edge_t is 32-bit integer)
seunghwak Feb 28, 2024
94a48c5
Merge branch 'branch-24.04' of https://github.com/rapidsai/cugraph in…
seunghwak Feb 28, 2024
dea19e1
resolve merge conflicts
seunghwak Mar 4, 2024
5206cb0
Merge branch 'branch-24.04' of https://github.com/rapidsai/cugraph in…
seunghwak Mar 4, 2024
def17db
clang-format
seunghwak Mar 5, 2024
6640c0d
add to_std_optional
seunghwak Mar 5, 2024
c78f123
edge mask support update
seunghwak Mar 5, 2024
6105ba8
address FIXMEs
seunghwak Mar 6, 2024
9bf5ad5
add tetes in CMakeFiles.txt
seunghwak Mar 6, 2024
bf89d92
bug fix
seunghwak Mar 6, 2024
a9dacdb
clang-format
seunghwak Mar 6, 2024
8dfc337
bug fix
seunghwak Mar 7, 2024
aa7b666
Merge branch 'upstream_pr4221' into fea_more_prim_edge_masking
seunghwak Mar 7, 2024
7232aec
Merge branch 'branch-24.04' of https://github.com/rapidsai/cugraph in…
seunghwak Mar 7, 2024
e26ea63
fix typo
seunghwak Mar 8, 2024
0407613
update documentation
seunghwak Mar 8, 2024
4eac0cd
fix compile errors
seunghwak Mar 8, 2024
8c87dbc
add tests
seunghwak Mar 8, 2024
67cc175
bug fix
seunghwak Mar 8, 2024
7458a64
bug fix
seunghwak Mar 9, 2024
ffae466
resolve merge conflicts
seunghwak Mar 9, 2024
82adb9c
Merge branch 'branch-24.04' of https://github.com/rapidsai/cugraph in…
seunghwak Mar 11, 2024
1e7f717
clang-format
seunghwak Mar 11, 2024
0bfe0dd
add a FIXME comment
seunghwak Mar 11, 2024
93711a2
Merge branch 'branch-24.04' of https://github.com/rapidsai/cugraph in…
seunghwak Mar 11, 2024
71127a6
Merge branch 'branch-24.04' into fea_more_prim_edge_masking
seunghwak Mar 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions cpp/include/cugraph/graph_view.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -613,7 +613,7 @@ class graph_view_t<vertex_t, edge_t, store_transposed, multi_gpu, std::enable_if
major_value_range_start_offset);
}

// FIXME: deprecated, replaced with copmute_number_of_edges (which works with or without edge
// FIXME: deprecated, replaced with compute_number_of_edges (which works with or without edge
// masking)
edge_t number_of_edges() const
{
Expand Down Expand Up @@ -923,7 +923,7 @@ class graph_view_t<vertex_t, edge_t, store_transposed, multi_gpu, std::enable_if
offsets_, indices_, this->number_of_vertices());
}

// FIXME: deprecated, replaced with copmute_number_of_edges (which works with or without edge
// FIXME: deprecated, replaced with compute_number_of_edges (which works with or without edge
// masking)
edge_t number_of_edges() const
{
Expand Down
8 changes: 8 additions & 0 deletions cpp/include/cugraph/utilities/misc_utils.cuh
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,14 @@ thrust::optional<T> to_thrust_optional(std::optional<T> val)
return ret;
}

template <typename T>
std::optional<T> to_std_optional(thrust::optional<T> val)
{
std::optional<T> ret{std::nullopt};
if (val) { ret = *val; }
return ret;
}

template <typename idx_t, typename offset_t>
rmm::device_uvector<idx_t> expand_sparse_offsets(raft::device_span<offset_t const> offsets,
idx_t base_idx,
Expand Down
11 changes: 7 additions & 4 deletions cpp/src/prims/detail/nbr_intersection.cuh
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
#include <thrust/iterator/transform_iterator.h>
#include <thrust/iterator/zip_iterator.h>
#include <thrust/optional.h>
#include <thrust/reduce.h>
#include <thrust/remove.h>
#include <thrust/scan.h>
#include <thrust/set_operations.h>
Expand Down Expand Up @@ -1232,9 +1233,11 @@ nbr_intersection(raft::handle_t const& handle,
rx_v_pair_nbr_intersection_sizes.size() + 1, handle.get_stream());
rx_v_pair_nbr_intersection_offsets.set_element_to_zero_async(size_t{0},
handle.get_stream());
auto size_first = thrust::make_transform_iterator(
rx_v_pair_nbr_intersection_sizes.begin(), cugraph::detail::typecast_t<edge_t, size_t>{});
thrust::inclusive_scan(handle.get_thrust_policy(),
rx_v_pair_nbr_intersection_sizes.begin(),
rx_v_pair_nbr_intersection_sizes.end(),
size_first,
size_first + rx_v_pair_nbr_intersection_sizes.size(),
rx_v_pair_nbr_intersection_offsets.begin() + 1);

rx_v_pair_nbr_intersection_indices.resize(
Expand Down Expand Up @@ -1344,8 +1347,8 @@ nbr_intersection(raft::handle_t const& handle,
}

thrust::inclusive_scan(handle.get_thrust_policy(),
rx_v_pair_nbr_intersection_sizes.begin(),
rx_v_pair_nbr_intersection_sizes.end(),
size_first,
size_first + rx_v_pair_nbr_intersection_sizes.size(),
rx_v_pair_nbr_intersection_offsets.begin() + 1);

std::vector<size_t> h_rx_v_pair_lasts(rx_v_pair_counts.size());
Expand Down
529 changes: 375 additions & 154 deletions cpp/src/prims/per_v_transform_reduce_dst_key_aggregated_outgoing_e.cuh

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
#include "prims/detail/nbr_intersection.cuh"
#include "prims/property_op_utils.cuh"

#include <cugraph/detail/decompress_edge_partition.cuh>
#include <cugraph/edge_partition_device_view.cuh>
#include <cugraph/edge_partition_endpoint_property_device_view.cuh>
#include <cugraph/edge_src_dst_property.hpp>
Expand Down Expand Up @@ -130,7 +131,9 @@ std::tuple<rmm::device_uvector<vertex_t>, ValueBuffer> sort_and_reduce_by_vertic
vertices.end(),
get_dataframe_buffer_begin(value_buffer),
reduced_vertices.begin(),
get_dataframe_buffer_begin(reduced_value_buffer));
get_dataframe_buffer_begin(reduced_value_buffer),
thrust::equal_to<vertex_t>{},
property_op<value_t, thrust::plus>{});

vertices.resize(size_t{0}, handle.get_stream());
resize_dataframe_buffer(value_buffer, size_t{0}, handle.get_stream());
Expand Down Expand Up @@ -201,14 +204,14 @@ struct accumulate_vertex_property_t {
* @param graph_view Non-owning graph object.
* @param edge_src_value_input Wrapper used to access source input property values (for the edge
* sources assigned to this process in multi-GPU). Use either cugraph::edge_src_property_t::view()
* (if @p e_op needs to access source property values) or cugraph::edge_src_dummy_property_t::view()
* (if @p e_op does not access source property values). Use update_edge_src_property to fill the
* wrapper.
* (if @p intersection_op needs to access source property values) or
* cugraph::edge_src_dummy_property_t::view() (if @p intersection_op does not access source property
* values). Use update_edge_src_property to fill the wrapper.
* @param edge_dst_value_input Wrapper used to access destination input property values (for the
* edge destinations assigned to this process in multi-GPU). Use either
* cugraph::edge_dst_property_t::view() (if @p e_op needs to access destination property values) or
* cugraph::edge_dst_dummy_property_t::view() (if @p e_op does not access destination property
* values). Use update_edge_dst_property to fill the wrapper.
* cugraph::edge_dst_property_t::view() (if @p intersection_op needs to access destination property
* values) or cugraph::edge_dst_dummy_property_t::view() (if @p intersection_op does not access
* destination property values). Use update_edge_dst_property to fill the wrapper.
* @param intersection_op quinary operator takes edge source, edge destination, property values for
* the source, property values for the destination, and a list of vertices in the intersection of
* edge source & destination vertices' destination neighbors and returns a thrust::tuple of three
Expand Down Expand Up @@ -260,8 +263,6 @@ void transform_reduce_dst_nbr_intersection_of_e_endpoints_by_v(
typename EdgeDstValueInputWrapper::value_iterator,
typename EdgeDstValueInputWrapper::value_type>>;

CUGRAPH_EXPECTS(!graph_view.has_edge_mask(), "unimplemented.");

if (do_expensive_check) {
// currently, nothing to do.
}
Expand All @@ -272,6 +273,7 @@ void transform_reduce_dst_nbr_intersection_of_e_endpoints_by_v(
init);

auto edge_mask_view = graph_view.edge_mask_view();

for (size_t i = 0; i < graph_view.number_of_local_edge_partitions(); ++i) {
auto edge_partition =
edge_partition_device_view_t<vertex_t, edge_t, GraphViewType::is_multi_gpu>(
Expand Down Expand Up @@ -484,7 +486,9 @@ void transform_reduce_dst_nbr_intersection_of_e_endpoints_by_v(
merged_vertices.end(),
get_dataframe_buffer_begin(merged_value_buffer),
reduced_vertices.begin(),
get_dataframe_buffer_begin(reduced_value_buffer));
get_dataframe_buffer_begin(reduced_value_buffer),
thrust::equal_to<vertex_t>{},
property_op<T, thrust::plus>{});
merged_vertices.resize(size_t{0}, handle.get_stream());
merged_vertices.shrink_to_fit(handle.get_stream());
resize_dataframe_buffer(merged_value_buffer, size_t{0}, handle.get_stream());
Expand Down
Loading
Loading