Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BENCH] Bron-Kerbosch clique detection #157

Merged
merged 63 commits into from
Oct 19, 2023
Merged
Changes from 62 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
3fe9e21
- added DFS cycle detection for directed and undirected graph
Hromz Jul 24, 2023
a629352
- corrected code according to comments
Hromz Jul 25, 2023
99f77cd
Apply suggestions from code review
bobluppes Jul 25, 2023
956eac3
Merge branch 'bobluppes:main' into main
Hromz Jul 28, 2023
3ec396d
Merge branch 'bobluppes:main' into main
Hromz Jul 29, 2023
3ba348d
Merge branch 'bobluppes:main' into main
Hromz Aug 1, 2023
432b6b4
- First example of potential use of the library
Hromz Aug 1, 2023
97ccea7
- small fixes
Hromz Aug 1, 2023
3e55e23
Update transport-example.md
Hromz Aug 2, 2023
286e9ce
Merge remote-tracking branch 'upstream/main'
bobluppes Aug 5, 2023
419b663
clang format fix
bobluppes Aug 5, 2023
2408835
Merge remote-tracking branch 'upstream/main'
bobluppes Aug 5, 2023
7e459bf
reflect breaking changes on main
bobluppes Aug 5, 2023
6fc3af0
clang format fix
bobluppes Aug 5, 2023
767a532
- corrected according to git comments
Hromz Aug 12, 2023
8ebb5f4
small fixes
Hromz Aug 12, 2023
ffe4553
small fixes
Hromz Aug 12, 2023
bb28f8c
Merge remote-tracking branch 'upstream/main'
bobluppes Aug 13, 2023
ce5df25
fix issues after merge
bobluppes Aug 13, 2023
1c82f04
fix syntax highlighting code blocks
bobluppes Aug 13, 2023
1c644a0
clang format
bobluppes Aug 13, 2023
a0fde77
Merge branch 'bobluppes:main' into main
Hromz Aug 13, 2023
2ae7d97
Merge branch 'bobluppes:main' into main
Hromz Aug 13, 2023
a3a3b31
Merge branch 'bobluppes:main' into main
Hromz Aug 15, 2023
c05cd27
- Added MST algorithm
Hromz Aug 18, 2023
b5db653
clang style correction
Hromz Aug 18, 2023
057ee77
Apply suggestions from code review
bobluppes Aug 31, 2023
9e92f3f
Merge remote-tracking branch 'upstream/main'
bobluppes Aug 31, 2023
665a484
Merge branch 'bobluppes:main' into main
Hromz Sep 1, 2023
5dce08b
Added topological sort algorithm DFS-based
Hromz Sep 5, 2023
03200f7
Merge branch 'main' into main
Hromz Sep 5, 2023
f8cd4fb
small fixes
Hromz Sep 5, 2023
21cde40
Merge branch 'main' of https://github.com/Hromz/graaf
Hromz Sep 5, 2023
14f0b8d
clang format fix
Hromz Sep 5, 2023
7efd134
added test
Hromz Sep 5, 2023
0bcad82
more tests
Hromz Sep 5, 2023
f96849a
Minor fixes
Hromz Sep 10, 2023
57d3c47
Clang format
Hromz Sep 10, 2023
c5ddba1
Update topological-sort.md
Hromz Sep 10, 2023
3c94ab6
Merge branch 'bobluppes:main' into main
Hromz Sep 13, 2023
19ea438
DOCS for DFS based cycle detection
Hromz Sep 14, 2023
68c2c2b
small fixes
Hromz Sep 14, 2023
55d3972
Update dfs-based.md
Hromz Sep 18, 2023
39487bd
Merge branch 'bobluppes:main' into main
Hromz Sep 19, 2023
da97d62
Added Floyd-WArshall algorithm
Hromz Sep 24, 2023
7a1ca3c
clang tidy
Hromz Sep 24, 2023
4a43bce
removed redundant negative cicle check and according tests, added two…
Hromz Sep 27, 2023
773a708
Merge branch 'bobluppes:main' into main
Hromz Sep 28, 2023
e0a2978
Merge branch 'bobluppes:main' into main
Hromz Sep 30, 2023
8441998
Merge branch 'bobluppes:main' into main
Hromz Sep 30, 2023
4e843a9
Merge branch 'bobluppes:main' into main
Hromz Oct 2, 2023
f3274c9
Bron-Kerbosch algorithm
Hromz Oct 6, 2023
9ac7524
Added to readme
Hromz Oct 6, 2023
14ccc5a
corrected according to comments
Hromz Oct 9, 2023
27e6894
Update README.md
Hromz Oct 9, 2023
f7ce62d
Merge branch 'main' into main
Hromz Oct 9, 2023
6969839
folder name correction, added json to docs
Hromz Oct 9, 2023
7f19ab7
Merge branch 'main' of https://github.com/Hromz/graaf
Hromz Oct 9, 2023
0f9ffe8
small correction in name and headers
Hromz Oct 9, 2023
0fb11b1
Merge branch 'bobluppes:main' into main
Hromz Oct 10, 2023
116e1f5
Merge branch 'bobluppes:main' into main
Hromz Oct 11, 2023
ef490b8
benchmark test Bron-Kerbosch
Hromz Oct 12, 2023
7b42950
Changed according to comments
Hromz Oct 16, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
245 changes: 245 additions & 0 deletions perf/graaflib/bron_kerbosch_benchmark.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,245 @@
#include <benchmark/benchmark.h>
#include <graaflib/algorithm/clique_detection/bron_kerbosch.h>
#include <graaflib/graph.h>

#include <random>
#include <vector>

namespace {

// Generating random number of vertices (1- 30) for a clique
std::random_device dev;
std::mt19937 rng(dev());
std::uniform_int_distribution<std::mt19937::result_type> random_clique_size(1,
30);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am always a little hesitant adding such non deterministic benchmarks, as it makes comparing results difficult. It might still be nice to have since we use it to determine the runtime complexity as well, just something to be aware of.


template <typename EDGE_T>
[[nodiscard]] std::vector<graaf::vertex_id_t> create_vertices(
graaf::undirected_graph<int, EDGE_T>& graph, size_t n) {
std::vector<graaf::vertex_id_t> vertices{};
vertices.reserve(n);

for (size_t i{0}; i < n; ++i) {
vertices.push_back(graph.add_vertex(i));
}

return vertices;
}

template <typename EDGE_T>
void add_clique(graaf::undirected_graph<int, EDGE_T>& graph,
const std::vector<graaf::vertex_id_t>& vertices,
size_t start_vertex, size_t clique_size) {
// We are in range of vector of vertices
size_t end_vertex = std::min(start_vertex + clique_size, vertices.size());
std::vector<graaf::edge_id_t> clique{};
clique.reserve((clique_size * (clique_size - 1)) / 2);

// Constructing a clique
for (size_t i{start_vertex}; i < end_vertex; ++i) {
for (size_t j{i + 1}; j < end_vertex; ++j) {
clique.emplace_back(vertices[i], vertices[j]);
}
}

for (auto& edge_t : clique) {
graph.add_edge(edge_t.first, edge_t.second, 1);
}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the need of storing the edges in the vector clique first? Can we not directly add them to the graph? i.e.

Suggested change
std::vector<graaf::edge_id_t> clique{};
clique.reserve((clique_size * (clique_size - 1)) / 2);
// Constructing a clique
for (size_t i{start_vertex}; i < end_vertex; ++i) {
for (size_t j{i + 1}; j < end_vertex; ++j) {
clique.emplace_back(vertices[i], vertices[j]);
}
}
for (auto& edge_t : clique) {
graph.add_edge(edge_t.first, edge_t.second, 1);
}
// Constructing a clique
for (size_t i{start_vertex}; i < end_vertex; ++i) {
for (size_t j{i + 1}; j < end_vertex; ++j) {
clique.emplace_back(vertices[i], vertices[j]);
graph.add_edge(vertices[i], vertices[j], 1);
}
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh my bad, I overconfused myself.

}
} // namespace

static void bron_kerbosh_two_vertices_cliques(benchmark::State& state) {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand the code correctly, we create a graph with a linear path of connected vertices:
a -- b -- c -- ...

This was not immediately clear to me when looking at the benchmark name. What do you think about calling this something like bron_kerbosch_linear_graph or bron_kerbosch_two_vertices_per_clique? To make it extra clear we could also add a small comment on top of each benchmark explaining the setup.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, that definitely will be a better explanation of what code does.

const auto number_of_edges{static_cast<size_t>(state.range(0))};

graaf::undirected_graph<int, int> graph{};

// We create enough vertices to construct the requested number of edges
const auto number_of_vertices{number_of_edges + 1};
const auto vertices{create_vertices(graph, number_of_vertices)};

for (auto _ : state) {
for (size_t i{0}; i < number_of_edges; ++i) {
graph.add_edge(vertices[i], vertices[i + 1], 1);
}
}

graaf::algorithm::bron_kerbosch(graph);
state.SetComplexityN(state.range(0));
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the docs the to-be-benchmarked code should go into the body of this for-loop. I therefore think we should add the edges outside of the loop and call bron_kerbosch inside.

In order to make sure the compiler does not optimize the call to the algorithm away (since the result is unused) it would also be good to add benchmark::DoNotOptimize:

Suggested change
for (auto _ : state) {
for (size_t i{0}; i < number_of_edges; ++i) {
graph.add_edge(vertices[i], vertices[i + 1], 1);
}
}
graaf::algorithm::bron_kerbosch(graph);
state.SetComplexityN(state.range(0));
for (size_t i{0}; i < number_of_edges; ++i) {
graph.add_edge(vertices[i], vertices[i + 1], 1);
}
for (auto _ : state) {
auto result = graaf::algorithm::bron_kerbosch(graph);
benchmark::DoNotOptimize(result);
}
state.SetComplexityN(state.range(0));

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for the other benchmarks, maybe you could also take a look there

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright!

}

static void bron_kerbosh_dense_graph(benchmark::State& state) {
const auto number_of_edges{static_cast<size_t>(state.range(0))};

graaf::undirected_graph<int, int> graph{};

// We create enough vertices to construct the requested number of edges
const auto number_of_vertices{number_of_edges + 1};
const auto vertices{create_vertices(graph, number_of_vertices)};

for (auto _ : state) {
for (size_t i{0}; i < number_of_edges; ++i) {
for (size_t j{i + 1}; j < number_of_edges; ++j) {
if (i != j && !graph.has_edge(vertices[i], vertices[j]))
graph.add_edge(vertices[i], vertices[j], 1);
}
}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at these for loops, would the condition in the if-statement ever evaluate to false? I think this is equivalent to:

for (size_t i{0}; i < number_of_edges; ++i) {
      for (size_t j{i + 1}; j < number_of_edges; ++j) {
        graph.add_edge(vertices[i], vertices[j], 1);
      }
    }

Which in turn looks pretty similar to the add_clique function. Could we therefore simply do:

add_clique(graph, vertices, vertices.front(), number_of_edges);

Copy link
Contributor Author

@Hromz Hromz Oct 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^^ Oh, how I mised that! Thanks for emphasizing this!

}

graaf::algorithm::bron_kerbosch(graph);
state.SetComplexityN(state.range(0));
}

static void bron_kerbosh_graph_triangle_cliques(benchmark::State& state) {
const auto number_of_edges{static_cast<size_t>(state.range(0))};

graaf::undirected_graph<int, int> graph{};

// We create enough vertices to construct the requested number of edges
const auto number_of_vertices{number_of_edges + 1};
const auto vertices{create_vertices(graph, number_of_vertices)};

size_t clique_size = 4;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the clique_size not be equal to 3 if we want to make triangle cliques?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad with naming, basically,  it's a triangle with a dot inside (4 vertex clique).


for (auto _ : state) {
for (size_t i{0}; i < number_of_edges; i += clique_size) {
std::vector<graaf::edge_id_t> clique{};
add_clique(graph, vertices, i, clique_size);
}
}

graaf::algorithm::bron_kerbosch(graph);
state.SetComplexityN(state.range(0));
}

static void bron_kerbosh_graph_connected_triangle_cliques(
benchmark::State& state) {
const auto number_of_edges{static_cast<size_t>(state.range(0))};

graaf::undirected_graph<int, int> graph{};

// We create enough vertices to construct the requested number of edges
const auto number_of_vertices{number_of_edges + 1};
const auto vertices{create_vertices(graph, number_of_vertices)};

size_t clique_size = 4;

for (auto _ : state) {
// Connecting all cliques
for (size_t i{0}; i + clique_size < number_of_edges; i += clique_size) {
graph.add_edge(vertices[i], vertices[i + clique_size], 1);
}

for (size_t i{0}; i < number_of_edges; i += clique_size) {
std::vector<graaf::edge_id_t> clique{};
add_clique(graph, vertices, i, clique_size);
}
}

graaf::algorithm::bron_kerbosch(graph);
state.SetComplexityN(state.range(0));
}

static void bron_kerbosh_graph_large_cliques(benchmark::State& state) {
const auto number_of_edges{static_cast<size_t>(state.range(0))};

graaf::undirected_graph<int, int> graph{};

// We create enough vertices to construct the requested number of edges
const auto number_of_vertices{number_of_edges + 1};
const auto vertices{create_vertices(graph, number_of_vertices)};

size_t clique_size = 25;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also do this in a follow-up, but looking at these parameters it looks like we can pull out the clique_size as a second parameter. Then we might be able to consolidate a few of these benchmarks. I.e.

BENCHMARK(bron_kerbosh_connected_cliques)
    ->Ranges({{100, 10000}, {3, 100}})  // total number of edges, clique size
    ->Unit(benchmark::kMillisecond)
    ->Complexity();

Copy link
Contributor Author

@Hromz Hromz Oct 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want I would like to add this in current PR!


for (auto _ : state) {
for (size_t i{0}; i < number_of_edges; i += clique_size) {
std::vector<graaf::edge_id_t> clique{};
add_clique(graph, vertices, i, clique_size);
}
}

graaf::algorithm::bron_kerbosch(graph);
state.SetComplexityN(state.range(0));
}

static void bron_kerbosh_graph_mixed_random_cliques(benchmark::State& state) {
const auto number_of_edges{static_cast<size_t>(state.range(0))};

graaf::undirected_graph<int, int> graph{};

// We create enough vertices to construct the requested number of edges
const auto number_of_vertices{number_of_edges + 1};
const auto vertices{create_vertices(graph, number_of_vertices)};

size_t clique_size = 3;

for (auto _ : state) {
for (size_t i{0}; i + clique_size < number_of_edges; i += clique_size) {
std::vector<graaf::edge_id_t> clique{};
add_clique(graph, vertices, i, clique_size++);

clique_size = random_clique_size(rng);
}
}

graaf::algorithm::bron_kerbosch(graph);
state.SetComplexityN(state.range(0));
}

static void bron_kerbosh_graph_connected_mixed_random_cliques(
benchmark::State& state) {
const auto number_of_edges{static_cast<size_t>(state.range(0))};

graaf::undirected_graph<int, int> graph{};

// We create enough vertices to construct the requested number of edges
const auto number_of_vertices{number_of_edges + 1};
const auto vertices{create_vertices(graph, number_of_vertices)};

size_t clique_size = 3;

for (auto _ : state) {
// Connecting all cliques
for (size_t i{0}; i < number_of_edges; ++i) {
graph.add_edge(vertices[i], vertices[i + 1], 1);
}

for (size_t i{0}; i + clique_size < number_of_edges; i += clique_size) {
std::vector<graaf::edge_id_t> clique{};
add_clique(graph, vertices, i, clique_size++);

clique_size = random_clique_size(rng);
}
}

graaf::algorithm::bron_kerbosch(graph);
state.SetComplexityN(state.range(0));
}

BENCHMARK(bron_kerbosh_two_vertices_cliques)
->Range(100, 10000)
->Unit(benchmark::kMillisecond)
->Complexity();
BENCHMARK(bron_kerbosh_dense_graph) /* large amount of vertices to process */
->Range(100, 1000)
->Unit(benchmark::kMillisecond)
->Complexity();
BENCHMARK(bron_kerbosh_graph_triangle_cliques)
->Range(100, 10000)
->Unit(benchmark::kMillisecond)
->Complexity();
BENCHMARK(bron_kerbosh_graph_connected_triangle_cliques)
->Range(100, 10000)
->Unit(benchmark::kMillisecond)
->Complexity();
BENCHMARK(bron_kerbosh_graph_large_cliques)
->Range(100, 10000)
->Unit(benchmark::kMillisecond)
->Complexity();
BENCHMARK(bron_kerbosh_graph_mixed_random_cliques)
->Range(100, 10000)
->Unit(benchmark::kMillisecond)
->Complexity();
BENCHMARK(bron_kerbosh_graph_connected_mixed_random_cliques)
->Range(100, 10000)
->Unit(benchmark::kMillisecond)
->Complexity();