Overset intergrid boundary and hole search

[jbakosi]

Overset intergrid boundary and hole search

---

This change is 

[adityakpandare]

This is great work @jbakosi! Thank you! A couple of comments below.

Reviewable status: 0 of 24 files reviewed, 2 unresolved discussions (waiting on @jbakosi)

---

src/Inciter/OversetFE.cpp line 219 at r1 (raw file):

  }

  for (std::size_t e=0; e<inpoel.size()/4; e++) {

Can we invert this set of loops? i.e. loop over points in bp and flag elements surrounding these points using esup.

---

src/Inciter/OversetFE.cpp line 923 at r1 (raw file):

          if (blid == 103) {
            for (auto i : ndset) m_uc(i,iflag) = 1.0;
          }

We should now be able to remove this hard coding, since setupIntergridBoundaries() sets up these flags automatically (not based on blid==103 hardcoded.

[jbakosi]

Reviewable status: 0 of 24 files reviewed, 2 unresolved discussions (waiting on @adityakpandare)

---

src/Inciter/OversetFE.cpp line 219 at r1 (raw file):

Previously, adityakpandare (Aditya Pandare) wrote…

Can we invert this set of loops? i.e. loop over points in bp and flag elements surrounding these points using esup.

Now that I think about this more, I'm not sure if exchanging the loops is clearly better.

If the outer loop is over the set of already-tagged points (in bp), we now need to look in psup to see what edges are connected to the points in bp. That needs the data structure psup first, generated for the full mesh. This is not stored by default. We could store it and then that would be the price. There is always a price of course. Which one is better, only a measurement can decide.

[jbakosi]

@adityakpandare I think is ready to merge - after a final look. Thanks for the help with this.

[adityakpandare]

Thank you @jbakosi. The CI ran into the following problems:

1. The asynclogic_oversetfe tests fails with Exception: Quiescence or another error detected. I am sending the stack trace to your email presently.
2. All the builds (gnu, intel, release, debug) hang because one of the asynclogic_oversetfe tests deadlocks.
3. clang throws the following compile-time warnings (-Werror):

src/Inciter/Transporter.cpp:515:66: error: implicit conversion loses integer precision:
'unsigned long' to 'std::unordered_set<int, std::hash<int>, std::equal_to<int>, std::allocator<int>>::value_type' (aka 'int')
[-Werror,-Wshorten-64-to-32]

   for (const auto& ibs : ib) for (auto s : ibs) usedsets.insert( s );
                                                          ~~~~~~  ^
src/Inciter/Discretization.cpp:489:27: error: implicit conversion changes signedness:
'std::tuple_element<0, const std::pair<const int, std::vector<unsigned long, std::allocator<unsigned long>>>>::type' (aka 'const int')
to 'const std::unordered_set<unsigned long, std::hash<unsigned long>, std::equal_to<unsigned long>, std::allocator<unsigned long>>::key_type' (aka 'const unsigned long')
[-Werror,-Wsign-conversion]

             if ( is.count(setid) ) {
                     ~~~~~ ^~~~~

src/Inciter/OversetFE.cpp:187:19: error: implicit conversion changes signedness:
'std::tuple_element<0, const std::pair<const int, std::vector<unsigned long, std::allocator<unsigned long>>>>::type' (aka 'const int')
to 'const std::unordered_set<unsigned long, std::hash<unsigned long>, std::equal_to<unsigned long>, std::allocator<unsigned long>>::key_type' (aka 'const unsigned long')
[-Werror,-Wsign-conversion]

     if ( is.count(setid) ) {
             ~~~~~ ^~~~~

src/Inciter/OversetFE.cpp:288:15: error: unused variable 'ib' [-Werror,-Wunused-variable]

   const auto& ib = g_inputdeck.get< tag::param, tag::compflow,
               ^

Could you take a look at these?

Reviewed 9 of 24 files at r1, all commit messages.
Reviewable status: 9 of 42 files reviewed, 1 unresolved discussion (waiting on @jbakosi)

[jbakosi]

The asynclogic problems (partially) should be taken care of by 7bdcd1e. Those that migrate still deadlock, you could disable migration for those by

diff --git a/tests/regression/inciter/transport/SlotCyl/asynclogic/CMakeLists.txt b/tests/regression/inciter/transport/SlotCyl/asynclogic/CMakeLists.txt
index 9ded28ef3..aba6eff55 100644
--- a/tests/regression/inciter/transport/SlotCyl/asynclogic/CMakeLists.txt
+++ b/tests/regression/inciter/transport/SlotCyl/asynclogic/CMakeLists.txt
@@ -27,6 +27,7 @@ function(add_async_test scheme virt npes ppn migr)
     set(ARG "-c slot_cyl_${scheme}.q -v -u ${virt} -b ${migr_cmd}")
     set(TEXT_RES "diag.0;diag.1")
     set(TEXT_BAS "noop.ndiff.cfg;noop.ndiff.cfg")
+    set(migr_cmd "")
   else()
     set(ARG "-c slot_cyl_${scheme}.q -i unitcube_1k.exo -v -u ${virt} -b ${migr_cmd}")
     set(TEXT_RES diag)

[jbakosi]

I did find a couple of data structures in Discretization that I forgot to pup, fixed in 3d5a030, but that still does not fix the migrating overset asynclogic tests. At this point I'm not sure how to fix those. Unless we come up with something obvious, I vote for disabling those tests, and putting migration to the overset todo list.