Matrix-free Schwarz tests fail in the absence of DTK

[lxmota]

The matrix-free Schwarz test that uses this file:

tests/large/LCM/Schwarz/Cubes/cubes_matrix-free.yaml

fails with a segmentation fault here:

src/LCM/evaluators/bc/SchwarzBC_Def.hpp#L254

in the absence of DTK.

I'm not sure if matrix-free Schwarz is supposed to work without DTK, so until there is confirmation one way or another, I'm opening this issue.

Also, this other test fails in the same way:

tests/large/LCM/CrystalPlasticity/SchwarzBar/matrixFree

[ikalash]

I have begun to look into this issue. It seems the reason this issue shows up w/o DTK is that when DTK is off, coupled_solution is obtained from the application via the getX() routine, whereas w/ DTK it is obtained from the STK discretization. Here is a backtrace of the error:

#1 0x00007fffd40326b9 in Tpetra::MultiVector<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false>::get1dView (this=0xe2ecd0) at /home/ikalash/Trilinos/packages/tpetra/core/src/Tpetra_MultiVector_def.hpp:3719
#2 0x00007ffff2ca7e8b in LCM::SchwarzBC_Base<PHAL::AlbanyTraits::Residual, PHAL::AlbanyTraits>::computeBCs (this=0xa28420, ns_node=0,
x_val=@0x7fffffff3f58: 7.2551919556468696e-317, y_val=@0x7fffffff3f50: 2.1219714434102453e-314,
z_val=@0x7fffffff3f48: 2.3099243825617678e-317) at /home/ikalash/Albany/src/LCM/evaluators/bc/SchwarzBC_Def.hpp:254
#3 0x00007ffff2ca3794 in LCM::fillResidual<LCM::SchwarzBC<PHAL::AlbanyTraits::Residual, PHAL::AlbanyTraits>,

get1DView() is failing (SchwarzBC_Def.hpp:254) b/c there is something wrong with coupled_solution when obtained from the application. I haven't figured out what yet (it is not null). Thought I would post this in case @lxmota , this rings any bells, since you wrote the getX() routine and relevant code in SchwarzBC_Def.hpp that uses this routine. I will continue looking into the issue.

[lxmota]

This may be related to that other issue that I could not make DTK work with the TPetra vector that the getX() function returns. This is because the solution vector from STK and the one from getX(), although both TPetra vectors, have different template parameters.

The getX() function is trivial. It just returns a pointer:

https://github.com/gahansen/Albany/blob/master/src/Albany_Application.hpp#L1019

That pointer is passed to computeGlobalResidualImplT here:

https://github.com/gahansen/Albany/blob/master/src/Albany_Application.cpp#L1212

and copied here:

https://github.com/gahansen/Albany/blob/master/src/Albany_Application.cpp#L1263

so it just points to the solution that computeGlobalResidualImplT receives.

I'm not sure we want to spend too much time chasing this weird issue. The old Schwarz BC is now left in the code just for debugging, so I don't think it's critical that we make it work with matrix-free.

[ikalash]

Fixed this; see commit ee0dd11 .