Feature/power

[maddyscientist]

This PR is the initial step towards more power awareness in QUDA, as well as adding OMP threading for host kernels

• Adds power, temperature and clock monitoring
  • Monitoring is enabled with QUDA_ENABLE_MONITOR=1 (default is off)
  • Monitoring is performed on a spawned thread, maintaining the history in a linked list
  • Default monitoring period is QUDA_ENABLE_MONITOR_PERIOD=1
  • Monitor info, together with derived energy usage is dumped to a monitor_*****.tsv file, where the **** encodes the rank id, and the date_time of the dump. All ranks have identical times by construction.
• Add OpenMP threading support for all CPU kernels
  • Most kernels seem to get reasonable scaling
  • QUDA_OPENMP CMake parameter is no longer marked as advanced
• Fixes compiler warning introduced in staggered contraction code #1416
• Fixes bug introduced in staggered contraction code #1416
• Fixed an issue with endQuda if memory leaks were detected when running multi-GPU: printfQuda would fail since comm_rank() would be called after the comms have been torn down

[bjoo]

After a quick scan this looks fine to me, I had only a few OpenMP comments I left as single comments regarding the use of collapse() I saw you use it in some cases and not in others, but I was just wondering if it could be used in one or two more. The only thing I worry about is dropping a functor ( Arg::apply ) or some such into a reduction clause. I wonder if it ought to be a custom reduction (like done for Multi-Reductions).
This could be just me not being up to spec with my OpenMP though. Modulo these I approve.