Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get PFUnit working on melvin. #1844

Merged
merged 3 commits into from
Aug 30, 2017
Merged

Get PFUnit working on melvin. #1844

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a5e9f09
Get PFUnit working on melvin.
jgfouca Aug 28, 2017
3646998
Cmake needs to load macros
jgfouca Aug 29, 2017
35077ea
Additional fixes to V1 cmake macro generation
jgfouca Aug 30, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions config/acme/machines/config_compilers.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -673,6 +673,7 @@ for mct, etc.
<CONFIG_ARGS> --host=Linux </CONFIG_ARGS>
<NETCDF_PATH>$(NETCDFROOT)</NETCDF_PATH>
<PNETCDF_PATH>$(PNETCDFROOT)</PNETCDF_PATH>
<PFUNIT_PATH MPILIB="mpi-serial" compile_threaded="false">/sems-data-store/ACME/pfunit/pfunit-3.2.8</PFUNIT_PATH>
<ADD_SLIBS> $(shell $(NETCDF_PATH)/bin/nf-config --flibs) -lblas -llapack</ADD_SLIBS>
<CXX_LIBS>-lstdc++ -lmpi_cxx</CXX_LIBS>
<ALBANY_PATH>/projects/install/rhel6-x86_64/ACME/AlbanyTrilinos/Albany/build/install</ALBANY_PATH>
Expand Down
7 changes: 5 additions & 2 deletions config/acme/machines/config_machines.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -546,16 +546,19 @@
<command name="load">sems-env</command>
<command name="load">sems-git</command>
<command name="load">sems-python/2.7.9</command>
<command name="load">sems-cmake/2.8.12</command>
</modules>
<modules compiler="gnu">
<command name="load">sems-gcc/5.3.0</command>
</modules>
<modules compiler="intel">
<command name="load">sems-intel/16.0.3</command>
</modules>
<modules>
<modules mpilib="mpi-serial">
<command name="load">sems-netcdf/4.4.1/exo</command>
</modules>
<modules mpilib="!mpi-serial">
<command name="load">sems-openmpi/1.8.7</command>
<command name="load">sems-cmake/2.8.12</command>
<command name="load">sems-netcdf/4.4.1/exo_parallel</command>
</modules>
</module_system>
Expand Down
12 changes: 6 additions & 6 deletions scripts/fortran_unit_testing/run_tests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ def parse_command_line(args):
args.use_openmp, args.xml_test_list, args.verbose


def cmake_stage(name, test_spec_dir, build_optimized, use_mpiserial, mpirun_command, output,
def cmake_stage(name, test_spec_dir, build_optimized, use_mpiserial, mpirun_command, output, pfunit_path,
cmake_args=None, clean=False, verbose=False, enable_genf90=True, color=True):
"""Run cmake in the current working directory.

Expand Down Expand Up @@ -178,10 +178,12 @@ def cmake_stage(name, test_spec_dir, build_optimized, use_mpiserial, mpirun_comm

cmake_command = [
"cmake",
"-C Macros.cmake",
test_spec_dir,
"-DCIME_CMAKE_MODULE_DIRECTORY="+os.path.abspath(os.path.join(_CIMEROOT,"src","CMake")),
"-DCMAKE_BUILD_TYPE="+build_type,
"-DPFUNIT_MPIRUN='"+mpirun_command+"'",
"-DPFUNIT_PATH="+pfunit_path
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you understand why this is needed? I'm curious because it isn't needed on any of the 4 machines I've tested on. I don't remember for sure, but I thought that CMake typically picked up PFUNIT_PATH from the environment.

That said, it seems like this doesn't do any harm, so I'm okay with its being here if it truly seems to be needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@billsacks I didn't have PFUNIT_PATH set in my environment. I tried setting it and that didn't help.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see what's going on in my unit test builds: The Macros.cmake file that is written in the unit test build directory has lines like this:

  if("$ENV{MPILIB}" STREQUAL "mpich")
    if("$ENV{compile_threaded}" STREQUAL "true")
      set(PFUNIT_PATH "/usr/local/pfunit/pfunit-mpi")
    endif()
  endif()
  if("$ENV{MPILIB}" STREQUAL "mpi-serial")
    if("$ENV{compile_threaded}" STREQUAL "false")
      set(PFUNIT_PATH "/usr/local/pfunit/pfunit3.2.8-serial")
    endif()
  endif()

Do you not get something like that in your Macros.cmake file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@billsacks no, mine looks nothing like that

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that I know the issue - the configure generation of Macros.cmake requires the config_compilers.xml version that cesm but not acme is using - you need to update your config_compilers.xml file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jgfouca Did you ever determine if this setting of -DPFUNIT_PATH (and most of the other changes you made in run_tests.py) was still needed after you got the Macros.cmake file set up appropriately? My hunch is that this should not be needed, since PFUNIT_PATH should now come from Macros.cmake

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The V1 macros file doesn't set PFUNIT_PATH, so yes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, thanks for clarifying

]
if use_mpiserial:
cmake_command.append("-DUSE_MPI_SERIAL=ON")
Expand Down Expand Up @@ -241,6 +243,7 @@ def find_pfunit(compilerobj, mpilib, use_openmp):
"""PFUNIT_PATH not found for this machine and compiler, with MPILIB={} and compile_threaded={}.
You must specify PFUNIT_PATH in config_compilers.xml, with attributes MPILIB and compile_threaded.""".format(mpilib, attrs['compile_threaded']))
logger.info("Using PFUNIT_PATH: {}".format(pfunit_path.text))
return pfunit_path.text

#=================================================
# Iterate over input suite specs, building the tests.
Expand Down Expand Up @@ -306,7 +309,7 @@ def _main():

compilerobj = Compilers(machobj, compiler=compiler, mpilib=mpilib)

find_pfunit(compilerobj, mpilib=mpilib, use_openmp=use_openmp)
pfunit_path = find_pfunit(compilerobj, mpilib=mpilib, use_openmp=use_openmp)

debug = not build_optimized
os_ = machobj.get_value("OS")
Expand Down Expand Up @@ -375,7 +378,7 @@ def _main():
if not os.path.islink("Macros.cmake"):
os.symlink(os.path.join(build_dir,"Macros.cmake"), "Macros.cmake")
use_mpiserial = not use_mpi
cmake_stage(name, directory, build_optimized, use_mpiserial, mpirun_command, output, verbose=verbose,
cmake_stage(name, directory, build_optimized, use_mpiserial, mpirun_command, output, pfunit_path, verbose=verbose,
enable_genf90=enable_genf90, cmake_args=cmake_args)
make_stage(name, output, make_j, clean=clean, verbose=verbose)

Expand All @@ -399,8 +402,5 @@ def _main():
run_cmd_no_fail(" ".join(ctest_command), from_dir=label, arg_stdout=None, arg_stderr=subprocess.STDOUT)





if __name__ == "__main__":
_main()
2 changes: 1 addition & 1 deletion scripts/lib/CIME/BuildTools/configure.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@ def _copy_depends_files(machine_name, machines_dir, output_dir, compiler):
if os.path.isfile(dfile) and not os.path.isfile(outputdfile):
shutil.copyfile(dfile, outputdfile)


def _generate_env_mach_specific(output_dir, machobj, compiler, mpilib, debug,
sysos, unit_testing):
"""
Expand All @@ -79,6 +78,7 @@ def _generate_env_mach_specific(output_dir, machobj, compiler, mpilib, debug,
ems_file = EnvMachSpecific(output_dir, unit_testing=unit_testing)
ems_file.populate(machobj)
ems_file.write()
ems_file.load_env(compiler, debug, mpilib)
for shell in ('sh', 'csh'):
ems_file.make_env_mach_specific_file(compiler, debug, mpilib, shell)
shell_path = os.path.join(output_dir, ".env_mach_specific." + shell)
Expand Down
18 changes: 12 additions & 6 deletions scripts/lib/CIME/BuildTools/macrowriterbase.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import os
from abc import ABCMeta, abstractmethod
from CIME.XML.standard_module_setup import *
from CIME.utils import get_cime_root
logger = logging.getLogger(__name__)

def _get_components(value):
Expand Down Expand Up @@ -234,14 +235,15 @@ def write_macros_file_v1(macros, compiler, os_, machine, macros_file="Macros", o
'''#
# cmake Macros generated from $compiler_file
#
set(CMAKE_MODULE_PATH %s)
include(Compilers)
set(CMAKE_C_FLAGS_RELEASE "" CACHE STRING "Flags used by c compiler." FORCE)
set(CMAKE_C_FLAGS_DEBUG "" CACHE STRING "Flags used by c compiler." FORCE)
set(CMAKE_Fortran_FLAGS_RELEASE "" CACHE STRING "Flags used by Fortran compiler." FORCE)
set(CMAKE_Fortran_FLAGS_DEBUG "" CACHE STRING "Flags used by Fortran compiler." FORCE)
set(all_build_types "None Debug Release RelWithDebInfo MinSizeRel")
set(CMAKE_BUILD_TYPE "${CMAKE_BUILD_TYPE}" CACHE STRING "Choose the type of build, options are: ${all_build_types}." FORCE)
''')
''' % os.path.join(get_cime_root(), "src", "CMake"))

# print the settings out to the Macros file, do it in
# two passes so that path values appear first in the
Expand All @@ -254,7 +256,10 @@ def write_macros_file_v1(macros, compiler, os_, machine, macros_file="Macros", o
if key.endswith("_PATH"):
if value.startswith("$"):
value = "$ENV{}".format(value[1:])
fd.write("set({} {})\n".format(key, value))


cmake_var = key.replace("NETCDF_PATH", "NetCDF_PATH").replace("PNETCDF_PATH", "Pnetcdf_PATH")
fd.write("set({} {})\n".format(cmake_var, value))
fd.write("list(APPEND CMAKE_PREFIX_PATH {})\n\n".format(value))

for key, value in sorted(macros.iteritems()):
Expand All @@ -268,6 +273,7 @@ def write_macros_file_v1(macros, compiler, os_, machine, macros_file="Macros", o

idx = 0
for is_shell, component in components:
component = component.replace("NETCDF", "NetCDF").replace("PNETCDF_PATH", "Pnetcdf_PATH")
if is_shell:
fd.write('execute_process(COMMAND {} OUTPUT_VARIABLE TEMP{:d})\n'.format(component, idx))
fd.write('string(REGEX REPLACE "\\n$" "" TEMP{:d} "${{TEMP{:d}}}")\n'.format(idx, idx))
Expand All @@ -281,13 +287,13 @@ def write_macros_file_v1(macros, compiler, os_, machine, macros_file="Macros", o
fd.write('set(TEMP "{}")\n'.format(value))

if "CFLAGS" in key:
fd.write("add_flags(CMAKE_C_FLAGS ${TEMP})\n\n")
fd.write("add_flags(CFLAGS ${TEMP})\n\n")
elif "FFLAGS" in key:
fd.write("add_flags(CMAKE_Fortran_FLAGS ${TEMP})\n\n")
fd.write("add_flags(FFLAGS ${TEMP})\n\n")
elif "CPPDEFS" in key:
fd.write("list(APPEND COMPILE_DEFINITIONS ${TEMP})\n\n")
fd.write("list(APPEND CPPDEFS ${TEMP})\n\n")
elif "SLIBS" in key or "LDFLAGS" in key:
fd.write("add_flags(CMAKE_EXE_LINKER_FLAGS ${TEMP})\n\n")
fd.write("add_flags(LDFLAGS ${TEMP})\n\n")

# Recursively print the conditionals, combining tests to avoid repetition
_parse_hash(macros["_COND_"], fd, 0, output_format)
Expand Down
2 changes: 1 addition & 1 deletion src/CMake/Compilers.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ endfunction()
#=================================================

# Detect "STOP" for CTest.
if(${CMAKE_Fortran_COMPILER_ID} STREQUAL NAG)
if("${CMAKE_Fortran_COMPILER_ID}" STREQUAL NAG)
# NAG prints the stop code instead of yielding a non-zero return, so we
# have to use a regex to catch that.
function(define_Fortran_stop_failure test_name)
Expand Down