Y2 Roadmap PSU Use case

Roadmap for PSU Y2 use case

• Use case document: https://docs.google.com/document/d/1uYtPy_VqoZ24JeU8qgTIUVzCmoCN35gSXFCQpqKXYP0/edit#
• Notes from F2F meeting (Oct 26 2018): https://docs.google.com/document/d/1roJlvnUH1q-fO5TtInwQgMwhhr7020IGUbbIEF-6zS8/edit#heading=h.m15tpj9gerf4
• Cheyenne guide: https://www2.cisl.ucar.edu/resources/computational-systems/cheyenne/cheyenne
• EnTK version: 0.7.6 (released), Docs
• Website: https://weiming-hu.github.io/AnalogsEnsemble/
• Github repo: https://github.com/Weiming-Hu/AnalogsEnsemble
• Similarity kernel version:
• Selection kernel version:

1. Profiling of similarity and selection kernels

• Execution time (T_x) of similarity kernel as a function of number of threads for different amounts of search space
  • No. of threads: [1,2,4,8,16,32]
  • Link to plot:
• Execution time (T_x) of selection kernel as a function of number of threads
  • No. of threads: [1,2,4,8,16,32]
  • Link to plot:

Note: Profiling to be done on Cheyenne simply specifying the kernels and parameters in a pbs script.

2. Derive configuration from profiling and shape workflow

• Configuration derived
  • Similarity kernel
    • No. of threads = No. of grid points
    • No. of threads per process = ?
    • No. of processes per EnTK task = ?
    • No. of EnTK tasks per similarity stage = ?
  • Selection kernel
    • No. of threads = ?
    • No. of threads per process = ?
    • No. of processes per EnTK task = ?
    • No. of EnTK tasks per similarity stage = ?
  • No. of similarity and selection stages per pipeline =
  • No. of pipelines =
• EnTK workflow implemented
  • Variables in workflow shape:
  • Constants in workflow shape:

Note: Value for any key can be a constant or a range

3. Profiling of workflow on Cheyenne

• Execution time T_x and relevant overheads of the workflow execution as a function of the variables determined in 2
  • Link to plot:

Note: Steps 1-3 to use small scale but real data

4. Experiments with all data at full scale on Cheyenne

• Requirements:
  • TBD
• Experiment design
  • TBD

5. Opportunities for adaptive execution

* TBD