Cylc workflow integration #28
Assignees
Comments
|
Cylc will be a separate component from the CESM externals and the postprocessing externals. It will reside in the caseroot. It will only be supported on NCAR CMIP6 supported machines - initially yellowstone and cheyenne. As such, it will have it's own repo with documentation. |
|
This was successfully tested in /glade/p/work/aliceb/sandboxes/tests/b.e20.B1850.f09_g16.test.03.cylc Need to load the following modules: Then follow the README instructions from the http://github.com/NCAR/CESM-WF to
edit create_cylc_newcase... 31d30 32a32
The cylc suite is created in ~/cylc_suites and ~/cylc-run and to start the gui:
NOTE: I had to create a symlink in the CESM sandbox called postprocessing at the top level to point to the common postprocessing sandbox -OR- users will need to clone the CESM_postprocessing |
bertinia
added a commit
to bertinia/CESM_postprocessing
that referenced
this issue
Apr 24, 2017
769e587 Merge pull request NCAR#29 from NCAR/devel 95fcbe6 Merge pull request NCAR#28 from NCAR/chunking 7f1ebbb Adding ability to produce speedup plots 1a81993 Add ability to plot log plots and subselected jobs by data 4d65341 Adding functionality to plot log bar plots 17337ab Merge branch 'chunking' of https://github.com/NCAR/PyReshaper into chunking f5716ae Formatting only. db18732 New YS 8x16 (128 proc) timings. 8a27c17 New timings for 4x4 YS runs. f0ceb7e BUGFIX: Wrong checks when generating plots. Also resizing. 7a69d4e New Yellowstone timing numbers. d4cd015 Merge branch 'chunking' of https://github.com/NCAR/PyReshaper into chunking 5f366c1 Changing pop-0.1deg chunking to fit in yellowstone memory. 7f33829 Adding to change log. 883eeab Adding chunking documentation to manual 738d310 Bumping up pop-0.1deg chunk size 1d4598a Copying version 0.X timings into new database for comparisons 1d612ba Renaming new timings database to default name b960ddc Renaming old timings file to match old version c691b15 New timings database for new PyReshaper version 27b420c Changing to indicate read chunk sizes only (write are same) b8e49e2 Attempt to adjust chunk sizes. 81b0223 Merge branch 'chunking' of https://github.com/NCAR/PyReshaper into chunking 66ca398 Set camfv-1deg chunk sizes to none (read/write whole variables) e635b00 Merge branch 'chunking' of https://github.com/NCAR/PyReshaper into chunking 57b2fe6 Moving sync location in convert 82395f5 BUGFIX: Wrong capitalization when attempting to call getdb() 5627dca Adding a message about the chunk sizes used. 47109f1 Fixing typo in warning message 4eed760 Adding chunking parameters to testinfo database. b84c7ad Merge pull request NCAR#27 from NCAR/devel fb809a4 BUGFIX: Datatype must be converted to a numpy dtype c3c204c Revert "Working on updating to Python3" 0de3b37 Revert "Updating IOBackend tests to work with Python 2.6+" 3a3f1d8 Revert "Updating to work with Python2.6+" 2f0fc2c Revert "Updating to work with Python2.6+" 66374d9 Updating to work with Python2.6+ a8f0fc2 Updating to work with Python2.6+ 232406a Updating IOBackend tests to work with Python 2.6+ 549ddf9 Working on updating to Python3 3350d71 Simplifying chunking argument parsing f703b59 Suppressing PyDev error 82cbcc0 Cleanup and passing in chunks to Reshaper convert method 48b4740 Enabling chunking via command-line script 17e98ad Adding ability to create new timings file at runtime. 3556e9b Adding ability to read/write separate timings files 62d0173 Removing NetCDF compare tool. Try to use cprnc. d413515 Adding pretty-print to output 8b3c256 Allowing backwards-compatibility with older netCDF4 b34445f Removing conflict with parser's help option 93ab872 One more modification 59155ad Making the script executable from the command line ae172ca Adding a NetCDF compare tool 533756b Merge branch 'devel' of https://github.com/NCAR/PyReshaper into devel e5ed2ac Adding module load to init and ignoring results directory. 68a3e3c Bugfix. Need to set default chunking after unlimited dim is found. d1c4edd Some cleanup and restructuring. 4343273 First pass at actual chunking code 5a07d3a Not printing 'usage'. Printing 'use'. a66c288 Merge pull request NCAR#25 from NCAR/memory_diag af921ec Adding maximum memory usage diagnostic output 04ff328 Merge pull request NCAR#24 from NCAR/extract_only df9e99c Updated changelog file. No more dates. Reverse order. a2c3e47 Updating the manual to describe the new extract-only features f31a7a0 Adding command-line option to extract only some time-series variables c3d06e8 Extract only some variables seems to work from Python side. 79af819 Checking extract_only tests. Appears to work, but needs more testing. 58eeb21 Updating specification tests to include time-series name list 13578d2 Updated GitHub references 2567496 First pass as modifying reshaper to use explicit time-series name list 031cfda Modifying specification to take time-series name list 7f7fa13 Merge pull request NCAR#23 from NCAR/default_1D_metadata b4ec088 Updating manual for new feature c87bafc Finished adding option to assert all 1d time-var variables are metadata 568d534 Cleanup: Better allowances for netCDF4-python v1.1.1 95aaf34 attempting to add netCDF4 v1.1.1 support 38ab628 Updating bakeoff data location on GLADE 9f55a5e Need to repr the wmode argument when writing script (must be str) f78f002 Bugfix 670edfa Backporting checkresults to Python 2.6 cc0a772 Bugfix: more specfile extension changing cbed5cd Bugfix: specfile has extension s2s 38fef51 bugfix f1de114 Bugfix and changing usage statements cf29b54 More formatting of descriptions 601081d Updating descriptions of test utilities dbff464 Backporting YS tests to Python version 2.6 fed167e Merge pull request NCAR#22 from NCAR/iobackend 7aa5477 Updating README to be consistent with manual. 12aa9e3 Updating CHANGELOG 44823f9 Updating user documentation 0a8d18b Updating comments only. f5e7ad7 Modifying CHANGES log f3b6041 Formatting only. a38aa2a Updating changelog. 6eab81b Cleaning up iobackend branch, allowing I/O backend to be set by default f762a67 Moved I/O backend info output to initialization 69548a0 More output about the NetCDF output settings. 7b0f71b Allowing the use of Python 2.6 ordereddict module, if available 6cd1d9f Changing the default backend to PyNIO (tests suggest it's faster) 74e497c Removing unnecessary import 915bf1c Updating tests with new names 469a0e0 Implementing use of new get_value/assign_value functions b557895 Adding get_value and assign_value methods for general get/set e26ffb1 Attempting to convert typecode 'S' to 'c' when using PyNIO e8bd27a Casting datatype to string in Nio create_variable d82aacc Adding user-level output on which I/O backend is being used. 9cdad55 Adding option for I/O backend to s2smake (stored in specfile) 562f6c4 Updating to new iobackend changes 6df401a Removing debug output. Changing 'typecode' str to 'datatype' Numpy.dtype 19ced93 More debugging info. 0634559 Adding debug output. ee97a88 Switching PyReshaper objects over to new iobackend dae810d Adding specifier checks on backend 49c081f Adding shape and size methods to NCVariable 3ed54aa Adding append tests. All tests pass. 8b72ef5 Added write tests. All are passing. 7c56210 Cleaned up API. Switched to netCDF4 way of dealing with attributes. 0c3cfd8 Now adding iobackend and tests. 43cc6d8 Updated copyright info. Added iobackend module and started tests. 2257386 Removing MultiSpecReshaper reference from factory function 18b5359 Removing MultiSpecReshaper 139f708 Starting work on new memory-management system/partitioner git-subtree-dir: reshaper/pyReshaper git-subtree-split: 769e587dabe15c0ecaf4d0c2903152a12ae640ce
|
Cylc CESM workflow related issues should be reported to the https://github.com/NCAR/CESM-WF repo. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Successfully tested an end-to-end workflow using
Need to include an email address in env_postprocess.xml.
Need to update documentation for using and setting up cylc.
The text was updated successfully, but these errors were encountered: