Docs/ben/level max dirs

CHANGELOG.md

@@ -4,6 +4,11 @@ All notable changes to Merlin will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Added
+- Documentation on `level_max_dirs`.
+
 ## [1.6.2]
 
 ### Added

docs/source/faq.rst

@@ -377,6 +377,15 @@ num resource set: Number of resource sets
 launch_distribution : The distribution of resources (default: plane:{procs/nodes})
 lsf: Verbatim flags only for the lsf parallel launch (jsrun ... <lsf>)
 
+What is level_max_dirs?
+~~~~~~~~~~~~~~~~~~~~~~~
+``level_max_dirs`` is an optional field that goes under the ``merlin.samples`` section
+of a yaml spec. It caps the number of sample directories that can be generated
+at a single level of a study's sample hierarchy. This is useful for getting around
+filesystem constraints when working with massive amounts of data.
+
+Defaults to 25.
+
 Where can I learn more about merlin?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Check out `our paper <https://arxiv.org/abs/1912.02892>`_ on arXiv.

docs/source/merlin_specification.rst

@@ -290,3 +290,4 @@ see :doc:`./merlin_variables`.
       generate:
         cmd: |
         python $(SPECROOT)/make_samples.py -dims 2 -n 10 -outfile=$(INPUT_PATH)/samples.npy "[(1.3, 1.3, 'linear'), (3.3, 3.3, 'linear')]"
+      level_max_dirs: 25