Unique ID feature (#525)

* Infrastructure for large Unique IDs

* Alpha codes and counters

* Date Counter support

* Two new concepts: @memorable and PluginResultIterator

docs/extending.md

@@ -118,6 +118,9 @@ represents the values that would be available to a formula running in the same c
 and `self.context.current_filename` which is the filename of the YAML file being
 processed.
 
+
+### Plugin Function Return Values
+
 Plugins can return normal Python primitive types, `datetime.date`, `ObjectRow` or `PluginResult` objects. `ObjectRow` objects represent new output records/objects. `PluginResult` objects
 expose a namespace that other code can access through dot-notation. `PluginResult` instances can be
 initialized with either a dict or an object that exposes the namespace through Python
@@ -127,6 +130,251 @@ If your plugin generates some special kind of data value which should be seriali
 as a primitive type (usually a string), subclass `PluginResult` and add a `simplify()`
 method to your subclass. That method should return a Python primitive value.
 
+### Remebering Function Result Values
+
+Plugins can ask Snowfakery to store their result values. In that case, they
+will only be called once and the value will be used forever in a particular
+context:
+
+```python
+class TimeStampPlugin(SnowfakeryPlugin):
+    class Functions:
+        @memorable
+        def constant_time(self, _=None):
+            "Return the current time and then remember it."
+            return time.time()
+```
+
+Given the following YAML, there are 4 contexts and therefore 4
+unique timestamps:
+
+```yaml
+# examples/plugins/test_timestamp.recipe.yml
+- plugin: tests.test_custom_plugins_and_providers.TimeStampPlugin
+- object: parent
+  count: 2
+  friends:
+    - object: A
+      count: 2
+      fields:
+        foo: ${{TimeStampPlugin.constant_time()}}
+    - object: B
+      count: 2
+      fields:
+        foo: ${{TimeStampPlugin.constant_time()}}
+    - object: A
+      count: 2
+      fields:
+        foo: ${{TimeStampPlugin.constant_time()}}
+    - object: B
+      count: 2
+      fields:
+        foo: ${{TimeStampPlugin.constant_time()}}
+```
+
+```s
+$ snowfakery examples/plugins/test_timestamp.recipe.yml
+parent(id=1)
+A(id=1, foo=1626883921.109788)
+A(id=2, foo=1626883921.109788)
+B(id=1, foo=1626883921.1107168)
+B(id=2, foo=1626883921.1107168)
+A(id=3, foo=1626883921.111661)
+A(id=4, foo=1626883921.111661)
+B(id=3, foo=1626883921.1125162)
+B(id=4, foo=1626883921.1125162)
+parent(id=2)
+A(id=5, foo=1626883921.109788)
+A(id=6, foo=1626883921.109788)
+B(id=5, foo=1626883921.1107168)
+B(id=6, foo=1626883921.1107168)
+A(id=7, foo=1626883921.111661)
+A(id=8, foo=1626883921.111661)
+B(id=7, foo=1626883921.1125162)
+B(id=8, foo=1626883921.1125162)
+```
+
+If a function takes arguments and -- for whatever reason --
+the argument values change, the memorized value will
+be recomputed as if it were a new context.
+
+```yaml
+# examples/plugins/test_timestamp_args_change.recipe.yml
+- plugin: tests.test_custom_plugins_and_providers.TimeStampPlugin
+- object: A
+  count: 5
+  fields:
+    blah:
+      fake: Name
+    foo: ${{TimeStampPlugin.constant_time(blah)}}
+```
+
+```s
+$ snowfakery examples/plugins/test_timestamp_args_change.recipe.yml
+A(id=1, foo=1626884493.793962)
+A(id=2, foo=1626884493.794135)
+A(id=3, foo=1626884493.7942772)
+A(id=4, foo=1626884493.794427)
+A(id=5, foo=1626884493.794549)
+```
+
+If your function accepts a special argument called "name", Snowfakery
+will allow the user to reuse the value between contexts.
+
+```yaml
+# examples/plugins/test_timestamp_name_change.recipe.yml
+- plugin: tests.test_custom_plugins_and_providers.TimeStampPlugin
+- object: A
+  fields:
+    foo: ${{TimeStampPlugin.constant_time(name="first")}}
+- object: A
+  fields:
+    foo: ${{TimeStampPlugin.constant_time(name="second")}}
+- object: B
+  fields:
+    foo: ${{TimeStampPlugin.constant_time(name="second")}}
+- object: A
+  fields:
+    foo: ${{TimeStampPlugin.constant_time(name="first")}}
+```
+
+```s
+$ snowfakery examples/plugins/test_timestamp_name_change.recipe.yml
+A(id=1, foo=1626884822.317738)
+A(id=2, foo=1626884822.3186612)
+B(id=1, foo=1626884822.3186612)
+A(id=3, foo=1626884822.317738)
+```
+
+Memorized objects are recreated on each iteration of Snowfakery.
+
+### Plugins with state that changes
+
+Let's imagine we want a plugin that A) reads a file and
+B) outputs a different line every time it is referred to.
+
+The way to deal with part A is to make a `PluginResultIterator`
+that represents the open file. The iterator should have a `next` method
+which returns each line. This will fulfill our requirement B.
+
+Here is the whole plugin module, including the PluginResultIterator and Plugin.
+
+```python
+# examples/plugins/OpenFiles.py
+from snowfakery import SnowfakeryPlugin, PluginResultIterator, memorable
+from pathlib import Path
+
+
+class OpenFiles(SnowfakeryPlugin):
+    class Functions:
+        @memorable
+        def read_line(self, filename, name=None):
+            parent_directory = Path(
+                str(self.context.field_vars()["snowfakery_filename"])
+            ).parent
+            abspath = parent_directory / filename
+            return OpenFile(abspath)
+
+
+class OpenFile(PluginResultIterator):
+
+    # initialize the object's state.
+    def __init__(self, filename):
+        self.file = open(filename, "r")
+
+    # cleanup later
+    def close(self):
+        self.file.close()
+
+    # the main logic goes in a method called `next`
+    def next(self):
+        return self.file.readline().strip()
+```
+
+Here is an example YAML:
+
+```yaml
+# examples/plugins/test_open_file.yml
+- plugin: examples.plugins.OpenFiles
+- object: WithOpenFile
+  count: 3
+  fields:
+    poetry_line:
+      OpenFiles.read_line:
+        filename: poem.txt
+```
+
+And some output:
+
+```s
+$ snowfakery examples/plugins/test_open_file.yml
+WithOpenFile(id=1, poetry_line=You must never let truth get in the way of beauty,)
+WithOpenFile(id=2, poetry_line=or so e.e.cummings believed.)
+WithOpenFile(id=3, poetry_line="This is the wonder that's keeping the stars apart.")
+```
+
+Iterators are recreated on each iteration of Snowfakery.
+
+As described earlier, you could open two different files using context:
+
+```yaml
+# examples/plugins/test_multiple_open_files.yml
+- plugin: examples.plugins.OpenFiles
+- object: parent
+  count: 3
+  friends:
+    - object: WithOpenFile
+      fields:
+        poetry_line:
+          OpenFiles.read_line:
+            filename: poem.txt
+    - object: WithOpenFile2
+      fields:
+        poetry_line:
+          OpenFiles.read_line:
+            filename: poem.txt
+```
+
+```s
+$ snowfakery examples/plugins/test_multiple_open_files.yml
+parent(id=1)
+WithOpenFile(id=1, poetry_line=You must never let truth get in the way of beauty,)
+WithOpenFile2(id=1, poetry_line=You must never let truth get in the way of beauty,)
+parent(id=2)
+WithOpenFile(id=2, poetry_line=or so e.e.cummings believed.)
+WithOpenFile2(id=2, poetry_line=or so e.e.cummings believed.)
+parent(id=3)
+WithOpenFile(id=3, poetry_line="This is the wonder that's keeping the stars apart.")
+WithOpenFile2(id=3, poetry_line="This is the wonder that's keeping the stars apart.")
+```
+
+And you could share a context between templates by using a `name`:
+
+```yaml
+# examples/plugins/test_shared_open_files.yml
+- plugin: examples.plugins.OpenFiles
+- object: parent
+  friends:
+    - object: WithOpenFile
+      fields:
+        poetry_line:
+          OpenFiles.read_line:
+            filename: poem.txt
+            name: JohnGreenPoem
+    - object: WithOpenFile2
+      fields:
+        poetry_line:
+          OpenFiles.read_line:
+            name: JohnGreenPoem
+    - object: WithOpenFile3
+      fields:
+        poetry_line:
+          OpenFiles.read_line:
+            name: JohnGreenPoem
+```
+
+### Lazy evaluation
+
 In the rare event that a plugin has a function which need its arguments to be passed to it unevaluated, for later (perhaps conditional) evaluation, you can use the `@snowfakery.lazy` decorator. Then you can evaluate the arguments with `self.context.evaluate()`.
 
 For example: