Plugin dir cli (#385)

* ✨enabled plugins dir via command line

* 📚 update exit code

* 🔬 test -pd on command line

* 📚 update readme

* 🐛 fix unit test which failed to detect the missing feature

* 🔨 code refactor copy engine and strip content processor

* 📚 here is the way to write modelines remover. #36

* 🔨 code refactoring

.moban.cd/changelog.yml

@@ -1,6 +1,15 @@
 name: moban
 organisation: moremoban
 releases:
+  - changes:
+      - action: Added
+        details:
+          - "`-pd` for command line to include custom plugin directories"
+      - action: Fixed
+        details:
+          - "strip did not work in 0.7.6"
+    date: tbd
+    version: 0.7.7
   - changes:
       - action: Added
         details:

.moban.cd/moban.yml

@@ -4,8 +4,8 @@ organisation: moremoban
 author: C. W.
 contact: [email protected]
 license: MIT
-version: 0.7.6
-current_version: 0.7.6
+version: 0.7.7
+current_version: 0.7.7
 release: 0.7.6
 branch: master
 master: index

CHANGELOG.rst

@@ -1,6 +1,17 @@
 Change log
 ================================================================================
 
+0.7.7 - tbd
+--------------------------------------------------------------------------------
+
+**Added**
+
+#. `-pd` for command line to include custom plugin directories
+
+**Fixed**
+
+#. strip did not work in 0.7.6
+
 0.7.6 - 22.5.2020
 --------------------------------------------------------------------------------
 

README.rst

@@ -42,57 +42,6 @@ For existing moban users, python 2 support has been dropped. Please stay with
 versions lower than 0.7.0 if you are still using python 2.
 
 
-Introduction
-================================================================================
-
-**moban** enabled **continuous templating** in `pyexcel <https://github.com/pyexcel/pyexcel>`_ and
-`coala <https://github.com//coala/coala>`_ project to keep
-documentation consistent across the documentations of individual libraries in the same
-organisation. Here is the primary use case of moban, as of now:
-
-.. image:: https://github.com/moremoban/yehua/raw/dev/docs/source/_static/yehua-story.png
-   :width: 600px
-
-
-And here is a list of other usages:
-
-#. `Django Mobans <https://github.com/django-mobans>`_, templates for django, docker etc.
-#. `Math Sheets <https://github.com/chfw/math-sheets>`_, generate custom math sheets in pdf
-
-All use cases are documented `here <http://moban.readthedocs.org/en/latest/#tutorial>`_
-
-Support
-================================================================================
-
-If you like moban, please support me on github,
-`patreon <https://www.patreon.com/bePatron?u=5537627>`_
-or `bounty source <https://salt.bountysource.com/teams/chfw-pyexcel>`_ to maintain
-the project and develop it further.
-
-With your financial support, I will be able to invest
-a little bit more time in coding, documentation and writing interesting extensions.
-
-Vision
-================================================================================
-
-Any template, any data in any location
-
-**moban** started with bringing the high performance template engine (JINJA2) for web
-into static text generation. 
-
-**moban** can use other python template engine: mako, handlebars, velocity,
-haml, slim and tornado, can read other data format: json and yaml, and can access both
-template file and configuration file in
-any location: zip, git, pypi package, s3, etc.
-
-
-Credit
-================================================================================
-
-`jinja2-fsloader <https://github.com/althonos/jinja2-fsloader>`_ is the key component to enable PyFilesystem2 support in moban
-v0.6x. Please show your stars there too!
-
-
 Quick start
 ================================================================================
 
@@ -229,9 +178,63 @@ Then you can access your files in s3 bucket:
     $ cat moban.output
     world
 
-Where the configuration sits in a s3 bucket, the output is a file in a zip. The content of s3data.yaml is::
+Where the configuration sits in a s3 bucket, the output is a file in a zip. The content of s3data.yaml is:
+
+
+.. code-block:
 
     hello: world
+
+
+Introduction
+================================================================================
+
+**moban** enabled **continuous templating** in `pyexcel <https://github.com/pyexcel/pyexcel>`_ and
+`coala <https://github.com//coala/coala>`_ project to keep
+documentation consistent across the documentations of individual libraries in the same
+organisation. Here is the primary use case of moban, as of now:
+
+.. image:: https://github.com/moremoban/yehua/raw/dev/docs/source/_static/yehua-story.png
+   :width: 600px
+
+
+And here is a list of other usages:
+
+#. `Django Mobans <https://github.com/django-mobans>`_, templates for django, docker etc.
+#. `Math Sheets <https://github.com/chfw/math-sheets>`_, generate custom math sheets in pdf
+
+All use cases are documented `here <http://moban.readthedocs.org/en/latest/#tutorial>`_
+
+Support
+================================================================================
+
+If you like moban, please support me on github,
+`patreon <https://www.patreon.com/bePatron?u=5537627>`_
+or `bounty source <https://salt.bountysource.com/teams/chfw-pyexcel>`_ to maintain
+the project and develop it further.
+
+With your financial support, I will be able to invest
+a little bit more time in coding, documentation and writing interesting extensions.
+
+Vision
+================================================================================
+
+Any template, any data in any location
+
+**moban** started with bringing the high performance template engine (JINJA2) for web
+into static text generation. 
+
+**moban** can use other python template engine: mako, handlebars, velocity,
+haml, slim and tornado, can read other data format: json and yaml, and can access both
+template file and configuration file in
+any location: zip, git, pypi package, s3, etc.
+
+
+Credit
+================================================================================
+
+`jinja2-fsloader <https://github.com/althonos/jinja2-fsloader>`_ is the key component to enable PyFilesystem2 support in moban
+v0.6x. Please show your stars there too!
 
 
 Installation
@@ -258,7 +261,8 @@ CLI documentation
 .. code-block:: bash
 
     usage: moban [-h] [-c CONFIGURATION] [-t TEMPLATE] [-o OUTPUT]
-                 [-td [TEMPLATE_DIR [TEMPLATE_DIR ...]]] [-cd CONFIGURATION_DIR]
+                 [-td [TEMPLATE_DIR [TEMPLATE_DIR ...]]]
+                 [-pd [PLUGIN_DIR [PLUGIN_DIR ...]]] [-cd CONFIGURATION_DIR]
                  [-m MOBANFILE] [-g GROUP] [--template-type TEMPLATE_TYPE]
                  [-d DEFINE [DEFINE ...]] [-e EXTENSION [EXTENSION ...]] [-f]
                  [--exit-code] [-V] [-v]
@@ -285,6 +289,8 @@ CLI documentation
                             add more directories for template file lookup
       -cd CONFIGURATION_DIR, --configuration_dir CONFIGURATION_DIR
                             the directory for configuration file lookup
+      -pd [PLUGIN_DIR [PLUGIN_DIR ...]], --plugin_dir [PLUGIN_DIR [PLUGIN_DIR ...]]
+                            add more directories for plugin lookup
       -m MOBANFILE, --mobanfile MOBANFILE
                             custom moban file
       -g GROUP, --group GROUP
@@ -302,19 +308,8 @@ CLI documentation
     Developer options:
       For debugging and development
     
-      --exit-code           tell moban to change exit code
+      --exit-code           by default, exist code 0 means no error, 1 means error
+                            occured. It tells moban to change 1 for changes, 2 for
+                            error occured
       -V, --version         show program's version number and exit
       -v                    show verbose, try -v, -vv, -vvv
-
-Exit codes
---------------------------------------------------------------------------------
-By default:
-
-- 0 : no error
-- 1 : error occured
-
-With `--exit-code`:
-
-- 0 : no changes
-- 1 : has changes
-- 2 : error occured

docs/conf.py

@@ -25,7 +25,7 @@
 copyright = '2017-2020 Onni Software Ltd.'
 author = 'C. W.'
 # The short X.Y version
-version = '0.7.6'
+version = '0.7.7'
 # The full version, including alpha/beta/rc tags
 release = '0.7.6'
 

docs/extension.rst

@@ -1,6 +1,6 @@
 Development guide
 =======================
-                   
+
 Jinja2 extensions for Moban
 ------------------------------
 
@@ -14,7 +14,7 @@ your plugins will NOT be loaded.
 Jinja2 Filter
 *******************
 
-.. literalinclude:: ../moban/filters/repr.py
+.. literalinclude:: ../moban/plugins/jinja2/filters/repr.py
 
 split_length
 --------------------------------------------------------------------------------
@@ -66,7 +66,7 @@ Built-in Tests
 Jinja2 Test
 *******************
 
-.. literalinclude:: ../moban/tests/files.py
+.. literalinclude:: ../moban/plugins/jinja2/tests/files.py
 
 
 `exists`
@@ -104,16 +104,16 @@ is an example starting point for any template engine.
            """
            A list template directories will be given to your engine class
            """
-   
+
        def get_template(self, template_file):
            """
            Given a relative path to your template file, please return a templatable thing that does
            the templating function in next function below
            """
-   
+
        def apply_template(self, template, data, output):
             """
-            Given the template object from `get_template` function, and data as python dictionary, 
+            Given the template object from `get_template` function, and data as python dictionary,
             and output as intended output file, please return "utf-8" encoded string.
             """
 
@@ -122,5 +122,27 @@ After you will have finished the engine plugin, you can either place it in `plug
 in order to get it loaded, or make an installable python package. In the latter case,
 please refer to `yehua`_: doing that in less than 5 minutes.
 
+
+Custom content processors for Moban
+----------------------------------------
+
+Since version 0.7.7, it became easy to write a content processor for moban.
+What you need is a content processing function, which will be fed the
+content of `template_file` and which is expected to return a string. And
+decorate your function with `ContentProcessor`:
+
+.. code-block:: python
+
+    @ContentProcessor('strip', 'Stripping', 'Stripped'):
+    def strip(template_file: str) -> str:
+        ret = template_file.strip()
+        return ret
+
+
+Here is how `copy` template type is coded:
+
+.. literalinclude:: ../moban/plugins/copy.py
+
+
 .. _lml: http://lml.readthedocs.io
 .. _yehua: http://yehua.readthedocs.io

docs/level-12-use-template-engine-extensions/README.rst

@@ -21,6 +21,9 @@ For example::
 Please also note that the following extensions are included by default:
 `jinja2.ext.do`, `jinja2.ext.loopcontrols`
 
+.. note::
+
+   if you intend to use extensions for one off usage, please use '-e' cli option
 
 Evaluation
 --------------------------------------------------------------------------------

docs/level-7-use-custom-jinja2-filter-test-n-global/README.rst

@@ -23,6 +23,10 @@ It is very important to have `__init__.py`, otherwise, it will NOT work. Other t
 files are named to show case the feature. You can choose whichever name you prefer,
 as long as you and your team could make sense of the names.
 
+.. note::
+
+   if you intend to use extensions for one off usage, please use '-pd' cli option.
+   i.e. `moban -td my-templates/ -t filter.jj2 -pd custom-jj2-plugin`
 
 Evaluation
 --------------------------------------------------------------------------------

moban/_version.py

@@ -1,2 +1,2 @@
-__version__ = "0.7.6"
+__version__ = "0.7.7"
 __author__ = "C. W."

moban/constants.py

@@ -29,6 +29,7 @@
 LABEL_TEMPLATE = "template"
 POSITIONAL_LABEL_TEMPLATE = "template_in_string"
 LABEL_TMPL_DIRS = "template_dir"
+LABEL_PLUGIN_DIRS = "plugin_dir"
 LABEL_EXIT_CODE = "exit-code"
 LABEL_OUTPUT = "output"
 LABEL_TEMPLATE_TYPE = "template_type"

moban/core/content_processor.py

@@ -0,0 +1,51 @@
+import logging
+
+from lml.plugin import PluginInfo
+
+from moban import constants
+
+LOG = logging.getLogger(__name__)
+
+
+class ContentProcessor(PluginInfo):
+    """
+    @ContentProcessor('strip', 'Stripping', 'Stripped'):
+    def strip(template_file: str) -> str:
+        ret = template_file.strip()
+        return ret
+    """
+
+    def __init__(self, action, action_continuing_tense, action_past_tense):
+        super(ContentProcessor, self).__init__(
+            constants.TEMPLATE_ENGINE_EXTENSION, tags=[action]
+        )
+        self.action = action
+        self.action_continuing_tense = action_continuing_tense
+        self.action_past_tense = action_continuing_tense
+
+    def __call__(self, a_content_processor_function):
+        continuing_tense = self.action_continuing_tense
+        past_tense = self.action_past_tense
+
+        class CustomEngine(object):
+            ACTION_IN_PRESENT_CONTINUOUS_TENSE = continuing_tense
+            ACTION_IN_PAST_TENSE = past_tense
+
+            def __init__(self, template_fs, extensions=None):
+                self.template_fs = template_fs
+
+            def get_template(self, template_file):
+                content = self.template_fs.readbytes(template_file)
+                ret = a_content_processor_function(content)
+                return ret
+
+            def get_template_from_string(self, a_string):
+                ret = a_content_processor_function(a_string)
+                return ret
+
+            def apply_template(self, template, *_):
+                ret = a_content_processor_function(template)
+                return ret
+
+        super(ContentProcessor, self).__call__(CustomEngine)
+        return a_content_processor_function