Merge pull request #327 from moremoban/dev

Release 0.6.0 - PyFileSystem2 support
moremoban · Sep 10, 2019 · f5d36ec · f5d36ec
2 parents e64e83d + c62ea15
commit f5d36ec
Show file tree

Hide file tree

Showing 133 changed files with 2,754 additions and 1,090 deletions.
diff --git a/.github/FUNDING.yml b/.github/FUNDING.yml
@@ -0,0 +1,3 @@
+# These are supported funding model platforms
+
+patreon: chfw
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
@@ -1,11 +1,8 @@
 Before raising the PR, here is a check list:
 
-1) have you written unit tests for your code changes?
-2) have you run "make format"?
-3) are you requesting to "dev"?
-4) have you updated the change log?
-5) do you think that you can understand your changes after 6 month?
-   5.1) can someone else understand your changes without your explanation?
-6) are you pround of your code changes?
-   6.1) do you have the feeling of achievement?
-7) please add your name and github link to contributors.rst in alphabetical order.
+- [ ] have you written unit tests for your code changes?
+- [ ] have you updated the change log?
+- [ ] can someone else understand your changes without your explanation?
+- [ ] are you proud of your code changes?
+- [ ] please add your name and github link to contributors.rst in alphabetical order.
+
diff --git a/.gitignore b/.gitignore
@@ -506,6 +506,8 @@ cscope.in.out
 cscope.po.out
 
 
+docs/deprecated-level-9-moban-dependency-as-pypi-package/mytravis.yml
+docs/deprecated-level-10-moban-dependency-as-git-repo/mytravis.yml
 docs/level-1-jinja2-cli/testout
 docs/level-10-moban-dependency-as-git-repo/mytravis.yml
 docs/level-18-user-defined-template-types/b.output

diff --git a/.isort.cfg b/.isort.cfg
@@ -2,7 +2,7 @@
 line_length=79
 # Ignore generated files
 skip=setup.py, moban/__init__.py
-known_first_party=lml, crayons, jinja2, ruamel.yaml, mock, nose
+known_first_party=fs, lml, crayons, jinja2, ruamel.yaml, mock, nose
 indent='    '
 multi_line_output=3
 length_sort=1

diff --git a/.moban.cd/changelog.yml b/.moban.cd/changelog.yml
@@ -1,6 +1,19 @@
 name: moban
 organisation: moremoban
 releases:
+  - changes:
+      - action: Added
+        details:
+          - "`#205`: support `pyFilesystem2 <https://pyfilesystem2.readthedocs.io/>`_"
+          - "`#185`: -v will enable moban application logging for development. And -V is for version."
+          - "`#325`: -vv show debug trace"
+          - "`#126`: Allow mobanfile to include data from arbitrary config files"
+          - "`#256`: jinja2-cli parity: '-D hello=world' to define custom variable on cli"
+      - action: Updated
+        details:
+          - "`#275`: fix moban 0.4.5 test failures on openSUSE Tumbleweed"
+    date: 10.09.2019
+    version: 0.6.0
   - changes:
       - action: Updated
         details:

diff --git a/.moban.cd/moban.yml b/.moban.cd/moban.yml
@@ -4,9 +4,9 @@ organisation: moremoban
 author: C. W.
 contact: [email protected]
 license: MIT
-version: 0.5.0
-current_version: 0.5.0
-release: 0.5.0
+version: 0.6.0
+current_version: 0.6.0
+release: 0.6.0
 branch: master
 master: index
 command_line_interface: "moban"
@@ -24,10 +24,10 @@ dependencies:
   - ruamel.yaml>=0.15.98;python_version == '3.8'
   - jinja2>=2.7.1
   - lml>=0.0.9
-  - appdirs>=1.2.0
+  - appdirs>=1.4.3
   - crayons>= 0.1.0
-  - GitPython>=2.0.0
-  - git-url-parse>=1.2.2
+  - fs>=2.4.11
+  - jinja2-fsloader>=0.2.0
 description: Yet another jinja2 cli command for static text generation
 scm_host: github.com
 lint_command: make install_test format git-diff-check lint

diff --git a/.moban.d/moban_gitignore.jj2 b/.moban.d/moban_gitignore.jj2
@@ -1,6 +1,8 @@
 {% extends "gitignore.jj2" %}
 
 {% block extra %}
+docs/deprecated-level-9-moban-dependency-as-pypi-package/mytravis.yml
+docs/deprecated-level-10-moban-dependency-as-git-repo/mytravis.yml
 docs/level-1-jinja2-cli/testout
 docs/level-10-moban-dependency-as-git-repo/mytravis.yml
 docs/level-18-user-defined-template-types/b.output

diff --git a/.travis.yml b/.travis.yml
@@ -16,9 +16,9 @@ env:
   - MINREQ=1
 
 stages:
-  - test
   - lint
   - moban
+  - test
 
 .disable_global: &disable_global
   addons: false

diff --git a/CHANGELOG.rst b/CHANGELOG.rst
@@ -1,6 +1,28 @@
 Change log
 ================================================================================
 
+0.6.0 - 10.09.2019
+--------------------------------------------------------------------------------
+
+Added
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+#. `#205 <https://github.com/moremoban/moban/issues/205>`_: support
+   `pyFilesystem2 <https://pyfilesystem2.readthedocs.io/>`_
+#. `#185 <https://github.com/moremoban/moban/issues/185>`_: -v will enable moban
+   application logging for development. And -V is for version.
+#. `#325 <https://github.com/moremoban/moban/issues/325>`_: -vv show debug trace
+#. `#126 <https://github.com/moremoban/moban/issues/126>`_: Allow mobanfile to
+   include data from arbitrary config files
+#. `#256 <https://github.com/moremoban/moban/issues/256>`_: jinja2-cli parity:
+   '-D hello=world' to define custom variable on cli
+
+Updated
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+#. `#275 <https://github.com/moremoban/moban/issues/275>`_: fix moban 0.4.5 test
+   failures on openSUSE Tumbleweed
+
 0.5.0 - 14.07.2019
 --------------------------------------------------------------------------------
 

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -48,3 +48,7 @@ When you enable travis-ci on your own account, you shall see travis-ci running a
 6. Commit with a suitable message `git commit -m " Changes made "`
 7. Push `git push origin branch_name`
 8. Go to the Github Repository and create a pull request to the dev branch
+
+## Commit messages
+
+We use git emojis for commit messages. Please read [the guide](https://github.com/slashsBin/styleguide-git-commit-message).
diff --git a/Makefile b/Makefile
@@ -27,3 +27,10 @@ format:
 	git diff
 	black -l 79 tests
 	git diff
+
+uml:
+	plantuml -tsvg -o ./images/ docs/*.uml
+
+
+doc: uml
+	sphinx-build -b html docs build
diff --git a/README.rst b/README.rst
@@ -25,13 +25,20 @@ moban - 模板 Yet another jinja2 cli command for static text generation
 
 
 **moban** brings the high performance template engine (JINJA2) for web into
-static text generation. It is used in `pyexcel` and `coala` project to keep documentation
-consistent across the documentations of individual libraries.
+static text generation. It is used in `pyexcel` and `coala` project to keep
+documentation consistent across the documentations of individual libraries.
 
-Our vision is: any template, any data. Our current architecture enables moban
-to plugin any python template engine: mako, handlebars, velocity, haml, slim and tornado
-and to plugin any data format: json and yaml. Please look at our issues. We have many
-more template engines and data format on the road map.
+Our vision is: any template, any data in any location. Our current architecture
+enables moban to plugin any python template engine: mako, handlebars, velocity,
+haml, slim and tornado, to plugin any data format: json and yaml, and in
+any location: zip, git, pypi package, s3, etc. Please
+look at our issues. We have many more template engines and data format on the
+road map.
+
+Documentation
+=================================================================================
+
+All use cases are documented `here <http://moban.readthedocs.org/en/latest/#tutorial>`_
 
 Installation
 ================================================================================
@@ -93,114 +100,128 @@ moban.output will contain::
 
 Please note that data.yml will take precedence over environment variables.
 
-`the tutorial`_ has more use cases.
-
-.. _the tutorial: http://moban.readthedocs.org/en/latest/#tutorial
 
-
-Usage
+Work with files in a git repo
 ================================================================================
 
-.. code-block:: bash
+Please install `gitfs2 <https://github.com/moremoban/gitfs2>`_::
 
+    $ pip install gitfs2
 
-   usage: moban [-h] [-cd CONFIGURATION_DIR] [-c CONFIGURATION]
-                [-td [TEMPLATE_DIR [TEMPLATE_DIR ...]]] [-t TEMPLATE] [-o OUTPUT]
-                [-f] [-m MOBANFILE]
-                [template]
 
-   Yet another jinja2 cli command for static text generation
-   
-   positional arguments:
-     template              string templates
-
-   optional arguments:
-     -h, --help            show this help message and exit
-     -cd CONFIGURATION_DIR, --configuration_dir CONFIGURATION_DIR
-                           the directory for configuration file lookup
-     -c CONFIGURATION, --configuration CONFIGURATION
-                           the dictionary file. if not present, moban
-                           will try to use environment vars as data
-     -td [TEMPLATE_DIR [TEMPLATE_DIR ...]], --template_dir [TEMPLATE_DIR [TEMPLATE_DIR ...]]
-                           the directories for template file lookup
-     -t TEMPLATE, --template TEMPLATE
-                           the template file. this overrides any targets
-                           defined in a custom moban file
-     -o OUTPUT, --output OUTPUT
-                           the output file
-     --template_type TEMPLATE_TYPE
-                           the template type, default is jinja2
-     -f                    force moban to template all files despite of
-                           .moban.hashes
-     --exit-code           tell moban to change exit code                      
-     -m MOBANFILE, --mobanfile MOBANFILE
-                           custom moban file
-   
-
-Exit codes
---------------------------------------------------------------------------------
-By default:
+And then you can do the following:
 
-- 0 : no error
-- 1 : error occured
+.. code-block:: bash
 
-With `--exit-code`:
+    $ moban -t 'git://github.com/moremoban/pypi-mobans.git!/templates/_version.py.jj2' \
+            -c 'git://github.com/moremoban/pypi-mobans.git!/config/data.yml' \
+            -o _version.py
+    Info: Found repo in /Users/jaska/Library/Caches/gitfs2/repos/pypi-mobans
+    Templating git://github.com/moremoban/pypi-mobans.git!/templates/_version.py.jj2 to _version.py
+    Templated 1 file.
+    $ cat _version.py
+    __version__ = "0.1.1rc3"
+    __author__ = "C.W."
 
-- 0 : no changes
-- 1 : has changes
-- 2 : error occured
 
-Built-in Filters
+Work with files in a python package
 ================================================================================
 
-split_length
---------------------------------------------------------------------------------
+Please install `pypifs <https://github.com/moremoban/pypifs>`_::
 
-It breaks down the given string into a fixed length paragraph. Here is the syntax::
+    $ pip install pypifs
 
-    {% for line in your_string | split_length(your_line_with) %}
-    {{line}}
-    {% endfor %}
 
-It is used to keep changelog formatted in
-`CHANGELOG.rst.jj2 in pypi-mobans project <https://github.com/moremoban/pypi-mobans/blob/master/templates/CHANGELOG.rst.jj2#L15>`_
+And then you can do the following:
 
-github_expand
---------------------------------------------------------------------------------
+.. code-block:: bash
 
-It expands simple hashtags into github issues. Here is the syntax::
+    $ moban -t 'pypi://pypi-mobans-pkg/resources/templates/_version.py.jj2' \
+            -c 'pypi://pypi-mobans-pkg/resources/config/data.yml' \
+            -o _version.py
+    Collecting pypi-mobans-pkg
+    ....
+    Installing collected packages: pypi-mobans-pkg
+    Successfully installed pypi-mobans-pkg-0.0.7
+    Templating pypi://pypi-mobans-pkg/resources/templates/_version.py.jj2 to _version.py
+    Templated 1 file.
+    $ cat _version.py
+    __version__ = "0.1.1rc3"
+    __author__ = "C.W."
 
-    {{ your_github_string | github_expand }}
+Work with S3 and other cloud based file systems
+================================================================================
 
+Please install `fs-s3fs <https://github.com/PyFilesystem/s3fs>`_::
 
-It makes it easy to mention github reference in change log in all projects. Here is
-the place it is applied:
-`CHANGELOG.rst.jj2 in pypi-mobans project <https://github.com/moremoban/pypi-mobans/blob/master/templates/CHANGELOG.rst.jj2#L15>`_
+    $ pip install fs-s3fs
 
+.. code-block:: bash
 
-Here is Grammar in the changelog.yml::
+    $ moban -c s3://${client_id}:${client_secrect}@moremoban/s3data.yml \
+            -o 'zip://my.zip!/moban.output' {{hello}}
+    $ unzip my.zip
+    $ cat moban.output
+    world
 
-    =============== ==============================
-    Syntax          Meaning
-    =============== ==============================
-    `#1`            moban issues 1
-    `PR#1`          moban pull request 1
-    `pyexcel#1`     other project issues 1
-    `pyexcel#PR#1`  other project pulll request 1
-    =============== ==============================
+Where the configuration sits in a s3 bucket, the output is a file in a zip. The content of s3data.yaml is::
 
-More details can be found in `moban's changelog.yml <https://github.com/moremoban/moban/blob/master/.moban.cd/changelog.yml#L10>`_
+    hello: world
 
-`repr`
---------------------------------------------------------------------------------
+	
+Usage
+================================================================================
 
-Returns a single quoted string in the templated file
+.. code-block:: bash
 
 
-Built-in Tests
-================================================================================
+    usage: moban [-h] [-cd CONFIGURATION_DIR] [-c CONFIGURATION]
+                 [-td [TEMPLATE_DIR [TEMPLATE_DIR ...]]] [-t TEMPLATE] [-o OUTPUT]
+                 [--template_type TEMPLATE_TYPE] [-f] [--exit-code] [-m MOBANFILE]
+                 [-g GROUP] [-V] [-v] [-D DEFINE [DEFINE ...]]
+                 [template]
+    
+    Yet another jinja2 cli command for static text generation
+    
+    positional arguments:
+      template              string templates
+    
+    optional arguments:
+      -h, --help            show this help message and exit
+      -cd CONFIGURATION_DIR, --configuration_dir CONFIGURATION_DIR
+                            the directory for configuration file lookup
+      -c CONFIGURATION, --configuration CONFIGURATION
+                            the dictionary file
+      -td [TEMPLATE_DIR [TEMPLATE_DIR ...]], --template_dir [TEMPLATE_DIR [TEMPLATE_DIR ...]]
+                            the directories for template file lookup
+      -t TEMPLATE, --template TEMPLATE
+                            the template file
+      -o OUTPUT, --output OUTPUT
+                            the output file
+      --template_type TEMPLATE_TYPE
+                            the template type, default is jinja2
+      -f                    force moban to template all files despite of
+                            .moban.hashes
+      --exit-code           tell moban to change exit code
+      -m MOBANFILE, --mobanfile MOBANFILE
+                            custom moban file
+      -g GROUP, --group GROUP
+                            a subset of targets
+      -V, --version         show program's version number and exit
+      -v                    show verbose
+      -d DEFINE [DEFINE ...], --define DEFINE [DEFINE ...]
+                            to take a list of VAR=VALUEs
+    
 
-`exists`
+Exit codes
 --------------------------------------------------------------------------------
+By default:
+
+- 0 : no error
+- 1 : error occured
 
-Test if a file exists or not
+With `--exit-code`:
+
+- 0 : no changes
+- 1 : has changes
+- 2 : error occured
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,3 @@
		# These are supported funding model platforms

		patreon: chfw
-Original file line number
+Diff line change
@@ Expand Up / @@ -16,9 +16,9 @@ env: @@
       - MINREQ=1
     stages:
-      - test
       - lint
       - moban
+      - test
     .disable_global: &disable_global
       addons: false
@@ Expand Down @@