LexerImpl Unit Tests (#480)

* #422: Added inline verbatim description to Verbatim tag documentation

* Added tests for whitespace control for templates when New Line Trimming is enabled. Also added assertj and commons-io Maven dependencies for use in the tests. Commons IO is used to compare template output to expected output that is loaded from a file. Loading the expected template output from a file is useful for testing multiline template output, which is necessary to test various scenarios. These are a form of Integration Test rather than Unit Tests and so it is practical and reasonable to verify the template output in this way.

* Added javadoc to describe the purpose of several existing tests and changed the test method names in some cases to make the purpose of each test method more clear.

* Added a 3rd elseif in the template for the For Loop with Nested If Statement Thatis Skipped test

* Added tests with 2 ifelse in a nested if and also 3 ifelse in a nested if.

* Added test of Nested If with One ifelse statement

* Improved and added additional whitespace control tests.

* Changed Token.toString() implementation to include the line number, which I found useful to see in log statements.

* Added unit test and formatting standards to Contributing documentation page.

* Added a reference to the Jinja Python template engine, which I think provides more evidence that the Pebble syntax is good. Jinja is a very popular Python template engine.

* Added unit tests for the LexerImpl class. Also added log4j debug level log statements in LexerImpl, PebbleEngine and LexerImpl that were useful to me when trying to follow the code. Added logback-test.xml to configure the logging level. Disabled the debug logging by default.  Also removed Hamcrest Matchers and replaced references to Hamcrest in exception testing code with AssertJ. Also, changed the name of a few private methods in LexerImpl for improve clarity.

* Fixed minor text formatting in the summary statement at the top of the Home page.

docs/src/orchid/resources/homepage.md

@@ -1,9 +1,7 @@
 ---
 ---
 
-Pebble is a Java templating engine inspired by <a href="https://twig.symfony.com/">Twig</a>.
-It features templates inheritance and easy-to-read syntax, ships with built-in <br/>autoescaping 
-for security, and includes integrated support for internationalization.
+Pebble is a Java templating engine inspired by <a href="https://twig.symfony.com/">Twig</a> and similar to the Python [Jinja Template Engine](https://palletsprojects.com/p/jinja/) syntax. It features templates inheritance and easy-to-read syntax, ships with built-in autoescaping for security, and includes integrated support for internationalization.
 
 ## Features
 

docs/src/orchid/resources/pages/contributing.md

@@ -52,10 +52,36 @@ will call `render()` on it's root node which in turn will call `render()` on eac
 This process recurses throughout the whole node tree until all nodes have rendered themselves to the
 provided `Writer` object.
 
+## Documentation
+
+The documentation website is generated using [Orchid](https://orchid.netlify.com/). 
+
+The documentation files are under the `src/orchid/resources` directory.
+
 ## Contributing Code
 Currently there aren't any formal guidelines. Just ensure that your changes include any
 necessary unit tests and send me a pull request on [github](https://github.com/PebbleTemplates/pebble)!
 
+## Standards and Guidelines
+
+### Code Formatting Standards
+
+1. Use [Google Style Guide](https://google.github.io/styleguide/javaguide.html) formatting conventions.
+   1. Configuring IntelliJ to use the Google Style Guide Formatter.
+      1. Clone the [Google Style Git Repo](https://github.com/google/styleguide).
+      2. Open **Preferences -> Editor -> Code Style -> Java**. 
+      4. Click on the gear and select **Import Scheme -> IntelliJ IDEA code style XML** and import the file **intellij-java-google-style.xml** from the [Google Style Git Repo](https://github.com/google/styleguide).
+   2. Configuring Eclipse to use the Google Style Guide Formatter.
+      1. Clone the [Google Style Git Repo](https://github.com/google/styleguide).
+      2. Select **Preference -> Java -> Code Style -> Formatter**.
+      3. Select **Import** and specify the file **eclipse-java-google-style.xml** from the [Google Style Git Repo](https://github.com/google/styleguide).
+
+### Unit Test Guidelines
+
+1. Use [AssertJ]() for all Unit and Integration tests. 
+2. A javadoc description should be included for each test method.
+3. The purpose of each test should be clear from the test method name.
+
 ## Acknowledgements
 Thanks to all the following contributors who are helping to make Pebble the best template engine available for Java. 
 

pebble/pom.xml

@@ -48,23 +48,11 @@
       <version>1.2.3</version>
       <scope>test</scope>
     </dependency>
-    <dependency>
-      <groupId>org.hamcrest</groupId>
-      <artifactId>java-hamcrest</artifactId>
-      <version>2.0.0.0</version>
-      <scope>test</scope>
-    </dependency>
     <dependency>
       <groupId>junit</groupId>
       <artifactId>junit</artifactId>
       <version>4.12</version>
       <scope>test</scope>
-      <exclusions>
-        <exclusion>
-          <groupId>org.hamcrest</groupId>
-          <artifactId>hamcrest-core</artifactId>
-        </exclusion>
-      </exclusions>
     </dependency>
     <!-- https://mvnrepository.com/artifact/org.assertj/assertj-core -->
 	<dependency>
@@ -74,6 +62,7 @@
 	    <scope>test</scope>
 	</dependency>
     <!-- https://mvnrepository.com/artifact/commons-io/commons-io -->
+    <!-- Used for loading template files and expected result files for template tests -->
 	<dependency>
 	    <groupId>commons-io</groupId>
 	    <artifactId>commons-io</artifactId>

pebble/src/main/java/com/mitchellbosecke/pebble/PebbleEngine.java

@@ -47,6 +47,9 @@
 import java.util.Locale;
 import java.util.concurrent.ExecutorService;
 
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 /**
  * The main class used for compiling templates. The PebbleEngine is responsible for delegating
  * responsibility to the lexer, parser, compiler, and template cache.
@@ -55,6 +58,8 @@
  */
 public class PebbleEngine {
 
+  private final Logger logger = LoggerFactory.getLogger(PebbleEngine.class);
+
   private final Loader<?> loader;
 
   private final Syntax syntax;
@@ -148,13 +153,15 @@ private PebbleTemplate getTemplate(String templateName, Loader loader) {
   private PebbleTemplate getPebbleTemplate(String templateName, Loader loader, Object cacheKey) {
 
     Reader templateReader = loader.getReader(cacheKey);
-
+    
     try {
+      logger.debug("Tokenizing template named {}", templateName);
       LexerImpl lexer = new LexerImpl(this.syntax,
           this.extensionRegistry.getUnaryOperators().values(),
           this.extensionRegistry.getBinaryOperators().values());
       TokenStream tokenStream = lexer.tokenize(templateReader, templateName);
-
+      logger.trace("TokenStream: {}", tokenStream);
+
       Parser parser = new ParserImpl(this.extensionRegistry.getUnaryOperators(),
           this.extensionRegistry.getBinaryOperators(), this.extensionRegistry.getTokenParsers(),
           this.parserOptions);