Streaming parser

CHANGELOG.md

@@ -1,5 +1,63 @@
 # CHANGELOG
 
+## Unreleased
+
+* Re-implemented the parser from the ground up in a more efficient way. The new
+  parser implementation uses core.async channels to complete the stages of the
+  parsing pipeline in parallel.
+
+  Performance is roughly the same (only slightly better) for scores under ~100
+  lines, but significantly better for larger scores.
+
+  More importantly, parsing asynchronously opens the door for us to make playing
+  a score happen almost immediately in the near future.
+
+  See [#37](https://github.com/alda-lang/alda-core/pull/37) for more details.
+
+* An added benefit of the new parser implementation is that it fixes issue
+  [#12](https://github.com/alda-lang/alda-core/issues/12). Line and column
+  numbers are now correct, and error messages are more informative when a score
+  fails to parse.
+
+* The `alda.parser-util` namespace, which included the `parse-to-*-with-context`
+  functions, has been removed. See [this commit](https://github.com/alda-lang/alda-core/pull/37/commits/5f35d659927952e99ea7ec9ab0ee2f4bb2f681aa) for more details.
+
+* The Alda parser no longer generates alda.lisp code.
+
+  Originally, the Alda parser created a score by generating alda.lisp code and
+  then evaluating it.  This actually changed some time ago to a system where the
+  parser generated a sequence of events directly and then used them to build the
+  score. We kept the code that generates alda.lisp code, even though it was no
+  longer an implementation detail of the parser, just an alternate "mode" of
+  parsing.
+
+  With these changes to the parser, it would take some additional work to
+  generate alda.lisp code. Since it is no longer necessary to do that,
+  generating alda.lisp code is no longer a feature of Alda. We could
+  re-implement this feature in the future as part of the new parser, if there is
+  a demand for it.
+
+* Miscellaneous implementation changes that could be relevant if you use Alda
+  as a Clojure library:
+
+  * `alda.parser/parse-input` returns a score map, rather than an unevaluated
+    S-expression. Calling this function will require and refer `alda.lisp` for
+    you if you haven't already done so in the namespace where you're using it.
+
+  * `alda.lisp/alda-code` does not throw an exception by itself if the code is
+    not valid Alda; instead, the output contains an Exception object, which gets
+    thrown when used inside of a score
+
+  * Whereas `alda.lisp/pitch` used to return a function to be applied to the
+    current octave and key signature, now it returns a map that includes its
+    `:letter` and `:accidentals`. This is more consistent with other alda.lisp
+    functions, and it allows notes to have equality semantics.
+
+    In other words, whereas `(= (note (pitch :c)) (note (pitch :c)))` used to be
+    `false`, now it is `true` because we aren't comparing anonymous functions.
+
+  * `(alda.lisp/barline)` now returns `{:event-type :barline}` instead of `nil`.
+
 ## 0.1.2 (2016-12-05)
 
 * Fixed [#27](https://github.com/alda-lang/alda-core/issues/27), a bug where, when using note durations specified in seconds/milliseconds, the subsequent "default" note duration was not being set.

README.md

@@ -4,9 +4,9 @@ The core machinery of [Alda](https://github.com/alda-lang/alda), implemented in
 
 ## Components
 
-* **alda.parser** (reads Alda code and transforms it into Clojure code in the context of the `alda.lisp` namespace)
+* **alda.lisp**: a Clojure DSL for building a musical score
 
-* **alda.lisp** (a Clojure DSL which provides the context for evaluating an Alda score, in its Clojure code form)
+* **alda.parser**: reads Alda code, interprets it as music events, and builds a score using  `alda.lisp`
 
 For more details about how each component works, see the alda-core [development guide](doc/development-guide.md).
 
@@ -20,11 +20,12 @@ Development on the Alda core library requires that you have the [Boot](http://bo
 
 To run the unit test suite, run `boot test`.
 
-#### Adding tests
+When developing, you can run `boot watch test` in a separate terminal and the
+tests will re-run every time you make a change to a file.
 
-It is generally good to add to the existing tests wherever it makes sense, i.e. whenever there is a new test case that Alda needs to consider. [Test-driven development](https://en.wikipedia.org/wiki/Test-driven_development) is a good idea.
+#### Adding tests
 
-If you find yourself adding a new file to the tests, be sure to add its namespace to the list of test namespaces in `build.boot` so that it will be included when you run the tests.
+It is generally good to add new test cases when fixing bugs and adding features. [Test-driven development](https://en.wikipedia.org/wiki/Test-driven_development) is a good workflow when developing alda-core.
 
 The automated test battery includes smoke tests where we parse and evaluate all of the example Alda scores in the `examples/` directory. If you add an additional example score, be sure to add it to the list of score files in `test/alda/examples_test.clj`.
 
@@ -38,12 +39,12 @@ To override this setting (e.g. for development and debugging), you can set the `
 
 To see debug logs, for example, you can do this:
 
-    export TIMBRE_LEVEL=debug
+    export TIMBRE_LEVEL=:debug
 
-When running tests via `boot test` and troubleshooting a failing test, it may be helpful to use debug-level logging by running `TIMBRE_LEVEL=debug boot test`.
+When running tests via `boot test` and troubleshooting a failing test, it may be helpful to use debug-level logging by running `TIMBRE_LEVEL=:debug boot test`.
 
 ## License
 
-Copyright © 2016 Dave Yarwood et al
+Copyright © 2017 Dave Yarwood et al
 
 Distributed under the Eclipse Public License version 1.0.

build.boot

@@ -1,18 +1,19 @@
 (set-env!
   :source-paths   #{"src" "test"}
-  :resource-paths #{"grammar" "examples"}
+  :resource-paths #{"examples"}
   :dependencies   '[
                     ; dev
-                    [adzerk/bootlaces      "0.1.13" :scope "test"]
-                    [adzerk/boot-test      "1.1.2"  :scope "test"]
-                    [alda/sound-engine-clj "0.1.0"  :scope "test"]
+                    [adzerk/bootlaces            "0.1.13"       :scope "test"]
+                    [adzerk/boot-test            "1.2.0"        :scope "test"]
+                    [alda/sound-engine-clj       "0.1.2"        :scope "test"]
+                    [org.clojure/tools.namespace "0.3.0-alpha3" :scope "test"]
                     ; used in examples_test.clj
-                    [io.aviso/pretty       "0.1.33" :scope "test"]
+                    [io.aviso/pretty             "0.1.33"       :scope "test"]
 
                     ; alda.core
                     [org.clojure/clojure    "1.8.0"]
-                    [instaparse             "1.4.3"]
-                    [com.taoensso/timbre    "4.7.4"]
+                    [org.clojure/core.async "0.3.442"]
+                    [com.taoensso/timbre    "4.10.0"]
                     [djy                    "0.1.4"]
                     [potemkin               "0.4.3"]
                     [clj_manifest           "0.2.0"]])
@@ -39,34 +40,7 @@
 
   target  {:dir #{"target"}}
 
-  test    {:namespaces '#{; general tests
-                          alda.parser.barlines-test
-                          alda.parser.clj-exprs-test
-                          alda.parser.event-sequences-test
-                          alda.parser.comments-test
-                          alda.parser.duration-test
-                          alda.parser.events-test
-                          alda.parser.octaves-test
-                          alda.parser.repeats-test
-                          alda.parser.score-test
-                          alda.parser.variables-test
-                          alda.lisp.attributes-test
-                          alda.lisp.cram-test
-                          alda.lisp.chords-test
-                          alda.lisp.code-test
-                          alda.lisp.duration-test
-                          alda.lisp.global-attributes-test
-                          alda.lisp.markers-test
-                          alda.lisp.notes-test
-                          alda.lisp.parts-test
-                          alda.lisp.pitch-test
-                          alda.lisp.score-test
-                          alda.lisp.variables-test
-                          alda.lisp.voices-test
-                          alda.util-test
-
-                          ; benchmarks / smoke tests
-                          alda.examples-test}})
+  test    {:include #"-test$"})
 
 (deftask package
   "Builds jar file."