Adding defparser

[judepayne]

Adding a simple defparser macro that creates a parser in the pod and returns a reference to it, just like parse

[borkdude]

Is it possible to add a test in test.clj for this?

[judepayne]

tests added. Let me know if too 'fancy' but the time margin tested in the final test is very safe.
I also tweaked the defparser macro to only deliver a compile-time boost when the grammar is specified as a string rather than (str "first part of grammar" "..."). This brings it into line with how the macro is structured in instaparse itself (and is necessary to avoid the use of eval)

[borkdude]

@judepayne Cool, I'll look at this first thing my morning

[judepayne]

hold off pls. Am making further changes

[borkdude]

Alright!

[judepayne]

Hi @borkdude I'm trying to get every last bit of instaparse core out across a combination of pod.babashka.instaparse and instaparse-bb. Why should Babashka have less?? :)
I have got most of it. Stumped on one thing in the pod...

instaparse has a function to enable-tracing! and then you can call a parse with :trace true. Instaparse prints parse progress/ diagnostics to standard out (according to the docs here).

In the pod, I have a little macro to capture out

(defmacro capture-out
  "Redirects *out* into baos, a ByteArrayOutputStream, while body is evaluated.
   Any output printed by that evaluation will be collected in baos."
  [baos & body]
  `(with-open [ps#  (PrintWriter. ~baos)]
     (binding [*out* ps#]
       (let [res# ~@body]
         (flush)
         res#))))

and I use it in the -main function like so:

:invoke (let [baos (ByteArrayOutputStream.)]
      (do (try
	    (let [var (-> (get message "var")
			  read-string
			  symbol)
		  args (get message "args")
		  args (read-string args)
		  args (read-transit args)]
	      (if-let [f (lookup var)]
		(let [v  (capture-out baos (apply f args))
		      ;; _ (debug :value v :type (type v))
		      value (write-transit (serialize v))
		      reply {"value" value
			     "id" id
			     "status" ["done"]
			     "out" (.toString baos)}]
		  (write stdout reply))
		(throw (ex-info (str "Var not found: " var) {}))))
	    (catch Throwable e
	      (debug e)
	      (let [reply {"ex-message" (ex-message e)
			   "ex-data" (write-transit
				      (assoc (ex-data e)
					     :type (str (class e))))
			   "id" id
			   "status" ["done" "error"]}]
		(write stdout reply))))
	  (.reset baos)
	  (recur)))

If I modify the parse function in the pod to print something out..

(defn parse [ref & opts]
  (println "parse called")
  (let [id (::id ref)
        p (get @parsers id)]
    (-> (apply insta/parse p opts)
        mark-failure)))

I can see "parse called" in the bb test.clj script, when testing the compiled pod.
However, if I enable-tracing! and do a parse with :trace true I am not seeing the output of instaparse. Not capturing it evidently. Inside, the pod, the capture-out macro will capture the printed diagnostics of a :trace true just fine.
Looking in the instaparse code, it's just using standard println for the diagnostics.

Any tips on what I could be missing here?

[judepayne]

Ah I see.
enable-tracing! dynamically reloads the instaparse.gll namespace.

   (defn enable-tracing!
     "Recompiles instaparse with tracing enabled.
      This is called implicitly the first time you invoke a parser with
      `:trace true` so usually you will not need to call this directly."
     []
     (alter-var-root #'gll/TRACE (constantly true))
     (alter-var-root #'gll/PROFILE (constantly true))
     (require 'instaparse.gll :reload))

[judepayne]

Closing this while I see whether a PR to instaparse with native-image support for tracing is acceptable. Will revert with a new PR

[borkdude]

Sure, it might be best to chop up your PRs in smaller parts so we can merge them individually. Thanks for your contributions!

[judepayne]

Pleasure! and thanks for the advice

[judepayne]

Pushed a smaller commit. Feel free to review and merge @borkdude

Additional functionality: defparser macro, parser returns a reified object that implements IFn as well as a Parser protocol so now you can do:

(def p (insta/parser "S = AB*; AB = A B; A = 'a'+; B = 'b'+"))

(p "aaabbbaab")   ;; calling the parser as a function invokes parse (as per instaparse)

Additional tests. Updated Readme

[borkdude]

@judepayne When I run bb test.clj it crashes over here. Could you maybe add the .txt file to e.g. test-resources? I think we need to run these tests in CI as well, but that's different issue/PR.

[judepayne]

test-resources done. I'll have a look at circleci for this

[judepayne]

Please merge the circleci PR first, then I can re-push this one and see where the problem is with test.clj... as it's working with me.

[borkdude]

Done!

[borkdude]

Seems to work! I'll merge.

One minor optional follow up after this, there seems to be something odd in the CircleCI build:

Error computing cache key: template: cacheKey:1:19: executing "cacheKey" at <checksum "project.clj">: error calling checksum: open /home/circleci/repo/project.clj: no such file or directory

[judepayne]

test.clj run successfully!

[judepayne]

Thanks.

Yes it's from this:

  - save_cache:
      paths:
        - ~/.m2
      key: v1-dependencies-{{ checksum "project.clj" }}

Pretty sure I didn't put that there. Assumed you did? If not, not sure how it got there - I'll remove it and then we're good :)