KotlinEditor

A library for parsing Kotlin source code into a parse tree for semantic analysis, linting, and rewriting in-place. Based on the official Kotlin grammar. Supports normal Kotlin source, Kotlin scripts, and Gradle Kotlin DSL.

Quick start

Add the necessary dependencies:

// build.gradle(.kts)
dependencies {
  // Just the generated antlr Listener and Visitor implementations, based on the grammar 
  implementation("app.cash.kotlin-editor:grammar:<<latest version>>")
  // A set of models and utilities that make it easier to interact with parse trees
  implementation("app.cash.kotlin-editor:core:<<latest version>>")
}

Write a listener implementation that extends KotlinParserBaseListener. Here's a partial example from this repo that parses a Gradle Kotlin DSL build script and "normalizes" the plugin applications:

// PluginNormalizer.kt
class PluginNormalizer private constructor(
  private val input: CharStream,
  private val tokens: CommonTokenStream,
  private val parser: KotlinParser,
  private val errorListener: CollectingErrorListener,
) : KotlinParserBaseListener() {

  // TODO: implement various listener methods. See full implementation in `recipes/plugins/`

  companion object {
    fun of(buildScript: Path): PluginNormalizer {
      return of(Parser.readOnlyInputStream(buildScript))
    }

    fun of(buildScript: InputStream): PluginNormalizer {
      val errorListener = CollectingErrorListener()

      return Parser(
        file = buildScript,
        errorListener = errorListener,
        listenerFactory = { input, tokens, parser ->
          PluginNormalizer(
            input = input,
            tokens = tokens,
            parser = parser,
            errorListener = errorListener,
          )
        }
      ).listener()
    }
  }
}

Making sense of parse trees

Implementing anything interesting on top of the tools in this repo requires interacting with the parse trees that antlr generates during a parse. The quickest way to get started with that is to visualize that parse tree. The instructions below explain how to do that.

First, for IDEA users, ensure the ANTLR v4 IDEA plugin is installed.

Next, with the plugin installed, there should be a little ANTLR Preview icon on your left sidebar. If it's not there, tap shift-cmd-a (on macOS) and type "antlr preview" and access it that way. Navigate to one of the grammar (.g4) files in grammar/src/main/antlr, such as KotlinParser.g4. Under "Input" on the left side of the tool window, paste in some Kotlin code (as simple or complex as you like). On the right, you can switch between the "Parse tree" and "Hierarchy" views, each of which are useful. "Profiler" is for profiling performance issues with your grammar, which should not be necessary.

The ANTLR Preview tool seems to prefer to parse source by starting with the kotlinFile start rule. If you want it to parse your source as script instead, navigate to KotlinParser.g4, right-click on script, and select Test Rule script from the context menu.

Project overview

The project is split between three main components:

1. A grammar and the generated parser code, from the ANTLR tool;
2. A "core" library with some high-level concepts build on the parse tree; and
3. A set of "recipes," that do something interesting to or with Kotlin source. These are meant to be used, and should also serve as examples.

The grammar

The grammar itself is broken into three components, one parser and two lexers:

1. The parser, KotlinParser.g4.
2. A lexer, KotlinLexer.g4.
3. Another lexer, UnicodeClasses.g4.

These files were all originally borrowed from the official Kotlin Grammar.

Note that the .tokens files are all generated by the antlr tool, but are checked into the main source set to make the IDE experience better.

The recipes

Some sample recipes are in the recipes directory. Feel free to contribute new recipes if you believe them to be generally useful; otherwise, simply treat this library as a normal dependency for your own projects.

Gradle build scans

This project is configured to publish build scans to the public build scan service. Publication is disabled by default but can be enabled by creating a local.properties file with the following contents:

kotlin.editor.build.scans.enable=true

This file should not be checked into version control.