elang

Toy programming language written by yours truely.

• Purely Python with standard library
• Parser: recursive descent
• Expressions in polish notation
• Abstract Syntax Tree Walker
• Literals: integer, character, string, array, map
• Scope / name binding

Use CLI

# Create virtual environment
python3 -m venv venv

# Activate virual environment
source venv/bin/activate

# Install depedencies
pip3 install -r requirements.txt

# Add parent working directory to PATH
export PATH="$(pwd):$PATH"

# List help contents for elang
elang --help

# Run test suit
elang test

# Deactivate virtual environment
deactivate

Example

Produce Abstract Syntax Tree as standard output in JSON format.

$ echo 'print("Hello world");' | elang lex | elang parse --format json
{
  "type": "SEQUENCE",
  "left": {
    "type": "PRINT_STRING",
    "left": {
      "type": "STR",
      "value": "\"Hello world\""
    }
  }
}

Run program.

$ echo 'print("Hello world");' | elang run
Hello world

Python API

lex = Lexer()
tokens = Lexer.from_token_file(file)
tokens = lex.from_program_file(file)
tokens = lex.from_program_lines(lines)
tokens = lex(program_line)
line_generator = Lexer.as_lines(tokens)

parse = Parser()
ast = parse(tokens)
ast_dict = ast.as_dict()
json_str = ast.as_json()
line_generator = ast.as_lines()

walk = Walker()
walk(ast)

Run Tests

export PYTHONPATH="$(pwd)"

# Run all tests
python3 -m unittest discover --verbose --start-directory tests/

# Run specific test suite
python3 -m unittest --verbose tests/test_lexer_tokens.py

# Run specific test
python3 -m unittest --verbose tests.test_lexer_integer_literal.TestIntegerLiteral.test_positive_integers

References

• https://rosettacode.org/wiki/Compiler/lexical_analyzer
• https://en.wikibooks.org/wiki/Compiler_Construction/Syntax_Analysis
• https://docs.python.org/3/library/re.html#writing-a-tokenizer
• https://en.wikipedia.org/wiki/Operator-precedence_parser

TODO

• boolean

• implement static array

  • assign single value
  • assign row
  • binary operation (eg multiply) row
  • print
  • access single value
  • access row

• implement hash table

  • default value
  • assign single value
  • access single value
  • print
  • access keys
  • access values

• loop

  • while with condition and statement
  • halt loop
  • continue loop

• variables

  • int
  • char
  • string
  • array
  • map
  • constants / immutable value
  • typing / immutable type
  • variable scope

• provide token context with AST error

• print statement with multiple arguments

• handle print string escape characters

• assert statement throws error and causes walker to exist with non-zero status code

• test putc(''); => error

• test putc('x'); => 'x'

• test putc(120); => 'x'

• test print(""); => ''

• test print("x"); => 'x'

• test print('x'); => '120'

• test print(120); => '120'

• test print(["Hello", " ", "world", ]); => 'Hello world'

• test print([["Hello", " ", "wolrd",]]); => 'Hello world'