Skip to content

Commit

Permalink
merge with upstream
Browse files Browse the repository at this point in the history
  • Loading branch information
quangIO committed Aug 26, 2021
2 parents 9cb151b + 81adcdc commit aaac25d
Show file tree
Hide file tree
Showing 27 changed files with 766 additions and 211 deletions.
27 changes: 17 additions & 10 deletions .azure-pipelines/main.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,14 @@
# TODO: Add (smoke) tests: load the grammars and execute the queries.
# TODO: For a PR, if only grammars/queries change, compile and test only the affected languages.
trigger:
- master
- ci
branches:
include:
- master
- azure-pipelines # To test changes related to Azure Pipelines.
- ci # To test all CI changes.
paths:
exclude:
- README.org
- CHANGELOG.md

pr:
- master
Expand All @@ -13,12 +19,12 @@ resources:
type: github
name: ubolonton/emacs-module-rs
ref: azure-pipelines
endpoint: ubolonton
- repository: emacs-tree-sitter
endpoint: emacs-tree-sitter
- repository: elisp-tree-sitter
type: github
name: ubolonton/emacs-tree-sitter
name: emacs-tree-sitter/elisp-tree-sitter
ref: azure-pipelines
endpoint: ubolonton
endpoint: emacs-tree-sitter

jobs:
- job: Build
Expand All @@ -33,14 +39,15 @@ jobs:
IMAGE_NAME: 'macos-10.14'
EVM_EMACS: emacs-27.1
BUNDLE_TARGET: macos
ubuntu-16.04-emacs-27.1:
IMAGE_NAME: 'ubuntu-16.04'
ubuntu-18.04-emacs-27.1:
IMAGE_NAME: 'ubuntu-18.04'
EVM_EMACS: emacs-27.1
BUNDLE_TARGET: linux
steps:
- template: .azure-pipelines/steps/setup-emacs.yml@emacs-module-rs
- template: .azure-pipelines/steps/setup-cask.yml@emacs-module-rs
- template: .azure-pipelines/steps/setup-tree-sitter-cli.yml@emacs-tree-sitter
- template: .azure-pipelines/steps/setup-tree-sitter-cli.yml@elisp-tree-sitter
parameters:
version: 0.19.3
- template: steps/grammar-bundle/-create.yml
- template: steps/grammar-bundle/-test.yml
15 changes: 8 additions & 7 deletions .azure-pipelines/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ resources:
type: github
name: ubolonton/emacs-module-rs
ref: azure-pipelines
endpoint: ubolonton
- repository: emacs-tree-sitter
endpoint: emacs-tree-sitter
- repository: elisp-tree-sitter
type: github
name: ubolonton/emacs-tree-sitter
name: emacs-tree-sitter/elisp-tree-sitter
ref: azure-pipelines
endpoint: ubolonton
endpoint: emacs-tree-sitter

jobs:
- job: Build
Expand All @@ -31,18 +31,19 @@ jobs:
IMAGE_NAME: 'macos-10.14'
EVM_EMACS: emacs-27.1
BUNDLE_TARGET: macos
ubuntu-16.04-emacs-27.1:
IMAGE_NAME: 'ubuntu-16.04'
ubuntu-18.04-emacs-27.1:
IMAGE_NAME: 'ubuntu-18.04'
EVM_EMACS: emacs-27.1
BUNDLE_TARGET: linux
steps:
- template: .azure-pipelines/steps/setup-emacs.yml@emacs-module-rs
- template: .azure-pipelines/steps/setup-cask.yml@emacs-module-rs
- template: .azure-pipelines/steps/setup-tree-sitter-cli.yml@emacs-tree-sitter
- template: .azure-pipelines/steps/setup-tree-sitter-cli.yml@elisp-tree-sitter
parameters:
version: 0.19.3
- template: steps/grammar-bundle/-create.yml
- template: steps/grammar-bundle/-save.yml
- template: steps/grammar-bundle/-test.yml

- job: Publish
dependsOn: Build
Expand Down
13 changes: 13 additions & 0 deletions .azure-pipelines/steps/grammar-bundle/-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
steps:
- powershell: |
cask install
.\script\test.ps1
displayName: Run tests (Windows)
condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))

- bash: |
set -e
cask install
script/test
displayName: Run tests
condition: and(succeeded(), ne(variables['Agent.OS'], 'Windows_NT'))
2 changes: 1 addition & 1 deletion .azure-pipelines/steps/grammar-bundle/-upload.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ steps:
- task: GitHubRelease@0
inputs:
gitHubConnection: ${{ parameters.endpoint }}
repositoryName: ubolonton/tree-sitter-langs
repositoryName: emacs-tree-sitter/tree-sitter-langs
isDraft: true
assetUploadMode: replace
assets: |
Expand Down
2 changes: 1 addition & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
@@ -1 +1 @@
queries/* -linguist-detectable
queries/** -linguist-detectable
102 changes: 56 additions & 46 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -11,144 +11,154 @@
path = repos/agda
url = https://github.com/tree-sitter/tree-sitter-agda
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/bash"]
path = repos/bash
url = https://github.com/tree-sitter/tree-sitter-bash
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/c"]
path = repos/c
url = https://github.com/tree-sitter/tree-sitter-c
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/c-sharp"]
path = repos/c-sharp
url = https://github.com/tree-sitter/tree-sitter-c-sharp
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/cpp"]
path = repos/cpp
url = https://github.com/tree-sitter/tree-sitter-cpp
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/css"]
path = repos/css
url = https://github.com/tree-sitter/tree-sitter-css
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/elm"]
path = repos/elm
url = https://github.com/razzeee/tree-sitter-elm
update = none
ignore = dirty
branch = main
ignore = dirty
branch = main
[submodule "repos/fluent"]
path = repos/fluent
url = https://github.com/tree-sitter/tree-sitter-fluent
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/go"]
path = repos/go
url = https://github.com/tree-sitter/tree-sitter-go
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/html"]
path = repos/html
url = https://github.com/tree-sitter/tree-sitter-html
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/janet-simple"]
path = repos/janet-simple
url = https://codeberg.org/sogaiu/tree-sitter-janet-simple
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/java"]
path = repos/java
url = https://github.com/tree-sitter/tree-sitter-java
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/javascript"]
path = repos/javascript
url = https://github.com/tree-sitter/tree-sitter-javascript
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/jsdoc"]
path = repos/jsdoc
url = https://github.com/tree-sitter/tree-sitter-jsdoc
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/json"]
path = repos/json
url = https://github.com/tree-sitter/tree-sitter-json
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/julia"]
path = repos/julia
url = https://github.com/tree-sitter/tree-sitter-julia
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/ocaml"]
path = repos/ocaml
url = https://github.com/tree-sitter/tree-sitter-ocaml
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/php"]
path = repos/php
url = https://github.com/tree-sitter/tree-sitter-php
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/python"]
path = repos/python
url = https://github.com/tree-sitter/tree-sitter-python
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/ruby"]
path = repos/ruby
url = https://github.com/tree-sitter/tree-sitter-ruby
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/rust"]
path = repos/rust
url = https://github.com/tree-sitter/tree-sitter-rust
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/scala"]
path = repos/scala
url = https://github.com/tree-sitter/tree-sitter-scala
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/swift"]
path = repos/swift
url = https://github.com/tree-sitter/tree-sitter-swift
update = none
ignore = dirty
branch = master
ignore = dirty
branch = master
[submodule "repos/typescript"]
path = repos/typescript
url = https://github.com/tree-sitter/tree-sitter-typescript
update = none
ignore = dirty
branch = master
[submodule "repos/pgn"]
path = repos/pgn
url = https://github.com/rolandwalker/tree-sitter-pgn.git
ignore = dirty
branch = master
[submodule "repos/hcl"]
path = repos/hcl
url = https://github.com/mitchellh/tree-sitter-hcl.git
ignore = dirty
branch = master
[submodule "repos/haskell"]
Expand Down
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,20 @@
# Changelog

## Unreleased
- Improved highlighting of arrow function parameters in `javascript` and `typescript`.
- Improved general highlighting for `ruby`.

## 0.10.5 - 2021-08-15
- Upgraded `ocaml` and `pgn` grammars.
- Added hl query patterns for `json`, `ocaml`.
- Improved interpolation highlighting in `python`, `ruby`, `bash`.

## 0.10.2 - 2021-08-06
- Made Python variable highlighting for robust and consistent.
- Added `hcl` grammar and hl query patterns.

## 0.10.1 - 2021-08-01
- Added `pgn` grammar (chess's Portable Game Notation).

## 0.10.0 - 2021-03-14
- Updated most language grammars to the latest version.
Expand Down
28 changes: 15 additions & 13 deletions README.org
Original file line number Diff line number Diff line change
@@ -1,31 +1,33 @@
#+TITLE: tree-sitter-langs
#+TITLE: Tree-sitter Language Bundle for Emacs

This is a convenient language bundle for the Emacs package [[https://github.com/ubolonton/emacs-tree-sitter][tree-sitter]]. It serves as an interim distribution mechanism, until ~tree-sitter~ is widespread enough for language-specific major modes to incorporate its functionalities.
#+BEGIN_HTML
<a href="https://dev.azure.com/emacs-tree-sitter/tree-sitter-langs/_build/latest?definitionId=2&branchName=master" rel="nofollow">
<img src="https://dev.azure.com/emacs-tree-sitter/tree-sitter-langs/_apis/build/status/ci?branchName=master&label=build&api-version=6.0-preview.1" alt="Build Status" style="max-width:100%;">
</a>
#+END_HTML

This is a convenient language bundle for the Emacs package [[https://github.com/emacs-tree-sitter/elisp-tree-sitter][tree-sitter]]. It serves as an interim distribution mechanism, until ~tree-sitter~ is widespread enough for language-specific major modes to incorporate its functionalities.

For each supported language, this package provides:
1. Pre-compiled grammar binaries for 3 major platforms: macOS, Linux and Windows, on x86_64. In the future, ~tree-sitter-langs~ may provide tooling for major modes to do this on their own.
2. An optional ~highlights.scm~ file that provides highlighting patterns. This is mainly intended for major modes that are not aware of ~tree-sitter~. A language major mode that wants to use ~tree-sitter~ for syntax highlighting should instead provide the query patterns on its own, using the mechanisms defined by [[https://ubolonton.github.io/emacs-tree-sitter/syntax-highlighting/interface-for-modes/][tree-sitter-hl]].
3. Optional query patterns for other minor modes that provide high-level functionalities on top of ~tree-sitter~, such as code folding, evil text objects... As with highlighting patterns, major modes that are directly aware of ~tree-sitter~ should provide the query patterns on their own.

** Building grammars from source
It's more convenient to work with this repository as a submodule of [[https://github.com/ubolonton/tree-sitter-langs][emacs-tree-sitter]]. However, if you want to skip that, below are the steps to build grammars from source using only this repository.
It's more convenient to work with this repository as a submodule of [[https://github.com/emacs-tree-sitter/elisp-tree-sitter#building-grammars-from-source][emacs-tree-sitter]]. However, if you want to skip that, below are the steps to build grammars from source using only this repository.

*** Tools and dependencies
- Install [[https://cask.readthedocs.io][cask]].
- Install ~npm~. (Some grammars depend on another npm-packaged grammar. For example: ~cpp~, ~typescript~.)
- Install [[https://tree-sitter.github.io/tree-sitter/creating-parsers#installation][tree-sitter CLI tool]] (if you don't use NodeJS, you can download the binary directly from [[https://github.com/tree-sitter/tree-sitter/releases][GitHub]]).
- Clone this repository and install its dependencies.
#+begin_src bash
git clone https://github.com/ubolonton/tree-sitter-langs
cd tree-sitter-langs
cask install
#+end_src
- Install NodeJS. It is needed to generate the grammar code from the JavaScript DSL. The recommended tool to manage NodeJS is [[https://volta.sh/][volta]].
- Install [[https://tree-sitter.github.io/tree-sitter/creating-parsers#installation][tree-sitter CLI tool]]. (Its binary can also be downloaded directly from [[https://github.com/tree-sitter/tree-sitter/releases][GitHub]].) *Note: versions 0.20+ cannot be used, as they introduce [[https://github.com/tree-sitter/tree-sitter/pull/1157][a breaking change]] in binary storage location*.

*** Building a single grammar
Run ~script/compile <lang>~, where ~lang~ is one of the languages under [[./repos][repos/]].
#+begin_src bash
cask eval "(progn (require 'tree-sitter-langs-build) (tree-sitter-langs-compile 'rust))"
script/compile rust
#+end_src

*** Building the whole grammar bundle
#+begin_src bash
cask eval "(progn (require 'tree-sitter-langs-build) (tree-sitter-langs-create-bundle))"
script/compile all
#+end_src
Loading

0 comments on commit aaac25d

Please sign in to comment.