Skip to content

Commit

Permalink
Run an executable version of LemMinX
Browse files Browse the repository at this point in the history
Download a binary lemminx, check its integrity, then run it.

Includes:
 * Setting to specify a binary, `xml.server.binary.path`
 * Setting to specify args for the binary, `xml.server.binary.args`

Defaults to java server in cases such as:
 * The binary can't be downloaded
 * The file containing the expected hash of the binary is missing
 * The hash of the binary doesn't match the expected hash
 * Binary specified in setting can't be located and the above three fail

Signed-off-by: David Thompson <[email protected]>
  • Loading branch information
fbricon authored and datho7561 committed Jan 15, 2021
1 parent d73238c commit 5a6c534
Show file tree
Hide file tree
Showing 15 changed files with 1,315 additions and 694 deletions.
9 changes: 9 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
root = true

[*.ts]
[*.js]
indent_style = space
indent_size = 2
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = false
5 changes: 5 additions & 0 deletions .vscodeignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
.vscode
node_modules
src/
tsconfig.json
webpack.config.js
27 changes: 23 additions & 4 deletions Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,13 @@ def buildVscodeExtension(){
sh "npm run vscode:prepublish"
}

def buildLemMinXBinary() {
sh "curl -Lo NativeImage.jenkins https://raw.githubusercontent.com/${params.FORK}/vscode-xml/${params.BRANCH}/NativeImage.jenkins"
load "NativeImage.jenkins"
}

node('rhel7'){
sh "curl -Lo package.json https://raw.githubusercontent.com/redhat-developer/vscode-xml/master/package.json"
sh "curl -Lo package.json https://raw.githubusercontent.com/${params.FORK}/vscode-xml/${params.BRANCH}/package.json"
def packageJson = readJSON file: 'package.json'
def serverVersion = packageJson?.xmlServer?.version
def files = []
Expand All @@ -24,6 +29,8 @@ node('rhel7'){
files = findFiles(glob: 'org.eclipse.lemminx*-uber.jar')
}

buildLemMinXBinary()

if (!files[0]) {
stage 'Build XML LS'
git url: 'https://github.com/eclipse/lemminx.git'
Expand All @@ -39,7 +46,21 @@ node('rhel7'){
deleteDir()
def gitUrl = "${GIT_REPO}"

git url: gitUrl?:'https://github.com/redhat-developer/vscode-xml.git'
git url: gitUrl?:'https://github.com/redhat-developer/vscode-xml.git', branch: params.BRANCH?: 'master'

stage 'set the link to download the binary server'
def packageJson = readJSON file: 'package.json'
def binaryUploadFolder = 'snapshots'
if (publishToMarketPlace.equals('true')) {
binaryUploadFolder = 'stable'
}
sh "sed -i -e 's_${UPLOAD_LOCATION}/vscode/snapshots/lemminx-binary/LATEST_${UPLOAD_LOCATION}/vscode/${binaryUploadFolder}/lemminx-binary/${packageJson.version}-${env.BUILD_NUMBER}_g' package.json"

stage 'package binary hashes'
sh "mkdir ./server"
sh "wget ${UPLOAD_LOCATION}/vscode/${binaryUploadFolder}/lemminx-binary/${packageJson.version}-${env.BUILD_NUMBER}/lemminx-linux.sha256 -O ./server/lemminx-linux.sha256"
sh "wget ${UPLOAD_LOCATION}/vscode/${binaryUploadFolder}/lemminx-binary/${packageJson.version}-${env.BUILD_NUMBER}/lemminx-win32.sha256 -O ./server/lemminx-win32.sha256"
sh "wget ${UPLOAD_LOCATION}/vscode/${binaryUploadFolder}/lemminx-binary/${packageJson.version}-${env.BUILD_NUMBER}/lemminx-osx-x86_64.sha256 -O ./server/lemminx-osx-x86_64.sha256"

stage 'install vscode-xml build requirements'
installBuildRequirements()
Expand All @@ -48,11 +69,9 @@ node('rhel7'){
buildVscodeExtension()
unstash 'server_distro'
def files = findFiles(glob: '**/org.eclipse.lemminx*-uber.jar')
sh "mkdir ./server"
sh "mv ${files[0].path} ./server"

stage "Package vscode-xml"
def packageJson = readJSON file: 'package.json'
sh "vsce package -o vscode-xml-${packageJson.version}-${env.BUILD_NUMBER}.vsix"

//stage 'Test vscode-xml for staging'
Expand Down
148 changes: 148 additions & 0 deletions NativeImage.jenkins
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
pipeline {
agent none
parameters {
// TODO: move this param to the job definition
// This parameter only affects the GraalVM version used under macOS and Windows
// In order to update GraalVM for rhel8, you need to modify the yaml scripts that are used to generate the rhel8 vm image.
string (name: 'GRAALVM_VERSION', defaultValue: '20.3.0', description: 'The version of GraalVM to use. This parameter only affects the Windows and macOS builds. In order to update GraalVM for rhel8, you need to modify the yaml scripts that are used to generate the rhel8 vm image.')
// This param needs to be defined, but it will be inherited from the parent job
//string (name: 'UPLOAD_LOCATION', description: 'Where to upload the binaries and SHA256 hashes')
// Declare 'stable' as a variable in the Jenkinsfile that loads this one
//booleanParam (name: 'stable', defaultValue: false, description: 'Whether this build should be considered a stable build and be uploaded to the stable builds location')
}
stages {
stage("native-image") {
parallel {
// Assumes GraalVM is set up on the rhel8 agent, and the environment variable "GRAALVM_PATH" points to its location
stage("Linux native-image") {
agent {
label "rhel8"
}
steps {
sh "rm -f lemminx-linux"
sh "cd lemminx && git pull && cd .. || git clone https://github.com/eclipse/lemminx.git"
sh "cd lemminx && JAVA_HOME=\$GRAALVM_PATH ./mvnw clean package -Dnative -DskipTests && cd .."
sh "cp lemminx/org.eclipse.lemminx/target/lemminx-linux* lemminx-linux"
stash name: 'lemminx-linux', includes: 'lemminx-linux'
}
}
stage("Windows native-image") {
agent {
label "win10"
}
steps {
powershell "if (Test-Path lemminx-win32.exe) { Remove-Item lemminx-win32.exe }"
powershell """
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
if (Test-Path lemminx) {
cd lemminx
git pull
cd ..
} else {
git clone https://github.com/eclipse/lemminx.git
}
"""
powershell """
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
if (-not (Test-Path graalvm-windows-${params.GRAALVM_VERSION}.zip)) {
Invoke-WebRequest https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-${params.GRAALVM_VERSION}/graalvm-ce-java11-windows-amd64-${params.GRAALVM_VERSION}.zip -OutFile graalvm-windows-${params.GRAALVM_VERSION}.zip
Expand-Archive graalvm-windows-${params.GRAALVM_VERSION}.zip
.\\graalvm-windows-${params.GRAALVM_VERSION}\\graalvm-ce-java11-${params.GRAALVM_VERSION}\\bin\\gu install native-image
}
"""
powershell """
\$Env:JAVA_HOME = \"\$(Get-Location)\\graalvm-windows-${params.GRAALVM_VERSION}\\graalvm-ce-java11-${params.GRAALVM_VERSION}\"
Push-Location
"""
bat """
pushd .
setlocal
set JAVA_HOME="%cd%\\graalvm-windows-${params.GRAALVM_VERSION}\\graalvm-ce-java11-${params.GRAALVM_VERSION}"
call "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\vcvarsall.bat" x86_amd64
popd
cd lemminx
.\\mvnw.cmd clean package -Dnative -DskipTests
cd ..
"""
powershell "mv lemminx\\org.eclipse.lemminx\\target\\lemminx-windows*.exe lemminx-win32.exe"

stash name: 'lemminx-win32.exe', includes: 'lemminx-win32.exe'
}
}
stage("macOS native-image") {
agent {
label "mac"
}
steps {
sh "rm -f lemminx-osx-x86_64.zip lemminx-osx-x86_64.sha256"
sh "cd lemminx && git pull && cd .. || git clone https://github.com/eclipse/lemminx.git"
sh """
if [ ! -f graalvm-darwin-${params.GRAALVM_VERSION}.tar.gz ]; then
curl https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-${params.GRAALVM_VERSION}/graalvm-ce-java8-darwin-amd64-${params.GRAALVM_VERSION}.tar.gz -L --output graalvm-darwin-${params.GRAALVM_VERSION}.tar.gz
tar -xzf graalvm-darwin-${params.GRAALVM_VERSION}.tar.gz
./graalvm-ce-java8-${params.GRAALVM_VERSION}/Contents/Home/bin/gu install native-image
fi
"""
sh "cd lemminx && JAVA_HOME=../graalvm-ce-java8-${params.GRAALVM_VERSION}/Contents/Home ./mvnw clean package -Dnative -DskipTests && cd .."
sh "cp lemminx/org.eclipse.lemminx/target/lemminx-osx-x86_64* lemminx-osx-x86_64"

stash name: 'lemminx-osx-x86_64', includes: 'lemminx-osx-x86_64'
}
}
}
}
stage ("UPX and upload") {
agent {
label "rhel8"
}
steps {
unstash name: 'lemminx-linux'
unstash name: 'lemminx-win32.exe'
unstash name: 'lemminx-osx-x86_64'

// Run UPX to reduce binary size
sh "upx lemminx-linux"
sh "upx lemminx-win32.exe"
sh "upx lemminx-osx-x86_64"

// get the sha256 hash of the files
sh "sha256sum lemminx-linux > lemminx-linux.sha256"
sh "sha256sum lemminx-win32.exe > lemminx-win32.sha256"
sh "sha256sum lemminx-osx-x86_64 > lemminx-osx-x86_64.sha256"

// TODO: "Upload to JBoss Tools"
// Move artifacts into a folder name unique to this build
sh "curl -Lo package.json https://raw.githubusercontent.com/${params.FORK}/vscode-xml/${params.BRANCH}/package.json"
script {
def packageJson = readJSON file: 'package.json'
def vscodeXmlVersion = packageJson?.version
def uploadFolderName = 'snapshots'
if (params.publishToMarketPlace.equals('true')) {
uploadFolderName = 'stable'
}
sh "mkdir ${vscodeXmlVersion}-${env.BUILD_NUMBER}"
sh """
cp lemminx-linux ${vscodeXmlVersion}-${env.BUILD_NUMBER}/lemminx-linux
cp lemminx-linux.sha256 ${vscodeXmlVersion}-${env.BUILD_NUMBER}/lemminx-linux.sha256
cp lemminx-win32.exe ${vscodeXmlVersion}-${env.BUILD_NUMBER}/lemminx-win32.exe
cp lemminx-win32.sha256 ${vscodeXmlVersion}-${env.BUILD_NUMBER}/lemminx-win32.sha256
cp lemminx-osx-x86_64 ${vscodeXmlVersion}-${env.BUILD_NUMBER}/lemminx-osx-x86_64
cp lemminx-osx-x86_64.sha256 ${vscodeXmlVersion}-${env.BUILD_NUMBER}/lemminx-osx-x86_64.sha256
"""
if (!params.publishToMarketPlace.equals('true')){
sh """
ln --symbolic ${vscodeXmlVersion}-${env.BUILD_NUMBER} LATEST
"""
}
sh """
# Upload the uniquely named folder to JBossTools
rsync -Pzrlt --rsh=ssh --protocol=28 ${vscodeXmlVersion}-${env.BUILD_NUMBER} ${params.UPLOAD_LOCATION}/vscode/${uploadFolderName}/lemminx-binary
rsync -Pzrlt --rsh=ssh --protocol=28 LATEST ${params.UPLOAD_LOCATION}/vscode/${uploadFolderName}/lemminx-binary
rm -rf ${vscodeXmlVersion}-${env.BUILD_NUMBER}
rm -f LATEST
"""
}
}
}
}
}
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,11 @@ or you can read this documentation inside vscode with the command `Open XML Docu

## Requirements

For running the binary version:
* Windows, macOS, or Linux, on a x86_64 CPU
* We don't make a binary specific for Apple ARM (Apple Silicon), but the x86_64 binary seems to work through the Rosetta 2 translation layer.

For running the Java version (required if you want to run [extensions](./docs/Extensions#custom-xml-extensions) to the base XML features):
* Java JDK (or JRE) 8 or more recent
* Ensure Java path is set in either:
* `xml.java.home` in VSCode preferences
Expand All @@ -60,6 +65,10 @@ The following settings are supported:
* [`xml.server.vmargs`](https://github.com/redhat-developer/vscode-xml/blob/master/docs/Preferences.md#server-vm-arguments): Specifies extra VM arguments used to launch the XML Language Server.
Eg. use `-Xmx1G -XX:+UseG1GC -XX:+UseStringDeduplication` to bypass class verification, increase the heap size to 1GB and enable String deduplication with the G1 Garbage collector.
* [`xml.server.workDir`](https://github.com/redhat-developer/vscode-xml/blob/master/docs/Preferences.md#server-cache-path): Set a custom folder path for cached XML Schemas. An absolute path is expected, although the `~` prefix (for the user home directory) is supported. Default is `~/.lemminx`.
* [`xml.server.preferBinary`](https://github.com/redhat-developer/vscode-xml/blob/master/docs/Preferences.md#server-binary-mode): If this setting is enabled, a binary version of the server will be launched even if Java is installed.
* [`xml.server.binary.path`](https://github.com/redhat-developer/vscode-xml/blob/master/docs/Preferences.md#server-binary-mode): Specify the path of a custom binary version of the XML server to use. A binary will be downloaded if this is not set.
* [`xml.server.binary.args`](https://github.com/redhat-developer/vscode-xml/blob/master/docs/Preferences.md#server-binary-mode): Command line arguments to supply to the binary server when the binary server is being used. Takes into effect after relaunching VSCode. Please refer to [this website for the available options](https://www.graalvm.org/reference-manual/native-image/HostedvsRuntimeOptions/). For example, you can increase the maximum memory that the server can use to 1 GB by adding `-Xmx1g`
* [`xml.server.silenceExtensionWarning`](https://github.com/redhat-developer/vscode-xml/blob/master/docs/Preferences.md#server-binary-mode): If this setting is enabled, do not warn about launching the binary server when there are extensions to the XML language server installed.
* `xml.trace.server`: Trace the communication between VS Code and the XML language server in the Output view. Default is `off`.
* `xml.logs.client`: Enable/disable logging to the Output view. Default is `true`.
* [`xml.catalogs`](https://github.com/redhat-developer/vscode-xml/blob/master/docs/Preferences.md#catalogs): Register XML catalog files.
Expand Down
6 changes: 6 additions & 0 deletions docs/Extensions.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,12 @@ You can debug your Java LemMinX extension in any Java IDE that supports remote d
8. Debug

![](images/Extensions/DebugLemMinXExtensionInEclipse.png)
## Binary Server

Unfortunately, classes cannot be loaded dynamically through SPI in the binary version of the LemMinX language server.
This means that you can't use LemMinX extensions with the binary server.
vscode-xml will always launch the Java version of the server if LemMinX extensions are detected and Java is present.
If you launch vscode-xml in binary mode with LemMinX extensions installed, a warning will be displayed that extensions are found but can't be used without Java.

## XML extension API (TypeScript)

Expand Down
20 changes: 18 additions & 2 deletions docs/Preferences.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,22 @@
-Dhttp.proxyPassword= <password> -Dhttps.proxyHost=<proxy_host> -Dhttps.proxyPort=<proxy_port>
```

## Server Binary Mode

By default, if Java is not installed, a binary version of the LemMinX language server will be downloaded and run.
If you want to use the binary version even if Java is installed, enable the `xml.server.preferBinary` option.
If something goes wrong with downloading the binary, then vscode-xml will fall back to trying to use the Java server.

Unfortunately, extensions to LemMinX such as lemminx-maven, liquibase-lsp, and lemminx-liberty cannot be used with the binary version of the server.
vscode-xml will always launch the Java version of the server if extensions are detected and the user has Java installed.
A warning will be displayed if you launch vscode-xml in binary server mode with extensions installed.
This warning can be disabled with the `xml.server.silenceExtensionWarning` setting.
Please see the [extensions page](./Extensions.md) for more information.

You can set a path for a custom binary with the `xml.server.binary.path` setting.

Like with the Java version of the server, you can set memory limits and other command line arguments to the server through the `xml.server.binary.args` setting. These will take into effect after relaunching VSCode. Please refer to [this website for the available options](https://www.graalvm.org/reference-manual/native-image/HostedvsRuntimeOptions/). For example, you can increase the maximum memory that the server can use to 1 GB by adding `-Xmx1g`

## Server Cache Path

vscode-xml maintains a cache of the schemas (eg: XSD, DTD) that are referenced using an internet URL.
Expand Down Expand Up @@ -81,9 +97,9 @@ Please see [XSD file associations](Validation.md#xml-file-association-with-xsd)
"catalog2.xml"
]
```

Please see [XML catalog with XSD](Validation.md#XML-catalog-with-XSD) and [XML catalog with DTD](Validation.md#XML-catalog-with-DTD) for more information.

## Grammar

Can be accessed through: `xml.problems.noGrammar`
Expand Down
26 changes: 22 additions & 4 deletions docs/Troubleshooting.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Troubleshooting

## Issues
## Issues

Please create [vscode-xml issues](https://github.com/redhat-developer/vscode-xml/issues) for any problem.

Expand All @@ -13,11 +13,29 @@ You can check if the server is not working in VSCode by going to:
2) `View -> Output -> XML Support` (drop down menu top right)
If it is not working it will indicate the server has Shutdown.

You can kill the process by:
You can kill the process by:

1) Run `jps` command in terminal
2) Check if multiple instances of `org.eclipse.lemminx-uber.jar` or `XMLServerLauncher`
3) According to your OS:

* on Windows OS: run `taskkill /F /PID ...` all instances
* on other OS: run `kill -9 ...` all instances
* on other OS: run `kill -9 ...` all instances

### Warning "The binary is not trusted" when running a binary

This warning means that the sha256sum of the binary server did not match expected sha256sum.

If you get this error when you have not set `xml.server.binary.path`, this means that the downloaded version of the binary server is corrupted.
Please try reinstalling vscode-xml.

If you get this error when you have set the `xml.server.binary.path`, this means that the version you are running is different than the one that vscode-xml.
If you trust this binary, calculate the sha256sum of the binary, then put it in a file in the same folder as the binary server that you are running.
The file name has to be the one in the script below.
Here's how to do this in different operating systems:

| | |
| -------------------- | ----------------------------------------------------------------------------------------------- |
| Linux | `sha256sum lemminx-linux > lemminx-linux.sha256` |
| macOS | `shasum -a 256 lemminx-osx-x86_64 > lemminx-osx-x86_64.sha256` |
| Windows (Powershell) | `(Get-FileHash lemminx-win32 -Algorithm SHA256).Hash | Out-File -FilePath lemminx-win32.sha256` |
Loading

0 comments on commit 5a6c534

Please sign in to comment.