Apache Daffodil™ Extension for Visual Studio Code: v1.2.0
The Apache Daffodil™ Extension for Visual Studio Code is an extension to the Microsoft® Visual Studio Code (VS Code) editor which enables Data Format Description Language (DFDL) syntax highlighting, code completion, and the interactive debugging of DFDL Schema parsing operations using Apache Daffodil™.
DFDL is a data modeling language used to describe file formats. The DFDL language is a subset of eXtensible Markup Language (XML) Schema Definition (XSD). Just as file formats are rich and complex, so is the modeling language to describe them. Developing DFDL Schemas can be challenging, requiring a lot of iterative development, and testing.
The purpose of Apache Daffodil™ Extension for Visual Studio Code is to ease the burden on DFDL Schema developers, enabling them to develop high quality, DFDL Schemas, in less time. VS Code is free, open source, cross-platform, well-maintained, extensible, and ubiquitous in the developer community. These attributes align well with the Apache Daffodil™ project and the Apache Daffodil™ Extension for Visual Studio Code.
DFDL is rich and complex. Developers using modern code editors expect some degree of built-in language support for the language in which they are developing, and DFDL should be no different. The Apache Daffodil™ Extension for Visual Studio Code provides syntax highlighting to improve the readability and context of the text. In addition, the syntax highlighting provides feedback to the developer indicating the structure and code appear syntactically correct.
The Apache Daffodil™ Extension for Visual Studio Code provides code completion, also known as “Intellisense”, offering context-aware code segment predictions that can dramatically speed up DFDL Schema development by reducing keyboard input, memorization by the developer, and typos.
The Apache Daffodil™ Extension for Visual Studio Code provides a Daffodil Data Parse Debugger which enables the developer to carefully control the execution of Apache Daffodil™ parse operations. Given a DFDL Schema and a target data file, the developer can step through the execution of a parse line by line, or until the parse reaches some developer-defined location, known as a break point, in the DFDL Schema. What is particularly helpful is that the developer can watch the parsed output, known as the "infoset", as it’s being created by the parser, and see where the parser is parsing in the data file. This enables the developer to quickly discover and correct issues, improving DFDL Schema development and testing cycles.
The Apache Daffodil™ Extension for Visual Studio Code provides an integrated data editor as a new experimental feature that is currently under development. It is akin to a hex editor, but tuned specifically for challenging Daffodil use cases. It is designed to support virtually any sized file, well beyond the limits of the standard text editor in VS Code, and it can handle non-text data just as well as text data. It has support for setting Daffodil debugger breakpoints on offset positions in the data file in addition to the positions in the DFDL Schema. It handles non-standard byte sizes, non-aligned bytes, and byte ordering where the Least Significant Byte (LSB) can be the first or last bit in a byte. As an editor designed for Daffodil developers by Daffodil developers, features of the tool will evolve quickly to address the specific needs of the Daffodil community.
This guide assumes VS Code and a Java Runtime Environment (Java 8 or greater) are installed.
- Install VS Code
- Install Java Runtime 8 or greater
- On Linux, glibc 2.31 or greater is required
The Apache Daffodil™ Extension for Visual Studio Code can be installed using one of two methods.
Option 1: Install the Apache Daffodil™ Extension for Visual Studio Code From the Visual Studio Code Extension Marketplace
The Apache Daffodil™ Extension for Visual Studio Code is available in the Visual Studio Code Extension Marketplace.
Option 2: Install the Latest .Vsix File From the Apache Daffodil™ Extension for Visual Studio Code Release Page
The latest .vsix (the file extension used for VS Code extensions) file can also be downloaded from the Apache Daffodil™ Extension for Visual Studio Code releases page and installed by either:
- Using the command-line via
code --install-extension <path-to-downloaded-vsix-file>; or - Using the "Extensions: Install from VSIX" command from within VS Code by opening the Command Palette (Mac = Command+Shift+P, Windows/Linux = Ctrl+Shift+P), and typing
vsixto bring up the command and pointing it at the downloaded.vsixfile, as demonstrated in the following animation.
Since DFDL Schema files end with .xsd (XML Schema Definition or XSD), the editor needs to be informed specifically that DFDL mode is desired over the more general XML mode, the following animation demonstrates how to set the desired mode for DFDL.
Auto suggest is triggered using control space or typing the beginning characters of an item, as demonstrated in the following animation.
📝 NOTE: Intellisense is context aware, so there is no need to begin a block with <, just start typing the tag name and code completion will automatically handle it as appropriate.
Typing one or more unique characters will further limit the results, as demonstrated in the following animation.
Code completion can be used to add the schema block, with just a couple of keystrokes, as demonstrated in the following animation.
Code completion can make short work out of completing a DFDL Format Block, offering context-sensitive suggestions for the format attribute values, as demonstrated in the following animation.
The > or / characters are used to close XML tags. Use tab to select an item from the drop down and to exit double quotes, as demonstrated in the following animation.
Code completion supports creating self-defined dfdl:complextypes and dfdl:simpleTypes, as demonstrated in the following animation.
The tab key can be used to complete an auto-complete item within an XML tag. After auto-complete is triggered, typing the initial character or characters will limit the suggestion results. Inside an XML tag a space or carriage return will trigger a list of context sensitive attribute suggestions, as demonstrated in the following animation.
The following animation demonstrates how code completion can be used to efficiently help create self-defined types.
The following animation demonstrates how code completion can be use to efficiently create xs:choices and dfdl:discriminators.
The following animation demonstrates how code completion can help authors use hidden references and dfdl:inputValueCalc.
The following animation demonstrates how code completion can help with creating elements using dfdl:outputValueCalc.
The following animation demonstrates examples of code completion assisting in the creation of more user-defined types.
XPath expressions can be code completed. The following animation demonstrates how the Path expressions are completed when calculating dfdl:Length values.
The following animation demonstrates how code completion can be used to help create dfdl:assert blocks.
The following animation demonstrates another couple of examples of dfdl:assert block creation using code completion.
- The Apache Daffodil™ Extension for Visual Studio Code uses a clunky method to auto complete curly braces within quotes. It is anticipated that this will be better addressed in the future. The auto complete method blocks suggestions while typing between the beginning quote, opening curly brace and the closing curly brace, ending quote.
Debugging a DFDL Schema Using the Apache Daffodil™ Extension for Visual Studio Code’s Bundled Daffodil Data Parse Debugger
Debugging a DFDL Schema needs both the DFDL Schema to use and a data file to parse. Instead of having to select the DFDL Schema and the data file each time from a file picker, a "launch configuration" can be created, which is a JSON description of the debugging session.
To create the launch profile:
-
Select
Run -> Open Configurationsfrom the VS Code menubar. This will load alaunch.jsonfile into the editor. There may be existingconfigurations, or it may be empty. -
Press
Add Configuration...and select theDaffodil Debug - Launchoption.
Once the launch.json file has been created it will look something like this
{
"type": "dfdl",
"request": "launch",
"name": "Ask for file name",
"program": "${command:AskForProgramName}",
"stopOnEntry": true,
"data": "${command:AskForDataName}",
"infosetOutput": {
"type": "file",
"path": "${workspaceFolder}/infoset.xml"
},
"debugServer": 4711
}This default configuration will prompt the user to select the DFDL Schema and data files. If desired, the "program" and "data" elements can be mapped specifically to the user's files to avoid being prompted each time.
📝 Note: Use ${workspaceFolder} for files in the VS Code workspace and use absolute paths for files outside of the workspace.
{
"type": "dfdl",
"request": "launch",
"name": "DFDL parse: My Data",
"program": "${workspaceFolder}/schema.dfdl.xsd",
"stopOnEntry": true,
"data": "/path/to/my/data",
"infosetOutput": {
"type": "file",
"path": "${workspaceFolder}/infoset.xml"
},
"debugServer": 4711
}Using the launch profile above a DFDL parse: My Data menu item at the top of the Run and Debug pane (Command-Shift-D) will display. Then press the play button to start the debugging session.
In the Terminal, log output from the DFDL debugger backend service will display. If something is not working as expected, check the output in this Terminal window for hints.
The DFDL Schema file will also be loaded in VS Code and there should be a visible marking at the beginning where the debugger has paused upon entry to the debugging session. Control the debugger using the available VS Code debugger controls such as setting breakpoints, removing breakpoints, continue, step over, step into, and step out.
-
Option 1:
- Open the DFDL Schema file to debug
- From inside the file open the Command Palette (Mac = Command+Shift+P, Windows/Linux = Ctrl+Shift+P)
- Once the command Palette is opened start typing
Daffodil Debug:- Option 1 =
Daffodil Debug: Debug File- This will allow for the user to fully step through the DFDL Schema. Once fully completed, it will produce an infoset to a file namedSCHEMA-infoset.xmlwhich it then opens as well. - Option 2 =
Daffodil Debug: Run File- This will run the DFDL Schema, producing the infoset to a file namedSCHEMA-infoset.xml.
- Option 1 =
-
Option 2:
- Open the schema file to debug
- Click the play button in the top right, two options will be provided:
- Option 1 =
Debug File- This will allow for the user to fully step through the schema (WIP). Once fully completed, it will produce a infoset to a file namedSCHEMA-infoset.xmlwhich it then opens as well. - Option 2 =
Run File- This will run the DFDL Schema, producing the infoset to a file namedSCHEMA-infoset.xmlwhich it then opens as well.
- Option 1 =
Find the infoset tools from the command menu (Mac = Command+Shift+P, Windows/Linux = Ctrl+Shift+P)
Find the hex view from the command menu (Mac = Command+Shift+P, Windows/Linux = Ctrl+Shift+P)
To enable the Apache Daffodil™ Extension for Visual Studio Code experimental features, from the command menu start typing 'daffodil', then select Daffodil Debug: Enable Experimental Features, then select Yes.
🧪 Warning: This is currently an experimental feature in development.
Ωedit is being integrated as the experimental data editor in the Apache Daffodil™ Extension for Visual Studio Code. Once experimental features are enabled, find the Data Editor in the command menu by typing 'omega', then select OmegaEdit: Data Editor.
After selecting a file to edit, a Data Editor tab will appear.
As of v1.2.0, this experimental feature is far from functional, but will be improving over time.
- Wrong Java Development Kit (JDK). Be sure Java 11+ is running.
On MacOS, using Homebrew:
# Install Java 11 from a macOS terminal
brew install java11Add change JAVA_HOME in the ~/.zshrc file (or equivalent):
# Java 11
export JAVA_HOME=/usr/local/Cellar/openjdk@11/11.0.12Be sure code is in the PATH by following the instructions here.
With JAVA_HOME set to the Java 11 install, run code in the terminal.
If problems are encountered or new features are desired, create tickets here.
If additional help or guidance on using Daffodil and its tooling is needed, please engage with the community on mailing lists and/or review the archives.