A Domain-Specific Language IDE in the Cloud Using Open-Source Only

This is a tutorial on how to build a web-based IDE with support for an example domain-specific language (DSL) using open-source technologies only. In particular we employ the following technologies from the Eclipse project:

• Xtext to implement a language server for a domain-specific language,
• Theia to build an IDE that runs in the cloud or as desktop client,
• Sprotty to visualize the DSL models in diagrams,
• Eclipse Layout Kernel to layout these diagrams automatically.

This example has been presented at EclipseCon Europe 2018 in Ludwigsburg (recording, slides).

Structure

Each branch in this repository represents a major stage, each commit a subtask to be implemented to reach the stage.

The four stages are as follows:

Stage 1: Implement a Language Server For a Statemachine DSL with Xtext

We create a new Xtext project, add a grammar for a statemachine DSL, and run the Xtext code generator to generate the language server (LS) for the DSL. We also add a bit of sugar in form of

• a parser test,
• a bit of scoping (customizing linking),
• a simple code generator that generates a Java class from a statemachine model,
• and some constraint checks.

Stage 2: Create a Theia Application and Integrate the DSL Language Server

We use a yeoman generator to create the skeleton for a theia-app with a custom extension. We turn the latter into a language extension for our DSL by copying the LS from the last stage, adding a grammar for highlighting and a bit of glue code. At the end of this stage, our IDE supports the textual statemachine language. Thanks to Xtext's, it already offers validation, content assist, hovers, navigation of references, etc.

Stage 3: Add Live-Diagrams for the DSL with Sprotty

Text is nice, but we want a visual representation of the statemachines as well. Using the Sprotty framework, we add diagram support to the LS, generate a diagram model from the DSL models, and extend the client with a diagram widget. The diagram updates automatically when the source model changes. We use the ELK to layout these diagrams and fine-tune the visual representation. To tie the graphics even stronger to the text, we add tracing and validation markers. Now, selection in the text editor and the diagram are synchronized.

Stage 4: Turn the Diagram View into a Graphical Editor (beta)

In this stage, we turn the graphical view into an editor. We add various interaction patterns to the diagrams to allow the creation of elements, the manipulation of edges, and the editing of labels. all these interactions are mapped to text modifications of the DSL model, such that the diagram is always generated from the text and never the other way around. It is far easier to keep the two representations consistent than with any bi-directional synchronization.

Note that the editing features of Sprotty are still in beta.

Build/Run Instructions

Gitpod

Gitpod is a cloud IDE that is based on Theia as well. Gitpod allows to experiment with the contents of this repository without installing anything on your local machine.

The buttons above open the respective stage in Gitpod. To run it, wait for the build to finish and then click on the Open Browser button. An example workspace is opened in a new browser tab.

Local

The prerequisites are defined here.

To build, run

cd language-server
./gradlew build
cd ../theia
yarn

From there, to run the browser app

cd browser-app
yarn start

and point your browser to localhost:3000.

Alternatively, run the electron app by running

yarn rebuild:electron
cd electron-app
yarn start

Troubleshooting

Note, this example application is single-sourced. Whenever you want to switch between the electron and the browser-based applications, you have to rebuild it manually. When switching to the electron, change the directory to theia and run the following:

yarn rebuild:electron

If you want to use the browser-based application, rebuild it for the browser from the theia folder:

yarn rebuild:browser