The Online IDEs Ecosystem

As part of my work in Wix on an online IDE, I wanted to research the field of online IDEs.
Turns out the story behing them is a really interesting and complex one.

For some more context, the product I'm working on is Velo.
The one liner for it goes as follows: Velo is a full-stack development platform that empowers you to rapidly build, manage and deploy professional web apps.

Velo is using the Monaco code editor, developed by Microsoft and used in VSCode. But other than that, it is a very different experience than VSCode.

Currently the experience of using Velo is lacking. To improve that, I wanted to see what other online editors are doing.

What are IDEs?

Let's start off with the terminology. There's a common confusion around IDEs and Text Editors.

  • A text editor is simply a computer program and a tool used for editing plain text.
  • An IDE, on the other hand, is a full-fledge software environment that consolidates basic developer tools required to build and test software.

Examples for Text Editors are:

  • Monaco - the text editor behind VSCode.
  • Notepad - we all know this one.
  • Vim - this is my personal work tool (NeoVim to be percise)

Examples of full-fledged IDEs:

  • IntelliJ IDEA by JetBrains - is an integrated development environment written in Java
  • VSCode by Micrsoft - is an open-source IDE that include support for debugging, syntax highlighting, intelligent code completion, snippets, code refactoring, and embedded Git.
  • Android Studio by Google - provides the fastest tools for building apps on every type of Android device.

What's at the core of modern IDEs?

  • LSP - the Language Server Protocol (LSP) - JSON based
  • Language Server - a process that provides language features
  • DAP - the Debug Adapter Protocol (DAP) defines the abstract protocol used between a development tool (e.g. IDE or editor) and a debugger.
  • Code Editor - the panel where the code is written
  • Workbench - all the UI that surrounds the code editor: file explorer, debug panel, source control panel etc.
  • Extensions - ability to add extra functionality to the IDE in the form of plugins
  • Language Clients - are extensions that add support for a language by running a Language Server and forwarding messages to the IDE

Why LSP?

LSP reduces the number of Language Servers each IDE has to support.

Before this protocol, in order to support a language, the IDE developers had two options:

  • Write code that understands the way that its language-server talks
  • Write their own custom language-server for it

With LSP, each language-server that talks using the protocal can be easily hooked into the IDE. IDEs that support the protocol can interface with all the language-servers the same way. So instead of implementing a whole language-server, or converting messages from one form to another, you could just implement the glue.

LSP Diagram

LSP Messages

So how does an LSP message look? Here's an example of a LSP message saying some text was edited:

tsx
interface TextEdit {
  // The range at which the message applies.
  range: Range;
  newText: string;
}

Here's another message informing of diagnostic errors in a document:

tsx
export interface Diagnostic {
  range: Range;
  severity?: DiagnosticSeverity;
  code?: integer | string;


  /**
   * The diagnostic's message.
   */
  message: string;
}

LSP in VSCode

This is the architecture of LSP in VSCode as illustrated in their website:

LSP Illustration

VSCode runs a node process called the "Extension Host". It runs a language-client for each language. All that client does is bootstrap the language-server for the language and pass messages to it. The language-servers send LSP message using JSON RPC.

More details on that later.

Let's go over options I've researched

So now, what does the market offer in terms of online IDEs? These are just some of the options:

  • Monaco
  • VScode
  • Codesandbox
  • Code-server
  • Theia
  • Stackblitz
  • Gitpod

Monaco

Monaco is a Text Editor. And with the additions of languages support, it can be more capable but still - not very similar to an IDE otherwise.

Monaco started as a browser based editor and later was pivoted to VSCode which started off as a desktop app. Monaco is extracted directly from the vscode codebase and exposed as a separate package. The devs at VSCode wanted to share this piece with the community and so they released it independently of VSCode.

There are some limitations to Monaco that assume that users of it will add the missing functionality themselves, on top of it. Just like VSCode does. Monaco is single file scoped. It wasn't meant for projects and never will be. This means that a you can't open multiple files with Monaco or have it report errors on the whole project. It's just aware of the current file.

Monaco in VSCode repository

Visual Studio Code

As mentioned - VSCode started off as Monaco.

It was later decided to build a full fledged desktop IDE as monaco at its base.

VSCode is roughly divided into:

  • The workbench - the surrounding UI
  • The editor - Monaco
  • The extensions host - a separate node process to run the extensions

The extensions host provides an isolated environment for extensions and doesn't block the UI. If an extension is doing some heavy computation, the user experience won't be hurt.

VSCode

VSCode is an Electron app at its core.

Electron is a cross-platform framework to build desktop apps using web technologies.

It runs a full browser called Chromium (Chrome is built on top of Chromium). That's why a lot of Electron based apps are very large in size.

So how does monaco seem so capable?

If VSCode uses Monaco, Monaco must be able to do a lot more stuff than I presented, right? Well actually, Monaco acts roughly like "Notepad", and VSCode fills in the gaps.

VSCode tells monaco:

  • How to add syntax highlighting
  • Where are the problems in the code
  • Which files to open, close, etc.
  • Most of this is done through VSCode extensions

For example, given a text file containing some code, Monaco doesn't know to understand that specific language. But if VSCode provides it with where the different tokens in the code are, it can apply CSS classes to the token to paint it accordingly. So, given there's a return statement on line 5, column 2, monaco can paint it pink.

There are ways to make Monaco slightly smarter, without running the full VSCode program.

Monaco can get smarter by using Language services (see monaco-typescript) Language services are scaled down Language Servers, very limited compared.

Other than Electron, VSCode also uses:

TextMate is a old code editor used in Macs.

It created a language grammer for defining what each word in a code is. Or in other words, tokenize some text.

Here's an example of a pattern, in lua, that "catalogues" the if, while, for and return statements as control keywords.

lua
patterns = ({  
  name = 'keyword.control.untitled';
  match = '\\b(if|while|for|return)\\b';
})

Given this information, Monaco can paint these tokens in some special way.

The VSCode Workbench itself is written in a very Microsoft-y way.

  • It doesn't use a UI framework like React but rather calls web APIs directly: document.createElement("div")
  • Uses dependency Injection
  • Has an MVVM like architecture

The reason Microsoft didn't use a UI framework like React was for performance reasons. At the end of the day, a framework always adds a performance panelty, for the benefit of developer ergonomics. On to the next product.

codesandbox

Codesandbox is an instantly ready, fully-featured online IDE for web development on any device with a browser. It was built with the idea of enabling users to code and run their webapps online.

Its creator dug heavily into the vscode repo and created a custom build that takes apart vscode and only uses what it needs. Since codesandbox is not running in the desktop or electron, he needed to make changes, hacking its way to a solution:

  • Emulate accessing the filesystem using BrowserFS
  • Writing complex build scripts that separates vscode to different parts and uses only what it needs (using gulp.js)

https://codesandbox.io/static/video-e6a34ad1b9555966b2b8be018e27c2a2.png

code-server

Code-server runs VS Code on any machine anywhere and allows accessing it in the browser. As you can see by now, a lot of the online IDEs are based on VSCode. On top of providing VSCode online, code-server also allows users to build and run their code among other things. It provides a server for for doing such things.

https://raw.githubusercontent.com/cdr/code-server/main/docs/assets/screenshot.png

Theia

Theia was built by the guys at Gitpod - another company that provides a browser IDE as a service. Theia is open sourced under https://github.com/eclipse-theia/theia.

https://raw.githubusercontent.com/eclipse-theia/theia/master/doc/images/theia-screenshot.png

There's an interesting story behind Theia.

Around 2017, VSCode was far away from being able to run in a browser. It relied heaily on Electron which only runs in the Desktop.

The guys at Gitpod decided to build their own browser based IDE. They started Theia around Monaco, which was open sourced by Microsoft at that time.

The entire workbench and extensions system was built from scratch - this involved A LOT of stuff. A terminal, debugger, file tree, etc. Contrary to VSCode, it uses React. It also uses dependency injection, namely InversifyJS.

At 2018, Gitpod gave the ownership of Theia to the Eclipse Open source Foundation. A couple of years later and Github would introduce the online version of VSCode. With that, Microsoft released more of VSCode's code to the open source community.

Despite that, there were still a few things missing. Namely, the server code for vscode, the extensions marketplace and some key extensions developed by Microsoft. Theia is still very activley developed to this day and backed by some serious companies.

It's important to note that:

  • Theia is more customizable than VSCode.
  • Even the workbench itself is a collection of extentions
  • They expose core services using DI

You can check out the extensive API they provide here.

Gitpod

Gitpod is a Cloud IDE as a service.

About a year ago, Gitpod announced that they are moving away from Theia to VSCode.. In 2020, the missing parts from VSCode were open sourced by Microsoft, except for the marketplace.

So what did Gitpod do after their move to VSCode? They kept the repo with their Theia based version of the VSCode server but trimmed it down to only contain the missing pieces: the marketplace. This is open sourced under github.com/openvscode-server and allows installing most VSCode extensions.

https://www.gitpod.io/images/index/vscode-browser.png

Stackblitz

StackBlitz is an online IDE where you can create Angular & React projects that are immediately online & shareable via link in just one click. It automatically takes care of installing dependencies, compiling, bundling, and hot reloading as you type. Runs fully in the browser - can work offline

The core of the tech powering StackBlitz is on source. The core is composed of:

  1. Turbo NPM client - 5x faster than Yarn & NPM, and runs natively in-browser
  2. WebContainers - a way to run Node.js in the browser using WebAssembly

Conclusion

  • Online IDEs have become a valid option for editing code
  • Feature parity between desktop and online IDEs very close
  • VSCode is insanely popular and provides a familiar user experience for web developers
  • All (with minor exceptions) cloud / web-based IDEs are based on it

Taken from the Stackoverflow 2021 developer survey: IDEs Usage Graph

So, which online IDE would I integrate into Velo?

After my research I would narrow it down to:

  • Gitpod's vscode based openvscode-server
  • Theia's highly customizable vscode-like IDE

Both can run in the Desktop or the Browser and are fully open sourced. They are welcoming PRs from the community as well.

← Back