Elm-beam

I've been exploring the idea of an Elm compiler that produces assembly for the Erlang Virtual Machine. You can find the code for this project on GitHub. This essay documents some interesting parts of the project.

Background

According to the website, Elm is "a delightful language for reliable webapps." I am a huge fan of the language and its goals, and I encourage you to give it a try. Elm helped me enjoy writing front-end code again, and many in the community share that experience. The Elm ecosystem is focused on the front-end webapp domain. This domain focus is much of what makes Elm delightful—the tooling, documentation, and community largely coalesce around this goal.

This experience with Elm produces a particular effect in many folks. I've heard it called the X-is-great-so-we-should-use-X-everywhere effect. A large part of our small community is excited by the idea of "Elm on the server", though that phrase likely means different things for different people. I enjoy how the Elm author responds to this idea in the roadmap writeup:

Many folks tell me “Elm should compile to X” where X is a thing they like. Here are people suggesting Go, Lua and Erlang... But why not go through OCaml or C# or Java or Scala or F# or Haskell or ES6 or C++ or Rust or node.js?

The hard part of supporting a domain is not the compiler. It is making a good ecosystem. Python is nice for scientific computing because of things like NumPy and SciPy, not because of whatever backend. Elm is nice for front-end because of the ecosystem, like the HTML library and The Elm Architecture, not the particulars of code generation. Just putting a typed functional language in a domain does not mean it will be fun and productive in practice!

... Writing a compiler backend is like 5% of what it takes to make a project like this worthwhile.

I echo this idea completely. The project discussed here is not "Elm on the server". Nor is it a suggested direction for the Elm team. Instead, it's a fun exploration in compilers and code generation that I'm excited to share with others. Perhaps the work here will help someone interested in building a delightful language for reliable web servers.

Project overview

$ tree
.
├── elm-compiler
│   └── # GIT SUBMODULE OF 0.18 ELM COMPILER
└── src
    ├── ElmBeam
    │   └── # BRIDGE BETWEEN SUBMODULE AND src/
    ├── Generate
    │   └── # CODEGEN MODULES
    └── Main.hs

elm-beam is a Haskell project with 2 major dependencies: the 0.18 Elm compiler and a BEAM codegen library I wrote called codec-beam. Integration with the Elm compiler is a bit untraditional. The Cabal file points both to files in this project and files in the git submodule. Building the project results in an executable that takes a .elm file and produces a file called elm.beam.

The Erlang Virtual Machine

In many ways, Elm matches the semantics of Erlang and Elixir. These languages all feature immutable data structures, pattern matching, and lambda functions. However, the similarities run even deeper—to the implementation of the Elm runtime itself. Here is a quote from the documentation of Elm's Process module:

Right now, this library is pretty sparse. For example, there is no public API for processes to communicate with each other. This is a really important ability, but it is also something that is extraordinarily easy to get wrong!

I think the trend will be towards an Erlang style of concurrency, where every process has an “event queue” that anyone can send messages to.

Internally, the Elm runtime is built upon processes and messages. It seems this implementation is explicitly inspired by Erlang's design. Elm programmers will recognize this design from their use of ports, Elm's means of JavaScript interop. In order to communicate between JavaScript and your Elm application, you send messages! Much like inter-process communication in Erlang:

  SENDING A MESSAGE

app.ports.fromJS.send(x)  // Elm ↔ JavaScript
App_pid ! X               %% Erlang ↔ Erlang


  RECEIVING A MESSAGE

app.ports.toJS.subscribe(x => f(x))  // Elm ↔ JavaScript
receive X -> f(X) end                %% Erlang ↔ Erlang

elm-beam embraces these similarities. The compiled Elm application is a BEAM module that defines an OTP gen_server. Starting a gen_server and communicating with it uses Erlang functions from the standard library. You can see these functions in action in the following demo:

gen_server design implications

Since elm-beam commits to the gen_server protocol, Elm-written modules can exist within a supervision tree. To me, this may be the most exciting aspect of this exploration. Imagine your team has spent several years writing software in the Erlang/Elixir ecosystem. During that time, your organization has developed tooling and workflow around the Erlang Virtual Machine. Now, you are interested in exploring what static types have to offer. Must you forfeit all of the tooling and workflow just to try it out?

The design of the project thus far suggests an answer to the hypothetical concern. elm-beam allows you to incrementally introduce Elm into Erlang/Elixir systems. Elm-written gen_server's act just like Erlang/Elixir-written gen_server's, living inside the same supervision trees and applications.

Generating BEAM

elm-beam is written in Haskell because the Elm compiler is written in Haskell. The Erlang compiler, however, is written in Erlang. While the Erlang toolchain exposes an API into its compiler, your program must be written in Erlang in order make use of it. For this project, I decided to generate the BEAM files myself. That's why I created codec-beam – a library for assembling syntactically valid BEAM files.

Though elm-beam is an experiment, I plan to continue supporting codec-beam. I'm excited to see how other Haskell programmers will use it to create their own compile-to-BEAM languages. If you are interested in learning more about codec-beam, please refer to my talk: Getting to BEAM without going through Erlang.

A previous version of elm-beam compiled to an Erlang intermediate form called Core Erlang. You could then use erlc +from_core to compile the .core file into a .beam file. I decided that the experience would be much nicer if I could generate BEAM myself, so I abandoned that direction. If you'd like to learn more about my time with Core Erlang, please refer to my blog post: The Core of Erlang.

Is BEAM best for Elm?

No. In fact, Elm's author suggests as much in the aforementioned roadmap writeup:

Ultimately, the ideal form of a project like this has its own bespoke backend, going to assembly directly.

I agree. In many ways, going through BEAM was a "shortcut". The similarities between Erlang and Elm mean that Elm "fits in" with BEAM languages, but Elm can certainly do better. For instance, an Elm-specific garbage collector could use a more precise strategy than that of Erlang's. This is a natural difference between languages with/without exhaustive type information at compile-time.

The Erlang platform includes a native compiler called the High Performance Erlang (HiPE). HiPE reads BEAM modules and produces native programs that generally run faster than their VM-interpreted counterparts. The existence of HiPE is further support for the "going to assembly" Elm compiler. With exhaustive type information at compile-time, Elm could create faster, more densely packed programs than BEAM even allows.

---

At this point, I feel like my work with elm-beam has come to an end. I'm satisfied with the results of the exploration, and I'm ready to continue with other projects. Hopefully you've found value in some these ideas. If so, I'd love to hear from you. And to those interested in continuing the exploration: join us in the #elm-beam channel on the Elm Slack.

Finally, I want to express the sincerest admiration for all the projects I referenced: Elm and Erlang in particular. Languages are large projects, and I tend to take them for granted. Thanks to all the folks involved in each of those projects, who spend time and energy creating awesome tools that I can use for free.