Elixir for the old and for the young

I was amused to announce Lisp; I am thrilled to announce Elixir support on Platform.sh--excited, jubilant. I'll explain why.

By language standards, Elixir is one of the new kids on the block. And like with Lisp, many of you may not have heard about it or were put off and never played with it. I'll explain why I love it, what it's useful for, and how you can get started.

A sorrowful anniversary

But first a bit of personal history. Elixir is one of those things in programming that are, for me, emotional. Maybe even more so now as I write this on the 20th of April. A year ago to the day, Joe Armstrong passed away. Joe was the creator of Erlang, the language on top of which Elixir has been built.

I first encountered Elixir nine years ago at the Rupy 2011 conference in Poznan, Poland, organized by @Zaiste, who, afterward, became a dear friend. I share my chlidlike excitement with him every time we meet a new technology (and even more so, a new programming language) that does things just a bit differently, that solves old problems with new elegance.

Competing affections

I was doing some Erlang for several years, while also getting into the Ruby world. I loved both. I loved the Erlang philosophy and its power to create simple, scalable distributed systems that "fail correctly." I loved Ruby (and Rails) for its productivity and elegance.

My love for Erlang grew through meeting people like Joe Armstrong: approachable, honest, and passionate, a lens through which my eyes were opened to some of the fundamentals of what it is we do. My infatuation with Ruby was ever stronger. Unlike any other language I programmed in, I rarely felt I needed to look at the docs: the APIs fit my mind like a glove.

I could whip up a Rails project in a day, include the devise gem, and have a full-blown application, with user management and all, ready for production. Of course that thing wouldn't scale, and Ruby is an unsafe soup, so making it live would require six layers of testing (good thing Ruby makes writing tests fun). I could also sit down with some smart people, plan ahead, and in a few months produce a beautiful Erlang real-timey massive thing that would be unbreakable.

Erlang has the most mature, but also maybe the simplest, concurrency model you can find. There's a runtime that runs lightweight, concurrent processes, allowing you to run those not only on multiple processes, but also on multiple hosts. Erlang is a "shared nothing" functional language that has built-in primitives to handle failure (when you understand "Let it crash," you will love "Let it crash").

I was in love, and I was frustrated. One language was productive as hell, but would grow into an unmanageable mess unless you applied the strictest of hygiene rules to you and to your team. The other would naturally evolve into a strong, maintainable codebase, but everything was hard: the awkward syntax always in your way, the tooling poor and unnatural.

In case you're not familiar with either, here's an Erlang hello world:

-module(hello).

-export([hello/0]).

hello() ->
   io:format("Hello World!~n", []).

And here is Ruby:

puts "Hello World"

Now, part of the problem of Erlang is that for an object-oriented programmer--someone coming from Python or Ruby or Java, or even JavaScript and PHP--it's quite a mind bend. It takes time until it "clicks."

For example, Erlang allows you to define the same function multiple times (each time with a different signature), and it will automatically call the correct implementation based on the parameters you pass. For a while, you're still looking for your "default values" for if/else control structures. For a while, you find it hard to think in this new, flowing paradigm of processes communicating, of everything immutable. And while you're doing this, you get hit by the awkward syntax, all the time. So getting converted is hard.

And converting is hard. Erlangers, like Haskell lovers, do come off often as religious zealots, full of good intentions, all the while screaming, "YOU ARE ALL GOING TO HELL, MISERABLE MISCREANTS!" When Erlangers do manage to convince others of their pure intentions, that their religion is good ... well, the scriptures turn out to be too mystical for most.

And if you are one of those that really did buy into the problem, you're probably alone; no one else on the team drank a full bottle of Kool-Aid to the last drop.

The awkward syntax, the weird tooling added to the idiosyncratic terminology, however powerful--they act as a barrier. And software projects are really as much about social collaboration and removing barriers between people as they are about primitives.

The zeitgeist

You get the drift. So I began looking for stuff that would bridge the gap between Erlang and Ruby. It was in the zeitgeist. One project that I really loved was https://celluloid.io/ (now, unmaintained) by Tony Arcieri, another one of my heroes. With Dcell, he added the distributed nature of Erlang to Ruby, bringing the actor model to Ruby so we could have a sane concurrency model. It was good. It was even very good. I used it on a couple of projects.

But the thing is that the capabilities Erlang gets from the design of its virtual machine can't really be slapped on top of something like Ruby. Erlang is not only a language, but it's also a runtime. It has a unique framework to build distributed network services (OTP) that's part of the language. And so Erlang is still the only distributed actor model implementation with a preemptive scheduler. It's the only one that really allows you to handle code running on multiple processes on multiple hosts transparently and still have reasonable error handling. Erlang allows you to hot-swap code in production and allows processes to upgrade themselves. Handling tens of thousands of concurrent HTTP connections in Erlang is not something you break a sweat about. With "Ruby with lipstick," you will.

The discovery of Elixir

Enters José Valim. Rupy 2011. The presentation is incredible. Elixir. Wow.

IO.puts("Hello World!")

Code that runs in the Erlang BEAM virtual machine, inheriting all of Erlang's capabilities, that is as sweet and elegant as Ruby. As usual, Joe Armstrong put it best:

He also said:

So I had my solution. This was as compelling a story as you could get. And José Valim was not just anyone. He was one of my Ruby heroes. So I felt that I could trust not only the implementation, but the way forward for the project. I went back to Paris happier than I had been for a while. All I needed was to wait a bit and my new shiny toy would be there. The idea of running Ruby-like code in the Erlang VM was incredibly compelling. Because I knew to trust the runtime. I knew it would handle the concurrency.

And then José Valim decided to put it on hold. (I can't really find the reference to that, but I remember it with sadness.) If I recall correctly, he said at the time that he didn't feel he was competent enough in language design and needed to learn more.

More than a year passed before Elixir had its real first release. This is the same year Go went out, and I believe it stole some of Elixir's thunder. Suddenly, there was another language that was very simple to use and had very strong concurrency primitives.

Adoption

Go only had single machine concurrency, no package management, and no error management to speak of. Still, it was an incredibly simple thing, well suited to the proletariat. And it compiled to a static binary you could just drop anywhere. Nothing graceful, no built-in scaling, but damn simple and damn useful. We're all working in the cloud now, right? Everything is just stateless proxies, and someone else is managing your data and state ... so, who cares.

Elixir, with all its elegance, had some baggage. At the time, the tooling and packaging around Erlang were not that good. So on one side, you had to do complicated things and understand what you were doing and configure quite a beast of a runtime; on the other, "deploy a static binary in a binary blob." And with Docker coming along, the "deploy a static binary in a binary blob" side became the fashionable one.

Also, the version of Elixir that finally materialized didn't try too hard to hide away the underlying Erlang. On the contrary, it was much more of a beautiful synthesis. New syntax, new ideas, all the power and platform breadth of Erlang. But less of a beginner thing.

José Valim and the Elixir community brought more to the table, and the tooling around the language is probably the most sublime you could find. When Phoenix, the Web framework of Elixir, came out I was even more excited. It inherited a lot of the good things about Sinatra and Ruby on Rails, while avoiding many of their pitfalls. Still, Elixir stayed very much a niche thing for the literati.

Over the years, it did get its following with people like Discord, WhatsApp, and AdRoll leveraging its unique capabilities. (Can you imagine anything common between those?)

But Elixir now suffers from the woes of languages that are "almost there." According to the StackOverflow 2019 survey, Elixir ranks just above Clojure (a Lisp!) and just below Dart (wat? Dart is still alive?) at 1.4%.

When choosing a programming language for a project, it's not only about the capabilities of the language, but how easy it is to hire for it. Will it be maintainable in the future? How easy is it to run and maintain in production? And there is a rolling snowball effect to this. Widely used languages get to have much wider library ecosystems to cover more use cases.

Stuff that isn't hard, like having this or that API client for this or that service, is something that can shave off weeks from a project. People got used to "just importing a lib," even when implementing a simple HTTP call is all that's needed. Hey, I implemented a full-blown "Config Reader Library" in Elixir for you, so you can write this:

Platformsh.Config.ecto_dsn_formatter("database")

Instead of something like this:

relationships = Poison.decode!(Base.decode64!(System.get_env("PLATFORM_RELATIONSHIPS")))
[postgresql_config | _tail] = relationships["postgresql"]

Hex, the Elixir package manager, may have 10,000 packages where npm has 1,000,000. (Seriously, check the number...a million!) Still, Elixir brings decades of Erlang with it along with the promise of a scalable, rock-solid production.

So should you bet on Elixir in 2020 ?

Well, as always, this depends on your use case. If you're writing a stateless network proxy, you should very probably use Go. Go is basically a DSL for writing stateless network proxies.

Is there a use case like this for Elixir? Something brain dead? Where the relative unpopularity gets compensated by some super powers? Other than the fact it's a beautiful, beautiful thing?

I think so: Presence and notifications. If your use case has anywhere in its perimeter the idea that you could have hundreds, maybe thousands, maybe millions of users (or for example IOT devices) that are going to be intermittently online, and you want to have the capability to send them messages, probably nothing will do the job better.

Phoenix, our go-to web framework, even has a presence module, which is a full-blown thing. It uses one of my other favorite technologies: CRDTs (Conflict-Free Replicated Data Type invented by Marc Shapiro), that grew and got popular through the Erlang community. Presence uses these data structures to store and broadcast topic-specific information to all channels with a given topic.

defmodule MyApp.MyChannel do
  use MyAppWeb, :channel
  alias MyApp.Presence

  def join("some:topic", _params, socket) do
    send(self(), :after_join)
    {:ok, assign(socket, :user_id, ...)}
  end

  def handle_info(:after_join, socket) do
    push(socket, "presence_state", Presence.list(socket))
    {:ok, _} = Presence.track(socket, socket.assigns.user_id, %{
      online_at: inspect(System.system_time(:second))
    })
    {:noreply, socket}
  end
end

Something like this is basically the full code that you would need to manage the presence of users and to route messages to them from a topic. And because of the underlying Erlang runtime, you would probably be able to handle tens of thousands of users on a single machine. But the day you need to have millions, the same code, with no extra infrastructure elements, would still work. There is a great blog post from discord on how they Scaled Elixir to 5,000,000 Concurrent Users.

So there you have it. And unlike with my Lisp story, woefully at Platform.sh we don't run any Elixir. We truly only have Python and Go as blessed languages. Yes, we are from those rational people that chose rationally. But the second we have to handle the first feature that deals with presence, I'll fight for us to look at Elixir. But also unlike the Lisp story, we do have clients that have asked us for Elixir support. So here goes.

I am jubilant to announce Elixir support on Platform.sh. The Elixir runtime is based on the beautifully maintained distribution from our great friends at Erlang Solutions. We base it off the Enterprise Erlang, so you have a lot of stuff built in (from cowboy to rebar3).

How do you use it?

Well, reading the documentation helps, but here is the gist of it: like for any other language or framework, you'll want to get the runtime configuration from the environment. So in your config/prod.secret.exs you'd want something like:

use Mix.Config

secret_key_base =
  System.get_env("SECRET_KEY_BASE") ||
    raise """
    environment variable SECRET_KEY_BASE is missing.
    """

config :template_elixir, TemplateElixirWeb.Endpoint,
  http: [port: String.to_integer(System.get_env("PORT") || "8888")],
  secret_key_base: secret_key_base

Next, we need the Platform.sh config files, a .platform.app.yaml with something such as:

name: app
type: elixir:1.10
disk: 512
web:
  commands:
    start: mix run --no-halt

Note: The --no-halt is important, the process launched must stay in the foreground .

And in .platform/routes.yaml

"https://{default}/":
  type: upstream
  upstream: "app:http"

And for good form, create an empty .platform/services.yaml.

Git commit, push, enjoy.

Closing words

What I really hope is that support from companies like my own Platform.sh will help remove barriers and ease adoption for beautiful languages like Elixir. I have nothing against Python or Go, but they are just a tiny bit boring, aren't they? We're allowed to have some fun.