Elixir for Pythonistas

This post was originally published on Brian’s SaaS application architecture blog.

For years, Python has been my go-to language. I’ve written all sorts of applications with it:

• Django web apps
• Client side GUI applications with PyQt
• Data science stuff with numpy, pandas, etc.
• Alexa applications
• Serverless systems and web APIs
• Microservices and a microservice library
• Many backend services to power various SAAS and non-SAAS applications

Suffice it to say, I’m a Python guy. Sure, I can be effective in other languages, but if there’s a problem that I need to solve programmatically, I reach for Python.

The Problem

Over the past seven years, I’ve worked at three companies, and they all had three things in common:

• Python web stacks
• Django
• Spaghetti code which is (extremely) hard to reason about and evolve

I’ve found that both Python and Django allow developers to make poor decisions when they’re architecting a web application. To be clear, it’s not necessarily a problem with Python or Django themselves. But it’s incredibly easy for good developers to go down the wrong path when writing a large Django application – and Python is guilty by association.

Node and JavaScript are probably worse offenders when it comes enabling bad patterns. Rails and Ruby aren’t far behind – although there are such strong conventions around the “Rails way” that people generally can figure out how to use it. (Which isn’t necessarily a good thing: I believe this creates a community that panics if they don’t have an existing pattern to copy/paste.)

But Django makes it easy to couple different parts of your application together, in spite of your best intentions. I would venture to say that most of this comes down to the ORM. Want to import some models from a completely different part of your application and start firing off queries? No problem. Want to write a triple nested loop over a queryset and fire off 1M+ DB statements for related records? Sure! (Note: I have seen this done and spent weeks fixing it.) Need some data in your template? Just shove an all() queryset in your template and iterate to your heart’s content. This works great when you have 10 rows in your table; not so much when you have 1M rows.

I’ve decided there must be a better way.

The (Possible) Solution

Slowly, I’ve been digging into Elixir. I want to see if a functional language can solve the problems I’ve hit with imperative languages like Python. A few of the reasons I’m exploring Elixir:

• Erlang VM (BEAM)
• Ability for some massive concurrency.
• Built-in messaging (hello microservices!)
• Possibility of hot-loading new code.
• Growing interest (which is mostly from the Rails community — but the consensus that “there must be a better way” is shared by a lot of different communities.)
• Ringing endorsements from other web influencers.

I’m completely open to being wrong: Elixir may not be the solution I’m looking for. But I need a new way of building web applications and microservices, and I’d venture to guess that other people do, too. So I’d like to do a series of posts focused on Elixir from the perspective of a Python developer.

Elixir for Pythonistas

While I’ll need to touch on Elixir syntax here and there, an entire post probably isn’t necessary: there are plenty of resources online about the Elixir language itself, and the official docs are quite good. If you’re starting from scratch, I’d recommend the following:

• Erlang/Elixir Syntax: A Crash Course
• My Elixir Fundamentals talk from the Boulder Elixir Meetup
• Programming Elixir by Dave Thomas

Very quickly, I’d like to get rolling into the distributed nature of Elixir/Erlang. Let’s start with some basics.

Note: I’m writing from the Pythonista’s perspective, but any of my comparisons should be made with procedural or OO languages. Here, I’ll just use Python to represent that class of languages, unless I’m discussing something truly unique to Python.

Immutability and Variables

Unlike Python, variables are immutable. For example:

>>> d = {‘name’: ‘bz’, ‘height’: 67}
>>> some_function(d)

Now, what is the value of d without knowing the details of some_function? It’s impossible to answer this. The reason is that you’re passing the d dictionary by reference, which means some_function can mutate any mutable object it’s given (lists, sets, class, instances, etc.)

What about Elixir:

iex> d = %{name: “brian”, height: 67}
%{height: 67, name: “brian”}    
iex> some_function.(d)
%{height: 67, name: “Fred”}  

You’ll notice that the Elixir shell spits out values while it’s evaluating commands. Here, we can see that some_function is replacing the name key with “Fred.” But, look at d after all of this.

iex> d
%{height: 67, name: “brian”}

That’s right…our original map (dict in Python terms) is unchanged. That’s pretty great. All of a sudden it became much easier to reason about what your program is doing since we’re dealing with data rather than behavior.

So, if we really did want to update our map, how would we handle this? We’ll simply reassign the d variable to the results returned from some_function:

iex> d = some_function.(d)                                                     │:yes
%{height: 67, name: “Fred”}
iex> d
%{height: 67, name: “Fred”}

Let’s just try to manhandle this thing:

iex> Map.put(d, :name, “sam”)
%{height: 67, name: “sam”}  
iex> d
%{height: 67, name: “Fred”}  

Doh! You cannot mutate an existing object. You will always be creating new objects. What you do with those is up to you.

I like this very much. Tracing code is now a matter of looking at what is occurring to the data, rather than trying to track down what code is changing this class instance, dict, etc from under me. There are other implications and advantages to immutable data types I won’t cover here.

Pattern Matching

You’ll hear the term “pattern matching” a lot with Elixir (and, I’d guess, with Erlang). This will likely be the biggest shift in thinking when coming from Python to Elixir, but it’s easy to understand as you work with it.

Above, we seemingly assigned a map to a variable d. Don’t be fooled here: what we did was pattern match the left side of the equality operator with the right side. What does that mean exactly?

Elixir will take an expression and attempt to match whatever is on the left side of the equals sign with that is on the right side. In this case, Elixir is matching the variable d with the map on the right:

iex> d = %{name: “brian”, height: 67}

There is one item on the left, d and one on the right, %{name: “brian”, height: 67}…so d ends up being pointed at this map.

Let’s look an Elixir tuple:

iex> tup = {1, 2, 3}
{1, 2, 3}

Makes sense. But we can also do this:

iex> {a, b, c} = {1, 2, 3}
{1, 2, 3}
iex> a
1
iex> b
2
iex> c
3

Here, Elixir attempts to match the left and right side. Because we have the same number of arguments, the right side values are assigned to the left side variables. This is “unpacking” in Python. We can do the same thing, so you may not be very impressed (yet).

>>> tup = (1, 2, 3)
>>> (a, b, c) = tup

Going back to our Map / dict, how would you extract the value of a dictionary key and assign it into a variable?

>>> d
{‘name’: ‘bz’, ‘height’: 67}
>>> myheight = d.get(‘height’)
>>> myheight
67

With Elixir, we can extract a value by matching a key on the left with the right:

iex> %{height: myheight} = d
%{height: 67, name: “Fred”}
iex> myheight
67

So we’re saying: “Elixir, please match a Map with a key of “height” on the left with whatever is on the right. If that matches, assign the variable “my height” to whatever the value is on the right side.”

That may seem trivial now, but it’s the underpinning of Elixir, and it helps prevent code like this:

def view_function(request, user, reports=None):
  reports = reports or {}
  for key, f in reports.items():
      perm_key = ‘user.can_view_%s_report’ % key
      if key == ‘unsigned’ and not user.sig_on:
          continue
      if key in (‘foo’, ‘br’) and not user.new_user:
          continue
      if key in (‘payments’, ‘all’, ‘client’) and not user.track:
          continue
      if key == ‘authorizations’ and not user.cms_user:
          continue
      if key == ‘totalcostof’ and not user.is_foobar:
          continue
      if key == ‘premiums’ and not \
              (user.some_attribute and request.user.admin and request.user.admin.is_manager):
          continue

In Elixir, all of these conditionals could be handled with pattern matching, resulting in multiple functions that handle some specific part of our domain logic. The code above becomes very (very) hard to reason about, test and debug. Sure, this can be refactored, but to my previous points: because it’s possible to write code like this, it’s inevitable that people will.