Profiling Elixir - Part 2

Last week I wrote about my learning experience in writing an implementation of Conway’s Game of Life in Elixir and my experiments with Elixir performance profiling tools. This week I’ll continue and describe one more Elixir profiling.

Using eflame with Elixir

I found the eflame profiler which describes itself as a “flame graph profiler for erlang”. I was intrigued as I had never heard of the flame graph representation before. Eflame’s github page linked to a fascinating page describing flame graphs. I recommend reading the page but in summary, flame graphs are diagrams in which the width of boxes represent the time spend in a function call. The call-chain from that function is stacked up on top of the box. I decided to give eflame a try.

Including eflame in an Elixir Project

Integration of eflame into my Elixir project was a little trickier than it was with the other profilers. Mostly, because eflame’s simple build system doesn’t line up with Mix’s expectations.

First, I simply tried adding

      { :eflame, git: "https://github.com/proger/eflame.git" }

to the deps section my mix.exs file. But mix complained

Could not compile eflame, no mix.exs, rebar.config or Makefile (pass :compile
as an option to customize compilation, set it to false to do nothing)

As recommended, I tried adding compile: false and that suppressed the build error but also left eflame out of my project. I was a bit confused as to how to proceed from here. I was not that familiar with how Mix works, expecially not at this level and was even less familiar with Erlang build systems like rebar.

But after some searching I found the rebar documentation and realized that eflame is using rebar but that it doesn’t need a rebar.config file. I also found that rebar compile is enough to build it. Putting it all together I updated my Mix deps to:

      { :eflame, ".*", git: "https://github.com/proger/eflame.git", compile: "rebar compile"}

and though I was ready to go. But Mix complained again:

** (Mix) The application eflame specified a non Semantic Version aa1e0c3. Mix can only
match the requirement .* against Semantic Versions, to match against any version, please
use a regex as requirement

Ok, a little more searching and I came up with

      { :eflame, ~r/.*/, git: "https://github.com/proger/eflame.git", compile: "rebar compile"}

and Mix was happy. I learned a lot about customizing Mix dependencies and a little about rebar. While I was hoping integration would be easier it was a great experience in my journey of learning Elixir.

With the setup work behind me, updating my Profile class to run eflame on my Elixir code was simple:

defmodule Profile do
  import ExProf.Macro

  def go do
    :eflame.apply(&run_test/0, [])
  end 
  
  def run_test, do: Life.CLI.main(["--animated", "--iterations", "5", "--seed", "test_data/glider1.dat"])
end

and gave me this pretty image to study:

Zooming into the image, I again saw time spent in ensure_compiled?/1 but it was less time than before. More time was show in Life.count_live_neighbors/3. I don’t know if this is a more accurate profile but it at least seemed like something I could work with. Also, the structure of the flame graph gave me the deeper call-chains I was looking for.

Cleanup

I did a little more work to clean up my Profile class. I wanted to go back and try ExProf and fprof again and I decided it didn’t make sense for me to keep changing the code to enable one profiler or the other so I just created functions for each of them:

defmodule Profile do
  import ExProf.Macro

  def eflame do
    :eflame.apply(&run_test/0, [])
  end 
  
  def fprof do
    :fprof.apply(&run_test/0, [])
    :fprof.profile()
    :fprof.analyse(
      [
        callers: true,
        sort: :own,
        totals: true,
        details: true
      ]
    )
  end
  
  def eprof, do:
    profile do: run_test

  def run_test, do: Life.CLI.main(["--animated", "--iterations", "5", "--seed", "test_data/glider1.dat"])
end

Next Steps

Over the next few posts I plan to continue performance analysis of this application. The first step will be to develop a consistent benchmarking methodology so that I can make accurate measurments and track progress. The second step will be to analyze the profiler results and experiment with optmization.