I have been writing reusable modules, like Blocking Queue which could be used in an appliction. That is, Blocking Queue is not an application on its own. It’s more like a library (to borrow a term form other languages).
I’ve been working on a few projects like this lately and I’ve found that I have a particular workflow. I do two things to try to maintain the quality of the code in these modules
- I write tests, mostly integration tests.
- I write typespecs and check them with Dialyzer
But, I have found that this workflow is lacking. I really want to be able to run Dialyzer over the integration tests. I want to do this in order to make sure that the tests and the module’s API match up. There are several reasons for this: first it makes sure the tests are correctly, but more importantly it makes sure that the types I’ve chosen for the API are correct or at least usable. If I can’t write tests that conform to the API then the API is too hard to use.
By default Dialyzer doesn’t run on tests because Dialyzer analyzes .beam files and tests are written as .exs scripts which are not compiled to BEAM.
In this post I will explain how I setup my tests so that I can compile then and run Dialyzer on them
I asked about how to do this on the elixir-lang-talk mailing list. and got a very helpful reply from José Valim:
You can use the elixirc_paths configuration to compile extra directories alongside your code:
Then you can define a module with assertions inside your elixirc_paths, for example, test/integration/foo.ex:
And define a regular test case that calls the functions in the new module, for example, test/my_app/foo_test.exs:
This way you have a module only for tests, that would be compiled like regular code in the test environment, that would integrate with dialyzer, and another module that run its tests. The only note is that you will need to run dialyzer in the test environment:
In the rest of the post I’ll follow José’s advice and will explain in great detail.
Compiling ExUnit Tests
The first step, is move the integration tests to another directory and rename it .ex instead of .exs. I’ve moved test/blocking_queue_test.exs to test/integeration/blocking_queue_test.ex.
Next, I add the following to mix.exs:
This differs slightly from the example José cited because I’m not using Phoenix and without Phoenix the default elixirc_paths is just
I also added this line to the Mix project:
I tested this out by running
mix test but what I have isn’t enough. The tests in blocking_queue_test.ex can’t use the
test macro they need to be regular functions. As regular functions, the tests can’t use strings for their names. After fixing these issue
mix test compiles correctly. I ended up with:
Bringing the tests back to ExUnit
The next step is to add a test file that just calls my integration test functions within ExUnit tests. I ended up with:
Certainly this the format of each test is regular enough that I could write a macro for this. I’ll put this down as a topic to visit later.
Running Dialyzer on ExUnit Tests
Finally, I can try running Dialyzer (in the test environment) to see how the types line up between my API and the usage in the tests.
It works! And it uncovered an error! Though, the error that it uncovered is just a small problem in the test itself. It seems that the types described by my Blocking Queue API and the tests are consistent. But it’s good that I am able to check (and keep them consistent going forward).
I can fix the one error reported by assinging the value to
_ like this:
In this post I’ve followed José Valim’s advice and have been able to run Dialyzer over my integration tests. I’m happy with the results and feel like this will improve my ability to test and write high quality Elixir code.
However, there are some redundancies in the test code itself. The file test/integration_test.exs provides no new information: it just imports external test functions into a file that ExUnit can work with. I would like to experiment with this a bit to see if this workflow could be simplified. The management of the test files and configuration of the test list should be easier to use.
Since I now have the original, recommended process working, next week I can try simplifying and automating it.