CI Pipelines in Elixir Projects

Continuous Integration (CI) is one of the most important practices introduced by the DevOps revolution, the goal is to reduce feedback loops while developing software.

For this, pipelines with quality and compliance checks executed by a continuous integration server have become a fundamental part. They allow us to verify, to a certain extent, that the changes we are integrating meet the established criteria.

Elixir has its own tools, and today we have various tools that allow us to automate different aspects of our code, put them in a pipeline, and execute them automatically with each commit.

On the other hand, I would like to take the opportunity to put this entire pipeline into a single script, so we can use it easily both in development and on the CI server.

The first step is to configure the preferred_envs option in our mix.exs file so that we don’t have to include MIX_ENV=test in every task.

def cli do
  [
    preferred_envs: [ci: :test]
  ]
end

Additionally, include an alias in the same mix.exs file to run our task using mix ci.

defp aliases do
  [
    # ... other aliases
    ci: ["CI"]
  ]
end

Now we create our lib/mix/tasks/ci.ex file and start writing our task.

defmodule Mix.Tasks.CI do
  @moduledoc """
  Run all the checks and tests for the project
  """
  use Mix.Task

  @preferred_cli_env :test

  @impl Mix.Task
  def run(_args) do
   ...
  end
end

Dependency Checks

The first step in our pipeline is fetching dependencies.

We can check our mix.exs file for retired dependencies using the mix hex.audit task, so we can replace them.

Then, by default, we have the deps.unlock --check-unused task that checks that we do not include dependencies in our mix.exs file that are not being used.

Additionally, the mix_audit dependency allows us to scan dependencies for reported vulnerabilities using the mix deps.audit task.

We can include them in our CI file:

defmodule Mix.Tasks.CI do
  @moduledoc """
  Run all the checks and tests for the project
  """
  use Mix.Task

  @preferred_cli_env :test

  @impl Mix.Task
  def run(_args) do
    Mix.Task.run("deps.get")

    Mix.Task.run("cmd", ["MIX_ENV=dev", "mix hex.audit"])

    Mix.Task.run("deps.unlock", ["--check-unused"])

    Mix.Task.run("deps.audit", ["--format", "human"])
end

Compiler Checks

Elixir includes a very useful argument called --warnings-as-errors. This allows us to treat compiler warnings as errors, preventing us from proceeding until they are resolved.

Although I personally like to enable this option within the mix.exs file, making it work all the time, some people find this option somewhat intrusive¹, as it can create friction during development by stopping the flow, for example, due to forgetting a _ in a variable that we are not using at the moment.

Despite the initial discomfort, I find that this friction disappears after a short adaptation period, making us more disciplined when writing code. However, I consider it essential to have this option active (at least) on the CI server.

Otherwise, our console will be plagued with warnings that make it difficult to read what is really important.

defmodule Mix.Tasks.CI do
  @moduledoc """
  Run all the checks and tests for the project
  """
  use Mix.Task

  @preferred_cli_env :test

  @impl Mix.Task
  def run(_args) do
    Mix.Task.run("deps.get")

    Mix.Task.run("cmd", ["MIX_ENV=dev", "mix hex.audit"])

    Mix.Task.run("deps.unlock", ["--check-unused"])

    Mix.Task.run("deps.audit", ["--format", "human"])

    Mix.Task.run("compile", ["--warnings-as-errors"])
  end
end

Format Checks

Elixir has a mix format tool that allows us to automatically format our code. However, it’s not uncommon to forget to format our code before committing, so we can include it in our script.

Additionally, we can include Credo, a static analysis tool that allows us to check the quality of our code. It has many rules that we can configure or disable, and we can even create our own rules. Although not all rules are enabled by default, so it’s worth thoroughly reviewing the documentation.

Credo has a strict mode that we can activate with the --strict option, allowing us to see all rules regardless of their priority.

defmodule Mix.Tasks.CI do
  @moduledoc """
  Run all the checks and tests for the project
  """
  use Mix.Task

  @preferred_cli_env :test

  @impl Mix.Task
  def run(_args) do
    Mix.Task.run("deps.get")

    Mix.Task.run("cmd", ["MIX_ENV=dev", "mix hex.audit"])

    Mix.Task.run("deps.unlock", ["--check-unused"])

    Mix.Task.run("deps.audit", ["--format", "human"])

    Mix.Task.run("compile", ["--warnings-as-errors"])

    Mix.Task.run("format", ["--check-formatted"])

    Mix.Task.run("credo", ["--strict"])
  end
end

Type Checks

Elixir is a dynamically typed language, so while we have options like type annotations and type specifications, these are not checked by the compiler. However, we can use dialyxir or one of its alternatives² to check the typing of our code.

defmodule Mix.Tasks.CI do
  @moduledoc """
  Run all the checks and tests for the project
  """
  use Mix.Task

  @preferred_cli_env :test

  @impl Mix.Task
  def run(_args) do
    Mix.Task.run("deps.get")

    Mix.Task.run("cmd", ["MIX_ENV=dev", "mix hex.audit"])

    Mix.Task.run("deps.unlock", ["--check-unused"])

    Mix.Task.run("deps.audit", ["--format", "human"])

    Mix.Task.run("compile", ["--warnings-as-errors"])

    Mix.Task.run("format", ["--check-formatted"])

    Mix.Task.run("credo", ["--strict"])

    Mix.Task.run("dialyzer")
  end
end

Verificaciones de migraciones

The excellent_migrations library allows us to review migrations for potentially problematic operations. I think it’s a very useful tool to avoid problems in production.

Another useful check we can add is that migrations are reversible. As Saša Jurić3 mentions³

defmodule Mix.Tasks.CI do
  @moduledoc """
  Run all the checks and tests for the project
  """
  use Mix.Task

  @preferred_cli_env :test

  @impl Mix.Task
  def run(_args) do
    Mix.Task.run("deps.get")

    Mix.Task.run("cmd", ["MIX_ENV=dev", "mix hex.audit"])

    Mix.Task.run("deps.unlock", ["--check-unused"])

    Mix.Task.run("deps.audit", ["--format", "human"])

    Mix.Task.run("compile", ["--warnings-as-errors"])

    Mix.Task.run("format", ["--check-formatted"])

    Mix.Task.run("credo", ["--strict"])

    Mix.Task.run("dialyzer")

    Mix.Task.run("ecto.create")

    Mix.Task.run("excellent_migrations.migrate")

    Mix.Task.run("ecto.rollback", ["--all"])
  end
end

Test Checks

After the previous checks, we can now run our tests. Additionally, we can include the --cover option to check our test coverage and the --warnings-as-errors option to treat warnings as errors.

defmodule Mix.Tasks.CI do
  @moduledoc """
  Run all the checks and tests for the project
  """
  use Mix.Task

  @preferred_cli_env :test

  require Logger

  @impl Mix.Task
  def run(_args) do
    Mix.Task.run("deps.get")

    Mix.Task.run("cmd", ["MIX_ENV=dev", "mix hex.audit"])

    Mix.Task.run("deps.unlock", ["--check-unused"])

    Mix.Task.run("deps.audit", ["--format", "human"])

    Mix.Task.run("compile", ["--warnings-as-errors"])

    Mix.Task.run("format", ["--check-formatted"])

    Mix.Task.run("credo", ["--strict"])

    Mix.Task.run("dialyzer")

    Mix.Task.run("ecto.create")

    Mix.Task.run("excellent_migrations.migrate")

    Mix.Task.run("ecto.rollback", ["--all"])

    Mix.Task.run("test", ["--cover", "--warnings-as-errors"])
  end
end

Finally, we can include some messages to provide some information about what is happening.

defmodule Mix.Tasks.CI do
  @moduledoc """
  Run all the checks and tests for the project
  """

  @preferred_cli_env :test

  use Mix.Task

  require Logger

  @impl Mix.Task
  def run(_args) do
    Logger.info("Getting dependencies", ansi_color: :cyan)

    Mix.Task.run("deps.get")

     Logger.info("Checking for retired dependencies", ansi_color: :cyan)

    Mix.Task.run("cmd", ["MIX_ENV=dev", "mix hex.audit"])

    Logger.info("Checking unused dependencies", ansi_color: :cyan)

    Mix.Task.run("deps.unlock", ["--check-unused"])

    Logger.info("Checking for reported vulnerabilities", ansi_color: :cyan)

    Mix.Task.run("deps.audit", ["--format", "human"])

    Logger.info("Compiling code", ansi_color: :cyan)

    Mix.Task.run("compile", ["--warnings-as-errors"])

    Logger.info("checking formatting", ansi_color: :cyan)

    Mix.Task.run("format", ["--check-formatted"])

    Logger.info("Running credo", ansi_color: :cyan)

    Mix.Task.run("credo", ["--strict"])

    Logger.info("Running dialyzer", ansi_color: :cyan)
    
    Mix.Task.run("dialyzer")

    Logger.info("Creating test database", ansi_color: :cyan)

    Mix.Task.run("ecto.create")

    Logger.info("Checking for migrations", ansi_color: :cyan)

    Mix.Task.run("excellent_migrations.migrate")

    Logger.info("Checking for reversibility of migrations", ansi_color: :cyan)

    Mix.Task.run("ecto.rollback", ["--all"])

    Logger.info("Running tests", ansi_color: :cyan)

    Mix.Task.run("test", ["--cover", "--warnings-as-errors"])
end

This task is a reference; depending on the project or team, we can add or remove checks. Another aspect is that it will probably reach a point where running the entire pipeline locally becomes too slow, so we can create a task intended to be run locally with the most important checks and leave the full pipeline for the CI server.

Dependency Checks#

Compiler Checks#

Format Checks#

Type Checks#

Verificaciones de migraciones#

Test Checks#