This article was originally published on Medium.

TL;DR If you don’t want to read how I’ve got to use Nix and general information about it but only focus on its use to setup a development environment, please jump to Using Nix. Even from that part, the reading can take some time as I wanted to share what I’ve learned and found useful accross two months of intensive usage. I had some initial questions I have answered to after reading a great portion of the documentation or asking to people. I wanted to share them with you so you can get into using Nix quickly, still understanding what you are doing.

Two years ago, whenever I needed to use some language or tool which wasn’t available on my machine, I would have used the system’s package manager to install it. It worked on my computer, but when I needed to reproduce the setup elsewhere it was another story. I was still a student then.

I remember a group project in which we were using Node.js. I had a recent version installed on my Mac via Homebrew, while at the university and on other student’s computer based on Ubuntu Linux, it was installed from the default repos. Then, my code using default parameters in functions would run on my machine but not on other’s, since their Node.js version was greatly outdated.

Using this approach can quickly become a nightmare, especially when you try to write a documentation on how to set up the development environment. You end up writing things like: “If you are on macOS please do this, on Debian do that, and Fedora run this other command.” You can verify it works now, but there is no guarantee it will still work in a few months.

asdf

At the end of my studies, while starting my internship, I started to do a lot more software development at home. Mainly because I didn’t have to worry anymore about not doing my homework. This is back then I discovered Elixir and started to build up my workflow, seeking for quality, reproducibility and joy of use. With my Node.js story in mind, I discovered language version managers. I went for asdf as it would handle for me quite a bunch of languages, including Erlang, Elixir, Node.js and Ruby.

If you are not familiar with it, asdf can install exactly the version you want for each language it handles through a plugin system. You can even have multiple versions installed on the same machine and set the wanted one for every project using a .tool-versions file which looks like this:

erlang 21.0
elixir 1.7.3-otp-21

It is simple to use, and you can set up easily your build environment on different machines, running asdf install in a project directory.

Yet, if the setup process is simplified, there are still issues:

after installing asdf, you need to install all the needed plugins. It is only a matter of asdf plugin-install <plugin-name> commands, but it is not automated;
some languages require dependencies to build. Erlang, for instance, needs some packages to be installed before asdf install erlang <version> works. You need a compiler, ncurses, OpenSSL and quite a few more. Setup instructions depends on the system you use;
If your project depends on other tools like, say, fwup for Nerves projects, you still have to provide manual setup instructions.

This is where Nix enters the game.

Nix

Nix defines itself as The Purely Functional Package Manager. Having learned the advantages of Functional Programming while using Elixir and looking at Haskell and other FP languages over the past year, its concept was very tempting to me.

Being purely functional means given an input, you always get the same output. That is, given a version of nixpkgs and a set of packages, you always get the same environment. In fact, packages are named derivations in the Nix jargon: they are functions that takes other derivations—their dependencies—as input and produce a derived result. They are built in isolation, so all dependencies must be explicitely stated. This ensures reproducibility.

Nix stores all the built derivations in the Nix store, usually located at /nix/store. A same package can be present multiple times in the Nix store, at different versions or even at the same version but using different versions of its dependencies. Remember: a built derivation is the product of all its dependencies; if you change something, it is a different product.

To achieve a unique naming for each derivation, a hash is computed from the set of its dependencies. You then get a path like:

/nix/store/k13mm9jqxm2ndlwzsj7zicsq7lpmmjlg-elixir-1.7.3

Unlike other package managers, Nix does not use the conventional /{,usr,usr/local}/{bin,sbin,lib,share,etc} directories. Instead, it uses a lot of symbolic links to create profiles. A profile is a kind of derivation used to setup a user environment. In a profile, you get a standard Unix tree with symbolic links to executables and configuration files stored in other derivation outputs. For instance, ~/.nix-profile/bin/elixir is a symbolic link to:

/nix/store/k13mm9jqxm2ndlwzsj7zicsq7lpmmjlg-elixir-1.7.3/bin/elixir

Actually, ~/.nix-profile is itself a link:

~/.nix-profile -> /nix/var/nix/profiles/per-user/***/profile

Which itself points to profile-56-link wich finally points to somewhere in the Nix store:

profile-56-link -> /nix/store/5yw8dnp9908ia6sdfvx01jzis4l2hni7-user-environment

That is, as I have said above, a profile is a derivation. It derives from a set of packages, that themselves derive from other packages. Depends on becomes in Nix derives from. This is conceptual but you get the idea.

Updating a symbolic link has the interesting property of being an atomic operation. This enables atomic transactions: when performing an upgrade, a new user-environment derivation is built, with a different hash. Then, a new generation is created for the profile—understand: a new symbolic link profile-57-link pointing to the new derivation. Then, and only then, the profile link is updated to point to profile-57-link. You’ve just performed an atomic upgrade. If things went south, you’ve also got atomic rollbacks for free: just update again the profile link to point to profile-56-linkand you are back in the past.

Moreover, only what you asked for is made available in the environment. For instance, Elixir depends on Erlang. Erlang is then installed somewhere in the Nix store and the Elixir installation is aware of it so it can work correctly. But unless you explicitely asked to also install Erlang, only Elixir binaries will be linked in your user environment.

Nix as a declarative configuration manager

Package managers usually work in an imperative way. That is, you ask them to install this, to perform an upgrade or to uninstall that. One really neat feature of Nix is Nix, the language. It is a purely functional domain-specific language that comes with Nix.

The primary use of Nix, the language, is to write derivations. Yet, different applications of Nix also leverage the language to manage packages and configuration declaratively. In NixOS—a special GNU/Linux distribution based on Nix—, all the system configuration and globally installed packages are declared in /etc/nixos/configuration.nix. It can look like:

{ config, pkgs, ... }:

{
  network.hostname = "nixos-test";
  time.timeZone = "Europe/Paris";

  # Omitting a lot of options, this is just a sample.

  environment.systemPackages = with pkgs; [
    curl
    git
    gnupg
    htop
    ...
  ];
}

To change the state of the system, you just have to edit the file, then ask NixOS to switch to the new environment:

# nixos-rebuild switch

It derives a new system environment, switches to it and reloads services as needed. With this kind of configuration, you can easily reproduce your system setup on different machines.

Another example is home-manager. It aims to provide the same kind of declarative configuration as NixOS but at the user level. I personally use it to manage my dotfiles and my set of user-wide-available utilities accross different machines.

Apart system and user environments, Nix can be used to setup a third kind of environment. Let’s talk about it.

Nix as a reproducible environment builder

It is neither practical nor wantable to update your system or user environment each time you need a particular dependency for a given project. First, the given dependency can be required only by a specific project: you do not need to have it available (system|user)-wide. Second, you want your environment to be shareable with other developers. Asking them to update their global environment with this or that requirement is not the best thing to do. This is where Nix shells enter.

A Nix shell is a temporary environment where build inputs of a derivation are made available to the user. Let’s say it more simply with an example: you can create a shell.nix file at the root of your project containing something like:

{ pkgs ? import <nixpkgs> {} }:

with pkgs;

mkShell {
  buildInputs = [ ocaml git ];
}

Then, run the nix-shell command. You are in a Nix shell, with OCaml and Git made available to you. The shell.nix can be committed, thus shared between developers.

Using Nix

To start working on a project using Nix, the first step is to install Nix itself. From the official documentation, all you have to do is running:

$ curl https://nixos.org/nix/install | sh

There are in fact two kinds of installation for Nix:

the single-user installation, where /nix/store is owned by the user installing Nix. This is the simplest way to install Nix if you are the only user on your machine and don’t want to use Nix extensively outside of setting your development environment. It is also the easiest to uninstall as you just have to delete the /nix directory;
the multi-user installation, where /nix/store is owned by root and a nixbld group has write access to it. All Nix operations are then performed by nix-daemon. Different users can use Nix simultaneously and you gain a system environment like on NixOS. In the multi-user installation, builds are performed by special builder users in complete isolation. It is a little bit more complex to manage, but this is the recommended way to install Nix if you plan to use it a lot, even if you are the only user on your machine. It is supported on macOS and all Linux running systemd with SELinux disabled.

As far as I know, the default installer currently choses automatically the type of installation. It could change in the near future to prompting the user for a choice. In the meantime, you can force how to install Nix by doing, for a single-user installation:

$ sh <(curl https://nixos.org/nix/install) --no-daemon

For a multi-user installation it is:

$ sh <(curl https://nixos.org/nix/install) --daemon

Nix being installed on you machine, you can create a shell.nix in your project:

# This defines a function taking `pkgs` as parameter, and uses
# `nixpkgs` by default if no argument is passed to it.
{ pkgs ? import <nixpkgs> {} }:

# This avoids typing `pkgs.` before each package name.
with pkgs;

# Defines a shell.
mkShell {
  # Sets the build inputs, i.e. what will be available in our
  # local environment.
  buildInputs = [ elixir git ];
}

Then, all you have to do is running:

$ nix-shell

Nix will copy or build derivations, then run a shell in which Elixir and Git are available. By default, the Nix shell is a bash. If you are like me and want to keep your good old zsh for your day-to-day environment, there is an interesting plugin for that.

In a standard Nix shell, your system-wide environment is still available. While it is quite useful for day-to-day work, you can easily miss a dependency when first building your environment. If you want to be sure not to miss any dependency and ensure reproducibility, you should run a pure Nix shell, that is a Nix shell where only the inputs explicitely stated in the shell.nix are available:

$ nix-shell --pure

You loose all your environment, all your aliases, all your usually available programs. You are in a standardised bash that will be the same on every machine where you run a pure shell. It is not a comfortable place to work, but it is really comfortable when your application builds in such a shell. It means any other developer or the future you will be able to setup the same environment and build the application as expected. But for your comfort, you can use a standard Nix shell most of the time.

In addition to Nix shells, you may sometimes want to make a tool globally available in your user environment. If you don’t use home-manager to manage it in a declarative way, you always can do it in an imperative way:

$ nix-env -i <package>

If you are looking for a better user experience in installing packages, nox is the way to go:

$ nix-env -i nox

Then just call nox with some search string:

$ nox gcc
1 avr-gcc-8.2.0 (nixpkgs.avrgcc)
    GNU Compiler Collection, version 8.2.0 for AVR microcontrollers
2 gcc-wrapper-7.3.0 (nixpkgs.gcc)
    GNU Compiler Collection, version 7.3.0 (wrapper script)
3 gcc-arm-embedded-6-2017-q2-update (nixpkgs.gcc-arm-embedded)
    Pre-built GNU toolchain from ARM Cortex-M & Cortex-R processors (Cortex-M0/M0+/M3/M4/M7, Cortex-R4/R5/R7/R8)
4 gcc-7.3.0 (nixpkgs.gcc-unwrapped)
    GNU Compiler Collection, version 7.3.0
5 gcc-wrapper-4.8.5 (nixpkgs.gcc48)
    GNU Compiler Collection, version 4.8.5 (wrapper script)
6 gcc-wrapper-5.5.0 (nixpkgs.gcc5)
    GNU Compiler Collection, version 5.5.0 (wrapper script)
7 gcc-wrapper-6.4.0 (nixpkgs.gcc6)
    GNU Compiler Collection, version 6.4.0 (wrapper script)
8 stdenv-darwin (nixpkgs.gcc7Stdenv)
    The default build environment for Unix packages in Nixpkgs
9 gcc-wrapper-8.2.0 (nixpkgs.gcc8)
    GNU Compiler Collection, version 8.2.0 (wrapper script)
...
Packages to install:

It will print you different alternatives so you can choose which one to install directly by entering its number. It is also a good way to quickly search a package name before adding it to a shell.nix.

In the example above, you can see several packages are available for gcc. The real package name to include in a shell.nix is the one in parentheses. If you’ve done with pkgs;, pkgs being by default nixpkgs in the shell.nix examples I’ve shown you before, you can then omit nixpkgs. from package names.

Keeping nixpkgs up to date

When you first install Nix, it also automatically installs the nixpkgs channel for you. nixpkgs is the main channel in the Nix community, and efforts are concentrated to it. You can event participate if you want as it is as easy as opening a pull-request on GitHub. You are likely willing to update it sometimes to get fresh versions of your packages. To do so, run:

$ nix-channel --update

Don’t forget to run the command as the user managing nixpkgs. If you’ve done a single-user installation, it is your standard user. In a multi-user installation, root is responsible for managing this channel, so you must run the command as root.

If you have installed derivations via nix-env -i or nox, once you have updated nixpkgs, you should run nix-env -u to rebuild them—understand: update them—on top of the last nixpkgs. If you have installed derivations on NixOS at the system level, or at the user level using home-manager, you have to run nixos-rebuild switch or home-manager switch to rebuild your environment.

When you update nixpkgs, remember that you are changing an input. The environments you build are now using derivations from this new version. It is not fully reproducible since you can’t now which version of nixpkgs another user have. In most cases, this is not an issue. Packages that could cause incompatibilities between versions, like languages or libraries, often provide different packages ready to use. For instance, while elixir provides a given version of Elixir—theorically the last one—built on a given version of Erlang that is not fixed and will evolve with nixpkgs updates, there is beam.packages.erlangR21.elixir_1_7 which provides you the latest Elixir 1.7 on the latest Erlang 21. At the time I am writing this article, it is namely Elixir 1.7.3 on Erlang 21.0. Yes, a nixpkgs update could bring you to Elixir 1.7.4 on Erlang 21.1, say, but they are compatible versions. Using this derivation, you will never end up with Elixir 1.8 or 2.0. However, if you really want to set nixpkgs to a given version to provide a fully-reproducible environment, this is possible. All you have to do is setting the pkgs variable differently in your shell.nix:

let
  pkgs = import (fetchTarball {
    url = https://github.com/NixOS/nixpkgs/archive/<rev>.tar.gz;
  }) {};

  # I’m also showing here you can define any variable.
  elixir = pkgs.beam.packages.erlangR20.elixir_1_6;
in

with pkgs;

mkShell {
  # `elixir` is not the latest Elixir anymore, but the latest
  # version of the 1.6 branch built on the latest Erlang 20 at the
  # time when `nixpkgs` was at <commit-hash>, as stated above.
  # Same for the version of Git.
  buildInputs = [ elixir git ];
}

Collecting garbage

As we have seen before, Nix stores the derivations outputs in /nix/store. It then builds environments linking to files in the store. But what happens when you update, uninstall a program or when you exit a Nix shell? Nothing. New environments are built, but nothing is removed from the store. After some time of use, the Nix store can grow a bit too much for your taste. While it is good to keep a cache between two nix-shell calls to avoid fetching again all dependencies, it can be useful to clean a bit the store sometimes.

Nix has a built-in garbage collector that looks for unreferenced derivations—derivation that are not part of an environment or a running Nix shell. You can call it by running:

$ nix-collect-garbage

Doing this has nevertheless a limitation: it does not clean old generations of profiles. Remember: when doing changes to environments, you don’t mutate its state: you create a new generation and switch to it. This is what makes rollbacks possible. To reclame space on disk, old generation need to be removed. You can ask the garbage collector to delete them before to collect garbage:

$ nix-collect-garbage -d

However, it is a good pratice do keep old generations for some time, just in case. You can tell the garbage collector to delete old generation only if they are older than a given amount of time. To keep old generation 30 day, for instance, you would do:

$ nix-collect-garbage --delete-older-than 30d

Garbage collection has one downside though: as shell environments are not symlinked in the GC roots like profiles, the garbage collector systematically deletes them. In the next section about direnv, we’ll see there is a way to persist them and avoid this kind of issue.

Apart from collecting garbage, there is another way to optimise the Nix store in term of disk space. When some derivation gets updated, all derivations depending on it have to be rebuilt since one of their inputs has changed. Many of their files—if not all—remain the same, so it is possible to reclaim space by hard-linking them:

$ nix-store --optimise -v

You can also make Nix auto-optimise its store when writing new files to it:

$ mkdir -p ~/.config/nix
$ echo "auto-optimise-store = true" >> ~/.config/nix/nix.conf

Please use this option with caution though. If it works really well on a single-user installation, I’ve seen race conditions on /nix/store/.links files creation when using several builder simultaneously. I prefer to run it by hand then. With a good alias like nso it’s quick.

direnv

While Nix provides a much richer developer experience, asdf has a very comfortable feature: you don’t have to run a command each time you want to get your environment configured. The right versions of your languages are automatically picked up depending on which directory you are in. As Nix lacks of this feature—is not its role anyway—let me introduce direnv.

direnv is a tool for automatically switching between environments, based on your current directory. To cite its documentation:

Before each prompt, direnv checks for the existence of a “.envrc” file in the current and parent directories. If the file exists (and is authorized), it is loaded into a bash sub-shell and all exported variables are then captured by direnv and then made available to the current shell.

One really neat thing about direnv is its awareness of Nix. You can just put use nix in a .envrc and direnv knows how to update you current shell to behave like a Nix shell. Let’s start with installing direnv:

$ nix-env -i direnv

For direnv to work, you then need to hook it into your shell. Edit your ~/.<shell>rc and put:

eval "$(direnv hook <shell>)"

Replace <shell> with zsh, bash or any other supported shell.

In any project directory, you can then create a .envrc by running:

$ direnv edit .

As your environment configuration is already handled by the shell.nix, simply tell direnv to use Nix by writing:

use nix

After saving the file and quitting your editor, direnv should automatically update your shell environment whenever you enter or exit the project tree.

If you change the .envrc contents or clone a project where a .envrc is already present, direnv will ask you to allow it. It is for security purpose: you should always check what a .envrc contents do. To allow it:

$ direnv allow

This is great, but you can however notice two things:

the environment can take some time to build each time you run a shell in a project directory;
the environment is not (yet) persistent: garbage collection would delete it from the Nix store.

In the direnv wiki page about Nix, you can find a script that builds a persistent and cached shell, thus avoiding both the two remarks I have written above. All you have to do to use it is adding it at the top of your .envrc files, like in this one. This script creates a .direnv directory at your project root, containing symbolic links to derivation outputs in the Nix store. It also links them in the GC roots, thus avoiding the shell environment to be garbage-collected as long as the .direnv directory exists. For this to work as expected, you must tell Nix to keep derivation outputs. Put this in your ~/.config/nix/nix.conf:

keep-outputs = true
keep-derivations = true

If you are on a Mac, this script does not work out of the box: it indeed relies on the GNU version of readlink. To make it work, you can install coreutils in your user profile via Nix:

$ nix-env -i coreutils

Please note that coreutils replaces some of the standard shell commands. While GNU versions are good, they may lack of some refinements like ls being able to show extended attributes. If you want to keep using your system ls while installing coreutils, you can achive this by aliasing ls to /usr/bin/ls.

A few words about comfort

TL;DR I have written some aliases for Nix and direnv. You can also add a visual indicator to your prompt by checking if $IN_NIX_SHELL is set.

As developers, we spend most of our day doing tasks on a computer. While some of these tasks can be long and require a great focusing, some of them like using tools as Git, Nix and direnv should be really quick and effortless. When in your shell, you shouldn’t have to ask if you are in a Git repository, on each branch, or wether you are in a Nix shell. Your shell should provide you information without you asking for. In the same way, interacting with these tools should be really quick. You shouldn’t end up typing things like:

$ git status
$ git checkout master
$ nix-shell --pure

These commands are not long, but they are too long to type—even with completion—for what they achieve. On my machines, I would do instead:

$ gcm
$ nisp

Note that I have omitted an alias for git status. This is because my prompt already shows me on which branch I am, and if there are changes or something to push or pull. Also, if I am in a Nix shell, my prompt is blue instead of green.

The purpose of this is not anymore about sharing a development environment with others. It is all about personal comfort. The common base is made of full commands: they can be run everywhere. You can always check the status of a repository with git status and run a pure Nix shell with nix-shell --pure. But on top of this common base, you should customise your environment so that your computer really becomes an ergonomic tool. When I take photos I love to think about my cameras like extensions of my arm, some of them being truly ergonomic—hello Fuji GW690 II! When I am on a computer, I like to feel the same way with my keyboard. Shortcuts and aliases, thanks to the muscular memory, are really helpful to keep you in the flow.

This being said, if you want to get some inspiration, I have written some aliases for Nix and direnv. The direnv ones even contains some aliases dedicated to the script for persistent cached shells. The idea is to help cleaning old local environments by doing dcl or at the end of a project, cleaning the environment by doing dar && ngco. dar is for “direnv archive”, that is, it denies the current .envrc and deletes the .direnv directory. Running ngco then deletes the environment from the Nix store, as the .direnv directory is gone. This is all about personal taste, customise as you prefer.

Last but not least: starting to use Nix and direnv in an existing project is as easy as running nixify, which is defined here. It simply creates a .envrc and a shell.nix if they don’t exist.

Conclusion

Migrating from asdf to Nix has been an interesting journey. They both share some ideas when it comes to managing the build environment. However, Nix is several orders of magnitude more powerful than asdf.

If we only focus on Nix shells, Nix can not only manage the languages you use, but also every other tool you would need for your development process. You can use pure shells to check if your environment is complete, avoiding to use tools available globally on your machine. Also, if a tool or version is not available in nixpkgs, you can easily write your own derivations.

When it comes to environment switching, asdf seems to be a winner in the first run, but once you have set up direnv you quickly forget about that.

Generally speaking, Nix is a more broader tool: you can use it to install software on your machine without messing your system, manage your configuration or automate your builds. If you are interested in Nix, you should also learn about these aspects.

I still have to figure about using Nix as part of my Rust workflow. I have especially questions about the RLS setup and cargo commands like cargo watch. Once it is done, I may come with an article about it. I also plan to write a much shorter article focused on Elixir projects—including Phoenix and Nerves—as I have managed to build and run them in pure Nix shells.

I hope this article has been useful to you in your understanding of Nix. If something is not clear, please tell me in a response: I will update the article accordingly.

Edit: if you are an Elixir developer, you can read my article about Using Nix in Elixir projects.

Contents

About using Nix in my development workflow