Published 2021-08-02
on Farid Zakaria’s Blog
—
Permalink
We are currently using nix-shell to create a reproducible environment for our developers and CI infrastructure. Can we minimize the dependency closure to make our CI jobs faster?
If you are familiar with Nix, undoubtedly you have come across nix-shell. Originally designed to help debug a failing deriviation by entering the user into a shell, it has quickly become a beloved way to create reproducible developer environments via mkShell.
mkShell is a special kind of derivation that is only useful when using it combined with nix-shell. It will in fact fail to instantiate when invoked with nix-build.
mkShell however comes with a predefined set of dependencies via the standard build environment (stdenv).
Let’s look at an empty mkShell and explore the transitive closure.
let
nixpkgs = import (builtins.fetchTarball {
name = "nixos-unstable-2020-09-24";
url =
"https://github.com/nixos/nixpkgs/archive/8ecc61c91a596df7d3293603a9c2384190c1b89a.tar.gz";
sha256 = "0vhajylsmipjkm5v44n2h0pglcmpvk4mkyvxp7qfvkjdxw21dyml";
}) { };
in nixpkgs.mkShell {
name = "basic-shell";
buildInputs = [ ];
shellHook = ''
'';
}
We can even visualize the dependency closure using graphviz.
nix-store --query --graph $(nix-build --no-out-link -A inputDerivation basic-shell.nix)
There is quite a lot there including GCC and we see the total closure size is ~270MiB.
❯ nix path-info --closure-size --human-readable $(nix-build --no-out-link -A inputDerivation basic-shell.nix)
/nix/store/zg33vr1apaq341c9bdfbhlwbp8l22qm6-basic-shell 268.3M
Let’s see how much better we can do with mkShellNoCC.
let
nixpkgs = import (builtins.fetchTarball {
name = "nixos-unstable-2020-09-24";
url =
"https://github.com/nixos/nixpkgs/archive/8ecc61c91a596df7d3293603a9c2384190c1b89a.tar.gz";
sha256 = "0vhajylsmipjkm5v44n2h0pglcmpvk4mkyvxp7qfvkjdxw21dyml";
}) { };
in nixpkgs.mkShellNoCC {
name = "shell-no-cc";
buildInputs = [ ];
shellHook = ''
'';
}
Looks like we are down to ~53MiB
❯ nix path-info --closure-size --human-readable $(nix-build --no-out-link -A inputDerivation shell-no-cc.nix).
/nix/store/7afg0p2kyc8qb5a0nv7mlvpf1mbpqkdx-shell-no-cc 53.5M
🕵️ Can we do better?
Thank you to siraben via Matrix to help share some of the following derivation.
Let’s str
