Building a NixOS container from macOS

About

On my way to create a new NixOS module to run RhodeCode Enterprise CE I did arrive at a point where I wanted to be able to just test out the module quickly and build a machine image.

After studying the NixOS manual I got the idea to just create a test NixOS configuration which configures a machine as a container and uses the one module I am about to develop.

This post describes the details of how I got this working, so that you can apply the same technique for you own NixOS endeavors.

Preparation: Remote building

For my case I am working usually on a macbook and I dislike to work inside of a virtual machine, so I used the remote building feature of Nix to be able to more or less ignore that fact that I am building things inside of a virtual NixOS machine. If you work directly on a Linux based system, then this is not needed.

The details of the remote building setup are described in a different post.

Minimal configuration

Starting with a minimal configuration allows to verify first if the basics are working. Here is what is needed to start the build:

{ config, pkgs, ... }:

{
  boot.isContainer = true;
}

Test the build

Building a machine is straight forward once you know which parameters to tweak.

-A system: Since I am on macOS I have to set the parameter system to a value like x86_64-linux. This is not needed if you are on a Linux box already.
<nixpkgs/nixos>: This refers to the folder nixos inside of my nixpkgs channel.
-I nixos-config=./test-machine.nix: Adds nixos-config into the include path. This is the mechanism by which Nix will find the configuration file which it has to use.
--show-trace: I left it here. Things did blow up initially and I did use this parameter to see a proper trace of the error's cause.

Combining these fragments into a command line will result in the following result:

nix-build --argstr system x86_64-linux \
    '<nixpkgs/nixos>' \
    -A system \
    -I nixos-config=./test-machine.nix \
    --show-trace

Inspecting the build result

On a successful result, there should be the regular symlink called result. It will point to the directory which contains all the bits and pieces which together form the container.

Inspecting the result of the build should show something like the following:

johannes@dev-johbo: ~/wo/nix-rhodecode/nixos master
$ ls -l result/
total 48
-r-xr-xr-x 1 johannes staff 12940 Jan  1  1970 activate
dr-xr-xr-x 3 johannes staff   102 Jan  1  1970 bin
-r--r--r-- 1 johannes staff     0 Jan  1  1970 configuration-name
lrwxr-xr-x 1 johannes staff    51 Jan  1  1970 etc -> /nix/store/akm6m0l54llibabzqfqkx1va8i8m5rhm-etc/etc
-r--r--r-- 1 johannes staff     0 Jan  1  1970 extra-dependencies
dr-xr-xr-x 2 johannes staff    68 Jan  1  1970 fine-tune
-r-xr-xr-x 1 johannes staff  4952 Jan  1  1970 init
-r--r--r-- 1 johannes staff     9 Jan  1  1970 init-interface-version
-r--r--r-- 1 johannes staff    25 Jan  1  1970 nixos-version
lrwxr-xr-x 1 johannes staff    55 Jan  1  1970 sw -> /nix/store/hwk74rwnlmswnkidj3yhfmys3vav89wd-system-path
-r--r--r-- 1 johannes staff    12 Jan  1  1970 system
lrwxr-xr-x 1 johannes staff    55 Jan  1  1970 systemd -> /nix/store/dbay1civ36b8hkr26wsxh586r331wy94-systemd-234

It's quite interesting to poke around in these generated bits and pieces, they contain all what's needed to bring a new machine build up.

How to continue from this state

To test a module or some modules, I started by adding the import as follows:

{ config, pkgs, ... }:

{
  # Makes testing easier
  boot.isContainer = true;

  imports = [
    ./vcsserver.nix
  ];

}

When I do build this again, it will pull in my new module under development and I can inspect the results via the update symlink result after each build without actually having to deploy the container yet.