'Why you should never ever use NixOS': a rebuttal

I first heard of NixOS around two years ago, when a friend decided to run it on her personal VPS instances. As with most new things, I immediately hated it. I didn’t like that they were attempting to generate config files for different services in a variety of languages using the nix language. I didn’t like the idea of an OS pretending to be a standard Linux distribution by strategically taping together symlinks and environment variables. I dislike the functional programming paradigm (if reading that made you feel the urge to evangelize, you’re part of the problem). I didn’t like an OS having to garbage collect unused packages. Most of all, I didn’t like the whole patchelf ld replacement voodoo. I could think of so many ways in which NixOS could break, given all the weird things it was doing.

It took almost two years of abuse from Puppet and constant shilling by this friend for me to give nix a fair shake. Testing it out on servers, I was pleased to see that in practice, the things I was worried about didn’t really matter. If a service you want to deploy already has a module, odds are that it exposes all configurations you would want to change as nix options. In modules where this isn’t available, you can just pass it a configuration file directly. I even went out of my way to come up with weird fringe use-cases like building all packages from source with -O3 and -march native CFLAGS, or overriding package versions to use the repo HEAD directly from git. One of the things I liked a lot about the declarative nature of NixOS was that the removal of a service, user, package, or anything else added through the nix configuration file is as simple as just removing the line from the nix config file. On puppet, installing a package requires adding an ensure=>present statement, and removal would require an ensure=>absent.

After spending some time on it, I liked it enough to make it my primary OS, running my laptop, desktop, and servers. Using NixoOS unstable as my daily driver made a lot of sense, as it was rolling like Arch, and allowed me to build packages with custom CFlags easily like Gentoo. The idea of replacing (Debian+)puppet with NixOS stable in prod was also being discussed. It was at this point that many of my colleagues brought this blog post to my attention. It is the second result on google for nixos devops, right after the NixOps repository.

The hit piece

The post was written by a cloud infrastructure architect with 15 years of experience, so I went over it thoroughly to understand exactly what kind of problems we might run into when using NixOS for our infrastructure. In the end, I concluded that this person either completely misunderstood NixOS and tried to use it like Debian/Docker (yes, I know, that’s a weird grouping), or decided they hated NixOS and tried to come up with rationalizations to feel better. Since my colleagues keep referencing this factually incorrect and misleading article, I’ll go over every point in it and try to give a different perspective. here we go…

Determinism

This one made me chuckle a little, as the author’s attempt to list a positive aspect (according to him) turns into a gripe literally (as in literally) one sentence in.

The system state NixOS generates for a given version of nix channels and a nix configuration is pretty much always the same. Now, this isn’t to say you can’t craft a nix package in such a way that every build generates a different output, say, by using a random number generator or something, but there are no such packages (as far as I know) in the Nixpkgs repo, and making one yourself with the intent of doing this is about the same as, say, making a Debian package that runs shutdown -h now during the dpkg install stage.

The next qualm of the author is that packages aren’t OS agnostic. This is true, in the sense that the way you build, say, wget on OSX is different from how you build it on Linux. They are entirely different operating systems with different ABIs and syscall interfaces. But from an end-user or sysadmin perspective, in both cases, you do environment.systemPackages = [pkgs.wget] to have wget in PATH.

The author then goes on to list fetchgit/fetchFromGithub as examples of this supposed false determinism. fetchgit and fetchFromGithub are fetcher methods to pull source code from git repos and GitHub respectively. You can read more about them here. If you’re writing your own packages, you can use any of these fetchers to get the source code for the package from a variety of sources. He then links to a series of discussions about using fetchFromGithub instead of fetchgit for different packages. I don’t understand what he’s upset about here, this is about the same as if a Debian package maintainer decided to shallow clone a repo instead of doing a full clone when building a package. What functions they use internally for fetching the source code shouldn’t matter for anyone pulling the result of the build. It doesn’t even matter if you use fetchgit or fetchFromGithub in your packages too. A GitHub repo is (you might want to sit down, this is a shocking revelation) a git repo. You can use the method for fetching a git repo by providing it the git URL and revision, or you can use the more convenient fetchFromGitHub method by providing a username, repo name, and revision. Someone decided to send in a pull request converting a lot of fetchgit calls to fetchFromGithub for readability (and possibly because fetchFromGithub does shallow clones by default, I can’t be bothered to check). This change got merged into the repo and was included in the next stable release (18.03 I think) of NixOS. I’ll be honest, I have no idea what the author’s problem with this is. It makes no difference to nix’s promise of deterministic builds. I can only assume the author doesn’t understand what determinism means.

Small community

This is true. I was very active on Ubuntu forums back when Ubuntu was a thing. After starting with Solaris (ew…), then jumping from Fedora to Mandriva, I had finally settled on Ubuntu because of how active and helpful the community was. In retrospect, less handholding and more getting told to RTFM might have helped me understand things faster, but it was nice to have a reliable place to get help from.

With NixOS, it has been significantly more difficult to get any help. My forum posts and cries for help on IRC often go unanswered. (Although to be fair, every single one of those posts has been about some weird fringe use case that most people would not have run into.)

Hiring people with experience in nix to manage your existing infrastructure would also be harder than finding people with experience in Terraform, ASnsible, or Puppet.

Documentation

Maybe the documentation was bad for 18.09, I haven’t checked. The current documentation was good enough for me to reach a reasonable level of competence within a month of hands-on experimentation. That being said, the author does something very disingenuous here and tries to scare you by showing what a nix derivation looks like. Specifically, he shows the Nginx derivation.

If you’re setting up an Nginx server on NixOS, it is straightforward and convenient. The NixOS Nginx module even supports enabling SSL with ACME by simply adding enableACME=true; to your conf. This wiki page shows a variety of Nginx configuration examples that demonstrate how easy it is to deploy and manage NGINX on NixOS. In my opinion, NGINX is easier to manage through nix, compared to manual configuration management, or solutions like puppet/ansible.

What the author has done in this section is tantamount to showing you the PKGBUILD for Nginx and telling you Arch is complicated. Let me demonstrate, using Nginx as an example.

Exhibit A: nginx on debian

Common syntax of describing a debian package looks like this (nginx derivation example):

#!/usr/bin/make -f

CFLAGS = `dpkg-buildflags --get CFLAGS`
CFLAGS += -Wall -DFORTIFY_SOURCE=2 -fstack-protector
CFLAGS += `dpkg-buildflags --get CPPFLAGS`
LDFLAGS = `dpkg-buildflags --get LDFLAGS`
# export necessary for (hardening) flags for perl
# (src/http/modules/perl/Makefile.PL).
export CFLAGS LDFLAGS

FLAVOURS:=full light extras naxsi

BUILDDIR_full = $(CURDIR)/debian/build-full
BUILDDIR_light = $(CURDIR)/debian/build-light
BUILDDIR_extras = $(CURDIR)/debian/build-extras
BUILDDIR_naxsi = $(CURDIR)/debian/build-naxsi
MODULESDIR = $(CURDIR)/debian/modules
BASEDIR = $(CURDIR)

DEB_BUILD_ARCH ?=$(shell dpkg-architecture -qDEB_BUILD_ARCH)
ifeq ($(DEB_BUILD_ARCH),sparc)
	CONFIGURE_OPTS = --with-cc-opt="-m32 -mcpu=ultrasparc"
endif


ifneq (,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
	NUMJOBS = $(patsubst parallel=%,%,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
	ifeq (${NUMJOBS}, 0)
		NUMJOBS = 1
	endif
else
	NUMJOBS = 1
endif

config.sub:
	dh_testdir
ifneq "$(wildcard /usr/share/misc/config.sub)" ""
		cp -f /usr/share/misc/config.sub config.sub
endif

config.guess:
	dh_testdir
ifneq "$(wildcard /usr/share/misc/config.guess)" ""
		cp -f /usr/share/misc/config.guess config.guess
endif

config.env.%:
	dh_testdir
	mkdir -p $(BUILDDIR_$*)
	cp -Pa $(CURDIR)/auto $(BUILDDIR_$*)/
	cp -Pa $(CURDIR)/conf $(BUILDDIR_$*)/
	cp -Pa $(CURDIR)/configure $(BUILDDIR_$*)/
	cp -Pa $(CURDIR)/contrib $(BUILDDIR_$*)/
	cp -Pa $(CURDIR)/src $(BUILDDIR_$*)/
	cp -Pa $(CURDIR)/man $(BUILDDIR_$*)/

config.status.full: config.env.full config.sub config.guess
	cd $(BUILDDIR_full) && CFLAGS="$(CFLAGS)" CORE_LINK="$(LDFLAGS)" ./configure  \
	    --prefix=/usr/share/nginx \
	    --conf-path=/etc/nginx/nginx.conf \
	    --error-log-path=/var/log/nginx/error.log \
	    --http-client-body-temp-path=/var/lib/nginx/body \
	    --http-fastcgi-temp-path=/var/lib/nginx/fastcgi \
	    --http-log-path=/var/log/nginx/access.log \
	    --http-proxy-temp-path=/var/lib/nginx/proxy \
	    --http-scgi-temp-path=/var/lib/nginx/scgi \
	    --http-uwsgi-temp-path=/var/lib/nginx/uwsgi \
	    --lock-path=/var/lock/nginx.lock \
	    --pid-path=/run/nginx.pid \
	    --with-pcre-jit \
	    --with-debug \
	    --with-http_addition_module \
	    --with-http_dav_module \
	    --with-http_gzip_static_module \
	    --with-http_realip_module \
	    --with-http_stub_status_module \
	    --with-http_ssl_module \
	    --with-http_sub_module \
	    --with-http_xslt_module \
	    --with-ipv6 \
	    --with-mail \
	    --with-mail_ssl_module \
	    --add-module=$(MODULESDIR)/nginx-upload-module \
	    --add-module=$(MODULESDIR)/nginx-auth-pam \
	    --add-module=$(MODULESDIR)/nginx-dav-ext-module \
	    --add-module=$(MODULESDIR)/nginx-echo \
	    --add-module=$(MODULESDIR)/nginx-upstream-fair \
	    --add-module=$(MODULESDIR)/ngx_http_substitutions_filter_module \
            $(CONFIGURE_OPTS) >$@
	touch $@

config.status.light: config.env.light config.sub config.guess
	cd $(BUILDDIR_light) && CFLAGS="$(CFLAGS)" CORE_LINK="$(LDFLAGS)" ./configure  \
	    --prefix=/usr/share/nginx \
	    --conf-path=/etc/nginx/nginx.conf \
	    --error-log-path=/var/log/nginx/error.log \
	    --http-client-body-temp-path=/var/lib/nginx/body \
	    --http-fastcgi-temp-path=/var/lib/nginx/fastcgi \
	    --http-log-path=/var/log/nginx/access.log \
	    --http-proxy-temp-path=/var/lib/nginx/proxy \
	    --lock-path=/var/lock/nginx.lock \
	    --pid-path=/run/nginx.pid \
	    --with-pcre-jit \
	    --with-http_gzip_static_module \
	    --with-http_ssl_module \
	    --with-ipv6 \
	    --without-http_browser_module \
	    --without-http_geo_module \
	    --without-http_limit_req_module \
	    --without-http_limit_zone_module \
	    --without-http_memcached_module \
	    --without-http_referer_module \
	    --without-http_scgi_module \
	    --without-http_split_clients_module \
	    --with-http_stub_status_module \
	    --without-http_ssi_module \
	    --without-http_userid_module \
	    --without-http_uwsgi_module \
	    --add-module=$(MODULESDIR)/nginx-echo \
            $(CONFIGURE_OPTS) >$@
	touch $@

config.status.extras: config.env.extras config.sub config.guess
	cd $(BUILDDIR_extras) && CFLAGS="$(CFLAGS)" CORE_LINK="$(LDFLAGS)" ./configure  \
	    --prefix=/usr/share/nginx \
	    --conf-path=/etc/nginx/nginx.conf \
	    --error-log-path=/var/log/nginx/error.log \
	    --http-client-body-temp-path=/var/lib/nginx/body \
	    --http-fastcgi-temp-path=/var/lib/nginx/fastcgi \
	    --http-log-path=/var/log/nginx/access.log \
	    --http-proxy-temp-path=/var/lib/nginx/proxy \
	    --http-scgi-temp-path=/var/lib/nginx/scgi \
	    --http-uwsgi-temp-path=/var/lib/nginx/uwsgi \
	    --lock-path=/var/lock/nginx.lock \
	    --pid-path=/run/nginx.pid \
	    --with-pcre-jit \
	    --with-debug \
	    --with-http_addition_module \
	    --with-http_dav_module \
	    --with-http_flv_module \
	    --with-http_gzip_static_module \
	    --with-http_mp4_module \
	    --with-http_perl_module \
	    --with-http_random_index_module \
	    --with-http_realip_module \
	    --with-http_secure_link_module \
	    --with-http_spdy_module \
	    --with-http_stub_status_module \
	    --with-http_ssl_module \
	    --with-http_sub_module \
	    --with-http_xslt_module \
	    --with-ipv6 \
	    --with-mail \
	    --with-mail_ssl_module \
	    --add-module=$(MODULESDIR)/headers-more-nginx-module \
	    --add-module=$(MODULESDIR)/nginx-auth-pam \
	    --add-module=$(MODULESDIR)/nginx-cache-purge \
	    --add-module=$(MODULESDIR)/nginx-dav-ext-module \
	    --add-module=$(MODULESDIR)/nginx-development-kit \
	    --add-module=$(MODULESDIR)/nginx-echo \
	    --add-module=$(MODULESDIR)/ngx-fancyindex \
	    --add-module=$(MODULESDIR)/nginx-http-push \
	    --add-module=$(MODULESDIR)/nginx-upload-progress \
	    --add-module=$(MODULESDIR)/nginx-upstream-fair \
	    --add-module=$(MODULESDIR)/ngx_http_substitutions_filter_module \
            $(CONFIGURE_OPTS) >$@
	touch $@

config.status.naxsi: config.env.naxsi config.sub config.guess
	cd $(BUILDDIR_naxsi) && CFLAGS="$(CFLAGS)" CORE_LINK="$(LDFLAGS)" ./configure  \
	    --prefix=/usr/share/nginx \
	    --conf-path=/etc/nginx/nginx.conf \
	    --error-log-path=/var/log/nginx/error.log \
	    --http-client-body-temp-path=/var/lib/nginx/body \
	    --http-fastcgi-temp-path=/var/lib/nginx/fastcgi \
	    --http-log-path=/var/log/nginx/access.log \
	    --http-proxy-temp-path=/var/lib/nginx/proxy \
	    --http-scgi-temp-path=/var/lib/nginx/scgi \
	    --http-uwsgi-temp-path=/var/lib/nginx/uwsgi \
	    --lock-path=/var/lock/nginx.lock \
	    --pid-path=/run/nginx.pid \
	    --with-pcre-jit \
	    --with-http_ssl_module \
	    --without-mail_pop3_module \
	    --without-mail_smtp_module \
	    --without-mail_imap_module \
	    --without-http_uwsgi_module \
	    --without-http_scgi_module \
	    --with-ipv6 \
	    --with-http_stub_status_module \
	    --with-http_realip_module \
	    --add-module=$(MODULESDIR)/naxsi/naxsi_src \
	    --add-module=$(MODULESDIR)/nginx-cache-purge \
	    --add-module=$(MODULESDIR)/nginx-upstream-fair \
	    $(CONFIGURE_OPTS) >$@
	touch $@

config.status.%:
	echo "configuration for flavour $* not yet defined."

build-arch.%: config.status.%
	dh_testdir
	dh_prep
	$(MAKE) -j$(NUMJOBS) -C $(BUILDDIR_$*) build

build-arch: $(foreach flavour,$(FLAVOURS),build-arch.$(flavour))
	dh_testdir
	touch $@

build-dbg.%: install
	dh_testdir
	dh_strip --package=nginx-$(*) --dbg-package=nginx-$(*)-dbg

build-dbg: $(foreach flavour,$(FLAVOURS),build-dbg.$(flavour))
	dh_testdir
	touch $@

build-indep:

build: build-indep build-arch
	dh_testdir
	touch $@

clean:
	dh_testdir
	dh_testroot
	rm -f build-stamp
	rm -f config.sub config.guess
	dh_clean
	rm -rf $(CURDIR)/debian/build-*
mime-types:
	

install: mime-types
	dh_testdir
	dh_testroot
	dh_prep
	dh_installdirs
	dh_install

binary-indep: build install
	dh_testdir
	dh_testroot
	dh_installman -i
	dh_installchangelogs -i -k CHANGES
	dh_installdocs -i
	dh_installdebconf
	dh_installexamples -i
	dh_installinit -r --no-start -i --name=nginx
	dh_installinit -r --no-start -i --name=nginx-naxsi-ui
	cp debian/logrotate debian/nginx-common/etc/logrotate.d/nginx
	mkdir -p debian/nginx-common/lib/systemd/system
	cp debian/nginx-common.service debian/nginx-common/lib/systemd/system/nginx.service
	dh_link -i
	dh_compress -i
	dh_fixperms -i
	dh_installdeb -i
	dh_gencontrol -i
	dh_md5sums -i
	dh_builddeb -i

binary-arch: install build-dbg
	dh_testdir
	dh_testroot
	dh_installchangelogs -a -k CHANGES
	dh_installdocs -a
	dh_link -aA
	dh_compress -a
	dh_perl -a
	dh_fixperms -a
	dh_installdeb -a
	dh_shlibdeps -a
	dh_gencontrol -a
	dh_md5sums -a
	dh_builddeb -a

binary: binary-indep binary-arch

.PHONY: build clean binary-indep binary-arch binary install

github link Interesting, isn’t it?

If yes, get ready, you’ll not be able to live in the system without writing something like that.

Exhibit B: nginx on arch

Common syntax of describing an arch package looks like this (nginx derivation example):

# Maintainer: Martchus <[email protected]>
# Contributor: Alexander Kuznecov <[email protected]>

_pkgname=nginx
pkgname=nginx-custom
pkgver=1.14.0
pkgrel=1
pkgdesc='Lightweight HTTP server and IMAP/POP3 proxy server (ready for use with 3rd party modules)'
arch=('x86_64')
url='https://nginx.org'
license=('custom')
depends=('pcre' 'zlib' 'openssl' 'pam' 'geoip' 'geoip-database' 'gd' 'libxslt' 'mailcap')
checkdepends=('perl' 'perl-gd' 'perl-io-socket-ssl' 'perl-fcgi' 'perl-cache-memcached'
              'memcached' 'ffmpeg' 'inetutils')
makedepends=('libxslt' 'gd' 'mercurial')
conflicts=('nginx' 'nginx-unstable' 'nginx-svn' 'nginx-devel' 'nginx-custom-dev' 'nginx-full')
provides=("nginx=$pkgver")

_user='http'
_group='http'
_doc_root="/usr/share/${_pkgname}/html"
_sysconf_path='etc'
_conf_path="${_sysconf_path}/${_pkgname}"
_tmp_path="/var/spool/${_pkgname}"
_pid_path='/run'
_lock_path='/var/lock'
_access_log_path="/var/log/${_pkgname}/access.log"
#_error_log_path="/var/log/${_pkgname}/error.log"
_error_log_path='stderr'
backup=("${_conf_path}/fastcgi.conf"
        "${_conf_path}/fastcgi_params"
        "${_conf_path}/koi-win"
        "${_conf_path}/koi-utf"
        "${_conf_path}/nginx.conf"
        "${_conf_path}/scgi_params"
        "${_conf_path}/uwsgi_params"
        "${_conf_path}/win-utf"
        "etc/logrotate.d/nginx")

source=("nginx.sh"
        "nginx.conf"
        "nginx.logrotate"
        "nginx.service"
        "https://nginx.org/download/nginx-$pkgver.tar.gz"{,.asc}
        hg+http://hg.nginx.org/nginx-tests#revision=d6daf03478ad
)
validpgpkeys=('B0F4253373F8F6F510D42178520A9993A1C052F8') # Maxim Dounin <[email protected]>
sha256sums=('c147953bd4e10ea3a74d9b4add59f18fc0bed093016a53a1d33ea2b9065cdc23'
            '8d8e314da10411b29157066ea313fc080a145d2075df0c99a1d500ffc7e8b7d1'
            '06ebe161af3e761f2e2e35a67c6c0af27bf61aea7cd4ba8b28372ced5e3b3175'
            'c61cf4fefeb2a3b4c5eaba61123546e03f87c701466e8dad2c9262433f13b2d5'
            '5d15becbf69aba1fe33f8d416d97edd95ea8919ea9ac519eff9bafebb6022cb5'
            'SKIP'
            'SKIP')

check() {
  cd nginx-tests
  TEST_NGINX_BINARY="$srcdir/${_pkgname}-${pkgver}/objs/nginx" find -maxdepth 1 -iname '*.t' -exec prove {} \+
}

build() {
  local _src_dir="${srcdir}/${_pkgname}-${pkgver}"
  cd $_src_dir

  ./configure \
    --prefix="/${_conf_path}" \
    --conf-path="/${_conf_path}/nginx.conf" \
    --sbin-path="/usr/bin/${_pkgname}" \
    --pid-path="${_pid_path}/${_pkgname}.pid" \
    --lock-path=${_pid_path}/${_pkgname}.lock \
    --http-client-body-temp-path=${_tmp_path}/client_body_temp \
    --http-proxy-temp-path=${_tmp_path}/proxy_temp \
    --http-fastcgi-temp-path=${_tmp_path}/fastcgi_temp \
    --http-uwsgi-temp-path=${_tmp_path}/uwsgi_temp \
    --http-scgi-temp-path=${_tmp_path}scgi_temp \
    --http-log-path=${_access_log_path} \
    --error-log-path=${_error_log_path} \
    --user=${_user} \
    --group=${_group} \
    --with-compat \
    --with-debug \
    --with-ipv6 \
    --with-pcre-jit \
    --with-file-aio \
    --with-mail \
    --with-mail_ssl_module \
    --with-stream \
    --with-stream_ssl_module \
    --with-threads \
    --with-http_ssl_module \
    --with-http_stub_status_module \
    --with-http_dav_module \
    --with-http_gzip_static_module \
    --with-http_realip_module \
    --with-http_addition_module \
    --with-http_auth_request_module \
    --with-http_xslt_module \
    --with-http_image_filter_module \
    --with-http_sub_module \
    --with-http_v2_module \
    --with-mail \
    --with-mail_ssl_module \
    --with-stream \
    --with-stream_ssl_module \
    --with-threads \
    --with-http_flv_module \
    --with-http_mp4_module \
    --with-http_random_index_module \
    --with-http_secure_link_module \
    --with-http_slice_module \
    --with-http_perl_module \
    --with-http_degradation_module \
    --with-http_geoip_module \
    --with-http_gunzip_module \
    --with-http_auth_request_module \
    --with-cc-opt="$CFLAGS $CPPFLAGS" \
    --with-ld-opt="$LDFLAGS"
  make
}

package() {
  cd "${srcdir}/${_pkgname}-${pkgver}"
  make DESTDIR="$pkgdir/" install

  sed -e "s|\<user\s\+\w\+;|user $_user $_group;|g" \
    -e '44s|html|/usr/share/nginx/html|' \
    -e '54s|html|/usr/share/nginx/html|' \
    -i ${pkgdir}/$_conf_path/nginx.conf

  mkdir -p ${pkgdir}/$_conf_path/sites-enabled/
  mkdir -p ${pkgdir}/$_conf_path/sites-available/

  install -d "${pkgdir}/${_tmp_path}"
  install -d "${pkgdir}/${_doc_root}"

  mv "${pkgdir}/${_conf_path}/html/"* "${pkgdir}/${_doc_root}"
  rm -rf "${pkgdir}/${_conf_path}/html"
  rm "$pkgdir"/etc/nginx/mime.types # in mailcap

  install -D -m555 "${srcdir}/nginx.sh" "${pkgdir}/etc/rc.d/${_pkgname}"
  install -D -m644 "${srcdir}/nginx.logrotate" "${pkgdir}/etc/logrotate.d/${_pkgname}"
  install -D -m644 "${srcdir}/nginx.conf" "${pkgdir}/etc/conf.d/${_pkgname}"
  install -D -m644 "${srcdir}/nginx.service" "${pkgdir}/usr/lib/systemd/system/nginx.service"
  install -D -m644 "LICENSE" "${pkgdir}/usr/share/licenses/${_pkgname}/LICENSE"

  install -d "$pkgdir"/usr/share/man/man8/
  gzip -9c man/nginx.8 > "$pkgdir"/usr/share/man/man8/nginx.8.gz

  for i in ftdetect indent syntax; do
    install -Dm644 contrib/vim/${i}/nginx.vim \
      "${pkgdir}/usr/share/vim/vimfiles/${i}/nginx.vim"
  done
}

source Interesting, isn’t it?

If yes, get ready, you’ll not be able to live in the system without writing something like that.

Reality check

I really don’t understand what the author’s intent in the previous section was. Did he genuinely think that to run a webserver on NixOS he would need to build Nginx from source instead of simply doing services.nginx.enable=true;? This is the more charitable interpretation, the alternative being that he is willfully deceiving readers by showing them a scary-looking code snippet that they wouldn’t need to deal with unless they’re building Nginx from source.

I’ve had to build packages from source from time to time. NGINX in particular had to be built from source to add support for sticky sessions. On Debian, I did this by fetching Nginx sources with apt-get source nginx, followed by modifying the rules file to add the flags I needed. After this, I had to build the debs and distribute them to the web servers using puppet. I no longer have my .bash_history from when I built the debs, so if I ever have to update or rebuild that package, I will need to dig up the exact flags I used somehow. If I had used NixOS for those web servers, it would have been as simple as

services.nginx.package = pkgs.nginx.override(
    configureFlags = [
        "--with-http_ssl_module"
        "–with-pcre-jit"
    ];
);

The flags I used for this Nginx, by the nature of NixOS, are baked into the configuration files. Any server I deploy this config to will have Nginx built with these configurations.

Configuration management

As far as I can tell, here the author complains about the USP of NixOS, ie. the existence of a configuration file that describes the absolute state of the system. You don’t need to keep track of a dozen independent configuration files across multiple services and daemons. As long as you have the configuration.nix file (and all files referenced by it), you can reliably reproduce the state of the system.

I’ll not get into how great it is to be able to configure the entire system from one place, but I will leave a link to my daily driver laptop’s configuration to show how convenient it is. If you want to replicate that configuration, I strongly recommend commenting out (import ./srcbuild.nix) and the boot.kernelPackages block unless you have a really fast nix build server and want to build everything from source, optimized for an Intel Skylake CPU.

Somewhat more strangely, the author complains about Chef, Puppet, and Ansible not working on NixOS. When configuration management through nix is pretty much the main selling point of NixOS, why would you use another configuration management tool on top of it?

Kernel upgrade

Again, I don’t know if things were different back when this post was written, but changing your kernel version is as simple as boot.kernelPackages = pkgs.linuxPackages_latest;, boot.kernelPackages = pkgs.linuxPackages_5_10;, or boot.kernelPackages = pkgs.linuxPackages_4_9;. You can even build a custom kenrel by providing the source code, provide custom configs, and use out-of-tree kenrel modules with ease. For detailed examples, RTFW(iki).

Cloud support

I’m currently using NixOS on most of my personal AWS/GCP instances. You can even generate custom AMIs and upload them with ease. As always, RTFM.

I’ve tested the AMIs on t3a, t3, t4g, c6g, and a handful of other instance types. m5 is also supported as of now. It could just be that nobody had built AMIs for m5 back when the author tried it. If that was the case, building new AMIs to support these instances would have been trivial.

On the subject of 100% CPU util on autoscaling groups, it looks like the author was spinning up base NixOS AMI instances, pushing his configuration.nix, and building it on the instances. If your use-case requires building packages from source, building an AMI with that package already built, or hosting your own nix cache with this build result, would be the reasonable way to deploy. What the author has done here is the same as, say, building Nginx from source in the startup script of an autoscaling group.

I haven’t tested NixOps deployment to AWS yet. I’ll update this section if/when I have a reason to try it.

Cache

The author is right about this one. Cache cleanup is a problem with the nix binary cache. There is no clean way to GC the cache as of now. so manual cleanups or scheduled LRU cleanups with something like find -atime +30 is necessary. The nix wiki doesn’t show how to do this at the moment. I might add this to the binary cache wiki page eventually.

Security

The author starts talking about security and very quickly wraps up that line of thought with something along the lines of “It looks secure, but maybe it isn’t, who knows?”. He then quickly pivots to changes in docker based derivations between versions 18.03 and 18.09. I haven’t used nix docker images, so I can’t comment on that, but I have no idea what that has to do with security. Sure, the author is upset about what sound like breaking changes across versions that supposedly required a lot of manual work, but why is this under the heading of security?

Windows support

A moment of silence for our comrades unfortunate enough to have to use Windows in their infrastructure.

System requirements

CPU

I just don’t get it. Why does the author have to build everything from source? I did that for my ThinkPad because deep down I’m a wannabe /g/entoo neckbeard, but pretty much every package is already available in the nix cache. If you have the need to build things from source, use a cache to reuse the results, and maybe spin up a spot build instance on AWS. Use NixOps to build in one place and distribute the build results. None of these methods are great secrets that were revealed to me after meditating under a waterfall for decades. RTFM.

Storage

You know what’s coming… RTFM NixOS can automatically clean up older generations periodically. Personally, I like to keep the last few generations around, especially when making kernel/driver changes.

Once invested

• Get hands-on and learn by practice
• L(isten to )TFM, ideally before you start using nix in prod
• Don’t try to use a linear induction motor the way you would use a wheel
• Don’t spread FUD while simultaneously demonstrating a total lack of understanding

What to do next

Give it a try. Read the docs. Understand the nix way of doing things instead of trying to force nix to behave like Debian. Understand that your decades of experience on Debian won’t port to NixOS. Distro-agnostic Linux knowledge will.

Shill’s notes

I asked that friend to proofread this post before it went up, and she asked if she could add her perspective to it. Here’s what she had to say:

When I first started messing around with Nix and NixOS, I didn’t know much about Linux to begin with. As science proceeds funeral by funeral, so my brain was inducted into the halls of *nix OSs with the new and strange rubbing shoulders with the “stable”, “enterprise-ready” distros. The filesystem Hierarchy Standard never got a chance to bake itself into my brain, and its assumptions were never sacred to me. I was once asked in an interview how I might resolve a situation where packages A and B depended on different versions of a package C, and I genuinely failed to see what the issue was. Once you’ve been exposed to a system that isolates every single package runtime from every other as a matter of course, why would you ever get precious about a package’s “first-person citizenship” on a system?

All this to say, it took me a long while to understand why, for someone for whom FHS is baked in, this distro looks plain weird. The only choices where Nix can be said to really, truly deviate from every other package management system of our day are where things are located and who can access them. It sequesters control of every resource in your system that might be fairly called “configuration”, in order to provide guarantees about it, and by “sequester” I mean “put in the nix store and mount it as read-only and disallow changes that aren’t made via evaluating a *.nix file and checksum the outputs” and the consequences of this choice are indeed as far-reaching as the nightmare scenarios I’m sure you-the-experienced-debian-sysad are already envisioning.

It seems fair to me to ask whether it’s worth it. So far, I’ve found it to be; it helps me think about what I would consider to be config vs data, and where the responsibility for a particular detail of the system state actually lies, and the importance of the distinction between persistence and reproducibility. In allowing you to configure a system coherently, NixOS frees your brain up to actually think about its architecture. (Before the comparison is made: I really hate Puppet, would not call it coherent, and would not say it frees up anyone’s brain. Puppet is slavery, you are the Puppet, and ensure=>absent for your soul is written inline in the hieradata somewhere.)

What seems unfair is to hold the learning curve against the distro without trying to climb it at least partway to see what’s up there. This author mentions digging into the nixpkgs repository for examples of nix code; have they read through the nix pills, which teach you the language? Have they tried to do anything interesting with it, to see how it goes? Looked at a pro-con list to see what they ought to be getting out of it, and the mechanisms by which these benefits are provided?

This ecosystem is larger, stranger, and more powerful than this author seems to realize. The “unnecessary difficulties” they talk about all seem to be failures to dig deep enough to see the necessities in question.

tl;dr you get what you pay (in RTFM effort and open-mindedness) for, dude.

• 2021 1
• 2020 1
• 2019 1
• 2018 3
• 2017 1
• 2016 2