Pin the development and CI toolchain with a Nix flake #428

New issue

Open

opened 2026-08-21 12:42:23 +00:00 by mph · 0 comments

mph commented

2026-08-21 12:42:23 +00:00

Owner

Problem

The tool versions for development and for CI come from two unrelated sources, and
neither is pinned in this repository.

CI. .github/Dockerfile.ci builds the image from .github/deps/, which is
gitignored. To rebuild the image a person follows a manual procedure in
.github/ci-image.md:33-45 that runs docker cp to take binaries out of another
container by hand. The Dockerfile does not declare most versions — it globs
jdk-* and node-v20.*, so the real versions live only in a markdown table.
The workflow pulls iw-cli-ci:latest on a self-hosted runner, so the image can
change with no commit here.

Development. Each machine supplies its own tools. Nothing ties a developer's
JDK, scala-cli, mill, node or bats to the versions CI uses.

The two sets agree today by luck, not by construction. Tool drift is found only
when CI turns red.

Proposal

Declare the toolchain once in a Nix flake, commit flake.lock, and derive both
the developer shell and CI from it.

Tools to cover: JDK 21, scala-cli, coursier, scalafix, mill 1.1.5, Node 20 with
corepack, bats 1.12.0, tmux, jq, rsync, python3, git, gh, glab, fj.

CI route — decide during analysis, not now

Two routes are open. Choose one when the issue is analysed.

Keep a container, but build it from the flake with
dockerTools.buildLayeredImage. The image becomes a derived artifact and the
hand-assembly step disappears.
Drop the custom image. Give the self-hosted runner a Nix store and run
nix develop --command ./iw ./test. A self-hosted runner keeps a warm store,
so there is no image to build and none to push.

Known constraints

A flake pins tools, not dependencies. Coursier still resolves the Scala
libraries and Yarn 4 still resolves the Node packages (and still needs
WEBAWESOME_NPM_TOKEN). The result is one declared toolset, not a reproducible
build. Tool drift is the problem that actually occurred, so this still hits the
target.
Some tools need extra work. fj is unlikely to be in nixpkgs, and mill 1.1.5
and the pinned scala-cli version may not match the nixpkgs versions. These need
fetchurl derivations or an overlay.
./iw is the entry point for every command. Without direnv each call becomes
nix develop -c ./iw …. An .envrc with use flake solves this, but then each
developer must install direnv. Decide this with the CI route.

Origin

Found while triaging LESSONS.md for IW-422. A captured lesson claimed bats was
absent from the machine and had to be run through nix-shell -p bats. By triage
time the claim was already false — bats 1.12.0 was in the user profile. The
lesson was deleted, because the real problem is not a missing binary but an
unpinned toolchain.

## Problem The tool versions for development and for CI come from two unrelated sources, and neither is pinned in this repository. **CI.** `.github/Dockerfile.ci` builds the image from `.github/deps/`, which is gitignored. To rebuild the image a person follows a manual procedure in `.github/ci-image.md:33-45` that runs `docker cp` to take binaries out of another container by hand. The Dockerfile does not declare most versions — it globs `jdk-*` and `node-v20.*`, so the real versions live only in a markdown table. The workflow pulls `iw-cli-ci:latest` on a self-hosted runner, so the image can change with no commit here. **Development.** Each machine supplies its own tools. Nothing ties a developer's JDK, scala-cli, mill, node or bats to the versions CI uses. The two sets agree today by luck, not by construction. Tool drift is found only when CI turns red. ## Proposal Declare the toolchain once in a Nix flake, commit `flake.lock`, and derive both the developer shell and CI from it. Tools to cover: JDK 21, scala-cli, coursier, scalafix, mill 1.1.5, Node 20 with corepack, bats 1.12.0, tmux, jq, rsync, python3, git, gh, glab, fj. ## CI route — decide during analysis, not now Two routes are open. Choose one when the issue is analysed. 1. Keep a container, but build it from the flake with `dockerTools.buildLayeredImage`. The image becomes a derived artifact and the hand-assembly step disappears. 2. Drop the custom image. Give the self-hosted runner a Nix store and run `nix develop --command ./iw ./test`. A self-hosted runner keeps a warm store, so there is no image to build and none to push. ## Known constraints - A flake pins tools, not dependencies. Coursier still resolves the Scala libraries and Yarn 4 still resolves the Node packages (and still needs `WEBAWESOME_NPM_TOKEN`). The result is one declared toolset, not a reproducible build. Tool drift is the problem that actually occurred, so this still hits the target. - Some tools need extra work. `fj` is unlikely to be in nixpkgs, and mill 1.1.5 and the pinned scala-cli version may not match the nixpkgs versions. These need `fetchurl` derivations or an overlay. - `./iw` is the entry point for every command. Without direnv each call becomes `nix develop -c ./iw …`. An `.envrc` with `use flake` solves this, but then each developer must install direnv. Decide this with the CI route. ## Origin Found while triaging LESSONS.md for IW-422. A captured lesson claimed `bats` was absent from the machine and had to be run through `nix-shell -p bats`. By triage time the claim was already false — `bats` 1.12.0 was in the user profile. The lesson was deleted, because the real problem is not a missing binary but an unpinned toolchain.

mph referenced this issue from a commit

2026-08-21 20:59:52 +00:00

workflow: triage lessons for IW-422 (entries 1-4)