Pin the development and CI toolchain with a Nix flake #428

New issue

Open

opened 2026-08-21 12:42:23 +00:00 by mph · 1 comment

mph commented

2026-08-21 12:42:23 +00:00

Owner

Problem

The tool versions for development and for CI come from two unrelated sources, and
neither is pinned in this repository.

CI. .github/Dockerfile.ci builds the image from .github/deps/, which is
gitignored. To rebuild the image a person follows a manual procedure in
.github/ci-image.md:33-45 that runs docker cp to take binaries out of another
container by hand. The Dockerfile does not declare most versions — it globs
jdk-* and node-v20.*, so the real versions live only in a markdown table.
The workflow pulls iw-cli-ci:latest on a self-hosted runner, so the image can
change with no commit here.

Development. Each machine supplies its own tools. Nothing ties a developer's
JDK, scala-cli, mill, node or bats to the versions CI uses.

The two sets agree today by luck, not by construction. Tool drift is found only
when CI turns red.

Proposal

Declare the toolchain once in a Nix flake, commit flake.lock, and derive both
the developer shell and CI from it.

Tools to cover: JDK 21, scala-cli, coursier, scalafix, mill 1.1.5, Node 20 with
corepack, bats 1.12.0, tmux, jq, rsync, python3, git, gh, glab, fj.

CI route — decide during analysis, not now

Two routes are open. Choose one when the issue is analysed.

Keep a container, but build it from the flake with
dockerTools.buildLayeredImage. The image becomes a derived artifact and the
hand-assembly step disappears.
Drop the custom image. Give the self-hosted runner a Nix store and run
nix develop --command ./iw ./test. A self-hosted runner keeps a warm store,
so there is no image to build and none to push.

Known constraints

A flake pins tools, not dependencies. Coursier still resolves the Scala
libraries and Yarn 4 still resolves the Node packages (and still needs
WEBAWESOME_NPM_TOKEN). The result is one declared toolset, not a reproducible
build. Tool drift is the problem that actually occurred, so this still hits the
target.
Some tools need extra work. fj is unlikely to be in nixpkgs, and mill 1.1.5
and the pinned scala-cli version may not match the nixpkgs versions. These need
fetchurl derivations or an overlay.
./iw is the entry point for every command. Without direnv each call becomes
nix develop -c ./iw …. An .envrc with use flake solves this, but then each
developer must install direnv. Decide this with the CI route.

Origin

Found while triaging LESSONS.md for IW-422. A captured lesson claimed bats was
absent from the machine and had to be run through nix-shell -p bats. By triage
time the claim was already false — bats 1.12.0 was in the user profile. The
lesson was deleted, because the real problem is not a missing binary but an
unpinned toolchain.

## Problem The tool versions for development and for CI come from two unrelated sources, and neither is pinned in this repository. **CI.** `.github/Dockerfile.ci` builds the image from `.github/deps/`, which is gitignored. To rebuild the image a person follows a manual procedure in `.github/ci-image.md:33-45` that runs `docker cp` to take binaries out of another container by hand. The Dockerfile does not declare most versions — it globs `jdk-*` and `node-v20.*`, so the real versions live only in a markdown table. The workflow pulls `iw-cli-ci:latest` on a self-hosted runner, so the image can change with no commit here. **Development.** Each machine supplies its own tools. Nothing ties a developer's JDK, scala-cli, mill, node or bats to the versions CI uses. The two sets agree today by luck, not by construction. Tool drift is found only when CI turns red. ## Proposal Declare the toolchain once in a Nix flake, commit `flake.lock`, and derive both the developer shell and CI from it. Tools to cover: JDK 21, scala-cli, coursier, scalafix, mill 1.1.5, Node 20 with corepack, bats 1.12.0, tmux, jq, rsync, python3, git, gh, glab, fj. ## CI route — decide during analysis, not now Two routes are open. Choose one when the issue is analysed. 1. Keep a container, but build it from the flake with `dockerTools.buildLayeredImage`. The image becomes a derived artifact and the hand-assembly step disappears. 2. Drop the custom image. Give the self-hosted runner a Nix store and run `nix develop --command ./iw ./test`. A self-hosted runner keeps a warm store, so there is no image to build and none to push. ## Known constraints - A flake pins tools, not dependencies. Coursier still resolves the Scala libraries and Yarn 4 still resolves the Node packages (and still needs `WEBAWESOME_NPM_TOKEN`). The result is one declared toolset, not a reproducible build. Tool drift is the problem that actually occurred, so this still hits the target. - Some tools need extra work. `fj` is unlikely to be in nixpkgs, and mill 1.1.5 and the pinned scala-cli version may not match the nixpkgs versions. These need `fetchurl` derivations or an overlay. - `./iw` is the entry point for every command. Without direnv each call becomes `nix develop -c ./iw …`. An `.envrc` with `use flake` solves this, but then each developer must install direnv. Decide this with the CI route. ## Origin Found while triaging LESSONS.md for IW-422. A captured lesson claimed `bats` was absent from the machine and had to be run through `nix-shell -p bats`. By triage time the claim was already false — `bats` 1.12.0 was in the user profile. The lesson was deleted, because the real problem is not a missing binary but an unpinned toolchain.

mph referenced this issue from a commit

2026-08-21 20:59:52 +00:00

workflow: triage lessons for IW-422 (entries 1-4)

claude commented

2026-10-16 03:52:16 +00:00

Member

Intake: needs-input

1. Which route must CI use: an image built from the flake, or Nix on the self-hosted runner?

Each CI job runs in the container iw-cli-ci:latest on a self-hosted runner (ci.yml#L30-L32). Route b changes the runner host, which is not in this repository, and nothing here tells whether that host has Nix. Open issue #407 (glab is missing from the CI image) is related, but it is a different issue.

a) Build the image from the flake with dockerTools.buildLayeredImage. The workflows stay almost the same. A person must still build and push the image after each change to flake.lock, unless CI also builds it.
b) Remove the image and run nix develop --command ./iw ./test on the runner. No image to build or push. The runner host must get a Nix store, and each job must be changed to run without container:. If you choose b, paste the output of nix --version from the runner host, or write "no Nix".

Recommended: a) — It changes only this repository, so the analysis does not depend on a host it cannot see.

2. Must the repository have an `.envrc` with `use flake`, so that `./iw` works without `nix develop -c`?

Each developer command starts with ./iw. Without direnv, each call becomes nix develop -c ./iw ….

a) Add .envrc and document direnv as a development requirement. Commands stay the same. Each developer must install direnv and run direnv allow one time.
b) Add no .envrc. Document nix develop as the way to enter the shell. There is no new tool to install. Developers must open the shell before they start work.

Recommended: a) — You keep the habit of typing ./iw directly, and the file costs nothing to a person who does not use direnv.

Answer in a comment on this issue, per number, for example "1: a", or in your own words; intake reads it on its next iteration.

Intake: needs-input ### 1. Which route must CI use: an image built from the flake, or Nix on the self-hosted runner? Each CI job runs in the container `iw-cli-ci:latest` on a self-hosted runner ([ci.yml#L30-L32](https://code.iterative.works/iterative-works/iw-cli/src/branch/main/.github/workflows/ci.yml#L30-L32)). Route b changes the runner host, which is not in this repository, and nothing here tells whether that host has Nix. Open issue #407 (glab is missing from the CI image) is related, but it is a different issue. - **a) Build the image from the flake with `dockerTools.buildLayeredImage`.** The workflows stay almost the same. A person must still build and push the image after each change to `flake.lock`, unless CI also builds it. - **b) Remove the image and run `nix develop --command ./iw ./test` on the runner.** No image to build or push. The runner host must get a Nix store, and each job must be changed to run without `container:`. If you choose b, paste the output of `nix --version` from the runner host, or write "no Nix". **Recommended: a)** — It changes only this repository, so the analysis does not depend on a host it cannot see. ### 2. Must the repository have an `.envrc` with `use flake`, so that `./iw` works without `nix develop -c`? Each developer command starts with `./iw`. Without direnv, each call becomes `nix develop -c ./iw …`. - **a) Add `.envrc` and document direnv as a development requirement.** Commands stay the same. Each developer must install direnv and run `direnv allow` one time. - **b) Add no `.envrc`. Document `nix develop` as the way to enter the shell.** There is no new tool to install. Developers must open the shell before they start work. **Recommended: a)** — You keep the habit of typing `./iw` directly, and the file costs nothing to a person who does not use direnv. Answer in a comment on this issue, per number, for example "1: a", or in your own words; intake reads it on its next iteration.

claude added the

needs-input

label

2026-10-16 03:52:16 +00:00