Share a single bloop daemon per .bats file to speed up E2E tests #357

New issue

Closed

opened 2026-04-17 07:53:26 +00:00 by mprihoda · 1 comment

mprihoda commented

2026-04-17 07:53:26 +00:00

(Migrated from github.com)

Context

Each of the 6 .bats files that set XDG_DATA_HOME to a temp dir currently creates a fresh mktemp -d per test in setup(). When a test invokes ./iw-run → scala-cli, a new bloop daemon cold-starts under that temp XDG — one daemon per test, ~3–5 s startup overhead each time.

The orphan-daemon leak itself was fixed in 7d3ce1f (Option 2: pkill in teardown), but the performance cost of cold-starting a JVM build server per test remains.

Proposal

Refactor the 6 affected test files to use BATS setup_file / teardown_file:

setup_file(): Create a single FILE_TEST_DIR with its own XDG_DATA_HOME. This becomes the shared bloop data dir for the whole .bats file.
setup(): Create a per-test TEST_DIR for test-specific state (plugin dirs, git repos, configs). Set IW_PLUGIN_DIRS or create plugin dirs under $XDG_DATA_HOME/iw/plugins/ — but clean them each test to maintain isolation.
teardown(): Remove per-test TEST_DIR only.
teardown_file(): Call stop_test_bloop and remove FILE_TEST_DIR.

This way one bloop daemon serves all tests in a file, and stop_test_bloop kills it once at the end.

Affected files

test/plugin-discovery.bats
test/plugin-commands-list.bats
test/plugin-commands-describe.bats
test/plugin-commands-execute.bats
test/plugin-hooks.bats
test/version-check.bats

Complexity

The main challenge is that several tests create plugins inside $XDG_DATA_HOME/iw/plugins/ and expect test-to-test isolation. Each test's plugin state must be cleaned or namespaced so tests don't contaminate each other.

Expected benefit

~20–30 fewer bloop cold-starts per full E2E run, saving roughly 1–2 minutes of wall time.

## Context Each of the 6 `.bats` files that set `XDG_DATA_HOME` to a temp dir currently creates a fresh `mktemp -d` **per test** in `setup()`. When a test invokes `./iw-run` → `scala-cli`, a new bloop daemon cold-starts under that temp XDG — one daemon per test, ~3–5 s startup overhead each time. The orphan-daemon leak itself was fixed in 7d3ce1f (Option 2: pkill in teardown), but the performance cost of cold-starting a JVM build server per test remains. ## Proposal Refactor the 6 affected test files to use BATS `setup_file` / `teardown_file`: 1. **`setup_file()`**: Create a single `FILE_TEST_DIR` with its own `XDG_DATA_HOME`. This becomes the shared bloop data dir for the whole `.bats` file. 2. **`setup()`**: Create a per-test `TEST_DIR` for test-specific state (plugin dirs, git repos, configs). Set `IW_PLUGIN_DIRS` or create plugin dirs under `$XDG_DATA_HOME/iw/plugins/` — but clean them each test to maintain isolation. 3. **`teardown()`**: Remove per-test `TEST_DIR` only. 4. **`teardown_file()`**: Call `stop_test_bloop` and remove `FILE_TEST_DIR`. This way one bloop daemon serves all tests in a file, and `stop_test_bloop` kills it once at the end. ## Affected files - `test/plugin-discovery.bats` - `test/plugin-commands-list.bats` - `test/plugin-commands-describe.bats` - `test/plugin-commands-execute.bats` - `test/plugin-hooks.bats` - `test/version-check.bats` ## Complexity The main challenge is that several tests create plugins inside `$XDG_DATA_HOME/iw/plugins/` and expect test-to-test isolation. Each test's plugin state must be cleaned or namespaced so tests don't contaminate each other. ## Expected benefit ~20–30 fewer bloop cold-starts per full E2E run, saving roughly 1–2 minutes of wall time.

mprihoda commented

2026-06-10 20:43:40 +00:00

(Migrated from github.com)

Closing as superseded by Phase 3 of the test-rebalance plan (docs/plans/2026-06-03-test-rebalance.md). The bloop cold-start cost was eliminated at the root rather than by sharing daemons: phase 3.1, 3.5, and 3.6 dropped the dead cp -r core/ + ./mill show core.jar + stop_test_bloop setup from BATS files that never actually invoked scala-cli. Per-test bloop daemons are no longer the bottleneck — most affected files now complete in <2s. Where scala-cli IS legitimately needed, the four-test test/contract/scala_cli_contract.bats covers it in ~2s aggregate.

Closing as superseded by Phase 3 of the test-rebalance plan (docs/plans/2026-06-03-test-rebalance.md). The bloop cold-start cost was eliminated at the root rather than by sharing daemons: phase 3.1, 3.5, and 3.6 dropped the dead `cp -r core/` + `./mill show core.jar` + `stop_test_bloop` setup from BATS files that never actually invoked scala-cli. Per-test bloop daemons are no longer the bottleneck — most affected files now complete in <2s. Where scala-cli IS legitimately needed, the four-test `test/contract/scala_cli_contract.bats` covers it in ~2s aggregate.