Git.push is killed after 5 minutes and the timeout is reported as a plain push failure #585

Open
opened 2026-09-21 08:33:47 +00:00 by claude · 9 comments
Member

Found during IW-578 phase 1.

Git.push (core/adapters/Git.scala:171) runs under ProcessAdapter.run's 5-minute default timeout, and the kill is reported as a plain push failure — ProcessResult.timedOut is ignored. A cold pre-push hook (full unit suite + command compile) outran it during IW-578 phase 1's phase-pr --no-wait: stderr stopped after 33 of 36 commands compiled, ~6 min after the commit; the same push run by hand passed in 61 s.

Found during IW-578 phase 1. `Git.push` (`core/adapters/Git.scala:171`) runs under `ProcessAdapter.run`'s 5-minute default timeout, and the kill is reported as a plain push failure — `ProcessResult.timedOut` is ignored. A cold pre-push hook (full unit suite + command compile) outran it during IW-578 phase 1's `phase-pr --no-wait`: stderr stopped after 33 of 36 commands compiled, ~6 min after the commit; the same push run by hand passed in 61 s.
Author
Member

Possibly a second way the same cold pre-push hook breaks a push. On IW-491 phase 1 (first phase-pr --no-wait), the push failed with fatal: the remote end hung up unexpectedly. git opens the SSH connection before it runs the pre-push hook. The hook's full mill core.test + dashboard.test run on a cold cache seems to outlast the forge's SSH idle timeout, so the forge closes the connection. A manual push with warm caches then succeeded. If this is the cause, a longer process timeout does not fix it. The hook must be faster, or run before git opens the connection.

Possibly a second way the same cold pre-push hook breaks a push. On IW-491 phase 1 (first `phase-pr --no-wait`), the push failed with `fatal: the remote end hung up unexpectedly`. git opens the SSH connection before it runs the pre-push hook. The hook's full `mill core.test` + `dashboard.test` run on a cold cache seems to outlast the forge's SSH idle timeout, so the forge closes the connection. A manual push with warm caches then succeeded. If this is the cause, a longer process timeout does not fix it. The hook must be faster, or run before git opens the connection.
Author
Member

Seen again during IW-599 phase 1 (cold Mill daemon). phase-pr push failed with no failure line: the captured pre-push output stops between two command compilations. A plain git push of the same commit passed the hook in 55 s. Not proven to be the timeout, but it matches the pattern this issue describes. (Git.push is now at core/adapters/Git.scala:180.)

Seen again during IW-599 phase 1 (cold Mill daemon). `phase-pr` push failed with no failure line: the captured pre-push output stops between two command compilations. A plain `git push` of the same commit passed the hook in 55 s. Not proven to be the timeout, but it matches the pattern this issue describes. (`Git.push` is now at `core/adapters/Git.scala:180`.)
Author
Member

Seen again in IW-715 (2026-10-05):

  • phase-pr push is probably killed by ProcessAdapter.DefaultTimeoutMs (5 min) while the pre-push hook runs on a cold build, and reports only the hook's partial output — core/adapters/Git.scala:276 calls ProcessAdapter.run(args) with no timeoutMs; on 2026-10-05 the first phase-pr push of IW-715-phase-01 failed with hook output that stops after 35 of 37 command compiles (no PUSH BLOCKED banner); the same push by hand right after, warm, passed in 59 s. Not reproduced cold.
Seen again in IW-715 (2026-10-05): - [ ] phase-pr push is probably killed by ProcessAdapter.DefaultTimeoutMs (5 min) while the pre-push hook runs on a cold build, and reports only the hook's partial output — core/adapters/Git.scala:276 calls ProcessAdapter.run(args) with no timeoutMs; on 2026-10-05 the first phase-pr push of IW-715-phase-01 failed with hook output that stops after 35 of 37 command compiles (no PUSH BLOCKED banner); the same push by hand right after, warm, passed in 59 s. Not reproduced cold.
Author
Member

Two more cases from IW-671 phase 1. The cause is the same: the phase-pr push runs the pre-push hook's cold build, and the push fails for a reason that is not in the code.

  • phase-pr push over SSH fails when the pre-push hook runs a cold build — git opens the Forgejo connection before the hook, and a multi-minute ./iw ./test unit lets the server drop it ("fatal: the remote end hung up unexpectedly" / "Forgejo: Failed to execute git command"); the same push with warm caches took 62 s and passed (IW-671 phase 1).
  • A batch session without WEBAWESOME_NPM_TOKEN cannot push any branch — the pre-push hook runs ./iw ./test unit, whose dashboard.test needs frontend.viteBuild (yarn install fails: "Environment variable not found (WEBAWESOME_NPM_TOKEN)"); seen in IW-671 phase 1 phase-pr.
Two more cases from IW-671 phase 1. The cause is the same: the `phase-pr` push runs the pre-push hook's cold build, and the push fails for a reason that is not in the code. - [ ] `phase-pr` push over SSH fails when the pre-push hook runs a cold build — git opens the Forgejo connection before the hook, and a multi-minute `./iw ./test unit` lets the server drop it ("fatal: the remote end hung up unexpectedly" / "Forgejo: Failed to execute git command"); the same push with warm caches took 62 s and passed (IW-671 phase 1). - [ ] A batch session without `WEBAWESOME_NPM_TOKEN` cannot push any branch — the pre-push hook runs `./iw ./test unit`, whose `dashboard.test` needs `frontend.viteBuild` (`yarn install` fails: "Environment variable not found (WEBAWESOME_NPM_TOKEN)"); seen in IW-671 phase 1 `phase-pr`.
Author
Member

Found during IW-682 (port allocation for worktrees). Follow-up, verbatim:

  • phase-pr push dies when the pre-push hook runs longer than 5 minutes — GitAdapter.push (core/adapters/Git.scala:275) uses ProcessAdapter.run with DefaultTimeoutMs = 5 * 60 * 1000 (core/adapters/Process.scala:24); the hook's full unit run took longer, and phase-pr printed "Failed to push branch" with only the hook's partial output; a manual git push with the same hook passed.
Found during IW-682 (port allocation for worktrees). Follow-up, verbatim: - [ ] `phase-pr` push dies when the pre-push hook runs longer than 5 minutes — `GitAdapter.push` (core/adapters/Git.scala:275) uses `ProcessAdapter.run` with `DefaultTimeoutMs = 5 * 60 * 1000` (core/adapters/Process.scala:24); the hook's full unit run took longer, and phase-pr printed "Failed to push branch" with only the hook's partial output; a manual `git push` with the same hook passed.
Author
Member

Found during IW-513, same area (how a failed Git.push is reported):

  • Classify git push failures as transient or permanent. — IW-513 gave the forge reads a TrackerError with Unreachable/Refused; a push failure is still one undifferentiated error. IW-513 FOLLOW-UPS.md
Found during IW-513, same area (how a failed `Git.push` is reported): - [ ] Classify `git push` failures as transient or permanent. — IW-513 gave the forge reads a `TrackerError` with `Unreachable`/`Refused`; a push failure is still one undifferentiated error. IW-513 FOLLOW-UPS.md
Author
Member

Intake: agent-ready
Workflow: diagnostic
Reason: all gates passed
Analysis: project-management/issues/IW-585/analysis.md on branch IW-585

Intake: agent-ready Workflow: diagnostic Reason: all gates passed Analysis: project-management/issues/IW-585/analysis.md on branch IW-585
Author
Member

Implement: needs-human
Reason: batch-implement exit 1 — the kanon plugin (batch/ForgeCatchUp.scala:143) does not compile against iw-cli core at this branch: core now returns TrackerError from the tracker port (#778, #780), and the plugin still expects String. No phase ran. This blocks every iw-cli issue, not only this one; the kanon plugin must be updated first.

batch-implement.log was not written; the command output:

[warn]  Using directives detected in multiple files:
- /home/mph/.local/share/iw/plugins/kanon/lib/deps.scala:1:1-7:41
It is recommended to keep them centralized in the /home/mph/Devel/iw/iw-cli-IW-585/core/project.scala file.

[error] /home/mph/.local/share/iw/plugins/kanon/lib/batch/ForgeCatchUp.scala:143:7
[error] Found:    Either[iw.core.model.TrackerError, Option[batch.ForgeState]]
[error] Required: Either[String, Option[batch.ForgeState]]
Compilation failed
exit=1
Implement: needs-human Reason: batch-implement exit 1 — the kanon plugin (batch/ForgeCatchUp.scala:143) does not compile against iw-cli core at this branch: core now returns TrackerError from the tracker port (#778, #780), and the plugin still expects String. No phase ran. This blocks every iw-cli issue, not only this one; the kanon plugin must be updated first. batch-implement.log was not written; the command output: ``` [warn] Using directives detected in multiple files: - /home/mph/.local/share/iw/plugins/kanon/lib/deps.scala:1:1-7:41 It is recommended to keep them centralized in the /home/mph/Devel/iw/iw-cli-IW-585/core/project.scala file. [error] /home/mph/.local/share/iw/plugins/kanon/lib/batch/ForgeCatchUp.scala:143:7 [error] Found: Either[iw.core.model.TrackerError, Option[batch.ForgeState]] [error] Required: Either[String, Option[batch.ForgeState]] Compilation failed exit=1 ```
Author
Member

Release: the implement loop failed this issue for a reason outside it. The kanon plugin did not compile against iw-cli main, because the tracker port now returns TrackerError (iw-cli #778). The plugin fix is dev-docs #484, and iw-cli 0.22.1 carries the core change. The issue goes back to the implement loop.

Release: the implement loop failed this issue for a reason outside it. The kanon plugin did not compile against iw-cli main, because the tracker port now returns TrackerError (iw-cli #778). The plugin fix is dev-docs #484, and iw-cli 0.22.1 carries the core change. The issue goes back to the implement loop.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
iterative-works/iw-cli#585
No description provided.