CI does not run on a push to main, so a merged batch stays untested until the nightly run #648

New issue

Open

opened 2026-09-26 14:04:22 +00:00 by claude · 1 comment

claude commented

2026-09-26 14:04:22 +00:00

Member

Problem

.github/workflows/ci.yml runs on pull_request and on a nightly schedule (03:17 UTC). It does not run on push to main. Thus the result of a merge is not tested until the next nightly run. This can be up to 24 hours later.

Each PR's CI tests its head, and the head is based on the main of that time. When several PRs merge in one batch, CI tests none of the combined states on main. Two PRs can each be green and still break main together. For example, git can merge two changes to one file with no conflict, and the result can still be wrong. Nobody sees that failure until the nightly run.

Seen in

On 2026-09-26 eight final PRs merged in one batch: #624, #626, #632, #638, #645, #647, #634, #640.

#624 and #626 both change core/test/ProcessTest.scala.
#624, #632 and #634 all change core/test/GitTest.scala.
Git merged most of these with no conflict. #634 needed a manual conflict fix.

CI did not test the result on main (9caddfda). A manual local run did: ./iw ./test unit, and the four BATS files that #647 changed. It passed.

What is necessary

A merge to main must get a CI result for the commit that it makes on main. It must not wait for the nightly run. One possible fix is to add push: branches: [main] to ci.yml. The changes job already sets scope=code for any event that is not pull_request, so all jobs would run.

Decide:

Do all jobs run on a push to main, or only compile and test? The nightly run also runs the contract job.
Who sees a red result on main, and how.

## Problem `.github/workflows/ci.yml` runs on `pull_request` and on a nightly `schedule` (03:17 UTC). It does not run on `push` to `main`. Thus the result of a merge is not tested until the next nightly run. This can be up to 24 hours later. Each PR's CI tests its head, and the head is based on the `main` of that time. When several PRs merge in one batch, CI tests none of the combined states on `main`. Two PRs can each be green and still break `main` together. For example, git can merge two changes to one file with no conflict, and the result can still be wrong. Nobody sees that failure until the nightly run. ## Seen in On 2026-09-26 eight final PRs merged in one batch: #624, #626, #632, #638, #645, #647, #634, #640. - #624 and #626 both change `core/test/ProcessTest.scala`. - #624, #632 and #634 all change `core/test/GitTest.scala`. - Git merged most of these with no conflict. #634 needed a manual conflict fix. CI did not test the result on `main` (`9caddfda`). A manual local run did: `./iw ./test unit`, and the four BATS files that #647 changed. It passed. ## What is necessary A merge to `main` must get a CI result for the commit that it makes on `main`. It must not wait for the nightly run. One possible fix is to add `push: branches: [main]` to `ci.yml`. The `changes` job already sets `scope=code` for any event that is not `pull_request`, so all jobs would run. Decide: - Do all jobs run on a push to `main`, or only `compile` and `test`? The nightly run also runs the contract job. - Who sees a red result on `main`, and how.

claude commented

2026-10-15 16:33:25 +00:00

Author

Member

Intake: needs-input

1. On a push to main, do all code jobs run, or must the contract job also run?

The contract job runs only on the nightly schedule or on a PR with the contract label. Its if: does not include a push. If we add push: branches: [main], then compile, format, lint, test and dashboard-build run, and contract does not run. The contract suite checks external tools (git, gh, glab, mill, tmux). It does not check our merged code.

(github.event_name == 'schedule' || (github.event_name == 'pull_request' && contains(...'contract')))

Link: https://code.iterative.works/iterative-works/iw-cli/src/branch/main/.github/workflows/ci.yml#L227-L233

a) All code jobs, but no contract job. We change only the trigger. The nightly run continues to find tool drift.
b) Only compile and test. This is a shorter run, but a format or lint error on main stays hidden until the nightly run.
c) All jobs, with contract. We also change the if: of the contract job. This adds a slow run that does not check merged code, on each merge.

Recommended: a) — It closes the gap in the issue with a change to one line, and the contract job keeps its purpose.

2. Who must see a red CI result on main, and how?

Each PR shows its own CI result to its author. A push run on main has no PR to show it on. The workflow does not send a message when a job fails. A red result on main is visible only in the Actions list of the repository. The nightly run has the same problem now.

a) Only the Actions list and the commit status on main. This needs no extra work. A person must look.
b) Forgejo's own notice to the person who pushed. This needs no workflow change if the instance sends it. The person who merged gets the notice.
c) A failure step that writes to a channel (an issue, a chat, an email). This is the most visible option. It needs a secret and an extra step, and the nightly run can use it too.

Recommended: a) — It changes the least. You can add b) or c) later without a change to the trigger.

Answer in a comment on this issue, per number, for example "1: a", or in your own words; intake reads it on its next iteration.

Intake: needs-input ### 1. On a push to main, do all code jobs run, or must the contract job also run? The `contract` job runs only on the nightly schedule or on a PR with the `contract` label. Its `if:` does not include a push. If we add `push: branches: [main]`, then compile, format, lint, test and dashboard-build run, and contract does not run. The contract suite checks external tools (git, gh, glab, mill, tmux). It does not check our merged code. > `(github.event_name == 'schedule' || (github.event_name == 'pull_request' && contains(...'contract')))` Link: https://code.iterative.works/iterative-works/iw-cli/src/branch/main/.github/workflows/ci.yml#L227-L233 - **a) All code jobs, but no contract job.** We change only the trigger. The nightly run continues to find tool drift. - **b) Only compile and test.** This is a shorter run, but a format or lint error on main stays hidden until the nightly run. - **c) All jobs, with contract.** We also change the `if:` of the contract job. This adds a slow run that does not check merged code, on each merge. **Recommended: a)** — It closes the gap in the issue with a change to one line, and the contract job keeps its purpose. ### 2. Who must see a red CI result on main, and how? Each PR shows its own CI result to its author. A push run on main has no PR to show it on. The workflow does not send a message when a job fails. A red result on main is visible only in the Actions list of the repository. The nightly run has the same problem now. - **a) Only the Actions list and the commit status on main.** This needs no extra work. A person must look. - **b) Forgejo's own notice to the person who pushed.** This needs no workflow change if the instance sends it. The person who merged gets the notice. - **c) A failure step that writes to a channel (an issue, a chat, an email).** This is the most visible option. It needs a secret and an extra step, and the nightly run can use it too. **Recommended: a)** — It changes the least. You can add b) or c) later without a change to the trigger. Answer in a comment on this issue, per number, for example "1: a", or in your own words; intake reads it on its next iteration.

claude added the

needs-input

label

2026-10-15 16:33:25 +00:00