Filed for honesty rather than diagnosis: I could not reproduce this and cannot attribute it.
What was seen
During the final verification of the Bun migration branch, one full live run reported:
467 pass / 1 fail / 1 error
It did not recur in:
- 5 subsequent full live runs by the same agent
- 3 stress runs of the specific suite that had just gained spawned-process tests
- 2 further full live runs (473 pass / 0 fail each)
That is roughly 1 failure in 13 runs. The failing test's name was not captured, so there is nothing to point at.
Why it is being filed rather than dismissed
The suite drives a single shared live BookStack instance and includes spawned processes, a rate limiter with real timing, and health checks with a monotonic clock. Any of those can produce a rare ordering-dependent failure. Two concrete candidates, neither confirmed:
- Earlier in the same session, two separate agents observed transient failures caused by a concurrent edit landing mid-run (a compile error in
src/types.ts written while the suite was executing). Both passed on re-run. If the flake was that, it cannot happen in CI or a clean checkout.
- The suite intentionally exercises throttling and waiter-shedding, which are timing-sensitive by nature.
Reproduction
Unknown. To hunt it:
docker compose up -d db bookstack
for i in $(seq 1 20); do
RUN_INTEGRATION=1 bun test 2>&1 | tee /tmp/run-$i.log | tail -3
done
grep -l "fail" /tmp/run-*.log
Capture the failing test name and stderr when it fires.
Notes
- CI is unaffected: CI runs
RUN_INTEGRATION=0, which skips the live suite entirely (501 tests, exit 0, consistently).
- The live suite is a local/manual tool (
bun run test:integration), so a rare flake does not gate merges today — but it does undermine trust in the suite, which is the point of having it.
Filed for honesty rather than diagnosis: I could not reproduce this and cannot attribute it.
What was seen
During the final verification of the Bun migration branch, one full live run reported:
It did not recur in:
That is roughly 1 failure in 13 runs. The failing test's name was not captured, so there is nothing to point at.
Why it is being filed rather than dismissed
The suite drives a single shared live BookStack instance and includes spawned processes, a rate limiter with real timing, and health checks with a monotonic clock. Any of those can produce a rare ordering-dependent failure. Two concrete candidates, neither confirmed:
src/types.tswritten while the suite was executing). Both passed on re-run. If the flake was that, it cannot happen in CI or a clean checkout.Reproduction
Unknown. To hunt it:
Capture the failing test name and stderr when it fires.
Notes
RUN_INTEGRATION=0, which skips the live suite entirely (501 tests, exit 0, consistently).bun run test:integration), so a rare flake does not gate merges today — but it does undermine trust in the suite, which is the point of having it.