E2E: "Remaining Amount" column header disappears after enabling "Effective Amount" (invoices list, Issue #1876 test)

[steilerDev]

BUG: "Remaining Amount" column header disappears after enabling "Effective Amount" (Issue #1876 E2E test)

Severity: Minor
Component: Frontend UI — Invoices list DataTable column preferences (useColumnPreferences)
Found in: e2e/tests/invoices/invoices.spec.ts:841 — "Effective Amount" column (Issue #1876) › Toggling "Effective Amount" shows a deposit/refund-aware value distinct from "Remaining Amount" @responsive (desktop project)

Steps to Reproduce

1. As an authenticated user, navigate to /invoices on desktop viewport (≥1024px).
2. Reset the table.invoices.columns preference (DELETE /api/users/me/preferences/table.invoices.columns) so both "Remaining Amount" and "Effective Amount" start hidden (both are defaultVisible: false).
3. Open the column settings gear and enable "Remaining Amount" (checkbox click, then close popover with Escape).
4. Re-open the column settings gear and enable "Effective Amount" the same way.
5. Assert both columnheader "Effective Amount" and "Remaining Amount" are visible — this assertion passes.
6. Immediately read the "Remaining Amount" column's cell value for a known invoice row.

Expected Behavior

"Remaining Amount" stays visible and its cell value is readable after having just been confirmed visible in step 5.

Actual Behavior

The "Remaining Amount" header is no longer found among visible table headers a moment later — Error: Column "Remaining Amount" not found among visible table headers (thrown from a toPass({ timeout: 3_000 }) retry loop, so this isn't a one-frame race; the header stays gone for the full 3s window). This has reproduced deterministically on every single CI run since it was introduced — 9 consecutive beta-targeted PRs (#1880, #1883, #1885, #1887, #1894, #1902, #1903, #1909, #1916), always in E2E Tests (Shard 5/16), always the same test, same error, ~4 minute runtime, both the initial attempt and the automatic retry. The two PRs immediately prior (#1855, #1858) had this shard passing, so the regression window is exactly PR #1880 (feat/1876-deposit-refunds), which is also where this test and the "Effective Amount" column were introduced.

Because E2E Gates is non-blocking on beta (only required on main-targeted PRs), this has been silently merging red for weeks without being caught.

Environment

• Browser: Chromium (desktop project, 1440px viewport per CI config)
• Viewport: desktop
• Docker: yes, CI-built image (testcontainers)

Evidence

CI log excerpt (PR #1916, run 30675997438, job 91303406982):

✘ 96 [desktop] › tests/invoices/invoices.spec.ts:841:3 › "Effective Amount" column (Issue #1876) › Toggling "Effective Amount" shows a deposit/refund-aware value distinct from "Remaining Amount" @responsive (5.9s)
✘ 101 [desktop] › ... (retry #1) (6.2s)
Error: Column "Remaining Amount" not found among visible table headers
- Timeout 3000ms exceeded while waiting on the predicate
> 398 |     }).toPass({ timeout: 3_000 });
1 failed

Same shard/test/error also present in runs for PRs #1880 (90597079671), #1883 (90646967242, note: PR #1883 additionally had 3 other unrelated shard failures that day — 11/16, 15/16, 6/16 — those look like independent flakes, not this issue), #1885 (90682391864), #1887 (90773257757), #1894 (90887939821), #1902 (91115031204), #1903 (91134593031), #1909 (91175477994).

Notes

Likely root cause area: client/src/hooks/useColumnPreferences.ts. The hook's own local toggleColumn/savePreferences functional-setState logic looks race-free for two back-to-back toggles in the same component instance (each savePreferences(updated, columnOrder) call captures the full accumulated visibleColumns set, and the shared debounce ref just re-schedules to the latest call). Test failure timing (assertion at "both headers visible" passes, then the very next statement — reading "Remaining Amount"'s cell — fails to even find the header) points instead at the debounced upsert() (fires 500ms after the last toggle) racing against the "load preferences on mount" useEffect ([preferences, preferenceKey] deps) — i.e. once the debounced save lands and the preferences array from usePreferences() is refetched/updated, that load effect re-runs and may re-hydrate visibleColumns from a preferences payload that doesn't yet (or no longer) reflects both toggles, dropping "Remaining Amount" back to hidden. Recommend the frontend/backend agents trace usePreferences()'s upsert/refetch sequencing and whether the server-persisted table.invoices.columns value actually contains both keys at the moment the effect re-fires. This is unrelated to the #1895/#1896/#1918 claim/deposit-scope work in the current branch — filed here per E2E process because it was discovered during prior-failure triage and has never been fixed since introduction.

[steilerDev]

[product-owner] Root cause confirmed and split out as a user-facing bug: #1955.

The mechanism your Notes section hypothesised is correct — the debounced PATCH resolving into usePreferences.upsert()'s optimistic setPreferences() creates a new array reference, which re-triggers useColumnPreferences's load effect (deps [preferences, preferenceKey]) and re-applies that PATCH's own payload over newer local state. Verified against source.

One correction worth recording, because the intuitive reading is backwards: fast clicking is the safe case. Two toggles inside 500ms coalesce — the second clearTimeout cancels the first pending save, and functional setState means the surviving payload carries the accumulated set. The race needs a gap longer than 500ms, which is ordinary reading pace, so this is genuinely user-reachable and not a test-only artifact.

Scope split going forward:

• This issue (E2E: "Remaining Amount" column header disappears after enabling "Effective Amount" (invoices list, Issue #1876 test) #1920) = the E2E determinism problem. Closed by awaiting the PATCH response in InvoicesPage.enableColumn() (matching DashboardPage.dismissCard()), landing with the current E2E fix PR.
• BUG: enabling two DataTable columns in a row silently hides the second one (column-preference save races its own re-sync) #1955 = the production race in useColumnPreferences/usePreferences. Prioritised Should Have, S-sized, Backlog.

Please do not close #1955 when shard 5/16 goes green — the E2E fix stops the test from driving the app through the racy window, but leaves the user-facing defect entirely untouched.

[steilerDev]

[orchestrator] Fixed in PR #1956 — but this issue's diagnosis was wrong, and that matters for #1955, so recording the correction rather than closing silently.

What actually caused the failure

.tableHeader (client/src/components/DataTable/DataTable.module.css:305) applies text-transform: uppercase. Playwright's innerText() returns the browser's rendered text, so it yields REMAINING AMOUNT. getColumnCellText() compared that case-sensitively against the mixed-case label 'Remaining Amount', so the match could never succeed — on any run, in any schedule, for any column.

Confirmed from the CI trace (run 30790367863, job 91612852553) by joining before/after events on callId: all five header scans across the full 3-second toPass window returned the same uppercase string, with zero variation.

Corrections to this issue's analysis

• "Deterministically on every single CI run since it was introduced" — correct, but the framing is misleading. PR feat(invoices): add deposit refund entries with negative claim adjustments #1880 is where getColumnCellText() was written, not where a regression landed. The helper never worked. That's also why no amount of timeout widening or retry budget would have helped.
• "The header stays gone for the full 3s window" — the header was never absent. It was present the whole time and never matched. The columnheader role assertions two lines earlier in the same test passed throughout, because an accessible name ignores text-transform. error-context.md snapshots (also ARIA-based) likewise showed both columns present, which was read for two rounds as a transient, self-healing state. There was no healthy state to heal back to.
• The recommendation to trace usePreferences() upsert/refetch sequencing sent two fix rounds in the wrong direction: first a waitForResponse on the preferences PATCH inside enableColumn(), then a shared-user preference-contamination theory. Both were well-argued and neither was the cause.

Consequences for #1955

#1955 stays open and is being fixed. It does not depend on this issue's evidence. Its mechanism was traced independently against source and I re-verified it before routing: usePreferences.upsert() returns a new array reference in both branches (usePreferences.ts:67/:71), useColumnPreferences's load effect has deps [preferences, preferenceKey] (:77), and that effect re-applies the resolving PATCH's own payload — so a save landing after a later toggle clobbers it.

What #1955 loses is its empirical support. This CI failure was cited there as the observed symptom, and it wasn't one. #1955 is a real source-traced defect with no confirmed user report behind it.

What shipped in #1956

• Case-insensitive header matching in getColumnCellText() — the actual fix.
• A second, independent bug this exposed: the test read its invoice off page 1 of an unfiltered list (Showing 1–25 of 37 items, target invoice absent). Those assertions had never executed. Now searches first, and re-asserts column visibility on the filtered page.
• A positive control before the pre-toggle not.toBeVisible() pair, which would otherwise pass vacuously if the table never rendered.
• The waitForResponse in enableColumn() from round 1 was kept — it closes a real ordering hazard and is harmless — but it is not what fixed this.

All 16 E2E shards green on a6ed876b.

Generalisable lesson, since this cost two CI cycles: an ARIA-name assertion passing while an innerText() read fails is the signature of a text-transformation mismatch, not a race. And a test that has failed on every run since it was written is a bug in the test, not a flake — check whether an assertion has ever passed before reaching for timing.